Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

region splits, but not assigned to other region servers

Difficulty using importtsv tool
(25 lines)
Incoming Row Distribution Strategy/Algorithm Among Region Servers?
(16 lines)
Jun 15, 2011
Sujee Maniyam
Sujee Maniyam
Hi all

I am loading data into a table using a standalone java client (multiple
threads).  I see the table splits into regions, but for a while the new
regions are not distributed among region servers. (see screenshot, in this
case all regions are in SN8 machine).  So the updates go to one RS as
opposed to many RSs.

Eventually the regions do get spread around.

In RegionServer logs I do see messages like following:
2011-06-14 15:50:44,319 INFO
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Region split, ME
TA updated, and report to master.
Parent=logs,,1308091368394.a4e9b514dbb9061cd444d5a1104de5b9., new re
gions: logs,,1308091842570.87cc920ee45add9c094c9e872774e2d1.,
logs,\x00\x07\xA0\xBC\x00\x00\x00\x01\x7
F\xFF\xFE\xDA\x17\x03\x1Ac,1308091842570.39dc0057e0bdbcdaa8bd51b5da0dfd3f..
Split took 1sec

version : Hbase 0.90.1-cdh3u0, r
region size : 1G

is this the expected behavior?

thanks
Sujee
http://sujee.net


Reply
Tags: standalone javamultiple threadsloading datajava clientservers
Messages in this thread
region splits, but not assigned to other region servers
Region splits in 0.89...
August 26, 2010 02:00:44 PM
My hbase table issued a mass split after I loaded regions with greater sizes than maxfilesize.. (my bad..) Now, when I try accessing the master through the web interface, it just hangs... And, if I scan the META, I get the parent regions set to…
createTable with specified region splits: works great
February 15, 2011 12:53:40 PM
Pre-creating regions using the byte[][] overload of createTable more or less doubled the performance of our main index table generation. Our keys start with hashes of the original record IDs, so the data can be evenly distributed between all…
Re: bug report: opening hbase region takes too long , making the region not available for more than
August 19, 2010 11:38:45 AM
Jonathan: We saw similar issue using HBASE 0.20.6 with HBASE-2473 Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in .META. for region HB_INC_POST_0818-ERROR_SAMPLES-1282193650093,,1282193650831 at…
Re: Region servers down...
August 24, 2010 01:17:20 PM
The last log to look at would be the .out file. J-D 2010/8/23 xiujin yang <xiujin### @hotmail.com>: > > Thank you J-D, > > I posted today's whole RS log: > http://pastebin.com/djGnNJxk > > GC log: >…
Re: Region servers down...
August 24, 2010 11:10:12 AM
It would be beneficial to separate the RS on 192.168.158.179 onto another machine. 2010/8/23 xiujin yang <xiujin### @hotmail.com> > > Hi > > My cluster is in this way. > Hadoop & HBase are deployed on different machine.…
RE: Region servers down...
August 24, 2010 08:37:08 PM
Thank you J-D. The out file is like this. It has an "NullPointerException" error. 2010-08-24 02:30:14.187::INFO: Logging to STDERR via org.mortbay.log.StdErrLog 2010-08-24 02:30:14.187::INFO: jetty-6.1.14 2010-08-24 02:30:14.122::INFO: …
Re: Region servers down...
August 24, 2010 01:01:31 AM
I don't really see the cause of the shutdown in there, it seems it was already under way. Do you see messages starting with "We slept" and then telling how long it slept? It should be not very far from that in the log. J-D 2010/8/23 xiujin yang…
Re: region servers shutdown
February 10, 2011 03:00:12 PM
Keys are randomized..Requests are nearly equally distributed across region servers (use to have sequential but that created region hot spots)..However current scheme requires our map reduce job to look for events in all regions (using hbase time…
region servers crashing
July 14, 2010 05:03:52 PM
Hi all, We've been having issues for a few days with HBase region servers crashing when under load from mapreduce jobs. There are a few different errors in the region server logs - I've attached a sample log of 4 different region servers crashing…
region servers shutdown
February 10, 2011 12:44:14 PM
Hi I've had this before ..but not to 70% of the cluster..region servers all dying..Any insight is helpful. Using hbase-0.20.6, hadoop-0.20.2 I don't see any error in the datanode or the namenode many thanks Here's the relevant log entires …
Re: region servers shutdown
February 10, 2011 02:46:02 PM
Are your keys sequential or randomized? On Thu, Feb 10, 2011 at 12:35 PM, Venkatesh <vramana### @aol.com> wrote: > iii) Processing about 600 million events per day (real-time put) - 200 > bytes per put. Each event is a row in a…
Crashing Region Servers
December 9, 2010 01:28:41 PM
We have a 6 node cluster, 5 with region serves. 2 of the region servers have been stable for days, but 3 of them keep crashing. Here are the logs around around when the crash occurs. (btw, we are shoving approximately the twitter firehose into…
Region servers exiting, not recovering
September 21, 2010 07:21:38 PM
Hi, so in our production, we see temporary networking failures (we are not quite 100% sure what they are) but now and then region server's zookeeper session would get expired and in addition some ipc channels would throw 'channel closed'. This…
HBase 0.90.0 region servers dying
February 16, 2011 02:41:32 AM
Hi, We have a newly setup a cluster of 5 nodes, each with 16 GB rams. We use HBase 0.90.0 on top of Hadoop from CDH3. When testing HBase under heavy load generated bu YCSB, we consistently see region servers dying silently, without any logs or…
Re: HBase querying across region servers
April 28, 2011 05:12:58 PM
(please don't write back personally unless it's really personal) That's all fine, rows are always contained in a single region. By state I meant if you created some fancy filter yourself and decided to keep some state where the filtering of one…
hmaster reports 0 region servers
October 14, 2010 11:07:47 PM
I applied the patch for HBASE-2939. (The patch is for 0.89 but my code is 0.20.6, I checked the patch found it only changed one connection thread at client side to a pool strategy.) But when I rebuild the source and start hbase cluster. The master…
Region servers suddenly disappearing
October 10, 2010 04:14:43 PM
Some of the region servers suddenly dying..I've pasted relevant log lines..I don't see any error in datanodes Any ideas? thanks venkatsh ..... 2010-10-10 12:55:36,664 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception:…
region servers not shutting down (v 0.90.1-cdh3u0, r)
June 10, 2011 02:25:22 PM
Hi all I am running Hbase on a 6 node cluster. HBase comes up fine, I can create a test table and put rows and scan. But I can't cleanly shut it down. the stop-hbase command goes on for ever printing dots. And I can see a couple of…
Region server blocking updates for a region for 90s
September 24, 2010 11:12:05 AM
Hey, We seem to have come across a bug in hbase with how it is flushing the memstore when it is full, I think it's related to this…
hbase region server/region information
June 6, 2011 10:12:40 PM
Hi, I just loaded a bunch of data into Hbase , and want to know the number of regions created, the region servers they are assigned to, and the region size being managed by each region server. Are there any tools/utilities I can use to quickly…
How to set region with _SESSION
April 26, 2011
hi how can I set the region (en, da, de etc) with mod_rewrite? the rewrite condition has to look like this www.domain.com/en/ www.domain.com/dk/…
C# Regex for a match outside a specific region
April 8, 2011
I have to find occurrences of a certain string (needle) within another string (haystack) that don't occur between specific "braces". For example…
Should I Populate Country/Region Names at DB or Application Level?
January 9, 2011
I'm going through a tough decision (at least for me)... My website would have a list of countries in dropdown list and list of cities based on…
MySQL relations for country region and town tables
February 18, 2011
Hi all, I am trying to figure out the best method to relate country, region and town tables. On my website I want the user to be able to just enter…
Tool to do batch lookup of DNS servers assigned to specific domains
April 28, 2011
While there are a handful of tools that let you drop a list of domain names and return any DNS record pertaining to those domain names, I have not…
Read/Write splits using Zend_Db
May 30, 2011
I have a PHP application that has grown in size. The database used to be on a single master but we intend to change that with a fairly standard…
How are IP addresses actually assigned?
May 31, 2011
I'm having a hard time understanding how a governing body assigns IP addresses, companies use BGP to advertise those IPs, and how the internet…
Ruby regex that splits on newline, space, or comma
February 14, 2011
Input: f = ["happy days", "happy\ndays", "happy,days", "happy, days"] patt = /some_regex/ My desired output after splitting each string in f on patt…
This security ID may not be assigned as the owner of this object.
March 16, 2011
Hi, I am getting this error 1299 "This security ID may not be assigned as the owner of this object" when i try to change directory to FTP Folder…
Hadoop MapReduce - Pig/Cassandra - Unable to create input splits
November 16, 2010
Hey out there, I'm trying to run a MapReduce Job with Pig and Cassandra and I always get the error: ERROR 2118: Unable to create input splits for:…
How to get machine's browser to resolve to domain assigned to self?
May 20, 2011
I have a machine running Apache HTTP server which works fine hosting the website for all machines except itself. When trying to load itself, I get…
Is it beneficial to add more hbase servers without adding more hdfs servers?
May 20, 2011
I'm using Eucalyptus and am considering putting hdfs and hbase on our node controllers. Would running hbase on some of our instances improve…
Mongodb do i need to put my config servers on separate servers?
April 16, 2011
i assume yes since if an instance goes down all 3 go down?
Websockets between 2 servers?
April 1, 2011
I wonder if it's possible to use websockets between 2 webservers instead of between a browser and backend? Im running 2 node.js servers and want to…
Help with servers, hosting, ...
January 31, 2011
Hey guys, I'm in the process of creating a new website but I know nothing about servers and hostings... And so when I'm done with the website I have…
Two MySQL servers
April 20, 2011
Hi, I want to use 2 SQL servers. I can run one on port 3306 succesful, but now I'm trying to create a whole new server on 3307. This is my config in…
How to edit name servers
April 30, 2011
I purchased a domain name for my website but the files are hosted some other place. The domain registration website show four name servers…
How, and when, do I add servers for my website?
March 24, 2011
I'm going to release a website in a week or so, and I'm gonna upload it first on a public server. My question is, if I get many users, how would I…
DNS problem, old dns servers are not going away
February 7, 2011
I have a website , and I changed the nameservers of the domain of my site, now people see it with my new nameservers however I still see the…
Can someone look into a web servers folders?
December 20, 2010
More a web server security question. Is it possible for someone to "probe" into a folder on a web server, even if an index file is in place? I…