Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

Freeing Up Region and/or merging region

MapReduce configuration issues
(28 lines)
HBase Meetup during Hadoop World NYC '11
(29 lines)
Aug 26, 2011
Rohan Rai
Rohan Rai
On suggestion by JDC

Creating a separate thread

On a smaller cluster pre allocating a certain memory to region leads to
continuous increase in heap space usage
with increasing usage.

Hence clean up of data is performed by me.

But region having no data does not get de allocated automatically and it
still occupies the preallocated memory.

I was a looking for a clean mechanism to achieve that.


Either merging two continuous empty region or deleting a region cleanly.

Unfortunately my efforts to mess directly with metadata and cleaning up
of region dirs on cluster
has resulted in my tables getting corrupt.

Basically any way to cleanly get back those precious memory.

Regards
Rohan





Reply
Tags: heap spacecontinuous increasespace usagejdcsuggestionmemory
Messages in this thread
Freeing Up Region and/or merging region
reply Re: [Freeing Up Region and/or merging region
(53 lines) Aug 26, 2011 12:40
Re: bug report: opening hbase region takes too long , making the region not available for more than
August 19, 2010 11:38:45 AM
Jonathan: We saw similar issue using HBASE 0.20.6 with HBASE-2473 Caused by: org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in .META. for region HB_INC_POST_0818-ERROR_SAMPLES-1282193650093,,1282193650831 at…
hbase region server/region information
June 6, 2011 10:12:40 PM
Hi, I just loaded a bunch of data into Hbase , and want to know the number of regions created, the region servers they are assigned to, and the region size being managed by each region server. Are there any tools/utilities I can use to quickly…
region splits, but not assigned to other region servers
June 15, 2011 02:55:09 AM
Hi all I am loading data into a table using a standalone java client (multiple threads). I see the table splits into regions, but for a while the new regions are not distributed among region servers. (see screenshot, in this case all regions are…
Region server blocking updates for a region for 90s
September 24, 2010 11:12:05 AM
Hey, We seem to have come across a bug in hbase with how it is flushing the memstore when it is full, I think it's related to this…
Region locality and .META. region
May 16, 2011 01:26:10 PM
Hi, I have two questions: 1. Does HBase knows how to handle blocks moving. e.g does HBase can recognize that some local block deleted from machine and move that region to machine with that block? 2. What happen if the region server of the .META.…
Trying to contact region "Some region"
February 22, 2011 08:28:52 AM
What does this exception signify: org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server Some server, retryOnlyOne=true, index=0, islastrow=false, tries=9, numtries=10, i=0, listsize=405,…
region doesn't split after 32+ GB
September 27, 2010 06:27:03 PM
I'm sequentially importing ~1 billion small rows (32 byte keys) into a table called StatAreaModelLink. I realize that sequential insertion isn't efficient by design, but I'm not in a hurry so I let it run all weekend. It's been proceeding quickly…
region not served
October 23, 2010 04:01:26 PM
This is after namenode crash, and cold restart of everything, it seems that meta data is wrong about multiple regions; http://pastebin.com/yMVmqpUY http://pastebin.com/8hXqJM7R Any ideas? Maybe some flushing race condition when a region was…
What's the region server doing?
March 1, 2011 07:08:49 PM
My cluster (10 nodes, hbase-0.20.6 + hadoop 0.20.2) is very very slow for any operation like disable table or delete. Master's thread dump says they are blocked by the metaScanner thread. When I looked at the log file on the .META RS there are no…
Re: Region servers down...
August 24, 2010 01:01:31 AM
I don't really see the cause of the shutdown in there, it seems it was already under way. Do you see messages starting with "We slept" and then telling how long it slept? It should be not very far from that in the log. J-D 2010/8/23 xiujin yang…
Re: Region servers down...
August 24, 2010 11:10:12 AM
It would be beneficial to separate the RS on 192.168.158.179 onto another machine. 2010/8/23 xiujin yang <xiujin### @hotmail.com> > > Hi > > My cluster is in this way. > Hadoop & HBase are deployed on different machine.…
Re: Region servers down...
August 24, 2010 01:17:20 PM
The last log to look at would be the .out file. J-D 2010/8/23 xiujin yang <xiujin### @hotmail.com>: > > Thank you J-D, > > I posted today's whole RS log: > http://pastebin.com/djGnNJxk > > GC log: >…
RE: Region servers down...
August 24, 2010 08:37:08 PM
Thank you J-D. The out file is like this. It has an "NullPointerException" error. 2010-08-24 02:30:14.187::INFO: Logging to STDERR via org.mortbay.log.StdErrLog 2010-08-24 02:30:14.187::INFO: jetty-6.1.14 2010-08-24 02:30:14.122::INFO: …
Re: Region replication?
April 19, 2011 03:10:04 PM
This is kind of true. There is only one regionserver to handle the reads, but there are multiple copies of the data to handle fail-over. On Tue, Apr 19, 2011 at 12:33 PM, Otis Gospodnetic <otis_gosp### @yahoo.com> wrote: > My question…
Region server OME
April 17, 2011 08:02:51 PM
Hi, My HBase deployment has been working great but recently the regionservers always got out of memory error. The symptom is like this: region servers failed with OME, and then I restarted them. One of region servers used up the maximum memory I…
getting region PUT errors.
September 28, 2010 12:54:33 PM
I am using rest and getting following errors on some insertions: 2010-09-28 10:51:07,215 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out of range for HRegion…
Region splits in 0.89...
August 26, 2010 02:00:44 PM
My hbase table issued a mass split after I loaded regions with greater sizes than maxfilesize.. (my bad..) Now, when I try accessing the master through the web interface, it just hangs... And, if I scan the META, I get the parent regions set to…
Region loadbalancing
December 13, 2010 10:37:49 AM
Hi all, we are using HBase 0.20.6 on a cluster of about 25 nodes with about 30k regions and are experiencing as issue which causes running M/R jobs to fail. When we restart single RegionServer, then happens the following: 1) all regions of…
Region Balancing
February 2, 2011 01:52:04 PM
I know there were some changes in .90 in terms of how region balancing occurs. Is there a resource somewhere that describes the options for the configuration? Per Jonathan Gray's recommendation we are trying to keep our region count down to 100 per…
How to delete a region?
July 22, 2011 04:25:56 AM
Hi all, Is there a way to delete one region from table? We now have two regions with the same startkey in our table, one of them is wrong(it is empty), how can I delete it safely? Thanks, Yi
How to set region with _SESSION
April 26, 2011
hi how can I set the region (en, da, de etc) with mod_rewrite? the rewrite condition has to look like this www.domain.com/en/ www.domain.com/dk/…
HBase: How to delete region
June 20, 2011
Due to legal issues I need to delete old data from HBase. I found a way to split the region using my criteria and by that moving the old data to the…
C# Regex for a match outside a specific region
April 8, 2011
I have to find occurrences of a certain string (needle) within another string (haystack) that don't occur between specific "braces". For example…
MySQL relations for country region and town tables
February 18, 2011
Hi all, I am trying to figure out the best method to relate country, region and town tables. On my website I want the user to be able to just enter…
Should I Populate Country/Region Names at DB or Application Level?
January 9, 2011
I'm going through a tough decision (at least for me)... My website would have a list of countries in dropdown list and list of cities based on…
Ubuntu: Freeing up ports?
August 14, 2010
Hi everybody, I have tried to install cassandra on my Ubuntu test environment, but what happens when I try to start it is that I get the following…
Merging SQL Records
April 4, 2011
hey, i'm not sure if this is the right place to ask this, but I'm having troubles with a script of mine. i'm trying to do some database merging with…
Merging two 301 redirects into one in .htaccess
February 27, 2011
I want to redirect www.mysite.com/* to mysite.com/* and /forum/* to /* The lines I managed on .htaccess are: RewriteEngine on #www.mysite.com ->…
Mysql merging columns
April 13, 2011
Hi, Say i have the following MySql 'orders' table: id|car|car_qty|speakers|speakers_qty 1 | 2 | 15 | 5 | 16 2 | 2 | 19 | 5 | 40 What is the query to…
Merging 2 MySQL Databases
March 8, 2011
I have 2 Databases that basically have the same structure, but different data. (The older Database has 2 extra tables) Is there any way for me to…
Merging individual tables?
January 29, 2011
I do have 5 tables. table1,table2,table3,table4 and table5 all these tables index are same. which means for example for john75de table1 got personal…
Merging mongodb files in different location
March 15, 2011
Hi all, May i know how do i merge my mongodb data files? The following is my current situation: At /data/db: drwxr-xr-x 3 root root 4096 2011-03-15…
Merging XML file with MySQL table
April 8, 2011
Hi. I have XML file which looks like this: <?xml version="1.0"?> <results> <row id="100"><name>name…
Merging MongoDB MapReduce output
June 16, 2011
I am currently using MongoDB's MapReduce to generate hourly ad view counts like this: { _id : "4/1/2011 9:00 AM", value : { AdViews_Total : 4 } }…
Merging similar tables in SQL part 2
April 18, 2011
I think people were reluctant to help me last time because I didn't want to post up the columns, but I'm eager to solve this, so I'm putting aside…
Merging two sites into one, how to redirect from the domain that's going away?
April 29, 2011
I haven't been able to find any existing questions that cover my exact issue, so here goes: My client wants her two sites (domain1.com and…
Merging Multiple Tables in different databases
January 31, 2011
Hi All, I have multiple tables with the same format(same column names) in multiple databases (From sqlite3). Does anybody know how to merge the data…
Simplify two MySQL queries by merging them into 1?
January 8, 2011
I have two MySQL queries which are very similiar however they return a COUNT() upon a specific WHERE clause ( type ), so I was wondering if they…
Merging two SQLite database files (C# .NET)
December 28, 2010
Hello all, I'm using C#/.NET with the C# wrapper for SQLite. I'm attempting to merge two SQLite databases together while excluding duplicates. I…
Merging two folder with mod-rewrite in .htaccess
January 13, 2011
Hi there, I'm currently developing a drupal based website but will have to keep a legacy system and many static pages/folders alive. It would be…