Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

Rebalance: empty_runs

How to delete host?
(55 lines)
Re: 回复: Re: the question of mogilefs version 2.55
(26 lines)
Jan 24, 2012
MadCamel
MadCamel
I've been trying to rebalance for several days now, and no new fids are 
being queued. empty_runs continues to increment, along with sdev_lastfid.

Can someone describe what exactly an empty run in the context of rebalance 
is? I'm semi-familiar with some of mogile's internals, but not the new 
rebalance system.

I've tried fiddling with some settings as per the wiki's suggestion with no

luck, bouncing mogilefsd, etc.


Rebalance status:
             bytes_queued = 0
           completed_devs =
               empty_runs = 433
              fids_queued = 0
                    limit = 0
             sdev_current = 33
             sdev_lastfid = 427819311
               sdev_limit = 5368709120
              source_devs = 32,90,4,98,34,182,117,5,...(alot)
            time_finished = 0
    time_latest_empty_run = 1327389078
          time_latest_run = 1327389078
             time_started = 1327388688
             time_stopped = 0

mogadm settings list
         enable_rebalance = 1
     internal_queue_limit = 2000
      queue_rate_for_fsck = 2000
     queue_rate_for_rebal = 200
      queue_size_for_fsck = 40000
           schema_version = 14




Reply
Tags: fidsincrement
Messages in this thread
Rebalance: empty_runs
reply Re: Rebalance: empty_runs
(23 lines) Jan 24, 2012 19:59
Neo Rebalance!
September 19, 2010 01:58:50 AM
Yo, I've finally uploaded a working version of the rebalance implementation I've been thinking about for forever. It's inteface is very raw, and I'd like to hear feedback about ways to make it easier to use. I'll be filling in a wiki page with…
Re: Does anyone actively use rebalance?
July 31, 2010 03:33:06 AM
Rebalance has been immensely useful to us. When adding new storage nodes, we are able to mitigate the damage from HD failures, are able to add nodes more sparingly, and just generally balances out load across storage nodes. ( 12 storage nodes, 6…
Rebalance error
June 15, 2011 10:04:39 AM
Hi, !watch is reporting this while running rebalance: :: [replicate(11617)] Rebalance for DevFID[d=133;f=7675034] (http://10.0.1.103:7500/dev133/0/007/675/0007675034.fid) failed: HTTP delete (due to 'did_rebalance;ret=1') failed: can't connect…
Re: Rebalance error
June 16, 2011 01:43:20 AM
No. Just a few per minute... On 06/16/2011 08:39 AM, dormando wrote: > > On Thu, 16 Jun 2011, Arkadi Colson wrote: > >> Can this be the reason that rebalance does not cleanup the over-replicated >> files? > How many of…
Rebalance hangs
August 31, 2011 07:11:29 PM
I'm using 2.53 and rebalance is hanging. I'm not exactly sure where to poke to figure out why. Nothing indicitave of a problem is showing up in !watch or syslog. Any suggestions? To reproduce the problem I run the following commands: mogadm…
Rebalance does not work well
October 17, 2011 09:54:37 PM
Hi all,I have raised a issue ,see https://issues.apache.org/jira/browse/AMQ-3544.
t/30-rebalance.t failure with IO::AIO 4.11
November 29, 2011 02:51:45 PM
t/30-rebalance.t fails on my Debian testing laptop, which has libio-aio-perl 4.11-2+b1. Towards the end of the test, I get a lot of 404 errors from DELETE. The 404 errors shuffle the "ok 48" message into the middle of the line and hides it from…
Re: mogile rebalance
December 19, 2011 07:56:21 AM
Hi , after adding the new device into the cluster i believe mogile completed replicating the files to the new device dev5 and now I wanted to run rebalance in order to distribute the file system load equally .. this is how the devices look now…
Re: mogile rebalance
December 19, 2011 10:49:41 AM
> this is the policy I ran ..to test > > mogadm rebalance policy --options="from_devices=dev3,dev4 to_devices=dev2,dev5 limit_type=device limit_by=size limit=10g" > > mogadm rebalance test > Tested rebalance policy... >…
Rebalance cluster
January 11, 2012 01:32:49 PM
Hi All, We have 5 nodes cluster(on 0.8.6), but two machines are slower and have less memory, so the performance was not good on those two machines for large volume traffic.I want to move some data from slower machine to faster machine to ease…
Rebalance Stuck
May 23, 2011 01:33:56 PM
We have a mogilefs cluster with a stuck rebalance. It won't stop, I can't change settings, and because of this I can't do a fsck. kel### @mogtracker1prod:~$ mogadm rebalance settings rebal_host = mogtracker2prod …
mogilefs rebalance
May 9, 2011 03:58:35 AM
Dear Listmembers, I have a question about mogilefs rebalance. We're using mogilefs in a virtual machine, and we have to migrate it. The question is, that if the rebalance is too slow for us, can we use rsyncing instead? After rsync, we simply…
Rebalance / fsck
January 16, 2012 03:55:14 AM
This is a multi-part message in MIME format.Hi Our MogileFS setup has 2 strange problems. We are running mogileFS on 2 locations: GS and DCG (as you can see in the hostnames of the storage hosts). There are as many hosts on GS as on DCG. Also…
Rebalance stuck?
September 8, 2010 05:46:17 AM
Hi, I have a problem with enable_rebalance. It seems that after a while it stopped working. Now i read in Store.pm about List::Util::shuffle() not being really random, is this still true? I've also setup a test environment to analyse the…
Rebalance.pm patch
October 28, 2010 05:39:20 AM
Hi, I'm finally trying the new Rebalance stuff and stubbled upon an error when using a global limit. When the JobMaster tries to load the saved rebal_state, Rebalance.pm croaks because it doesn't recognize the saved setting 'limit'. bye, Martijn.…
Failure Detector and Rebalance
November 7, 2011 03:01:34 PM
Does failure-detector (any implementation) is rebalance-aware? If a node is added to the cluster, Is this node part of the nodes being checked in any of the failure detector implementations? thanks, --john
Reproductive for issue of rebalance
October 15, 2011 03:00:03 AM
Hi all,assuming there are two networked brokers A and B in both direction . Now setting up a client to connect to Broker A and making sure that the client has connected to Broker A via log information. Then shutdown Broker A and we expect for…
Rebalance docs, Roadmap, etc
September 19, 2010 07:51:34 PM
First off, I have posted initial documentation for the unreleased rebalance overhaul: http://code.google.com/p/mogilefs/wiki/Rebalance Next, I have posted a quick Roadmap of all the major features I've been planning on working on: …
source socket, rebalance issues
August 19, 2010 02:18:34 PM
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm running mogilefs 2.30 with 14 hosts with 4 devices each. When I !watch on one of my trackers, I see lots (several to many a minute) of messages like these: :: [replicate(9979)] Unable to create…
Re: source socket, rebalance issues
August 23, 2010 06:28:41 PM
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 8/23/10 12:24 PM, dormando wrote: >> It looks like the maxconns default is 10K and I don't set it explicitly. >> I assume that is connections that would show up in netstat? I'm seeing…
How can I get Cassandra to automatically rebalance?
April 24, 2010
I am looking at a cassandra cluster, but the administration effort seems to be quite high. Is there any way I can configure Cassandra to rebalance…