Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

2 Datacenter sharding advice wanted

MongoDB] What to do when periods exist in MapReduce key?
(22 lines)
Category and attributes
(17 lines)
Feb 7, 2012
Jon Schneider
Jon Schneider
We are currenty running out of one datacenter but will soon be moving
to another and our current datacenter will become the failover
location.

Currently running 2 shards made up of a replica set of 3 server each.

Just looking for an experienced opinion here to tell me if I am way
off in the bush or if my plans look sane.

I was thinking of adding a 4th member to the replica sets as well as
an arbiter to bring the total to 5 votes in each set.

Then I am going to move 2 of the set members and the arbiter instances
to the new primary datacenter.  Will also move 2 of the config servers
to the primary datacenter.

So in the end it will look something like this:
Datacenter A (primary):
Shard A - member 1
Shard A - member 2
Shard A - Arbiter
Shard B - member 1
Shard B - member 2
Shard B - Arbiter
Config 1
Config 2

Datacenter B (failover):
Shard A - member 3
Shard A - member 4
Shard B - member 3
Shard B - member 4
Config 3

On failover (which should never happen) the arbiter server would be
powered on in DataCenter B via VMWare SRM and all should be happy
other then only one config server being available so the shard config
would be frozen but that's fine since we'll be happy enough to even be
up.

As another note the Arbiter servers in Datacenter A will not be able
to talk directly to the set members in Datacenter B due to the
arbiters being in a special SRM network that exists in both
Datacenters.

Opinions welcome,





Reply
Tags: datacentershards
Messages in this thread
2 Datacenter sharding advice wanted
reply Re: 2 Datacenter sharding advice wanted
(15 lines) Feb 8, 2012 13:40
reply Re: 2 Datacenter sharding advice wanted
(45 lines) Feb 8, 2012 15:05
reply Re: Re: 2 Datacenter sharding advice wanted
(58 lines) Feb 8, 2012 16:24
reply Re: 2 Datacenter sharding advice wanted
(15 lines) Feb 8, 2012 16:40
Advice wanted on modeling
January 11, 2011 12:07:57 PM
Hi, I've been experimenting quite a bit with Cassandra and think I'm getting to understand it, but I would like some advice on modeling my data in Cassandra for an application I'm developing. The application will have a large number of records,…
Any advice for design non-sharding for future sharding?
March 30, 2011 10:30:00 PM
Now , because mongoDB cannot return matched elements in array, so we have to run js function, so we cannot use sharding although we really want to shard the collections May be it can be done in 2.0 (http://jira.mongodb.org/browse/ SERVER-828) Any…
MongoDB sharding / replica set / multiple datacenter / slaveok reads
August 9, 2010 05:29:46 PM
Hi, how do i place config servers + can i configure the mongos servers in each dc to use local slaveok reads for readonly connections in each dc? Writes use a 2nd standard replica set connection but aren't of interest in this case. e.g. A request…
Advice (sanity check) on sharding strategy for ad-hoc queries
December 8, 2010 01:22:00 PM
Hi all I would value some advice on how best to configure mongoDB for ad-hoc queries. We currently use mongoDB as the primary data store but use an MSSQL table for historical data. I would like to investigate the possibilities for removing MSSQL…
Datacenter migration
May 10, 2011 07:55:13 AM
Hi, We have a cluster of ~20 nodes all located in 1 datacenter running Cassandra 0.7.2. We are planning to move from this datacenter to another with as minimal downtime as possible. The first strategy that came into my mind is to use the…
datacenter ShardStrategy
May 8, 2011 10:09:41 PM
Hi All, I want to use datacenter ShardStrategy in my cassandra setup .Can someone please let me know what steps / configuration changes I need to make. Thanks Anurag
datacenter awareness
January 6, 2011 03:29:12 AM
This is a multi-part message in MIME format.Hi, I have a question about datacenter awareness. /Setup:/ -> 4 trackers, 2 trackers in each datacenter -> 8 storage, 4 storage servers in each datacenter DCG (*10.0.1.96/28*): mogfs1-dcg -…
Sharding setup and strategy for sharding key in a collection
November 10, 2010 02:07:41 PM
Hi all, A few questions about sharding setup and assigning sharding keys within a collection. 1. We have 2 shared machines allocated for mongoDB ( Shared meaning, other team members may use those servers for running hadoop or some ad-hoc…
Message Forwarding to another datacenter
June 9, 2011 04:27:46 AM
Hi all, What is the preferred way to reliably forward messages from an AMQ broker in one datacenter (DC1) to another AMQ in another datacenter (DC2)? Is it network of brokers - like: <networkConnector uri="static:(tcp://localhost:61617)" …
cross datacenter replication
February 11, 2011 12:26:32 PM
Hello. I'm in the process of trying out mongodb for the user data storage for our website. My current task is to evaluate possible approaches for asynchronous data replication between the 2 data centers we have. The link between those is…
Quorum and Datacenter loss
December 12, 2010 06:50:35 AM
Hi cassandra experts - We're planning a cassandra cluster across 2 datacenters (datacenter-aware, random partitioning) with QUORUM consistency. It seems to me that with 2 datacenters, if one datacenter is lost, the reads/writes to cassandra …
Datacenter replication/failover
May 4, 2011 12:33:51 PM
In March of last year there was discussion of supporting replication across datacenters * A single cluster that spans two data centers, with special allocation strategy that make sure that a shard and its replica do not exists on the same data…
Quorum, Hector, and datacenter preference
March 24, 2011 08:03:36 AM
Hi - Our cluster is spread between 2 datacenters. We have a straight-forward IP assignment so that OldNetworkTopology (rackinferring snitch) works well. We have cassandra clients written in Hector in each of those data centers. The Hector…
Re: Re: Re: Re: Quorum, Hector, and datacenter preference
March 25, 2011 12:33:24 PM
Glad it worked 2011/3/25 <jonathan### @gmail.com> > very cool. thanks for the info. this is exactly what we need. > > > On Mar 25, 2011 8:22am, Patricio Echagüe <patr### @gmail.com> wrote: > > > > It's a…
Added check for datacenter in cluster
August 18, 2011 04:04:57 AM
From: Francesco Vollero <fvoll### @redhat.com> Signed-off-by: Francesco Vollero <fvoll### @redhat.com> --- .../lib/deltacloud/drivers/rhevm/rhevm_client.rb | 10 ++++++++-- .../lib/deltacloud/drivers/rhevm/rhevm_driver.rb | 1 +…
Cassandra Multiple DataCenter Suitability - why?
June 18, 2010 03:20:40 AM
Hello, I keep reading everywhere that Cassandra has supported multiple datacenters from the beginning. I would like to know what does Cassandra do to achieve that. Is it just that the developers have written some code that supports that scenario,…
Cassandra Multiple DataCenter Suitability - why?
June 17, 2010 12:44:32 PM
Hello, I keep reading everywhere that Cassandra has supported multiple datacenters from the beginning. I would like to know what does Cassandra do to achieve that. Is it just that the developers have written some code that supports that scenario,…
Re: Cassandra Multiple DataCenter Suitability - why?
June 18, 2010 01:51:01 AM
Alright, I thought I'd post it here because it has to do with Cassandra internals. I will repost it correctly. Alexander > > Jonathan Shook > On Thu, 17 Jun 2010 12:58:34 -0700 > This is a (yet another) question for the users list.…
questions on cassandra (repair and multi-datacenter)
September 16, 2010 05:20:39 PM
Hi, I have a few questions and was looking for an answer. I have a cluster of 7 Cassandra 0.6.5 nodes in my test setup. RF=2. Original data size is about 100 gigs, with RF=2, i see the total load on the cluster is about 200 gigs, all good. 1. I…
multi datacenter cluster, without fibre speeds
November 11, 2011 12:15:31 PM
Has anyone experimented running cassandra clusters in geographicly separated locations connected thru ordinary broadband? By ordinary broadband I mean 30Mbps or 50Mbps Thanks Marco
Datacenter IP Addressing and DNS Management
January 1, 2011
Hello everyone Basically we are setting up a small Datacenter, about 300 amps power and max 50 racks, Im saying these coz I wanna u imagine the size…
MySQL, PHP help wanted
June 20, 2011
I am quite a beginner to MySQL. I have this simple expression which I need to filter. $a = "2"; $s1 = sprintf("%s,",$a)."%"; $s2 =…
Wanted: SSL certificate analyzer
April 4, 2011
We have a server that needs to work with SSL. We're having some problems with the SSL certificates. Does anybody know a good certificate analyzer…
Mod rewrite not working help wanted(htaccess) :)
April 5, 2011
Hello, I wanna rewrite this url fusion-stable-zendopen-4-01-204/upload/index.php?/Ontruimingsvonnis/Tickets/Submit to…
MySQL IN CLAUSE duplicates WANTED.
February 12, 2011
$sql = 'SELECT * FROM `courses` WHERE `id` IN ('. implode(",", $course[0]) .')'; This is the code I just had help with, I have used it before but…
Regexp wanted to format string of letters
June 9, 2011
I want to format the initials of a persons name in a standard way. Every letter in the initials should be in uppercase and every letter (from to…
MySQL question on fetching only wanted data
March 9, 2011
Hello. I've been asking couple of questions before concerning my database analysis. I am grateful for all the replies yet here is another question.…
I just wanted to DES 4096 bytes of data with a 128 bits key...
January 9, 2011
...and what the nice folks at OpenSSL gratiously provide me with is this. :) Now, since you shouldn't be guessing when using cryptography , I come…
MySQL asked during installation to Win7 whether I wanted TCP-IP connection.
March 27, 2011
I last installed MySQL 7 years ago, and don't remember this being asked. If it was, I don't remember my response. Anyway, I said no today, because…
Query for Users not in a specific group? (Wanted to use EXCEPT but it seems MySQL doesn't support it)
February 14, 2011
I have a many to many relationship between users and groups: CREATE TABLE IF NOT EXISTS `SecurityIdentifiers` ( `Guid` char(36) NOT NULL, PRIMARY…
Sharding Problem........
January 12, 2011
Hi, I am new to mongodb, I am undergoing testing based on sharding in single machine...I have two shards running on the machine...One config server…
How to Verify Sharding?
January 6, 2011
HI, Hey, Buddy.I have clearly mentioned that i have done with the sharding configuration and i have configured some ordinary backups in-order to…
SHARDING CONFIGURATION HELP...........
January 10, 2011
Hi, I am new to mongo-db,I tried to configure sharding by following the procedure in the mongo-db web-site..But i am getting some errors...... I…
MySQL sharding approaches?
April 4, 2011
What is the best approach for Sharding MySQL tables. The approaches I can think of are : Application Level sharding? Sharding at MySQL proxy layer?…
How can one configure MongoDB sharding using PHP?
June 18, 2011
I'm working on setting up a MongoDB replica set on EC2. I'd like to configure auto-sharding such that my nodes can easily come up and down. Is it…
Sharding at application level
March 31, 2011
I am designing a multi-tenant system and am considering sharding by tenant at the application layer level instead of database. Hypothetically, the…
MYSQL HORIZONTAL SHARDING
May 13, 2011
Which are the best projects for mysql Horizontal Sharding, I mean spread database into multiple servers (we already have partitioning tables, but it…
Sharding GridFS on MongoDB
March 17, 2011
I'm documenting about the GridFS and the possibility to shard it among different machines. Reading the documentation here , the suggested shard key…
Multiple tables sharding with mysql
January 15, 2011
Hi, I'm making a GPS app that will deals with 200 millions records in a table. My initial thought is to divide the table into multiple tables like…
100 Terrabytes database on PostgreSQL without sharding
March 28, 2011
Is it realistic to setup 100Tb database (about 90Tb actually) on PostgreSQL without data sharding between a number of nodes? Are there any success…