Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

Mongos socket exceptions against shards that are down - breaks connectivity to whole shard?

Re: MongoDB 2.0 Ram vs DataSize
(46 lines)
Dec 2, 2011
BernieD
BernieD
I am running mongos with three config servers against two shards.

When I bring a shard down (just by stopping mongod on all nodes of one
shard), I was expecting that that the mongos process will catch this
and then inserts to the missing shard will fail and queries to the
missing shard will fail. These requests may fail gracefully if I have
the partial atribute set in my cursors.

The MongoDB docs say "operations that can be resolved at other shards
will continue to work properly," but whenever I have stopped one shard
completely and then tried to connect to a mongos process by PHP, perl,
or python I get an exception:  "query error: socket exception". I
notice that PHP connections behind Apache (and because their
connections were fairly persistent?) did seem to work as expected for
the life of their connection, I believe.

The mongo command line is a bit more informative:
error: {
	"$err" : "could not initialize cursor across all shards because :
socket exception @ ec2-xxx-.compute-1.amazonaws.com:XXXXX",
	"code" : 14827
}

For me, this means that all the shards are unreachable from my
application if any one shard is completely down. Is there some setting
I am missing that lets mongos or these drivers be more robust?





Reply
Tags: shardshardsconfigservers
Messages in this thread
Connectivity problems with new shard.
February 12, 2011 05:00:15 PM
Today I set up a new replica set and tried to add it as a new shard to my cluster. As you can guess, I have failed :-) Here's what was happening in mongos log (errors in red, comments in blue): Sun Feb 13 01:22:23 [conn69123] addshard request {…
Windows socket connectivity issues - Java driver
October 5, 2011 09:55:29 AM
Windows 7 64-bit MongoDB version 2.0.0 Java driver version 2.6.5 Internet connected via WLAN to a ZOOM router I am running a very basic Play! webapp using the MongoDB Java driver. When I first start the mongod service on Windows and make my…
mongos could not initialize cursor across all shards error
October 14, 2011 03:37:28 PM
Hi, We have an issue showing up since we upgraded to 2.0 Every now an then we get this error on mongos logs and more often Fri Oct 14 17:40:24 [conn688] ns: masked could not initialize cursor across all shards because : ns: masked…
Bypassing mongos for inserts directly to shards?
October 29, 2010 04:59:09 PM
I am curious if it is possible to bypass the mongos process and insert data directly into individual shard machines when using a a sharded collection. The reason for this is that mongos is incredibly slow for writes at this time and we have to…
Shards and chunks (defining the right shard key for us)
February 17, 2011 12:45:29 PM
I'm putting up some of our first sharded systems and I have some observations on write loads. We are creating a database for user activity. We'd like to persist this data indefinitely since a user my return to us at anytime in the future. So we…
Socket Exception when Balancing shards
May 10, 2011 01:01:00 PM
Hi, I have 2 replica sets that I have enabled for sharding - (Shard1_rs : 2 nodes + 1 arbitrer & Shard2_rs : 2 nodes + 1 arbitrer). All nodes are running on EC2. I have one mongos running and I can see from mongos log that balancer is…
shard, mongos collection
December 29, 2010 07:32:02 AM
hi, when doing use config db.mongos.find() i get a list of mongos services, but the list also contains old mongos that doesn't exists any more? when this collection is cleaned? will it affect the selecrtion of which mongos will be the balancer? …
errmsg - can't add a mongos process as a shard.
August 26, 2011 09:19:15 AM
Hi, I have set up shard servers on two amazon ec2 ubuntu instances. I have followed the following steps to set up shard servers on each instance -: .1. # mongod --configsvr .2. # mongos --configdb ip_address_of_amazonec2_instance .3. # mongod…
Issue with mongos knowing that a shard has a new primary
October 14, 2010 02:48:43 PM
Hey all, Ran into an issue where a running mongos had issues issuing commands when a shards primary *vanished*, even though it appears the secondary was properly promoted. I have 3 config servers, 4 shards, with each shard having 2 hosts in a…
weird socket errors on mongos with wan connection
September 18, 2010 10:00:24 AM
Hello, this is my architecture, i have 2 sharded servers, 1 config server and mongos process on each app server. I also have one app servers outside of my LAN in other WAN (amazon), that mongos is always logging some errors. I did test the WAN…
mongos shutdown exceptions
December 13, 2010 06:34:35 PM
(mongos startup log messages): Wed Nov 24 05:17:20 /opt/mongodb/bin/mongos db version v1.7.3, pdfile version 4.5 starting (--help for usage) Wed Nov 24 05:17:20 git version: fcb5790e7528a101a49c52e442ded0e2b7a043d6 Wed Nov 24 05:17:20 sys info:…
Re: Re: How to reload shard information in mongos without restarting the process?
November 6, 2010 12:12:43 PM
Could you open a JIRA ticket for this? Alberto. On Sat, Nov 6, 2010 at 12:49 PM, Kraman <kra### @gmail.com> wrote: > Hi Albert, > > Yes, I did wait for the removeshard process to complete by calling it > multiple times till it…
How to reload shard information in mongos without restarting the process?
November 5, 2010 07:31:45 PM
Hello, Does anyone know of a way of reloading the shard information for a mongos process without having to reload it. I ran into problem with mongos having stale shard information and reporting an error. Basically I has 2 shards, and I removed one…
Newbie Sharding - couldn't connect to new shard mongos connectionpool
October 11, 2011 10:39:12 AM
This is a super basic question that has been driving me up a wall. I have been setting up a sharded and replicated environment using the instructions at http://www.mongodb.org/display/DOCS/Simple+Initial+Sharding+Architectu re. I did this once…
extremely poor shard performance, replica set diverges, mongos crash under load
December 21, 2010 05:52:04 PM
I tried to run a batch conversion script (written in perl, simple loop through some user IDs) on a newly balanced cluster, which was also servicing some low-volume writes from our web server pool. My setup: Linux 2.6.32-21-server #32-Ubuntu SMP…
9001 Socket Exceptions
August 19, 2011 01:56:59 PM
A few days ago I made a minor configuration change to our mongo servers -- I bumped the log level to -vvv -- and now I'm seeing lots of the following in our logs: Fri Aug 19 11:35:06 [conn11396] MessagingPort recv() conn closed?…
Mongos segfault (memory corruption) and "no availalable shards to take chunks"
April 12, 2011 10:25:58 AM
I have a cluster of 3 MongoDB shards. Right now the mongos process is behaving very weird. Since a few hours ago it started printing lots of "[Balancer] no availalable shards to take chunks" errors. However all 3 shards are online and…
"could not initialize cursor across all shards because" from mongos
November 2, 2011 07:45:58 AM
Wed Nov 2 12:35:41 [conn68536] ns: friends_photos.user_queue could not initialize cursor across all shards because : ns: friends_photos.user_queue ClusteredCursor::query @ shard5/mongodb1:10002,mongodb5:10001 attempt: 0Wed Nov 2 12:35:41…
Attachment indexing breaks shards or hangs during indexing
January 30, 2011 02:13:37 PM
Hi all, I'm looking at incorporating ES into our environment to allow us to search some large databases that simply don't do well with standard SQL queries to find things. First, I just want to say I am very impressed with ES so far. Great…
Server occurs socket exceptions after connection
February 18, 2011 04:38:51 AM
hi, I ran some commands in my code like insert, find etc. But there are always show these exceptions in logs. Here is some parts of my code: var connectionString = "mongodb://192.168.92.100:29000"+"/? connect=direct;slaveok=true"; MongoServer…
In 64-bit MongoDB When the sharding process starts from one shards to another shards?
February 18, 2011
Hi, I am undergoing a unit test in 64-bit mongodb in 64-bit linux machine.I have two shards both of the shards are configured in the local machine…
Socket exceptions filling up logs
April 7, 2011
Hello, I've recently had trouble with a piece of stack trace filling my tomcat stdout. It seems to start when the server starts, even though no one…
MongoDB Socket Exceptions when moving chunks
May 20, 2011
I have 2 shards in my mongodb db cluster, with 1 mongos server. I have a total of 8 servers, with one replica set having 5 and the other 3. I have a…
How to display FluentNHiberante Exceptions or SQL Exceptions?
May 11, 2011
Hi, I use FluentNHibernate and SQLite. Often I get Exception in the fluent configuration due to some problem with my Domain classes. But VS only…
In mongodb Autusharding, same data is available in all the shards?
February 1, 2011
Hi,I am dealing with mongodb Autosharding,Whether the same data will be available in all the shards or the data will be different in each shard? Plz…
In 64-mongoDB Whether i have to proveide any size limit for each shards?
February 18, 2011
I am performing the test with 64-bit MongoDB in 64-bit linux machine in my lab.I have configured two shards in my local-host.According to my concern…
Can mongos be configured to talk to more than one mongo cluster?
May 27, 2011
The rule of thumb is to have the 'mongos' process running on each of your application servers. This keeps your application talking to localhost…
Trouble connecting to Mongo Config Server from mongos
May 8, 2011
Folks, I am setting up a MongoDB Cluster. The problem is when I am starting up 'mongos' it fails to connect to Mongod Config Server. If I start…
I am getting error message in mongos server anyone help me to resolve the issue..
January 27, 2011
Hi, I am new to mongodb and i have configured sharding and i am working on it.I have two server, each server i have one shard and also i am running…
Why doesn't Python socket receive a final blank value during recv loop of HTTP socket?
June 20, 2011
I've written a miniature proxy module in Python 3 to simply sit between my browser and the web. My goal is to merely proxy the traffic going back…
Shard Size in 32-bit?
January 20, 2011
Hi, I am working with sharding and have two shards. MaxSize of Each Shard will be 1GB in 32-bit machine?....Shards are configured in single 32-bit…
Is it OK to shard mongodb without replica set?
May 15, 2011
Lets say I have 4 database servers. I want to add them to the "shard" cluster. That's it. No replica set, no nothing. Just 4 servers. If I turn…
How to shard existing key-value storage?
September 4, 2010
Hello, Let's suppose that we have 3Gb key-value storage on server A. I'm starting to feel that we need another server (server B). So, I have to…
MySQL InnoDB big table: to shard or to add more RAM ?
March 24, 2011
Folks, I'm a developer of a social game and there are already 700k players in the game, and about 7k new players are registered every day, about 5k…
For 32-bit sharding whats the size limit of each shard?
January 31, 2011
Hi,I am using 32-bit mongodb for sharding.I configured two shards on two 32-bit machines.Each machine has one shard...The max size of all shards is…
In 64-bit mongodb how AutomaticFailover works,when each shard has different data?
February 2, 2011
Hi,I am handling with Autosharding,and i had questioned about Whether the data in the shard "A" Will be available in shard "B".They have answered as…
Does multiple shard key helps performance in mongodb ?
April 13, 2011
Since sharding database use shard key to split chunk AND route queries, so I think maybe more shard key can helps to make more queries targeted I…
YCSB - why I can never find a shard configuration anywhere on internet
May 8, 2011
I see all kinds of reference to MongoDB as a client for the YCSB benchmarks to test NoSQL database server scalability / elasticity.…
Android connectivity with mysql
February 24, 2011
hey guys i am developing a final year project where i need to connect android emulator with mysql database in order to retrieve values. java file:…
How to detect WCF connectivity without try-catch?
February 24, 2011
I use WCF netNamedPipeBinding and wsHttpBinding . I would like to know whether a one-way method can execute successfully without try ing it. EDIT As…