Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account

Elasticsearch - EC2 Region/Availability zone problem

0

84 views
Hi there,
We've got a bit of a problem with our elasticsearch environment on AWS in the US-West-1 region. As far as we can tell, what's happening is that the cloud-aws plugin is treating availability zones as distinct regions, instead of including the AZs we are specifying.

elasticsearch.yml contains the following specific settings:
network.bind_host: _eth0:ipv4_
#cloud.aws.region: us-west-1
cloud.aws.access_key: <DELETED>
cloud.aws.secret_key: <DELETED>
cloud.node.auto_attributes: true
discovery.type: ec2
discovery.any_group: true
discovery.ec2.tag.stage: production
discovery.ec2.availability_zones: us-west-1a,us-west-1c
gateway.type: s3
gateway.s3.bucket: <our-bucket-name>
gateway.s3.concurrent_streams: 5
gateway.expected_nodes: 3
gateway.recover_after_time: 1m
gateway.recover_after_nodes: 1
indices.recovery.max_size_per_sec: 0
indices.recovery.concurrent_streams: 5
cluster.routing.allocation.node_concurrent_recoveries: 2
discovery.zen.ping_timeout: 30s

cloud.aws.region is commented out because when it isn't commented out, starting elasticsearch gives an error as follows and then crashes out:
[2012-01-03 18:42:24,333][WARN ][com.amazonaws.http.AmazonHttpClient] Unable to execute HTTP request: null
[2012-01-03 18:42:24,344][WARN ][com.amazonaws.http.AmazonHttpClient] Unable to execute HTTP request: null
{0.18.6}: Initialization Failed ...
1) AmazonClientException[Unable to execute HTTP request: null]
ProtocolException[Received redirect response HTTP/1.1 301 Moved Permanently but no location header]

With the above custom config, when starting up either node (both in different AZs) we get this in the logs on each:

[2012-01-03 19:06:14,454][INFO ][cluster.service ] [Bulldozer] new_master [Bulldozer][4QTO0FKfQkGiUOgGHncu_w][inet[/10.169.49.245:9300]]{aws_availability_zone=us-west-1a}, reason: zen-disco-join (elected_as_master)

[2012-01-03 19:06:23,301][INFO ][cluster.service ] [Mad Dog Rassitano] new_master [Mad Dog Rassitano][vPKkND4UTJ6o245oHL8efQ][inet[/10.171.49.112:9300]]{aws_availability_zone=us-west-1c}, reason: zen-disco-join (elected_as_master)

We only want one to be a master for the cluster - they simply won't cluster at all.

Can anyone point out what we are doing wrong here?
James.
asked January 4, 2012 1:32 am AZOT
/users/65084/jjwc

0 Answers

Be the first to answer this question

Join with account you already have


Sign in with Twitter account
Sign in with Facebook account
Sign in with Google Friend Connect

Preview
Similar questions