Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

Problem with insert data to elasticsearch

hi
(12 lines)
JSON wildcard search
(99 lines)
Oct 25, 2010
Jakub
Jakub
Hi,

I'm using latest elasticsearch release 0.12.0
I have problem with insert data to elasticsearch.
I'm using "elasticsearch" PHP client to insert data.

Insert data is done in loop.
PHP script insert 0 - 10000 entries and reload itself.
Next it insert entries from 10000 - 20000. And it reload and start
from 30000.
Script should works like this to the end. About 190000 entries.

Insert single entry works ok. Problem is that script stops every time
after insert 28233 entries.
I'm 100% sure it's not problem with inserted data of entry 28233
because starting from 10000 it will go thourgh entry 28233 without any
problem and will stop on entry 38233. So it always insert only 28233
entries.
PHP client return error.
Couldnt connect to host [123.123.123.123], ElasticSearch down?

(123.123.123.123 - is fake ip address)

This problem don't generate any log entry in log file.
ElasticSearch is not down after this problem happen.


Did anyone have same problem?
Can anyone help me to find what is the reason of this problem?

Thank You
Regards
Jakub


Reply
Messages in this thread
Problem with insert data to elasticsearch
reply Re: Problem with insert data to elasticsearch
(45 lines) Oct 25, 2010 05:27
reply Re: Problem with insert data to elasticsearch
(60 lines) Oct 25, 2010 09:02
reply Re: Problem with insert data to elasticsearch
(73 lines) Oct 25, 2010 09:12
reply Re: Problem with insert data to elasticsearch
(88 lines) Oct 26, 2010 10:59
reply Re: Problem with insert data to elasticsearch
(41 lines) Apr 22, 2011 20:35
reply Re: Problem with insert data to elasticsearch
(65 lines) Apr 23, 2011 15:34
Similar Threads
Re: Can't insert data into hbase, OOME.
Thanks. I found. I looked in upstream branch instead of stumbleupon branch in my local repo. 2010/9/30 Alexey Kovyrin <ale### @kovyrin.net>: >…
Re: Can't insert data into hbase, OOME.
Data is simple table with two column families. info: json object (small) rows: ~300 bigdecimal columns per row.
Re: Can't insert data into hbase, OOME.
Weird indeed, even after the WAL was rolled 4 times (theoretically 256MB of data) I don't even see a flush request... although you're running at INFO level instead of DEBUG. Could you switch that and send us just the full log. Thanks a lot! J-D …
Can't insert data into hbase, OOME.
Hi all, I'm stuck. I can't insert any valuable peace of data into hbase. Data is something around ~20mil rows (20G). I try to insert them into nondistributed hbase with 4 parallel jobs. MR job run until all memory given to hbase is exhaused and…
Append data by insert command
Hi, Not able to append data in hive tables .I know only one syntax of "INSERT" command which have keyword called "OVERWRITE" , It overwrites data in whole partition. Is there any other way by that I can append records in hive tables by HQL? …
Lost data which succeeded to insert right before server was down
Hi all, I'm now checking reliability when inserting. I set one cluster of 3 nodes with replication factor 2 and OrderPreservingPartitioner. I insert data (A,B,C in this order) with consistency level ONE to node No.3. I immediately shut off (turn…
Can i modify the stream of data and then insert into hive
This is a multi-part message in MIME format.Hi, Suppose i get a stream of data. Now I want to filter this 'stream data' and then insert into hive table. For example lets say a file "datas.txt" has following info
insert data into a column of complex type
Hi, Can we insert data into a column of complex type (eg map type) through hive command line? The only related article I can find is at http://www.mail-archive.com/common-comm### @hadoop.apache.org/msg01031.html: "The tables with columns that are…
CouchDB ElasticSearch Integration Howto
Hi, ElasticSearch has added a feature which directly indexes couchdb documents. ElasticSearch listens on the CouchDB _changes interface and indexes the docs. This feature has undergone some changes in the current development series. So,…
FW: Problem passing data through the SCXML
Hi! I`ve a problem with the SCXML, i use custom actions, and everything is fine, but, if i have to insert a data from the model data i had generated with the next code line: <cs:Myfunction valueA="NameOfValueInDataModel"/> …
Problem selecting new data object in combo box using DOComboBoxModel
I am using the current snapshot of the org.apache.cayenne.dataview classes in a swing application with Cayenne 3.x. I am having a problem that when I add (programmatically) an item to a combo box I can only select it programmatically. The object…
Created: (PIG-1595) casting relation to scalar- problem with handling of data from non PigStorage lo
casting relation to scalar- problem with handling of data from non PigStorage loaders
Updated: (HIVE-442) Partition is created before data is moved thus creating a window where data is i
[ https://issues.apache.org/jira/browse/HIVE-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-442:
Created: (DBCP-336) Incorect data from data base if connection pool size is minimum (say 1 to 3 )
Incorect data from data base if connection pool size is minimum (say 1 to 3 )
Closed: (FELIX-1938) Bad error message when an Event Subscriber does not set the data type and data
[ https://issues.apache.org/jira/browse/FELIX-1938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Clement Escoffier closed FELIX-1938.
Created: (HIVE-1540) Read-only, columnar data file for nested data structures
Read-only, columnar data file for nested data structures
About data locality (Was: Re: HBase on same boxes as HDFS Data nodes)
(changing the subject, let's not hijack threads) > will the data move over time though...for example if i have lots of access to data in DataNode A ? without the current work that is in progress.. HBase has no control on that, but data will be…
Question about data warehousing and data mining
Hi all, I am currently trying to find out what frameworks/software/product will support data warehousing/data mining the best. We get around 1.5+ TB of log data every month and we want to do some reporting on top of that and later on move on to…
Data from multiple tables (Join Data)
Hi all, I am new to Cassandra... I want to use to cassandra for a billing system. As I saw in many places that Joins won't work in BigTable implementation but i feel i needed it for my App. I am unable to get the data from multiple tables…
Re: Issue 199 in redis: Redis loses data when data is pushed after blocking (BRPOP/BLPOP) client ter
Updates: Status: Verified Comment #6 on issue 199 by antirez: Redis loses data when data is pushed after blocking (BRPOP/BLPOP) client terminates http://code.google.com/p/redis/issues/detail?id=199 This is now ok, closing