Thank you for your answer, I did some tests, by writing something simple that writes bulks using TransportClient to ES server in parallel mode (something like BulkProcessor...), and it looks that the hadoop job run pretty same time, so there is no big difference here with using "es-hadoop"
But it looks for me that ES server consume less CPU when it receive bulks via TransportClient Anyway looks like my bottleneck somewhere inside hadoop mapper job (maybe parser or reading lines from file on S3... :/ ), so I will research it... Thanks! Igor. On Thursday, April 17, 2014 11:16:33 PM UTC+3, Igor Romanov wrote: > > Hi all, > > Currently I am working with elasticsearch-hadoop library with > EsOutputFormat that is writing to elasticsearch, > But it looks to me like the writing is slow (elasticsearch-hadoop works > with HTTP bulks on port 9200) > So my question is it worth to try to write something of my own that will > use TransportClient of elasticsearch which will write bulks to > elasticsearch via tcp on 9300 ? > (I think it should reduce opening http sockets each bulk, also I will no > longer write to Hadoop Context writable objects, but directly to elastic > search bulks...) > > Anyone had experience with that? can share it? > > Thanks, > Igor > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d4a0d60d-0d8a-40cd-9bd7-52a5724587d1%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
