Yes, that is sufficient to clear out the documents. But... take the advice given by Jörg to heart.
Elasticsearch is already optimized to take a bulk request and optimally process it as fast as it can be done. There should not be more than one of them at a time; no gain will be seen, and (as you have seen) bad results will be seen. What you could do is use something like the LMAX Disruptor<http://lmax-exchange.github.io/disruptor/>and set it up for multiple producers and one handler thread (or worker thread, either one in this case). Your own 20 (or whatever) worker threads should publish to the disruptor's ring buffer. Then the handler thread would contain the BulkRequestBuilder and process incoming documents as you show in your code snippet. Or do the same thing but with a Java queue of some kind that your workers store into and your one processor thread pulls from and does the bulk request processing. I only recommend the Disruptor because it's an incredibly awesome thing that is small and very easy to use; once you get up to speed it takes a few lines of code to be able to pass through millions of events per second (yes, you read that right. I've seen it on my little old laptop for myself). Of course, ES won't keep up, but the Disruptor will not add any perceptible latency to your processing. It's a thing of beauty and joy, just like Elasticsearch is for search. Regards, Brian > > (there may be 20 workerThreads) > As you can see, as each thread submits work, the thread will do a > client.prepareBulk() ... is that sufficient clear out the documents? > > workerThread() { > Client client = getMyGlobalTransportClient(); > BulkRequestBuilder bulkRequest = client.prepareBulk(); > for (...) { > bulkRequest.add(...) > if (bulkRequest.numberOfActions() >= chunksize) { > BulkResponse bulkResponse = bulkRequest.execute().actionGet(); > if (bulkResponse.hasFailures()) { > ... > } else { > ... > } > bulkRequest = client.prepareBulk(); > } > > etc > > > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/36de8440-d3f8-4f2e-8e88-bd251e8d63cb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
