Axel,
Well, I use the Java API and not the REST API. And while there is a lot
more code around this, here's the core of my bulk load process. I use the
BulkRequestBuilder (a new one for each set of bulk updates!) and not the
BulkProcessor because (a) this code was already written and works and (b)
it gives me some additional control. For instance, I can configure the
maximum error display so that, if I'm loading 1M documents but they all
fail with some stupid fault of mine, I can see the first 128 errors but not
get swamped with all 1 million since they are more than likely to be very
similar.
I first wrote this code for ES 0.19.4 and it has remained unchanged and
working perfectly up to and including ES 1.1.0:
BulkResponse bulkResponse = bulkRequest.execute().actionGet();
if (bulkResponse.hasFailures())
{
if (failedDocuments <= failedMsgLimit)
Sysprint.err.println("BULK WARN: Bulk load failues: "
+ bulkResponse.buildFailureMessage());
BulkItemResponse[] items = bulkResponse.getItems();
for (BulkItemResponse item : items)
{
if (item.isFailed())
{
/* Failure detected */
failedDocuments++;
incrementFailedOpType(item.getOpType());
if (failedDocuments <= failedMsgLimit)
{
Sysprint.err.println("BULK WARN: Bulk failure for "
+ item.getOpType() + ": " + item.getFailureMessage());
}
}
else
{
/* Success: Keep track of version numbers > 1 */
if (item.getVersion() > 1)
setVersionGtOne++;
}
}
}
Hope this helps!
Brian
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7961fd8d-8b7e-431e-bce0-4ac16293b1d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
