12 hours is an absurdly long time for indexing 10 million docs. queue:1000 is much too high for production. For test it may be ok (it effectively disables queue rejections) but on production, you play with the risk of starving your cluster resources.
Do you rmonitor the resource usage of ES, especially the heap? Is GC starving your cluster? Do you see OOMs? Do you evaluate the bulk responses for errors? Do you throttle bulk request concurrency? Do you set refresh interval to -1? Hint: if 5 nodes is your maximum, you can also bulk index with 5 shards and replica level 0, after bulk, you can increase replica level to 1. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHj9E%2BDSsoz%3DXY%3DVc0N0s5QLhb6Ea-CQq6dwQOZRZWn0A%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
