Hi,
I'm currently posting updates via cURL, in batches of 1,000 docs in JSON
files.
My setup consists of 2 shards, 1 replica each, 50m docs in total.
These updates are hitting a node at random, from a server across the
Internet.
Apart from the obvious delay, I'm also seeing QTime's of 1,000 to 5,000.
This strikes me as quite high since I also sometimes see times of around
300-500, on similar cloud setups.
The setup is running on VMs with rotary disks, and enough RAM to hold
roughly half the entire index in disk cache (I'm in the process of
upgrading this).
I hard commit every 10 minutes but don't open a new searcher, just to
make sure data is "safe". I softCommit every 1 minute to make data
available.
Are there any obvious things I can do to improve my situation?
Thanks,
Rob
