Hi everyone,
I'm inserting around 265 000 documents into an elastic search cluster
composed of 3 nodes (real servers).
On two servers i give elastic search 20g of heap, on third one which has
64g ram, i set 30g of heap for elastic search.
I set elastic search configuration to :
- 3 shards (1 per server)
- 0 replicas
- discovery.zen.ping.multicast.enabled: false (and giving on each node the
unicast hostnames of the two other nodes);
- and this :
indices.memory.index_buffer_size: 50%
index.refresh_interval: 30s
threadpool:
index:
type: fixed
size: 30
queue_size: 1000
bulk:
queue_size: 1000
bulk:
type: fixed
size: 30
queue_size: 1000
search:
type: fixed
size: 100
queue_size: 200
get:
type: fixed
size: 100
queue_size: 200
Indexing is done by groups of 100 000 docs, and here is my application log :
INFO: Adding records to bulk insert batch
INFO: Added 100000 records to bulk insert batch. Inserting batch...
-- Bulk insert took 38.724 secondes
INFO: Adding records to bulk insert batch
INFO: Added 100000 records to bulk insert batch. Inserting batch...
-- Bulk insert took 31.134 secondes
INFO: Adding records to bulk insert batch
INFO: Added 64201 records to bulk insert batch. Inserting batch...
-- Bulk insert took 17.366 secondes
--- Import CSV file took 108.905 secondes ---
I'm wondering if this time is correct or not, or if there is something i
can do to improve performances ?
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3a38e79e-9afb-4146-a7e1-7984ec082e22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
