Can you yry increasing your bulk count to 1000, or more? Regards, Mark Walkom
Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 9 May 2014 05:01, slushi <[email protected]> wrote: > We are testing out release 1.1.1 and during our indexing performance > testing, we seemed to get significantly slower throughput, the > document/second rate is about 30% slower. We used the exact same yml file > and startup settings. The code is also identical except for the breaking > changes in the java client api (in this case minor naming changes) and > different elasticsearch/lucene jars. > > We have a 4 node test cluster. The test basically creates an index with 4 > shards and no replicas/refreshing. We are indexing documents that are about > 2KB each using the bulk api (500 documents per request). > > Below is some environment info and some settings that we changed from the > default. > > MemTotal: 99016988 kB > > ES_HEAP_SIZE=24g > > MAX_OPEN_FILES=65535 > > > */etc/elasticsearch/elasticsearch.yml :* > > cluster.name: estest > > node.name: "es1" > > node.rack: rack2 > > bootstrap.mlockall: true > > network.host: 1.2.3.4 > > gateway.recover_after_nodes: 3 > > gateway.expected_nodes: 3 > > discovery.zen.minimum_master_nodes: 1 > > discovery.zen.ping.multicast.enabled: false > > discovery.zen.ping.unicast.hosts: ["es1", "es2", "es3", "es4"] > > indices.memory.index_buffer_size: 50% > > index.translog.flush_threshold_ops: 50000 > > threadpool.search.type: fixed > > threadpool.search.size: 20 > > threadpool.search.queue_size: 100 > > threadpool.index.type: fixed > > threadpool.index.size: 60 > > threadpool.index.queue_size: 200 > > threadpool.bulk.type: fixed > > threadpool.bulk.size: 50 > > threadpool.bulk.queue_size: 1000 > > cluster.routing.allocation.awareness.attributes: rack > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/edea9e8f-e2f3-424e-b381-3d6dc4b96979%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/edea9e8f-e2f3-424e-b381-3d6dc4b96979%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YYMESG9zYtb9shwEHU1Pkt9Dzn9n%3D6TMegRcx4pyZSkw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
