I tried various counts, after a certain point they didn't make much difference. Also I am not necessarily concerned with improving performance as much as figuring out why I got a slowdown with the exact same settings in 1.1.1. I want to be sure we didn't miss some configuration somewhere or some other issue. Thanks!
On May 8, 2014, at 6:32 PM, Mark Walkom <[email protected]> wrote: > Can you yry increasing your bulk count to 1000, or more? > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [email protected] > web: www.campaignmonitor.com > > > On 9 May 2014 05:01, slushi <[email protected]> wrote: > We are testing out release 1.1.1 and during our indexing performance testing, > we seemed to get significantly slower throughput, the document/second rate is > about 30% slower. We used the exact same yml file and startup settings. The > code is also identical except for the breaking changes in the java client api > (in this case minor naming changes) and different elasticsearch/lucene jars. > > We have a 4 node test cluster. The test basically creates an index with 4 > shards and no replicas/refreshing. We are indexing documents that are about > 2KB each using the bulk api (500 documents per request). > > Below is some environment info and some settings that we changed from the > default. > > MemTotal: 99016988 kB > > ES_HEAP_SIZE=24g > > MAX_OPEN_FILES=65535 > > /etc/elasticsearch/elasticsearch.yml : > > cluster.name: estest > > node.name: "es1" > > node.rack: rack2 > > bootstrap.mlockall: true > > network.host: 1.2.3.4 > > gateway.recover_after_nodes: 3 > > gateway.expected_nodes: 3 > > discovery.zen.minimum_master_nodes: 1 > > discovery.zen.ping.multicast.enabled: false > > discovery.zen.ping.unicast.hosts: ["es1", "es2", "es3", "es4"] > > indices.memory.index_buffer_size: 50% > > index.translog.flush_threshold_ops: 50000 > > threadpool.search.type: fixed > > threadpool.search.size: 20 > > threadpool.search.queue_size: 100 > > threadpool.index.type: fixed > > threadpool.index.size: 60 > > threadpool.index.queue_size: 200 > > threadpool.bulk.type: fixed > > threadpool.bulk.size: 50 > > threadpool.bulk.queue_size: 1000 > > cluster.routing.allocation.awareness.attributes: rack > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/edea9e8f-e2f3-424e-b381-3d6dc4b96979%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to a topic in the Google > Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/qydm3PG4Jxw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAEM624YYMESG9zYtb9shwEHU1Pkt9Dzn9n%3D6TMegRcx4pyZSkw%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/E64BA391-4124-4CAC-BC70-360F91B549BF%40gmail.com. For more options, visit https://groups.google.com/d/optout.
