For starters, Java doesn't compress any pointers over 32GB, you are way over that limit and losing efficiency there.
What version of java are you using? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 23 April 2014 07:09, John Smith <[email protected]> wrote: > Hi I downloaded the latest ES 1.1.1 > > I have a 200GB RAM with 2 x 8 cores hyper threaded. "32" cores total > machine 1.6T of disk space. > > I start elastic search as follows... > > ./elasticsearch -Xms100g -Xmx100g -Des.index.store.type=memory > Using Java 1.7_51 > > I then create my index as follows... > > $ curl -XPUT http://localhost:9200/myindex/ -d \ > ' > index : > store: > type: memory > ' > > And my Java web app (Using vertx.io) > > // On app startup... Ensure we have one instance of client. Regardless how > many app threads may write to the index. > synchronized(clientCreated) { > if(clientCreated.compareAndSet(false, true)) { > node = nodeBuilder().clusterName("elasticsearch").client(true).node(); > client = node.client(); > } > } > > > // Per request coming into my web application. Using vertx for the web > framework. > // For each request we use the one client instance. > client.prepareIndex("myindex", "doc", request.getString("id")) > .setSource(bodyStr) // Already sending Json so no need to convert > it! > .execute(new ActionListener<IndexResponse>(){ > > @Override > public void onFailure(Throwable t) { > req.response().end("Error: " + t.toString()); > } > > @Override > public void onResponse(IndexResponse res) { > req.response().end(res.getIndex()); > }}); > > > Both the webapp and ES running on same server. So all write/read requests > are localhost. > > Testing as follows > > JMeter (100 users, running on my desktop) ------ Remote ----> WebApp ----- > localhost ----> ES > > I get about 6000 writes/sec and it seems to get lower as the number of > docs that get indexed increases. > Average request/response latency is about 15-20ms. > Network Time/Jmeter data generation( Each document is about 1000 > bytes)/web app is about 5 ms. I know this because I also have a simple > hello world response to test the average latency of those 3 "parameters". > So it seems that in-memory takes average 15ms I would think ES can do much > better then that? > > Is there any tuning settings I can try for strictly in-memory index? > > Thanks > > > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/dc7c69f8-d4ab-42f7-88dc-c165472c2892%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/dc7c69f8-d4ab-42f7-88dc-c165472c2892%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624babfptgf1m8H3Lh4n53ST5UV_dKHj%2BPkB-jKF7nrX%2B7Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
