Ok so I decided to skip in-memory for now just to test bassic functionality.
I'm running elastic search with defaults as ./elasticsearch -Xms32g -Xmx32g I also got bigdesk installed. Either I'm not getting something... But why as I write more documents to the index the Indexing requests per second (Δ) goes down and the Indexing time per second (Δ) Is going up So basically it's getting slower and slower. Is their any sensible tuning parameters. I would expect that the insertion should be stable and not getting slower. It's a 32 core machine and enough ram, standard drives though. There has to be a way to setup buffers and queues to alliviate the issue of disks "being slow" On Wednesday, April 23, 2014 10:42:37 AM UTC-4, John Smith wrote: > > On a 32 core machine? Plus I think 1.7_51 uses G1 > > I have tested another "indexing" api up to 190GB or so with 30,000,000 > objects and my latency was 3ms overall including network and app logic. > > And I haven't tested that many records with elastic search yet ;) > > > On Wednesday, 23 April 2014 10:20:20 UTC-4, Jörg Prante wrote: >> >> The ES "memory" or "ram" store (Lucene RAMDirectory) puts enormous >> pressure on JVM garbage collection. >> >> You can not expect that standard JVM with CMS GC can give the best >> performance. >> >> More info in this great article by Mike McCandless >> >> >> http://blog.mikemccandless.com/2012/07/lucene-index-in-ram-with-azuls-zing-jvm.html >> >> Maybe Java 8 with G1 GC is giving slightly better numbers. But do not >> expect too much. >> >> Jörg >> >> >> >> >> >> On Wed, Apr 23, 2014 at 3:19 PM, John Smith <[email protected]> wrote: >> >>> 1.7_51 but i dont see how their could be a limitation. >>> >>> I used java up to 200GB easily and with no issues either... >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/cf8950c9-923e-4cea-90f5-c51a09bf1b16%40googlegroups.com >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c6e486c7-6076-4c9e-9073-0f0b808a36e8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
