How much heap are you giving to ES? With this many requests, if your setup is not falling over it is probably not garbage collect related because that would result in very noticable delays/unavailability of es. 32GB should be a good value given how much memory you have. Also, you probably want to use doc_values in your mapping so that you can utilize the os file cache and move some of the memory pressure on the heap. You seem to have plenty of ram, so your entire dataset should easily fit in RAM.
Also, don't use G1 for elasticsearch. There are known issues with that particular garbage collector in combination with lucene. CMS is the best option for ES. 500% of 20 cores doesn't sound that bad; you'd max them out at 4000%. Still, it would be nice to know what it is doing. In any case, you might want to try out marvel to find out where your setup is bottlenecked. Also, you might want to consider scaling horizontally instead of vertically. Many smaller servers can be nicer than one big one. On Wednesday, January 28, 2015 at 2:09:22 PM UTC+1, Oto Iashvili wrote: > > Hi > > I have a website for classified. For this I'm using elasticsearch, > postgres and rails on a same ubuntu 14.04 dedicated server, with 256go of > RAM and 20 cores, 40 threads . > > I have 10 indexes on elasticsearch, each have default numbers of shardes > (5). They have between 1000 and 400 000 classifieds dependings on which > index. > approximatly 5000 requests per minute, 2/3 making an elasticsearch request. > > according to htop, jvm is using around 500% of CPU > I try different options, I reduce number of shardes per index, I also try > to change JAVA_OPTS of followed > > #JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC" > #JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC" > > #JAVA_OPTS="$JAVA_OPTS -XX:CMSInitiatingOccupancyFraction=75" > #JAVA_OPTS="$JAVA_OPTS -XX:+UseCMSInitiatingOccupancyOnly" > > JAVA_OPTS="$JAVA_OPTS -XX:+UseG1GC" > > but it doesnt seems to change anything. > > so to questions : > - when you change any setting on elasticsearch, and then restart, should > the improvement (if any) be visible immediatly or can it arrive a bit later > thanks to cache or any thing else ? > - can any one help me to find good configuration for JVM / elasticsearch > so it will not take that many ressources > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/49d08b16-276c-4639-8c1b-26cb5d74530f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
