Re: Solr High GC issue
@rulinma: I've analyzed all your messages and noticed you keep sending useless mark and good. Are you trying to spam the mailing list? Could you please stop doing this? E-mail clients these days have bookmarking / staring in case you want to save the good post for later reading or mark it otherwise. Thanks, Dmitry On Wed, Jun 4, 2014 at 3:25 PM, rulinma ruli...@gmail.com wrote: mark. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-High-GC-issue-tp4138570p4139785.html Sent from the Solr - User mailing list archive at Nabble.com. -- Dmitry Kan Blog: http://dmitrykan.blogspot.com Twitter: http://twitter.com/dmitrykan
Re: Solr High GC issue
mark. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-High-GC-issue-tp4138570p4139785.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr High GC issue
BTW, memory usage is _much_ better in Solr 4.x, if it's at all an option you might consider upgrading. See: http://searchhub.org/2012/04/06/memory-comparisons-between-solr-3x-and-trunk/ Best, Erick On Thu, May 29, 2014 at 11:40 AM, Toke Eskildsen t...@statsbiblioteket.dk wrote: bihan.chandu [bihan.cha...@gmail.com] wrote: I am Currently using solr 3.6.1 and my system handle lot of request .Now we are facing High GC issue in system. Maybe it would help to get an idea of what is causing all the allocations? - How many documents in your index? - How many queries/sec? - How long does a typical query take? - How many cores does your machine have? - How many documents are returned/query? - How much faceting do you perform? How many unique terms/facet field? - Toke Eskildsen
Re: Solr High GC issue
Hi Bihan,
That's a lot of parameters and without trying one can't really give you
very specific and good advice. If I had to suggest something quickly I'd
say:
* go back to the basics - remove most of those params and stick with the
basic ones. Look at GC and tune slowly by changing/adding params one at a
time.
* consider using G1 GC with the most recent Java7.
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr Elasticsearch Support * http://sematext.com/
On Thu, May 29, 2014 at 1:36 AM, bihan.chandu bihan.cha...@gmail.comwrote:
Hi All
I am Currently using solr 3.6.1 and my system handle lot of request .Now we
are facing High GC issue in system. Please find the memory parameters in my
solr system . Can some on help me to identify is there any relationship
between my memory parameters and GC issue.
MEM_ARGS=-Xms7936M -Xmx7936M -XX:NewSize=512M -XX:MaxNewSize=512M
-Xss1024k
-XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+CMSParallelRemarkEnabled -XX:+AggressiveOpts
-XX:LargePageSizeInBytes=2m -XX:+UseLargePages -XX:MaxTenuringThreshold=15
-XX:-UseAdaptiveSizePolicy -XX:PermSize=256M -XX:MaxPermSize=256M
-XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGC
-Xloggc:${GCLOG} -XX:-OmitStackTraceInFastThrow -XX:+DisableExplicitGC
-XX:-BindGCTaskThreadsToCPUs -verbose:gc -XX:StackShadowPages=20
Thanks
Bihan
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-High-GC-issue-tp4138570.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: Solr High GC issue
you will probably also want to get some better visibility into what is going on
with your JVM and GC
easiest way is to enable some GC logging options. the following additional
options will give you a good deal of information in gc logs
-Xloggc:$JETTY_LOGS/gc.log
-verbose:gc
-XX:+PrintGCDateStamps
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
-XX:+PrintTenuringDistribution
-XX:+PrintClassHistogram
-XX:+PrintHeapAtGC
-XX:+PrintGCApplicationConcurrentTime
-XX:+PrintGCApplicationStoppedTime
-XX:+PrintPromotionFailure
-XX:+PrintAdaptiveSizePolicy
-XX:+PrintTLAB
-XX:PrintFLSStatistics=1
-XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=5
-XX:GCLogFileSize=10m
you may find you have a particular portion of your heap which is undersized
using G1GC with the adaptive sizing is a very handy way to deal with the memory
usage in solr which can be somewhat difficult to tune optimally using the
traditional static ratios (what works well for data ingestion is probably not
optimal for searching)
once you have a baseline of logs using your existing JVM sizings and the
additional logging options above you might try switching from CMS to G1GC with
adaptive sizing, and removing all the static tunings for tenuring and ratios
and compare to a very minimal G1GC config
-XX:+UseG1GC
-XX:+UseAdaptiveSizePolicy -XX:MaxGCPauseMillis=1000
--
configuring the JMX interface is another way to get real time views into what
is going on using jconsole or jvisualvm tools
From: Otis Gospodnetic otis.gospodne...@gmail.com
Sent: Thursday, May 29, 2014 07:20
To: solr-user@lucene.apache.org
Subject: Re: Solr High GC issue
Hi Bihan,
That's a lot of parameters and without trying one can't really give you
very specific and good advice. If I had to suggest something quickly I'd
say:
* go back to the basics - remove most of those params and stick with the
basic ones. Look at GC and tune slowly by changing/adding params one at a
time.
* consider using G1 GC with the most recent Java7.
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr Elasticsearch Support * http://sematext.com/
On Thu, May 29, 2014 at 1:36 AM, bihan.chandu bihan.cha...@gmail.comwrote:
Hi All
I am Currently using solr 3.6.1 and my system handle lot of request .Now we
are facing High GC issue in system. Please find the memory parameters in my
solr system . Can some on help me to identify is there any relationship
between my memory parameters and GC issue.
MEM_ARGS=-Xms7936M -Xmx7936M -XX:NewSize=512M -XX:MaxNewSize=512M
-Xss1024k
-XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+CMSParallelRemarkEnabled -XX:+AggressiveOpts
-XX:LargePageSizeInBytes=2m -XX:+UseLargePages -XX:MaxTenuringThreshold=15
-XX:-UseAdaptiveSizePolicy -XX:PermSize=256M -XX:MaxPermSize=256M
-XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGC
-Xloggc:${GCLOG} -XX:-OmitStackTraceInFastThrow -XX:+DisableExplicitGC
-XX:-BindGCTaskThreadsToCPUs -verbose:gc -XX:StackShadowPages=20
Thanks
Bihan
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-High-GC-issue-tp4138570.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr High GC issue
Hi All Thanks for the Suggestion. I will implement this changes and let us know the update Regards Bihan -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-High-GC-issue-tp4138570p4138694.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr High GC issue
Agreed, that is a LOT of options.
First, check the defaults and remove any flags that are setting something to
the default. You can see all the flags and the default values with this command:
java -XX:+PrintFlagsFinal -version
For example, the default for ParallelGCThreads is 8, so you do not need to set
that.
We set a fairly large new generation, about 1/4 of heap. 512 Meg is way too
small. Solr will allocate a lot of objects that are only used to handle one
HTTP request. You want all of those to fit in the new space, even when there
are simultaneous requests. If new is not big enough, they will be allocated in
tenured space and will cause more frequent major GCs. For an 8G heap, we use a
2G new size.
wunder
On May 29, 2014, at 7:20 AM, Otis Gospodnetic otis.gospodne...@gmail.com
wrote:
Hi Bihan,
That's a lot of parameters and without trying one can't really give you
very specific and good advice. If I had to suggest something quickly I'd
say:
* go back to the basics - remove most of those params and stick with the
basic ones. Look at GC and tune slowly by changing/adding params one at a
time.
* consider using G1 GC with the most recent Java7.
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr Elasticsearch Support * http://sematext.com/
On Thu, May 29, 2014 at 1:36 AM, bihan.chandu bihan.cha...@gmail.comwrote:
Hi All
I am Currently using solr 3.6.1 and my system handle lot of request .Now we
are facing High GC issue in system. Please find the memory parameters in my
solr system . Can some on help me to identify is there any relationship
between my memory parameters and GC issue.
MEM_ARGS=-Xms7936M -Xmx7936M -XX:NewSize=512M -XX:MaxNewSize=512M
-Xss1024k
-XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=80 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+CMSParallelRemarkEnabled -XX:+AggressiveOpts
-XX:LargePageSizeInBytes=2m -XX:+UseLargePages -XX:MaxTenuringThreshold=15
-XX:-UseAdaptiveSizePolicy -XX:PermSize=256M -XX:MaxPermSize=256M
-XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintGC
-Xloggc:${GCLOG} -XX:-OmitStackTraceInFastThrow -XX:+DisableExplicitGC
-XX:-BindGCTaskThreadsToCPUs -verbose:gc -XX:StackShadowPages=20
Thanks
Bihan
RE: Solr High GC issue
bihan.chandu [bihan.cha...@gmail.com] wrote: I am Currently using solr 3.6.1 and my system handle lot of request .Now we are facing High GC issue in system. Maybe it would help to get an idea of what is causing all the allocations? - How many documents in your index? - How many queries/sec? - How long does a typical query take? - How many cores does your machine have? - How many documents are returned/query? - How much faceting do you perform? How many unique terms/facet field? - Toke Eskildsen
