Robert, Step 1 is to get the latest Java 7 or if you have to remain on 6 then use the latest 6.
Otis -- SOLR Performance Monitoring - http://sematext.com/spm On Dec 18, 2012 7:54 PM, "Petersen, Robert" <rober...@buy.com> wrote: > Hi solr user group,**** > > ** ** > > Sorry if this isn’t directly a Solr question. Seems like once in a blue > moon the GC crashes on a server in our Solr 3.6.1 slave farm. This seems > to only happen on a couple of the twelve slaves we have deployed and only > very rarely on those. It seems like this doesn’t directly affect solr > because in the logs it looks like solr keeps working after the time of the > exception but our external monitoring tool reports that the solr service is > down so our operations department restarts solr on that box and alerts me. > The solr logs show nothing unusual. The exception does show up in the > catalina.out log file though. Does this happen to anyone else? Here is > the basic error and I have attached the crash dump file also. Our total > uptime on these boxes is over a year now BTW.**** > > ** ** > > #**** > > # A fatal error has been detected by the Java Runtime Environment:**** > > #**** > > # SIGSEGV (0xb) at pc=0x00002b5379346612, pid=13724, tid=1082353984**** > > #**** > > # JRE version: 6.0_25-b06**** > > # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode > linux-amd64 )**** > > # Problematic frame:**** > > # V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned > long)+0x82**** > > #**** > > # An error report file with more information is saved as:**** > > # /var/LucidWorks/lucidworks/hs_err_pid13724.log**** > > #**** > > # If you would like to submit a bug report, please visit:**** > > # http://java.sun.com/webapps/bugreport/crash.jsp**** > > #**** > > ** ** > > VM Arguments:**** > > jvm_args: > -Djava.util.logging.config.file=/var/LucidWorks/lucidworks/tomcat/conf/logging.properties > -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > -Xmx32768m -Xms32768m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode > -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false > -Dcom.sun.management.jmxremote.authenticate=false > -Dcom.sun.management.jmxremote.port=6060 > -Djava.endorsed.dirs=/var/LucidWorks/lucidworks/tomcat/endorsed > -Dcatalina.base=/var/LucidWorks/lucidworks/tomcat > -Dcatalina.home=/var/LucidWorks/lucidworks/tomcat > -Djava.io.tmpdir=/var/LucidWorks/lucidworks/tomcat/temp **** > > java_command: org.apache.catalina.startup.Bootstrap -server > -Dsolr.solr.home=lucidworks/solr start**** > > Launcher Type: SUN_STANDARD**** > > ** ** > > Stack: [0x0000000000000000,0x0000000000000000], sp=0x0000000040835eb0, > free space=1056983k**** > > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > code)**** > > V [libjvm.so+0x3c4612] Par_ConcMarkingClosure::trim_queue(unsigned > long)+0x82**** > > V [libjvm.so+0x3c481a] CMSConcMarkingTask::do_work_steal(int)+0xfa**** > > V [libjvm.so+0x3c3dcf] CMSConcMarkingTask::work(int)+0xef**** > > V [libjvm.so+0x8783dc] YieldingFlexibleGangWorker::loop()+0xbc**** > > V [libjvm.so+0x8755b4] GangWorker::run()+0x24**** > > V [libjvm.so+0x71096f] java_start(Thread*)+0x13f**** > > ** ** > > Heap**** > > par new generation total 345024K, used 180672K [0x00002aaaae120000, > 0x00002aaac5780000, 0x00002aaac5780000)**** > > eden space 306688K, 53% used [0x00002aaaae120000, 0x00002aaab8243c28, > 0x00002aaac0ca0000)**** > > from space 38336K, 40% used [0x00002aaac3210000, 0x00002aaac415c3f8, > 0x00002aaac5780000)**** > > to space 38336K, 0% used [0x00002aaac0ca0000, 0x00002aaac0ca0000, > 0x00002aaac3210000)**** > > concurrent mark-sweep generation total 33171072K, used 12144213K > [0x00002aaac5780000, 0x00002ab2ae120000, 0x00002ab2ae120000)**** > > concurrent-mark-sweep perm gen total 83968K, used 50650K > [0x00002ab2ae120000, 0x00002ab2b3320000, 0x00002ab2b3320000)**** > > ** ** > > Code Cache [0x00002aaaab054000, 0x00002aaaab9a4000, 0x00002aaaae054000)** > ** > > total_blobs=2800 nmethods=2273 adapters=480 free_code_cache=40752512 > largest_free_block=15808**** > > ** ** > > ** ** > > ** ** > > Thanks,**** > > ** ** > > *Robert (Robi) Petersen* > > Senior Software Engineer**** > > Search Department**** > > ** ** >
