Re: background merge hit exception while optimizing index (SOLR 4.4.0)

2014-01-14 Thread Ralf Matulat
It becomes just more spooky. The optimize-run this night was succesful. Yesterday I did two things: 1. Checked the index without any result (no problems found). 2. I did an expungeDelete on the mentioned index. So I have no idea what is going on here. Btw: Java version is still the old 1.6.0. f

Re: background merge hit exception while optimizing index (SOLR 4.4.0)

2014-01-14 Thread Ralf Matulat
I checked the index without any problems beeing found. So it is not obvious, what is going wrong here while the index itself looks okay. Next step, updating java, is work in progress. So I will come back after sorting out the java version as the cause for the failing optimize. The checkIndex

Re: background merge hit exception while optimizing index (SOLR 4.4.0)

2014-01-13 Thread Ralf Matulat
It sounds quite obvious to upgrade the java environment to go on with that. We are updating our index almost every second and so over time it counts up a lot of segment files (up to 32 in our case). So optimizing was always a good idea in the SOLR 3.X world. We regocnized that in SOLR 4.4.0 it

Re: background merge hit exception while optimizing index (SOLR 4.4.0)

2014-01-13 Thread Michael McCandless
I have trouble understanding J9's version strings ... but, is it really from 2008? You could be hitting a JVM bug; can you test upgrading? I don't have much experience with Solr faceting on optimized vs unoptimized indices; maybe someone else can answer your question. Lucene's facet module (not

Re: background merge hit exception while optimizing index (SOLR 4.4.0)

2014-01-13 Thread Ralf Matulat
> java -version java version "1.6.0" Java(TM) SE Runtime Environment (build pxa6460sr3ifix-20090218_02(SR3+IZ43791+IZ43798)) IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460-20081105_25433 (JIT enabled, AOT enabled) J9VM - 20081105_025433_LHdSMr JIT - r9_20081031_1330 GC

Re: background merge hit exception while optimizing index (SOLR 4.4.0)

2014-01-13 Thread Michael McCandless
Which version of Java are you using? That root cause exception is somewhat spooky: it's in the ByteBufferIndexCode that handles an "UnderflowException", ie when a small (maybe a few hundred bytes) read happens to span the 1 GB page boundary, and specifically the exception happens on the final read

background merge hit exception while optimizing index (SOLR 4.4.0)

2014-01-12 Thread Ralf Matulat
Hi, I am currently running into merge-issues while optimizing an index. To give you some informations: We are running 4 SOLR Servers with identical OS, VM-Hardware, RAM etc. Only one Server by now is having issues, the others are fine. We are running SOLR 4.4.0 with Tomcat 6.0 It was running si