Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-26 Thread mdz-munich 
It seems to work now. 


We simply added 

/ulimit -v unlimited /

to our tomcat-startup-script. 


@Yonik: Thanks again! 


Best regards,

Sebastian 


--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3200105.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread mdz-munich 
Maybe it's important:

- The OS (Open Suse 10) is virtualized on VMWare
- Network Attached Storage

Best regards

Sebastian


--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3191986.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread Yonik Seeley 
Yep, there ya go... your OS configuration is limiting you to 27G of
virtual address space per process.  Consider setting that to
unlimited.

-Yonik
http://www.lucidimagination.com



On Fri, Jul 22, 2011 at 1:05 PM, mdz-munich
 wrote:
> It says:
>
> /core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 257869
> max locked memory       (kbytes, -l) 64
> max memory size         (kbytes, -m) 28063940
> open files                      (-n) 8192
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 8192
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 257869
> virtual memory          (kbytes, -v) 27216080
> file locks                      (-x) unlimited/
>
>
> Best regards,
>
> Sebastian
>
>
>
> Yonik Seeley-2-2 wrote:
>>
>> OK, best guess is that you're going over some per-process address space
>> limit.
>>
>> Try seeing what "ulimit -a" says.
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> On Fri, Jul 22, 2011 at 12:51 PM, mdz-munich
>>  wrote:
>>> Hi Yonik,
>>>
>>> thanks for your reply!
>>>
 Are you specifically selecting MMapDirectory in solrconfig.xml?
>>>
>>> Nope.
>>>
>>> We installed Oracle's Runtime from
>>>
>>> http://java.com/de/download/linux_manual.jsp?locale=de
>>>
>>> /java.runtime.name = Java(TM) SE Runtime Environment
>>> sun.boot.library.path = /usr/java/jdk1.6.0_26/jre/lib/amd64
>>> java.vm.version = 20.1-b02
>>> shared.loader =
>>> java.vm.vendor = Sun Microsystems Inc.
>>> enable.master = true
>>> java.vendor.url = http://java.sun.com/
>>> path.separator = :
>>> java.vm.name = Java HotSpot(TM) 64-Bit Server VM
>>> tomcat.util.buf.StringCache.byte.enabled = true
>>> file.encoding.pkg = sun.io
>>> java.util.logging.config.file =
>>> /local/master01_tomcat7x_solr33x/conf/logging.properties
>>> user.country = DE
>>> sun.java.launcher = SUN_STANDARD
>>> sun.os.patch.level = unknown
>>> java.vm.specification.name = Java Virtual Machine Specification
>>> user.dir = /local/master01_tomcat7x_solr33x/logs
>>> solr.abortOnConfigurationError = true
>>> java.runtime.version = 1.6.0_26-b03
>>> java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
>>> java.endorsed.dirs = /local/master01_tomcat7x_solr33x/endorsed
>>> os.arch = amd64
>>> java.io.tmpdir = /local/master01_tomcat7x_solr33x/temp
>>> line.separator =  /
>>>
>>> But no success with 1000 docs/batch, this was thrown during optimize:
>>>
>>> /
>>> 22.07.2011 18:44:05 org.apache.solr.core.SolrCore execute
>>> INFO: [core.digi20] webapp=/solr path=/update params={} status=500
>>> QTime=87540
>>> 22.07.2011 18:44:05 org.apache.solr.common.SolrException log
>>> SEVERE: java.io.IOException: Map failed
>>>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>>>        at
>>> org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:303)
>>>        at
>>> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:217)
>>>        at
>>> org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
>>>        at
>>> org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:245)
>>>        at
>>> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:117)
>>>        at
>>> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:703)
>>>        at
>>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4196)
>>>        at
>>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3863)
>>>        at
>>> org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
>>>        at
>>> org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2715)
>>>        at
>>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2525)
>>>        at
>>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2462)
>>>        at
>>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:410)
>>>        at
>>> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85)
>>>        at
>>> org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:154)
>>>        at
>>> org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:177)
>>>        at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
>>>        at
>>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:67)
>>>        at
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
>>>        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
>>>        at
>>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
>>

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread mdz-munich 
It says:

/core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 257869
max locked memory   (kbytes, -l) 64
max memory size (kbytes, -m) 28063940
open files  (-n) 8192
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 8192
cpu time   (seconds, -t) unlimited
max user processes  (-u) 257869
virtual memory  (kbytes, -v) 27216080
file locks  (-x) unlimited/


Best regards,

Sebastian



Yonik Seeley-2-2 wrote:
> 
> OK, best guess is that you're going over some per-process address space
> limit.
> 
> Try seeing what "ulimit -a" says.
> 
> -Yonik
> http://www.lucidimagination.com
> 
> On Fri, Jul 22, 2011 at 12:51 PM, mdz-munich
>  wrote:
>> Hi Yonik,
>>
>> thanks for your reply!
>>
>>> Are you specifically selecting MMapDirectory in solrconfig.xml?
>>
>> Nope.
>>
>> We installed Oracle's Runtime from
>>
>> http://java.com/de/download/linux_manual.jsp?locale=de
>>
>> /java.runtime.name = Java(TM) SE Runtime Environment
>> sun.boot.library.path = /usr/java/jdk1.6.0_26/jre/lib/amd64
>> java.vm.version = 20.1-b02
>> shared.loader =
>> java.vm.vendor = Sun Microsystems Inc.
>> enable.master = true
>> java.vendor.url = http://java.sun.com/
>> path.separator = :
>> java.vm.name = Java HotSpot(TM) 64-Bit Server VM
>> tomcat.util.buf.StringCache.byte.enabled = true
>> file.encoding.pkg = sun.io
>> java.util.logging.config.file =
>> /local/master01_tomcat7x_solr33x/conf/logging.properties
>> user.country = DE
>> sun.java.launcher = SUN_STANDARD
>> sun.os.patch.level = unknown
>> java.vm.specification.name = Java Virtual Machine Specification
>> user.dir = /local/master01_tomcat7x_solr33x/logs
>> solr.abortOnConfigurationError = true
>> java.runtime.version = 1.6.0_26-b03
>> java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
>> java.endorsed.dirs = /local/master01_tomcat7x_solr33x/endorsed
>> os.arch = amd64
>> java.io.tmpdir = /local/master01_tomcat7x_solr33x/temp
>> line.separator =  /
>>
>> But no success with 1000 docs/batch, this was thrown during optimize:
>>
>> /
>> 22.07.2011 18:44:05 org.apache.solr.core.SolrCore execute
>> INFO: [core.digi20] webapp=/solr path=/update params={} status=500
>> QTime=87540
>> 22.07.2011 18:44:05 org.apache.solr.common.SolrException log
>> SEVERE: java.io.IOException: Map failed
>>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>>        at
>> org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:303)
>>        at
>> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:217)
>>        at
>> org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
>>        at
>> org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:245)
>>        at
>> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:117)
>>        at
>> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:703)
>>        at
>> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4196)
>>        at
>> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3863)
>>        at
>> org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
>>        at
>> org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2715)
>>        at
>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2525)
>>        at
>> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2462)
>>        at
>> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:410)
>>        at
>> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85)
>>        at
>> org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:154)
>>        at
>> org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:177)
>>        at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
>>        at
>> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:67)
>>        at
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
>>        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
>>        at
>> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
>>        at
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
>>        at
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>>        at
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>>        at
>> org.apache.catalina.core.StandardWrapperValve

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread Yonik Seeley 
OK, best guess is that you're going over some per-process address space limit.

Try seeing what "ulimit -a" says.

-Yonik
http://www.lucidimagination.com

On Fri, Jul 22, 2011 at 12:51 PM, mdz-munich
 wrote:
> Hi Yonik,
>
> thanks for your reply!
>
>> Are you specifically selecting MMapDirectory in solrconfig.xml?
>
> Nope.
>
> We installed Oracle's Runtime from
>
> http://java.com/de/download/linux_manual.jsp?locale=de
>
> /java.runtime.name = Java(TM) SE Runtime Environment
> sun.boot.library.path = /usr/java/jdk1.6.0_26/jre/lib/amd64
> java.vm.version = 20.1-b02
> shared.loader =
> java.vm.vendor = Sun Microsystems Inc.
> enable.master = true
> java.vendor.url = http://java.sun.com/
> path.separator = :
> java.vm.name = Java HotSpot(TM) 64-Bit Server VM
> tomcat.util.buf.StringCache.byte.enabled = true
> file.encoding.pkg = sun.io
> java.util.logging.config.file =
> /local/master01_tomcat7x_solr33x/conf/logging.properties
> user.country = DE
> sun.java.launcher = SUN_STANDARD
> sun.os.patch.level = unknown
> java.vm.specification.name = Java Virtual Machine Specification
> user.dir = /local/master01_tomcat7x_solr33x/logs
> solr.abortOnConfigurationError = true
> java.runtime.version = 1.6.0_26-b03
> java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
> java.endorsed.dirs = /local/master01_tomcat7x_solr33x/endorsed
> os.arch = amd64
> java.io.tmpdir = /local/master01_tomcat7x_solr33x/temp
> line.separator =  /
>
> But no success with 1000 docs/batch, this was thrown during optimize:
>
> /
> 22.07.2011 18:44:05 org.apache.solr.core.SolrCore execute
> INFO: [core.digi20] webapp=/solr path=/update params={} status=500
> QTime=87540
> 22.07.2011 18:44:05 org.apache.solr.common.SolrException log
> SEVERE: java.io.IOException: Map failed
>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
>        at
> org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:303)
>        at 
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:217)
>        at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
>        at
> org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:245)
>        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:117)
>        at 
> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:703)
>        at 
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4196)
>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3863)
>        at
> org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
>        at 
> org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2715)
>        at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2525)
>        at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2462)
>        at
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:410)
>        at
> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85)
>        at
> org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:154)
>        at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:177)
>        at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
>        at
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:67)
>        at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
>        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
>        at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
>        at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
>        at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
>        at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
>        at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
>        at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
>        at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
>        at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
>        at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
>        at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563)
>        at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
>        at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:403)
>        at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:301)
>        at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:162)
>        at
> org.apa

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread mdz-munich 
Hi Yonik,

thanks for your reply! 

> Are you specifically selecting MMapDirectory in solrconfig.xml? 

Nope.

We installed Oracle's Runtime from 

http://java.com/de/download/linux_manual.jsp?locale=de

/java.runtime.name = Java(TM) SE Runtime Environment
sun.boot.library.path = /usr/java/jdk1.6.0_26/jre/lib/amd64
java.vm.version = 20.1-b02
shared.loader = 
java.vm.vendor = Sun Microsystems Inc.
enable.master = true
java.vendor.url = http://java.sun.com/
path.separator = :
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
tomcat.util.buf.StringCache.byte.enabled = true
file.encoding.pkg = sun.io
java.util.logging.config.file =
/local/master01_tomcat7x_solr33x/conf/logging.properties
user.country = DE
sun.java.launcher = SUN_STANDARD
sun.os.patch.level = unknown
java.vm.specification.name = Java Virtual Machine Specification
user.dir = /local/master01_tomcat7x_solr33x/logs
solr.abortOnConfigurationError = true
java.runtime.version = 1.6.0_26-b03
java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
java.endorsed.dirs = /local/master01_tomcat7x_solr33x/endorsed
os.arch = amd64
java.io.tmpdir = /local/master01_tomcat7x_solr33x/temp
line.separator =  /

But no success with 1000 docs/batch, this was thrown during optimize: 

/
22.07.2011 18:44:05 org.apache.solr.core.SolrCore execute
INFO: [core.digi20] webapp=/solr path=/update params={} status=500
QTime=87540 
22.07.2011 18:44:05 org.apache.solr.common.SolrException log
SEVERE: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:303)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:217)
at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:245)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:117)
at 
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:703)
at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4196)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3863)
at
org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2715)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2525)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2462)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:410)
at
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85)
at
org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:154)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:177)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:67)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:403)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:301)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:162)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.ni

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread Yonik Seeley 
On Fri, Jul 22, 2011 at 9:44 AM, Yonik Seeley
 wrote:
>> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64
>
> I'm confused why MMapDirectory is getting used with the IBM JVM... I
> had thought it would default to NIOFSDirectory on Linux w/ a non
> Oracle JVM.

I verified that the MMapDirectory is selected by default with the IBM
JVM (it must also contain the right Sun internal classes).

Anyone else have experience with MMapDirectory w/ IBM's JVM?

-Yonik
http://www.lucidimagination.com


> Are you specifically selecting MMapDirectory in solrconfig.xml?
>
> Can you try the Oracle JVM to see if that changes things?
>
> -Yonik
> http://www.lucidimagination.com
>
>
>
> On Fri, Jul 22, 2011 at 5:58 AM, mdz-munich
>  wrote:
>> I was wrong.
>>
>> After rebooting tomcat we discovered a new sweetness:
>>
>> /SEVERE: REFCOUNT ERROR: unreferenced org.apache.solr.core.SolrCore@3c753c75
>> (core.name) has a reference count of 1
>> 22.07.2011 11:52:07 org.apache.solr.common.SolrException log
>> SEVERE: java.lang.RuntimeException: java.io.IOException: Map failed
>>        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1099)
>>        at org.apache.solr.core.SolrCore.(SolrCore.java:585)
>>        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:463)
>>        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316)
>>        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:207)
>>        at
>> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:130)
>>        at
>> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:94)
>>        at
>> org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:273)
>>        at
>> org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:254)
>>        at
>> org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:372)
>>        at
>> org.apache.catalina.core.ApplicationFilterConfig.(ApplicationFilterConfig.java:98)
>>        at
>> org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4584)
>>        at
>> org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5262)
>>        at
>> org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5257)
>>        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
>>        at java.util.concurrent.FutureTask.run(FutureTask.java:149)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
>>        at java.lang.Thread.run(Thread.java:736)
>> Caused by: java.io.IOException: Map failed
>>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
>>        at
>> org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
>>        at 
>> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
>>        at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
>>        at
>> org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)
>>        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
>>        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:92)
>>        at 
>> org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:113)
>>        at
>> org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirectoryReader.java:29)
>>        at
>> org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:81)
>>        at
>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:750)
>>        at 
>> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
>>        at org.apache.lucene.index.IndexReader.open(IndexReader.java:428)
>>        at org.apache.lucene.index.IndexReader.open(IndexReader.java:371)
>>        at
>> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
>>        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1088)
>>        ... 18 more
>> Caused by: java.lang.OutOfMemoryError: Map failed
>>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
>>        ... 33 more/
>>
>> Any ideas and/or suggestions?
>>
>> Best regards & thank you,
>>
>> Sebastian
>>
>> --
>> View this message in context: 
>> http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3190976.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread Yonik Seeley 
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64

I'm confused why MMapDirectory is getting used with the IBM JVM... I
had thought it would default to NIOFSDirectory on Linux w/ a non
Oracle JVM.
Are you specifically selecting MMapDirectory in solrconfig.xml?

Can you try the Oracle JVM to see if that changes things?

-Yonik
http://www.lucidimagination.com



On Fri, Jul 22, 2011 at 5:58 AM, mdz-munich
 wrote:
> I was wrong.
>
> After rebooting tomcat we discovered a new sweetness:
>
> /SEVERE: REFCOUNT ERROR: unreferenced org.apache.solr.core.SolrCore@3c753c75
> (core.name) has a reference count of 1
> 22.07.2011 11:52:07 org.apache.solr.common.SolrException log
> SEVERE: java.lang.RuntimeException: java.io.IOException: Map failed
>        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1099)
>        at org.apache.solr.core.SolrCore.(SolrCore.java:585)
>        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:463)
>        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316)
>        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:207)
>        at
> org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:130)
>        at
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:94)
>        at
> org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:273)
>        at
> org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:254)
>        at
> org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:372)
>        at
> org.apache.catalina.core.ApplicationFilterConfig.(ApplicationFilterConfig.java:98)
>        at
> org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4584)
>        at
> org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5262)
>        at
> org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5257)
>        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:149)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
>        at java.lang.Thread.run(Thread.java:736)
> Caused by: java.io.IOException: Map failed
>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
>        at
> org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
>        at 
> org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
>        at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
>        at
> org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)
>        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
>        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:92)
>        at 
> org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:113)
>        at
> org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirectoryReader.java:29)
>        at
> org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:81)
>        at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:750)
>        at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
>        at org.apache.lucene.index.IndexReader.open(IndexReader.java:428)
>        at org.apache.lucene.index.IndexReader.open(IndexReader.java:371)
>        at
> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
>        at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1088)
>        ... 18 more
> Caused by: java.lang.OutOfMemoryError: Map failed
>        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
>        ... 33 more/
>
> Any ideas and/or suggestions?
>
> Best regards & thank you,
>
> Sebastian
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3190976.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

RE: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread mdz-munich 
I was wrong. 

After rebooting tomcat we discovered a new sweetness: 

/SEVERE: REFCOUNT ERROR: unreferenced org.apache.solr.core.SolrCore@3c753c75
(core.name) has a reference count of 1
22.07.2011 11:52:07 org.apache.solr.common.SolrException log
SEVERE: java.lang.RuntimeException: java.io.IOException: Map failed
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1099)
at org.apache.solr.core.SolrCore.(SolrCore.java:585)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:463)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:207)
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:130)
at
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:94)
at
org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:273)
at
org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:254)
at
org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:372)
at
org.apache.catalina.core.ApplicationFilterConfig.(ApplicationFilterConfig.java:98)
at
org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4584)
at
org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5262)
at
org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5257)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
at java.util.concurrent.FutureTask.run(FutureTask.java:149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
at java.lang.Thread.run(Thread.java:736)
Caused by: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:92)
at 
org.apache.lucene.index.DirectoryReader.(DirectoryReader.java:113)
at
org.apache.lucene.index.ReadOnlyDirectoryReader.(ReadOnlyDirectoryReader.java:29)
at
org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:81)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:750)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:75)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:428)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:371)
at
org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1088)
... 18 more
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
... 33 more/

Any ideas and/or suggestions? 

Best regards & thank you,

Sebastian 

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3190976.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-22 Thread mdz-munich 

mdz-munich wrote:
> 
> Yeah, indeed.
> 
> But since the VM is equipped with plenty of RAM (22GB) and it works so far
> (Solr 3.2) very well with this setup, I AM slightly confused, am I?
> 
> Maybe we should LOWER the dedicated Physical Memory? The remaining 10GB
> are used for a second tomcat (8GB) and the OS (Suse). As far as I
> understand NIO (mostly un-far), this package "can directly use the most
> efficient operations of the underlying platform".
> 

After three days of banging my head against the wall: Problem solved! 

It seems there was not enough Memory left for NIO. 

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3190916.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-20 Thread mdz-munich 
Yeah, indeed.

But since the VM is equipped with plenty of RAM (22GB) and it works so far
(Solr 3.2) very well with this setup, I AM slightly confused, am I?

Maybe we should LOWER the dedicated Physical Memory? The remaining 10GB are
used for a second tomcat (8GB) and the OS (Suse). As far as I understand NIO
(mostly un-far), this package "can directly use the most efficient
operations of the underlying platform". 






 

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3186986.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-20 Thread Robert Petersen 
Says it is caused by a Java out of memory error, no?  

-Original Message-
From: mdz-munich [mailto:sebastian.lu...@bsb-muenchen.de] 
Sent: Wednesday, July 20, 2011 9:18 AM
To: solr-user@lucene.apache.org
Subject: Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

Here we go ...

This time we tried to use the old LogByteSizeMergePolicy and
SerialMergeScheduler:




We did this before, just to be sure ... 

~300 Documents:

/
SEVERE: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirector
y.java:264)
at
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at
org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreRead
ers.java:244)
at
org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4192)
at
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3859)
at
org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.
java:37)
at
org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2714)
at
org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2709)
at
org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2705)
at
org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3509)
at
org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1850)
at
org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1814)
at
org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1778)
at
org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:143)
at
org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHand
ler2.java:183)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.
java:416)
at
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpd
ateProcessorFactory.java:85)
at
org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:98)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Conte
ntStreamHandlerBase.java:67)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerB
ase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.ja
va:356)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j
ava:252)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
erChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
e.java:240)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
e.java:164)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(Authenticator
Base.java:462)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
:164)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
:100)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563
)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:4
03)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:30
1)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(
Http11Protocol.java:162)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(
Http11Protocol.java:140)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.j
ava:309)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecuto
r.java:897)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
va:919)
at java.lang.Thread.run(Thread.java:736)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
... 44 more

20.07.2011 18:07:30 org.apache.solr.core.SolrCore execute
INFO: [core.digi20] webapp=/solr path=/update params={} status=500
QTime=12302 
20.07.2011 18:07:30 org.apache.solr.common.SolrException log
SEVERE: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-20 Thread mdz-munich 
Here we go ...

This time we tried to use the old LogByteSizeMergePolicy and
SerialMergeScheduler:




We did this before, just to be sure ... 

~300 Documents:

/
SEVERE: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at 
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702)
at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4192)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3859)
at
org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2714)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2709)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2705)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3509)
at 
org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1850)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1814)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1778)
at 
org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:143)
at
org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:183)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:416)
at
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:85)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:98)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:67)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:240)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:164)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:164)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:563)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:403)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:301)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:162)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:140)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
at java.lang.Thread.run(Thread.java:736)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
... 44 more

20.07.2011 18:07:30 org.apache.solr.core.SolrCore execute
INFO: [core.digi20] webapp=/solr path=/update params={} status=500
QTime=12302 
20.07.2011 18:07:30 org.apache.solr.common.SolrException log
SEVERE: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)

Re: Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-20 Thread mdz-munich 
Update.

After adding 1626 documents without doing a commit or optimize:

/Exception in thread "Lucene Merge Thread #1"
org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: Map
failed
at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:517)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
Caused by: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at 
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702)
at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4192)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3859)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
... 10 more
/

Any ideas, any suggestions?

Greetz & thank you,

Sebastian



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-3-3-Exception-in-thread-Lucene-Merge-Thread-1-tp3185248p3185344.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr 3.3: Exception in thread "Lucene Merge Thread #1"

2011-07-20 Thread mdz-munich 
Dear Devs and Users,

it is I! 

Okay, it starts with that:

/Exception in thread "Lucene Merge Thread #1"
org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: Map
failed
at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:517)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
Caused by: java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at org.apache.lucene.index.FieldsReader.(FieldsReader.java:129)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
at 
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702)
at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4192)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3859)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:388)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
... 10 more/


And than quickly moves forward to that:

/SEVERE: auto commit error...
java.io.IOException: Map failed
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:782)
at
org.apache.lucene.store.MMapDirectory$MMapIndexInput.(MMapDirectory.java:264)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
at
org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:88)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:114)
at 
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:702)
at 
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:677)
at
org.apache.lucene.index.BufferedDeletesStream.applyDeletes(BufferedDeletesStream.java:249)
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3571)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3508)
at 
org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1850)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1814)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1778)
at 
org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:143)
at
org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:183)
at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:416)
at
org.apache.solr.update.DirectUpdateHandler2$CommitTracker.run(DirectUpdateHandler2.java:611)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:452)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
at java.util.concurrent.FutureTask.run(FutureTask.java:149)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:109)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:218)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
at java.lang.Thread.run(Thread.java:736)
Caused by: java.lang.OutOfMemoryError: Map failed
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:779)
... 24 more/


And at than that:
/
SEVERE: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed
out:
NativeFSLock@/bsbsolrdata/solrindex/master01_solr33x/core.digi20/data/index/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:84)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:1115)
at 
org.apache.solr.update.SolrIndexWriter.(SolrIndexWriter.java:83)
at
org.apache.solr.update.UpdateHandler.createMainIndexWriter(UpdateHandler.java:101)
at
org.apache.solr.update.DirectUpdateHandler2.openWriter(DirectUpdateHandler2.java:175)
at
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:223)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:99)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java: