Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2014-07-28 Thread Harald Kirsch 

Hi,

the stack trace points to tika, which is likely in the process of 
extracting indexable plain text from some document.


Tika's job is one of the dirtiest you can think of in the whole indexing 
business. We throw all kinds of more or less 
documented/broken/misguided/ill-designed/cruft/truncated documents at it 
and want it to do miracles in understanding that stuff and getting the 
plain text out.


It does quite a good job most of the time, but sometimes it just gets 
trapped. There is nearly no chance to get Tika bug-free (whatever that 
means when the requirements are ill-defined), so we must live with 
accidents like you have here, where seemingly Tika reckoned that it 
needs a large amount of memory to parse a document.


There are two ways out:

a) You very strictly control whatever document enters Tika by using a 
white-list.
b) You don't let Tika run as part of Solr, but take it into a seperate 
process, let it crash and restart it automatically.


Regards,
Harald.



On 25.07.2014 19:32, Ameya Aware wrote:

Please find below entire stack trace:


ERROR - 2014-07-25 13:14:22.202; org.apache.solr.common.SolrException;
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Requested
array size exceeds VM limit
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:790)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:439)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
at java.lang.AbstractStringBuilder.append(Unknown Source)
at java.lang.StringBuilder.append(Unknown Source)
at
org.apache.solr.handler.extraction.SolrContentHandler.characters(SolrContentHandler.java:303)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)
at
org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)
at
org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)
at
org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)
at
org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:278)
at org.apache.tika.parser.txt.TXTParser.parse(TXTParser.java:88)
at

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2014-07-25 Thread Greg Walters 
Would you include the entire stack trace for your OOM message? Are you seeing 
this on the client or server side?

Thanks,
Greg

On Jul 25, 2014, at 10:21 AM, Ameya Aware ameya.aw...@gmail.com wrote:

 Hi,
 
 I am in process of indexing lot of documents but after around 9
 documents i am getting below error:
 
 java.lang.OutOfMemoryError: Requested array size exceeds VM limit
 
 I am passing below parameters with Solr :
 
 java -Xms6144m -Xmx6144m -XX:MaxPermSize=512m
 -Dcom.sun.management.jmxremote -XX:+UseParNewGC -XX:+UseCompressedOops
 -XX:+UseConcMarkSweepGC
 -XX:+CMSIncrementalMode -XX:+CMSParallelRemarkEnabled
 -XX:+UseCMSInitiatingOccupancyOnly
 -XX:CMSInitiatingOccupancyFraction=70 -XX:ConcGCThreads=6
 -XX:ParallelGCThreads=6 -jar start.jar
 
 
 Also, i am Auto-committing after 2 documents.
 
 
 I searched on google for this but could not get any specific answer.
 
 
 Can anybody help with this?
 
 
 Thanks,
 Ameya

RE: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2014-07-25 Thread Matt Kuiper (Springblox) 
You might consider looking at your internal Solr cache configuration 
(solrconfig.xml).  These caches occupy heap space, and from my understanding do 
not overflow to disk.  So if there is not enough heap memory to support the 
caches an OOM error will be thrown.

I also believe these caches live in Old Gen.  So you might consider decreasing 
your CMSInitiatingOccupancyFraction to trigger a GC sooner.

Based on your description below every 20,000 documents your caches will be 
invalidated and rebuilt as part of a commit.  So a GC that occurs sooner may 
help free the memory of the old caches.  

Matt

-Original Message-
From: Ameya Aware [mailto:ameya.aw...@gmail.com] 
Sent: Friday, July 25, 2014 9:22 AM
To: solr-user@lucene.apache.org
Subject: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

Hi,

I am in process of indexing lot of documents but after around 9 documents i 
am getting below error:

java.lang.OutOfMemoryError: Requested array size exceeds VM limit

I am passing below parameters with Solr :

java -Xms6144m -Xmx6144m -XX:MaxPermSize=512m -Dcom.sun.management.jmxremote 
-XX:+UseParNewGC -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC 
-XX:+CMSIncrementalMode -XX:+CMSParallelRemarkEnabled 
-XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSInitiatingOccupancyFraction=70 -XX:ConcGCThreads=6
-XX:ParallelGCThreads=6 -jar start.jar


Also, i am Auto-committing after 2 documents.


I searched on google for this but could not get any specific answer.


Can anybody help with this?


Thanks,
Ameya

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2014-07-25 Thread Ameya Aware 
Please find below entire stack trace:


ERROR - 2014-07-25 13:14:22.202; org.apache.solr.common.SolrException;
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Requested
array size exceeds VM limit
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:790)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:439)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at java.util.Arrays.copyOf(Unknown Source)
at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
at java.lang.AbstractStringBuilder.append(Unknown Source)
at java.lang.StringBuilder.append(Unknown Source)
at
org.apache.solr.handler.extraction.SolrContentHandler.characters(SolrContentHandler.java:303)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
at
org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)
at
org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)
at
org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)
at
org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)
at
org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:278)
at org.apache.tika.parser.txt.TXTParser.parse(TXTParser.java:88)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
at
org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:219)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:241)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2014-07-25 Thread Greg Walters 
Using Tika to extract documents or content is something I don't have experience 
with but it looks like your issue is in that process. If you're able to 
reproduce this issue near the same place every time maybe you've got a document 
that has a lot of nested fields in it or otherwise causes the extractor/update 
processor to do something weird.

Thanks,
Greg

On Jul 25, 2014, at 12:32 PM, Ameya Aware ameya.aw...@gmail.com wrote:

 Please find below entire stack trace:
 
 
 ERROR - 2014-07-25 13:14:22.202; org.apache.solr.common.SolrException;
 null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Requested
 array size exceeds VM limit
 at
 org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:790)
 at
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:439)
 at
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
 at
 org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
 at
 org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
 at
 org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
 at
 org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
 at
 org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
 at
 org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
 at
 org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
 at
 org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
 at
 org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
 at
 org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
 at
 org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
 at
 org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
 at org.eclipse.jetty.server.Server.handle(Server.java:368)
 at
 org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
 at
 org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
 at
 org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
 at
 org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
 at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:636)
 at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
 at
 org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
 at
 org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
 at
 org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
 at
 org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
 at java.lang.Thread.run(Unknown Source)
 Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
 at java.util.Arrays.copyOf(Unknown Source)
 at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
 at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
 at java.lang.AbstractStringBuilder.append(Unknown Source)
 at java.lang.StringBuilder.append(Unknown Source)
 at
 org.apache.solr.handler.extraction.SolrContentHandler.characters(SolrContentHandler.java:303)
 at
 org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at
 org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)
 at
 org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at
 org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at
 org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
 at
 org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)
 at
 org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)
 at
 org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)
 at
 org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)
 at
 org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:278)
 at org.apache.tika.parser.txt.TXTParser.parse(TXTParser.java:88)
 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242)
 at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:120)
 at
 org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:219)
 at
 org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
 at

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-08-07 Thread Toke Eskildsen 
On Thu, 2013-08-01 at 15:24 +0200, Grzegorz Sobczyk wrote:
 Today I found in solr logs exception: java.lang.OutOfMemoryError: Requested
 array size exceeds VM limit.
 At that time memory usage was ~200MB / Xmx3g

[...]

 Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
 at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:64)
 at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:37)
 at
 org.apache.solr.handler.component.ShardFieldSortedHitQueue.init(ShardDoc.java:113)

Are you requesting a very large number of results? Integer.MAX_VALUE
perhaps? If so, you need to change that to a more manageable number.

- Toke Eskildsen, State and University Library, Denmark

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-08-02 Thread Grzegorz Sobczyk 
Solr parameters listed in dashboard:
-DzkHost=localhost:2181,172.27.5.121:2181,172.27.5.122:2181
-XX:+UseConcMarkSweepGC
-Xmx3072m
-Djava.awt.headless=true

Mem usage in last days: http://i42.tinypic.com/29z5rew.png

It's production system and there's too many requests to detect which query
is buggy
some lines before every error (result of: zgrep -B 200 SEVERE:
null:java.lang.RuntimeException: java.lang.OutOfMemoryError
daemon.err.2.gz  out.log )
https://gist.github.com/gsobczyk/65e2ab9bb13a99aec89e

I've tried invoking recent request before exception but without success of
reproduce error



On 1 August 2013 20:39, Erick Erickson erickerick...@gmail.com wrote:

 What are the memory parameters you start Solr with? The Solr admin page
 will tell you how much memory the JVM has.

 Also, cut/paste the queries you're running when you see this.

 Best
 Erick


 On Thu, Aug 1, 2013 at 9:50 AM, Grzegorz Sobczyk gsobc...@gmail.com
 wrote:

  after node starts in log I have only few requests:
  https://gist.github.com/gsobczyk/6131503#file-solr-oom-log
  this error occurred multiple times
 
 
 
 
  On 1 August 2013 15:33, Rafał Kuć r@solr.pl wrote:
 
   Hello!
  
   The exception you've shown tells you that Solr tried to allocate an
   array that exceeded heap size. Do you use some custom sorts? Did you
   send large bulks during the time that the exception occurred?
  
   --
   Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch
  
Today I found in solr logs exception: java.lang.OutOfMemoryError:
   Requested
array size exceeds VM limit.
At that time memory usage was ~200MB / Xmx3g
  
Env looks like this:
3x standalone zK (Java 7)
3x Solr 4.2.1 on Tomcat (Java 7)
Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux
One Solr and one ZK on single host: lmsiprse01, lmsiprse02,
 lmsiprse03
  
Before exception I start restarting Solr:
lmsiprse01:
[2013-08-01 05:23:43]: /etc/init.d/tomcat6-1 stop
[2013-08-01 05:25:09]: /etc/init.d/tomcat6-1 start
lmsiprse02 (leader):
2013-08-01 05:27:21]: /etc/init.d/tomcat6-1 stop
2013-08-01 05:29:31]: /etc/init.d/tomcat6-1 start
lmsiprse03:
[2013-08-01 05:25:48]: /etc/init.d/tomcat6-1 stop
[2013-08-01 05:26:42]: /etc/init.d/tomcat6-1 start
  
and error shows up at 2013 5:27:26 on lmsiprse01
  
Is anybody knows what happened?
  
fragment of log looks:
sie 01, 2013 5:27:26 AM org.apache.solr.core.SolrCore execute
INFO: [products] webapp=/solr path=/select
   
  
 
 params={facet=truestart=0q=facet.limit=-1facet.field=attribute_u-typfacet.field=attribute_u-gama-kolorystycznafacet.field=brand_namewt=javabinfq=node_id:1056version=2rows=0}
hits=1241 status=0 QTime=33
sie 01, 2013 5:27:26 AM org.apache.solr.common.SolrException log
SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError:
Requested array size exceeds VM limit
at
   
  
 
 org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
at
   
  
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
at
   
  
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
   
  
 
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
   
  
 
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
   
  
 
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
   
  
 
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
   
  
 
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
   
  
 
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
   
  
 
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
   
  
 
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
   
  
 
 org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
   
  
 
 org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at
   
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds
 VM
   limit
at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:64)
at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:37)
at
   
  
 
 org.apache.solr.handler.component.ShardFieldSortedHitQueue.init(ShardDoc.java:113)
at
   
  
 
 org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:766)
at
   
  
 
 org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:625)
at

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-08-02 Thread Erick Erickson 
Well, this is probably not a rogue query. You can test this, of course
by replaying all your queries on a test system. My guess is that it's
just too much stuff on too small a box.

Or you could have poorly configured Solr parameters. I've seen, for
instance, the filterCache sized at 1M. Which runs fine for a long time
until indexing stops and the filterCache fills up. Which from
your picture seems like a possibility, you have an increasing
memory usage for days.

I'd suggest something like jhat/jmap to find out where the memory
is being used up, a review of your configs, things like that.

Best
Erick


On Fri, Aug 2, 2013 at 7:44 AM, Grzegorz Sobczyk gsobc...@gmail.com wrote:

 Solr parameters listed in dashboard:
 -DzkHost=localhost:2181,172.27.5.121:2181,172.27.5.122:2181
 -XX:+UseConcMarkSweepGC
 -Xmx3072m
 -Djava.awt.headless=true

 Mem usage in last days: http://i42.tinypic.com/29z5rew.png

 It's production system and there's too many requests to detect which query
 is buggy
 some lines before every error (result of: zgrep -B 200 SEVERE:
 null:java.lang.RuntimeException: java.lang.OutOfMemoryError
 daemon.err.2.gz  out.log )
 https://gist.github.com/gsobczyk/65e2ab9bb13a99aec89e

 I've tried invoking recent request before exception but without success of
 reproduce error



 On 1 August 2013 20:39, Erick Erickson erickerick...@gmail.com wrote:

  What are the memory parameters you start Solr with? The Solr admin page
  will tell you how much memory the JVM has.
 
  Also, cut/paste the queries you're running when you see this.
 
  Best
  Erick
 
 
  On Thu, Aug 1, 2013 at 9:50 AM, Grzegorz Sobczyk gsobc...@gmail.com
  wrote:
 
   after node starts in log I have only few requests:
   https://gist.github.com/gsobczyk/6131503#file-solr-oom-log
   this error occurred multiple times
  
  
  
  
   On 1 August 2013 15:33, Rafał Kuć r@solr.pl wrote:
  
Hello!
   
The exception you've shown tells you that Solr tried to allocate an
array that exceeded heap size. Do you use some custom sorts? Did you
send large bulks during the time that the exception occurred?
   
--
Regards,
 Rafał Kuć
 Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch
   
 Today I found in solr logs exception: java.lang.OutOfMemoryError:
Requested
 array size exceeds VM limit.
 At that time memory usage was ~200MB / Xmx3g
   
 Env looks like this:
 3x standalone zK (Java 7)
 3x Solr 4.2.1 on Tomcat (Java 7)
 Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux
 One Solr and one ZK on single host: lmsiprse01, lmsiprse02,
  lmsiprse03
   
 Before exception I start restarting Solr:
 lmsiprse01:
 [2013-08-01 05:23:43]: /etc/init.d/tomcat6-1 stop
 [2013-08-01 05:25:09]: /etc/init.d/tomcat6-1 start
 lmsiprse02 (leader):
 2013-08-01 05:27:21]: /etc/init.d/tomcat6-1 stop
 2013-08-01 05:29:31]: /etc/init.d/tomcat6-1 start
 lmsiprse03:
 [2013-08-01 05:25:48]: /etc/init.d/tomcat6-1 stop
 [2013-08-01 05:26:42]: /etc/init.d/tomcat6-1 start
   
 and error shows up at 2013 5:27:26 on lmsiprse01
   
 Is anybody knows what happened?
   
 fragment of log looks:
 sie 01, 2013 5:27:26 AM org.apache.solr.core.SolrCore execute
 INFO: [products] webapp=/solr path=/select

   
  
 
 params={facet=truestart=0q=facet.limit=-1facet.field=attribute_u-typfacet.field=attribute_u-gama-kolorystycznafacet.field=brand_namewt=javabinfq=node_id:1056version=2rows=0}
 hits=1241 status=0 QTime=33
 sie 01, 2013 5:27:26 AM org.apache.solr.common.SolrException log
 SEVERE: null:java.lang.RuntimeException:
 java.lang.OutOfMemoryError:
 Requested array size exceeds VM limit
 at

   
  
 
 org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
 at

   
  
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
 at

   
  
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
 at

   
  
 
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at

   
  
 
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at

   
  
 
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at

   
  
 
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at

   
  
 
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at

   
  
 
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at

   
  
 
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at

   
  
 
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
 at

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-08-01 Thread Rafał Kuć 
Hello!

The exception you've shown tells you that Solr tried to allocate an
array that exceeded heap size. Do you use some custom sorts? Did you
send large bulks during the time that the exception occurred?

-- 
Regards,
 Rafał Kuć
 Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch

 Today I found in solr logs exception: java.lang.OutOfMemoryError: Requested
 array size exceeds VM limit.
 At that time memory usage was ~200MB / Xmx3g

 Env looks like this:
 3x standalone zK (Java 7)
 3x Solr 4.2.1 on Tomcat (Java 7)
 Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux
 One Solr and one ZK on single host: lmsiprse01, lmsiprse02, lmsiprse03

 Before exception I start restarting Solr:
 lmsiprse01:
 [2013-08-01 05:23:43]: /etc/init.d/tomcat6-1 stop
 [2013-08-01 05:25:09]: /etc/init.d/tomcat6-1 start
 lmsiprse02 (leader):
 2013-08-01 05:27:21]: /etc/init.d/tomcat6-1 stop
 2013-08-01 05:29:31]: /etc/init.d/tomcat6-1 start
 lmsiprse03:
 [2013-08-01 05:25:48]: /etc/init.d/tomcat6-1 stop
 [2013-08-01 05:26:42]: /etc/init.d/tomcat6-1 start

 and error shows up at 2013 5:27:26 on lmsiprse01

 Is anybody knows what happened?

 fragment of log looks:
 sie 01, 2013 5:27:26 AM org.apache.solr.core.SolrCore execute
 INFO: [products] webapp=/solr path=/select
 params={facet=truestart=0q=facet.limit=-1facet.field=attribute_u-typfacet.field=attribute_u-gama-kolorystycznafacet.field=brand_namewt=javabinfq=node_id:1056version=2rows=0}
 hits=1241 status=0 QTime=33
 sie 01, 2013 5:27:26 AM org.apache.solr.common.SolrException log
 SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError:
 Requested array size exceeds VM limit
 at
 org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
 at
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
 at
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
 at
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
 at
 org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
 at
 org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
 at
 org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
 at java.lang.Thread.run(Thread.java:724)
 Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
 at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:64)
 at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:37)
 at
 org.apache.solr.handler.component.ShardFieldSortedHitQueue.init(ShardDoc.java:113)
 at
 org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:766)
 at
 org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:625)
 at
 org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:604)
 at
 org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
 at
 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
 at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817)
 at
 org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
 at
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
 ... 13 more

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-08-01 Thread Grzegorz Sobczyk 
after node starts in log I have only few requests:
https://gist.github.com/gsobczyk/6131503#file-solr-oom-log
this error occurred multiple times




On 1 August 2013 15:33, Rafał Kuć r@solr.pl wrote:

 Hello!

 The exception you've shown tells you that Solr tried to allocate an
 array that exceeded heap size. Do you use some custom sorts? Did you
 send large bulks during the time that the exception occurred?

 --
 Regards,
  Rafał Kuć
  Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch

  Today I found in solr logs exception: java.lang.OutOfMemoryError:
 Requested
  array size exceeds VM limit.
  At that time memory usage was ~200MB / Xmx3g

  Env looks like this:
  3x standalone zK (Java 7)
  3x Solr 4.2.1 on Tomcat (Java 7)
  Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux
  One Solr and one ZK on single host: lmsiprse01, lmsiprse02, lmsiprse03

  Before exception I start restarting Solr:
  lmsiprse01:
  [2013-08-01 05:23:43]: /etc/init.d/tomcat6-1 stop
  [2013-08-01 05:25:09]: /etc/init.d/tomcat6-1 start
  lmsiprse02 (leader):
  2013-08-01 05:27:21]: /etc/init.d/tomcat6-1 stop
  2013-08-01 05:29:31]: /etc/init.d/tomcat6-1 start
  lmsiprse03:
  [2013-08-01 05:25:48]: /etc/init.d/tomcat6-1 stop
  [2013-08-01 05:26:42]: /etc/init.d/tomcat6-1 start

  and error shows up at 2013 5:27:26 on lmsiprse01

  Is anybody knows what happened?

  fragment of log looks:
  sie 01, 2013 5:27:26 AM org.apache.solr.core.SolrCore execute
  INFO: [products] webapp=/solr path=/select
 
 params={facet=truestart=0q=facet.limit=-1facet.field=attribute_u-typfacet.field=attribute_u-gama-kolorystycznafacet.field=brand_namewt=javabinfq=node_id:1056version=2rows=0}
  hits=1241 status=0 QTime=33
  sie 01, 2013 5:27:26 AM org.apache.solr.common.SolrException log
  SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError:
  Requested array size exceeds VM limit
  at
 
 org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
  at
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
  at
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
  at
 
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at
 
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at
 
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
  at
 
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
  at
 
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  at
 
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at
 
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  at
 
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
  at
 
 org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
  at
 
 org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
  at
  org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
  at java.lang.Thread.run(Thread.java:724)
  Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM
 limit
  at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:64)
  at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:37)
  at
 
 org.apache.solr.handler.component.ShardFieldSortedHitQueue.init(ShardDoc.java:113)
  at
 
 org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:766)
  at
 
 org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:625)
  at
 
 org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:604)
  at
 
 org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
  at
 
 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817)
  at
 
 org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
  at
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
  ... 13 more




-- 
Grzegorz Sobczyk

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-08-01 Thread Erick Erickson 
What are the memory parameters you start Solr with? The Solr admin page
will tell you how much memory the JVM has.

Also, cut/paste the queries you're running when you see this.

Best
Erick


On Thu, Aug 1, 2013 at 9:50 AM, Grzegorz Sobczyk gsobc...@gmail.com wrote:

 after node starts in log I have only few requests:
 https://gist.github.com/gsobczyk/6131503#file-solr-oom-log
 this error occurred multiple times




 On 1 August 2013 15:33, Rafał Kuć r@solr.pl wrote:

  Hello!
 
  The exception you've shown tells you that Solr tried to allocate an
  array that exceeded heap size. Do you use some custom sorts? Did you
  send large bulks during the time that the exception occurred?
 
  --
  Regards,
   Rafał Kuć
   Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch
 
   Today I found in solr logs exception: java.lang.OutOfMemoryError:
  Requested
   array size exceeds VM limit.
   At that time memory usage was ~200MB / Xmx3g
 
   Env looks like this:
   3x standalone zK (Java 7)
   3x Solr 4.2.1 on Tomcat (Java 7)
   Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux
   One Solr and one ZK on single host: lmsiprse01, lmsiprse02, lmsiprse03
 
   Before exception I start restarting Solr:
   lmsiprse01:
   [2013-08-01 05:23:43]: /etc/init.d/tomcat6-1 stop
   [2013-08-01 05:25:09]: /etc/init.d/tomcat6-1 start
   lmsiprse02 (leader):
   2013-08-01 05:27:21]: /etc/init.d/tomcat6-1 stop
   2013-08-01 05:29:31]: /etc/init.d/tomcat6-1 start
   lmsiprse03:
   [2013-08-01 05:25:48]: /etc/init.d/tomcat6-1 stop
   [2013-08-01 05:26:42]: /etc/init.d/tomcat6-1 start
 
   and error shows up at 2013 5:27:26 on lmsiprse01
 
   Is anybody knows what happened?
 
   fragment of log looks:
   sie 01, 2013 5:27:26 AM org.apache.solr.core.SolrCore execute
   INFO: [products] webapp=/solr path=/select
  
 
 params={facet=truestart=0q=facet.limit=-1facet.field=attribute_u-typfacet.field=attribute_u-gama-kolorystycznafacet.field=brand_namewt=javabinfq=node_id:1056version=2rows=0}
   hits=1241 status=0 QTime=33
   sie 01, 2013 5:27:26 AM org.apache.solr.common.SolrException log
   SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError:
   Requested array size exceeds VM limit
   at
  
 
 org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653)
   at
  
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366)
   at
  
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
   at
  
 
 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
   at
  
 
 org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
   at
  
 
 org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
   at
  
 
 org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
   at
  
 
 org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
   at
  
 
 org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
   at
  
 
 org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
   at
  
 
 org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
   at
  
 
 org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
   at
  
 
 org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
   at
   org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
   at java.lang.Thread.run(Thread.java:724)
   Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM
  limit
   at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:64)
   at org.apache.lucene.util.PriorityQueue.init(PriorityQueue.java:37)
   at
  
 
 org.apache.solr.handler.component.ShardFieldSortedHitQueue.init(ShardDoc.java:113)
   at
  
 
 org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:766)
   at
  
 
 org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:625)
   at
  
 
 org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:604)
   at
  
 
 org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:311)
   at
  
 
 org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
   at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817)
   at
  
 
 org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
   at
  
 
 org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
   ... 13 more
 
 


 --
 Grzegorz Sobczyk

Re: java.lang.OutOfMemoryError: Requested array size exceeds VM limit

2013-07-13 Thread Stephane Gamard 
Hi Ali,You are getting an error because of the number of rows you are trying to fetch. Solr will keep all results in its queue before submitting the results. Solution is to page thru your results (but be careful about deep paging)._Stephane On July 13, 2013 at 2:31:57 AM, Ali, Saqib (docbook@gmail.com) wrote: I am getting a java.lang.OutOfMemoryError: Requested array size exceeds VM
limit on certain queries.

Please advise:

19:25:02,632 INFO  [org.apache.solr.core.SolrCore]
(http-oktst1509.company.tld/12.5.105.96:8180-9) [collection1] webapp=/solr
path=/select
params={sort=sent_date+ascdistrib=falsewt=javabinversion=2rows=2147483647df=textfl=idshard.url=
"">12.5.105.96:8180/solr/collection1/NOW=1373675102627start=0q=thread_id:1439513570014188310isShard=truefq=domain:company.tld+AND+owner:11782344fsv=true}
hits=1 status=0 QTime=1
19:25:02,637 ERROR [org.apache.solr.servlet.SolrDispatchFilter]
(http-oktst1509.company.tld/12.5.105.96:8180-2)
null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Requested
array size exceeds VM limit
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:670)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:380)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at
org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:372)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:679)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:931)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit