[
https://issues.apache.org/jira/browse/SOLR-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15099168#comment-15099168
]
Greg Wilkins commented on SOLR-8539:
------------------------------------
Sean,
from the jetty point of view - the server is entirely designed with the
intention of being embedded. Jetty is primarily a software component before it
is a software container... it just so happens that we distribute with our
components assembled as a standard servlet container. You'll have noted that
our XML configuration is just calling the java API, so it is easy to convert to
embedded.
Embedding jetty is a very easy and sensible path to go down.
With regards to the servlet container, the vast majority of that complexity is
optional and if you are configuring your own server you don't need to
instantiate it. You can easily get rid of servlet classloaders, security,
sessions, even servlets themselves if you want to write to the jetty APIs.
So if you program to Jetty at the Server+Handler level, you get pretty much the
same level of abstraction as offered by Netty - but with the servlet request
API for familiarity. We provide the same kind of async capability - but
perhaps the servlet API is a little more clunky (but also has its good points
as well).
In short - if servlets & webapps are seen as a burden, then don't use them. We
are happy to give advice as to how to not use the full servlet container.
We are also keen to participate in discussions like this on OOME handling, to
improve our embeddability and adaptability for more use cases.
> Solr queries swallows up OutOfMemoryErrors
> ------------------------------------------
>
> Key: SOLR-8539
> URL: https://issues.apache.org/jira/browse/SOLR-8539
> Project: Solr
> Issue Type: Bug
> Reporter: Varun Thacker
> Fix For: 5.5, Trunk
>
> Attachments: SOLR-8539.patch
>
>
> I was testing a crazy surround query and was hitting OOMs easily with the
> query. However I saw that the OOM killer wasn't triggered. Here is the stack
> trace of the error on solr 5.4:
> {code}
> WARN - 2016-01-12 18:37:03.920; [ x:techproducts]
> org.eclipse.jetty.util.thread.QueuedThreadPool$3;
> java.lang.OutOfMemoryError: Java heap space
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1855)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2068)
> at
> org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
> at java.lang.Thread.run(Thread.java:745)
> ERROR - 2016-01-12 18:37:03.922; [ x:techproducts]
> org.apache.solr.common.SolrException; null:java.lang.RuntimeException:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:611)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:472)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:222)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.newTermState(Lucene50PostingsReader.java:149)
> at
> org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.<init>(SegmentTermsEnumFrame.java:100)
> at
> org.apache.lucene.codecs.blocktree.SegmentTermsEnum.getFrame(SegmentTermsEnum.java:215)
> at
> org.apache.lucene.codecs.blocktree.SegmentTermsEnum.pushFrame(SegmentTermsEnum.java:241)
> at
> org.apache.lucene.codecs.blocktree.SegmentTermsEnum.seekCeil(SegmentTermsEnum.java:728)
> at
> org.apache.lucene.index.FilterLeafReader$FilterTermsEnum.seekCeil(FilterLeafReader.java:185)
> at org.apache.lucene.index.TermsEnum.seekExact(TermsEnum.java:74)
> at org.apache.lucene.index.TermContext.build(TermContext.java:94)
> at
> org.apache.lucene.search.spans.SpanTermQuery.createWeight(SpanTermQuery.java:72)
> at
> org.apache.lucene.search.spans.SpanOrQuery.createWeight(SpanOrQuery.java:132)
> at
> org.apache.lucene.search.spans.SpanNearQuery.createWeight(SpanNearQuery.java:192)
> at
> org.apache.lucene.search.spans.SpanNearQuery.createWeight(SpanNearQuery.java:42)
> at
> org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:904)
> at
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:887)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:535)
> at
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:202)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1672)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1491)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:557)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:525)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:273)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:457)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:222)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> {code}
> So HttpSolrCall#sendError needs to throw an OOM exception instead of
> swallowing it and sending back and error in the response.
> Once I made HttpSolrCall#sendError throw an exception on OOM I still hit the
> following stack strace
> {code}
> WARN - 2016-01-13 07:43:39.449; [ x:techproducts]
> org.eclipse.jetty.servlet.ServletHandler; Error for
> /solr/techproducts%2Fselect
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockPostingsEnum.<init>(Lucene50PostingsReader.java:477)
> at
> org.apache.lucene.codecs.lucene50.Lucene50PostingsReader.postings(Lucene50PostingsReader.java:220)
> at
> org.apache.lucene.codecs.blocktree.SegmentTermsEnum.postings(SegmentTermsEnum.java:1002)
> at
> org.apache.lucene.search.spans.SpanTermQuery$SpanTermWeight.getSpans(SpanTermQuery.java:119)
> at
> org.apache.lucene.search.spans.SpanOrQuery$SpanOrWeight.getSpans(SpanOrQuery.java:160)
> at
> org.apache.lucene.search.spans.SpanNearQuery$SpanNearWeight.getSpans(SpanNearQuery.java:213)
> at
> org.apache.lucene.search.spans.SpanWeight.scorer(SpanWeight.java:133)
> at
> org.apache.lucene.search.spans.SpanWeight.scorer(SpanWeight.java:38)
> at org.apache.lucene.search.Weight.bulkScorer(Weight.java:135)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:667)
> at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:474)
> at
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:240)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1766)
> at
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1583)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:613)
> at
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:524)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:273)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2062)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:653)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:457)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> {code}
> So is Jetty not propagating the OOM Error back to the JVM?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
