Folks,
I ran 20,000,000 records into Solr via the extractingUpdateRequestHandler under
jetty. The previous problems with resources have apparently been resolved by
using Http1.1 with keep-alive, rather than creating and destroying 20,000,000
sockets. ;-) However, after the client terminates, I still find the Solr
process chewing away CPU - indeed, there were 5 threads doing this.
A thread dump yields the following partial trace for all 5 threads:
"btpool0-13" prio=10 tid=0x0000000041391000 nid=0xe7c runnable
[0x00007f4a8c789000]
java.lang.Thread.State: RUNNABLE
at
org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:925)
at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:897)
at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:977)
at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.close(MultipartStream.java:924)
at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.close(MultipartStream.java:904)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:119)
at org.apache.commons.fileupload.util.Streams.copy(Streams.java:64)
at
org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:362)
at
org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126)
at
org.apache.solr.servlet.MultipartRequestParser.parseParamsAndFillStreams(SolrRequestParsers.java:343)
at
org.apache.solr.servlet.StandardRequestParser.parseParamsAndFillStreams(SolrRequestParsers.java:396)
at
org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:114)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
...
I could be wrong, but it looks to me like either jetty or fileupload may have a
problem here. I have not looked at the jetty source code, but infinitely
spinning processes even after the socket has been abandoned do not seem
reasonable to me. Thoughts?
Karl
