Commenting out the following line in SolrCore fixes my problem... but of
course I don't get the result status info... but this isn't a problem for me
really.
-Sangraal
writer.write("<result status=\"" + status + "\"></result>");
On 7/27/06, sangraal aiken <[EMAIL PROTECTED]> wrote:
I'm running on Tomcat... and I've verified that the complete post is
making it through the SolrUpdate servlet and into the SolrCore object...
thanks for the info though.
--
So the code is hanging on this call in SolrCore.java
writer.write("<result status=\"" + status + "\"></result>");
The thread dump:
"http-8080-Processor24" Id=32 in RUNNABLE (running in native) total cpu
time= 40698.0440ms user time=38646.1680ms
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java
:92)
at java.net.SocketOutputStream.write (SocketOutputStream.java:136)
at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(
InternalOutputBuffer.java:746)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java
:433)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:348)
at
org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite
(InternalOutputBuffer.java:769)
at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite (
ChunkedOutputFilter.java:125)
at org.apache.coyote.http11.InternalOutputBuffer.doWrite(
InternalOutputBuffer.java:579)
at org.apache.coyote.Response.doWrite(Response.java:559)
at org.apache.catalina.connector.OutputBuffer.realWriteBytes (
OutputBuffer.java:361)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:324)
at org.apache.tomcat.util.buf.IntermediateOutputStream.write(
C2BConverter.java:235)
at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes (StreamEncoder.java
:336)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(
StreamEncoder.java:404)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java
:408)
at sun.nio.cs.StreamEncoder.flush (StreamEncoder.java:152)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
at org.apache.tomcat.util.buf.WriteConvertor.flush(C2BConverter.java
:184)
at org.apache.tomcat.util.buf.C2BConverter.flushBuffer (
C2BConverter.java:127)
at org.apache.catalina.connector.OutputBuffer.realWriteChars(
OutputBuffer.java:536)
at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java
:439)
at org.apache.tomcat.util.buf.CharChunk.append (CharChunk.java:370)
at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java
:491)
at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java
:161)
at org.apache.catalina.connector.CoyoteWriter.write (
CoyoteWriter.java:170)
at org.apache.solr.core.SolrCore.update(SolrCore.java:695)
at org.apache.solr.servlet.SolrUpdateServlet.doPost(
SolrUpdateServlet.java:52)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java :252)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke (
StandardContextValve.java:178)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:126)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:105)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:107)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:148)
at org.apache.coyote.http11.Http11Processor.process (
Http11Processor.java:869)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
PoolTcpEndpoint.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
LeaderFollowerWorkerThread.java:80)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java :684)
at java.lang.Thread.run(Thread.java:613)
On 7/27/06, Otis Gospodnetic <[EMAIL PROTECTED] > wrote:
>
> I haven't been following the thread, but....
> Not sure if you are using Tomcat or Jetty, but Jetty has a POST size
> limit (set somewhere in its configs) that may be the source of the problem.
>
> Otis
> P.S.
> Just occurred to me.
> Tomcat. Jetty. Tom & Jerry. Jetty guys should have called their thing
> Jerry or Jerrymouse.
>
> ----- Original Message ----
> From: Mike Klaas < [EMAIL PROTECTED]>
> To: [email protected]
> Sent: Thursday, July 27, 2006 6:33:16 PM
> Subject: Re: Doc add limit
>
> Hi Sangraal:
>
> Sorry--I tried not to imply that this might affect your issue. You
> may have to crank up the solr logging to determine where it is
> freezing (and what might be happening).
>
> It is certainly worth investigating why this occurs, but I wonder
> about the advantages of using such huge batches. Assuming a few
> hundred bytes per document, 6100 docs produces a POST over 1MB in
> size.
>
> -Mike
>
> On 7/27/06, sangraal aiken <[EMAIL PROTECTED]> wrote:
> > Mike,
> > I've been posting with the content type set like this:
> > conn.setRequestProperty("Content-Type",
> "application/octet-stream");
> >
> > I tried your suggestion though, and unfortunately there was no change.
> > conn.setRequestProperty ("Content-Type", "text/xml;
> charset=utf-8");
> >
> > -Sangraal
> >
> >
> > On 7/27/06, Mike Klaas <[EMAIL PROTECTED]> wrote:
> > >
> > > On 7/27/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:
> > >
> > > > class SolrConnection:
> > > > def __init__(self, host='localhost:8983', solrBase='/solr'):
> > > > self.host = host
> > > > self.solrBase = solrBase
> > > > #a connection to the server is not opened at this point.
> > > > self.conn = httplib.HTTPConnection (self.host)
> > > > #self.conn.set_debuglevel(1000000)
> > > > self.postheaders = {"Connection":"close"}
> > > >
> > > > def doUpdateXML(self, request):
> > > > try:
> > > > self.conn.request('POST', self.solrBase+'/update', request,
> > > > self.postheaders)
> > >
> > > Disgressive note: I'm not sure if it is necessary with tomcat, but
> in
> > > my experience driving solr with python using Jetty, it was necessary
> > > to specify the content-type when posting utf-8 data:
> > >
> > > self.postheaders.update({'Content-Type': 'text/xml; charset=utf-8'})
>
> > >
> > > -Mike
> > >
> >
> >
>
>
>
>
