RE: Index/Update Problems with Solrj/Tomcat and Larger Files
I am running against 1.2. Where would I get the 1.3-dev version? I will try different versions of Tomcat and/or Jetty. Thanks for all your suggestions, I'll let you know. -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 19, 2007 8:30 PM To: solr-user@lucene.apache.org Subject: Re: Index/Update Problems with Solrj/Tomcat and Larger Files > > However, if I go to the tomcat server and restart it after I have issued > the process command, the program returns and the documents are all > posted correctly! > > Very strange behavioram I somehow not closing the connection > properly? > What version is the solr you are connecting to? 1.2 or 1.3-dev? (I have not tested against 1.2) Does this only happen with tomcat? If you run with jetty do you get the same behavior? (again, just stabs in the dark) If you can make a small repeatable problem, post it in JIRA and I'll look into it. ryan
Re: Index/Update Problems with Solrj/Tomcat and Larger Files
However, if I go to the tomcat server and restart it after I have issued the process command, the program returns and the documents are all posted correctly! Very strange behavioram I somehow not closing the connection properly? What version is the solr you are connecting to? 1.2 or 1.3-dev? (I have not tested against 1.2) Does this only happen with tomcat? If you run with jetty do you get the same behavior? (again, just stabs in the dark) If you can make a small repeatable problem, post it in JIRA and I'll look into it. ryan
RE: Index/Update Problems with Solrj/Tomcat and Larger Files
Ok, I'll try to play with those. Any suggestion on the size? Something else that is very interesting is that I just tried to do an aggregate add of a bunch of docs, including the one that always returned the error. I called a function to create a SolrInputDocument and return it. I then did the following: Collection docs = new ArrayList(); SolrServer server = new CommonsHttpSolrServer(solrPostUrl); UpdateRequest ur = new UpdateRequest(); ur.setAction( UpdateRequest.ACTION.COMMIT, false, false ); //Auto Commits on Update... ur.add(docs); UpdateResponse rsp = ur.process(server); In doing this, the program simply hangs after the last command. If I let it sit there for an amount of time, it eventually returns with the error: class org.apache.solr.client.solrj.SolrServerException (java.net.SocketException: Connection reset by peer: socket write error) However, if I go to the tomcat server and restart it after I have issued the process command, the program returns and the documents are all posted correctly! Very strange behavioram I somehow not closing the connection properly? -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 19, 2007 11:49 AM To: solr-user@lucene.apache.org Subject: Re: Index/Update Problems with Solrj/Tomcat and Larger Files I'm stabbing in the dark here, but try fiddling with some of the other connection settings: getConnectionManager().getParams().setSendBufferSize( big ); getConnectionManager().getParams().setReceiveBufferSize( big ); http://jakarta.apache.org/httpcomponents/httpclient-3.x/apidocs/org/apac he/commons/httpclient/params/HttpConnectionManagerParams.html Daley, Kristopher M. wrote: > I tried 1 and 6, same result. > > -Original Message- > From: Ryan McKinley [mailto:[EMAIL PROTECTED] > Sent: Wednesday, September 19, 2007 11:18 AM > To: solr-user@lucene.apache.org > Subject: Re: Index/Update Problems with Solrj/Tomcat and Larger Files > > Daley, Kristopher M. wrote: >> I have tried changing those settings, for example, as: >> >> SolrServer server = new CommonsHttpSolrServer(solrPostUrl); >> ((CommonsHttpSolrServer)server).setConnectionTimeout(60); >> ((CommonsHttpSolrServer)server).setDefaultMaxConnectionsPerHost(100); >> ((CommonsHttpSolrServer)server).setMaxTotalConnections(100); >> >> However, still no luck. >> > > Have you tried anything larger then 60? 60ms is not long... > > try 1 (10s) and see if it works. > >
Re: Index/Update Problems with Solrj/Tomcat and Larger Files
I'm stabbing in the dark here, but try fiddling with some of the other connection settings: getConnectionManager().getParams().setSendBufferSize( big ); getConnectionManager().getParams().setReceiveBufferSize( big ); http://jakarta.apache.org/httpcomponents/httpclient-3.x/apidocs/org/apache/commons/httpclient/params/HttpConnectionManagerParams.html Daley, Kristopher M. wrote: I tried 1 and 6, same result. -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 19, 2007 11:18 AM To: solr-user@lucene.apache.org Subject: Re: Index/Update Problems with Solrj/Tomcat and Larger Files Daley, Kristopher M. wrote: I have tried changing those settings, for example, as: SolrServer server = new CommonsHttpSolrServer(solrPostUrl); ((CommonsHttpSolrServer)server).setConnectionTimeout(60); ((CommonsHttpSolrServer)server).setDefaultMaxConnectionsPerHost(100); ((CommonsHttpSolrServer)server).setMaxTotalConnections(100); However, still no luck. Have you tried anything larger then 60? 60ms is not long... try 1 (10s) and see if it works.
RE: Index/Update Problems with Solrj/Tomcat and Larger Files
I tried 1 and 6, same result. -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 19, 2007 11:18 AM To: solr-user@lucene.apache.org Subject: Re: Index/Update Problems with Solrj/Tomcat and Larger Files Daley, Kristopher M. wrote: > I have tried changing those settings, for example, as: > > SolrServer server = new CommonsHttpSolrServer(solrPostUrl); > ((CommonsHttpSolrServer)server).setConnectionTimeout(60); > ((CommonsHttpSolrServer)server).setDefaultMaxConnectionsPerHost(100); > ((CommonsHttpSolrServer)server).setMaxTotalConnections(100); > > However, still no luck. > Have you tried anything larger then 60? 60ms is not long... try 1 (10s) and see if it works.
Re: Index/Update Problems with Solrj/Tomcat and Larger Files
Daley, Kristopher M. wrote: I have tried changing those settings, for example, as: SolrServer server = new CommonsHttpSolrServer(solrPostUrl); ((CommonsHttpSolrServer)server).setConnectionTimeout(60); ((CommonsHttpSolrServer)server).setDefaultMaxConnectionsPerHost(100); ((CommonsHttpSolrServer)server).setMaxTotalConnections(100); However, still no luck. Have you tried anything larger then 60? 60ms is not long... try 1 (10s) and see if it works.
RE: Index/Update Problems with Solrj/Tomcat and Larger Files
I have tried changing those settings, for example, as: SolrServer server = new CommonsHttpSolrServer(solrPostUrl); ((CommonsHttpSolrServer)server).setConnectionTimeout(60); ((CommonsHttpSolrServer)server).setDefaultMaxConnectionsPerHost(100); ((CommonsHttpSolrServer)server).setMaxTotalConnections(100); However, still no luck. I took the SimplePostTool.java file from the wiki, changed the URL, compiled it and ran it with the output of the command: UpdateRequest ur = new UpdateRequest(); ur.add(addDoc); String xml = ur.getXML(); This works. It seems that it must be a communication setting, but I'm stumped. -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 19, 2007 10:31 AM To: solr-user@lucene.apache.org Subject: Re: Index/Update Problems with Solrj/Tomcat and Larger Files > > I have had this and other files index correctly using a different > combination version of Tomcat/Solr without any problem (using similar > code, I re-wrote it because I thought it would be better to use Solrj). > I get the same error whether I use a simple StringBuilder to created the > add manually or if I use Solrj. I have manually encoded each field > before passing it in to the add function as well, so I don't believe it > is a content problem. I have tried to change every setting in Tomcat > and Solr that I can think of, but I'm newer to both of them. > So it works if you build an XML file with the same content and send it to the server using the example post.sh/post.jar tool? Have you tried messing with the connection settings? SolrServer server = new CommonsHttpSolrServer( url ); ((CommonsHttpSolrServer)server).setConnectionTimeout(5); ((CommonsHttpSolrServer)server).setDefaultMaxConnectionsPerHost(100); ((CommonsHttpSolrServer)server).setMaxTotalConnections(100); a timeout of 5ms is probably too short... ryan
Re: Index/Update Problems with Solrj/Tomcat and Larger Files
I have had this and other files index correctly using a different combination version of Tomcat/Solr without any problem (using similar code, I re-wrote it because I thought it would be better to use Solrj). I get the same error whether I use a simple StringBuilder to created the add manually or if I use Solrj. I have manually encoded each field before passing it in to the add function as well, so I don't believe it is a content problem. I have tried to change every setting in Tomcat and Solr that I can think of, but I'm newer to both of them. So it works if you build an XML file with the same content and send it to the server using the example post.sh/post.jar tool? Have you tried messing with the connection settings? SolrServer server = new CommonsHttpSolrServer( url ); ((CommonsHttpSolrServer)server).setConnectionTimeout(5); ((CommonsHttpSolrServer)server).setDefaultMaxConnectionsPerHost(100); ((CommonsHttpSolrServer)server).setMaxTotalConnections(100); a timeout of 5ms is probably too short... ryan
