Please help - Solr Cell using 'stream.url'
I'm batching documents into solr using solr cell with the 'stream.url' parameter. Everything is working fine until I get to about 5k documents in and then it starts issuing 'read timeout 500' errors on every document. The sysadmin says there's plenty of CPU, memory, and no paging so it doesn't look like the OS is the problem. I can curl the documents that Solr is trying to index and failing just fine so it seems to be a Solr issue. There's only about 35K documents total so Solr should even blink. Can anyone help me diagnose this problem? I'd be happy to provide any more detail that is needed. Thanks - Tod
Re: stream.url
Hi Hoss, Thanks for reply and it got working The reason was as you said i was not double escaping i used %2520 for whitespace and it is working now Thanks, satya
Re: stream.url
:I used escape charaters and made it... It is not problem for : a single file of 'solr apache' but it shows the same problem for the files : like Wireless lan.ppt, Tom info.pdf. Since you haven't told us what the original URL is that you are trying to pass as a value for the stream.url value, it's impossible for us to guess wehter your URL escaping is working properly. bear in mind that you need to escape url metacharacters *twice* for this type of thing -- once to encode the URL in a way that the final server will recognize it, and once again to pass it as a value in a URL to Solr. since you explicitly mention having problems with white space, but i don't see any %25 or %2B sequences in your URL i'm going to guess that the porblem is you are not double escaping the white space properly -- the first time you escape it it should either be + or %20 which means the second time it should either be %2B or %2520 -Hoss -- http://lucenerevolution.org/ ... October 7-8, Boston http://bit.ly/stump-hoss ... Stump The Chump!
Re: stream.url
Hi all, I am unable to index the files of remote system that contains escaped characters in their file names i think there is a problem in solr for indexing the files of escaped characters in remote system... Has anybody tried to index the files in remote system that contain the escaped characters But solr is working good for files that has no escaped characters in their name. I sent the request through the curl by encoding the filename in url format but the problem is same... Regards, satya
stream.url
Hi all, I am using stream.url to index the files in the remote system. when i use the url as 1) curl http://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws?file=yaws_presentation.pdfliteral.id=schb4 it works and i get the response as the file got indexed. but when i use 2) curl http://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws?file=solr; apache.pdf literal.id=schb5 i get the error in the solr... i replaced the escaped characters with %20 for space and %26 for , but the error is same saying Unexpected end of file from server java.net.SocketException.. when i used without solr as http://remotehost:port/file_download.yaws?file=solr apache.pdf then i get the file downloaded to my system. I here enclose the entire error= HTTP Status 500 - Unexpected end of file from server java.net.SocketException: Unexpected end of file from server at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1368) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1362) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1016) at org.apache.solr.common.util.ContentStreamBase$URLStream.getStream(ContentStreamBase.java:88) at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:169) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:57) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:133) at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:242) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1355) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:340) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) Caused by: java.net.SocketException: Unexpected end of file from server at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:769) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:766) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1072) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2173) at java.net.URLConnection.getContentType(URLConnection.java:485) at org.apache.solr.common.util.ContentStreamBase$URLStream.init(ContentStreamBase.java:81) at org.apache.solr.servlet.SolrRequestParsers.buildRequestFrom(SolrRequestParsers.java:138) at org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:117) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226) ... 12 more can anybody provide information regarding this?? Regards, Satya
Re: stream.url
Hi, this has nothing to do with Solr... you can't use a filename containing as a URL parameter... if you really need to submit such a weird named file, you have to escape the , see http://www.december.com/html/spec/esccodes.html for the code... Cheers, Stefan Am 02.09.2010 09:35, schrieb satya swaroop: Hi all, I am using stream.url to index the files in the remote system. when i use the url as 1) curl http://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws?file=yaws_presentation.pdfliteral.id=schb4 it works and i get the response as the file got indexed. but when i use 2) curl http://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws?file=solr; apache.pdf literal.id=schb5 i get the error in the solr... i replaced the escaped characters with %20 for space and %26 for, but the error is same saying Unexpected end of file from server java.net.SocketException.. when i used without solr as http://remotehost:port/file_download.yaws?file=solr apache.pdf then i get the file downloaded to my system. I here enclose the entire error= HTTP Status 500 - Unexpected end of file from server java.net.SocketException: Unexpected end of file from server at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1368) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1362) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1016) at org.apache.solr.common.util.ContentStreamBase$URLStream.getStream(ContentStreamBase.java:88) at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:169) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:57) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:133) at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:242) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1355) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:340) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) Caused by: java.net.SocketException: Unexpected end of file from server at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:769) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:766) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1072) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2173) at java.net.URLConnection.getContentType(URLConnection.java:485) at org.apache.solr.common.util.ContentStreamBase$URLStream.init(ContentStreamBase.java:81) at org.apache.solr.servlet.SolrRequestParsers.buildRequestFrom(SolrRequestParsers.java:138) at org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:117) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226) ... 12 more can anybody provide information regarding this?? Regards, Satya -- *** Stefan Moises Senior Softwareentwickler shoptimax GmbH Guntherstraße 45 a 90461 Nürnberg Amtsgericht Nürnberg HRB 21703 GF Friedrich Schreieck Tel.: 0911/25566-25 Fax: 0911/25566-29 moi...@shoptimax.de http://www.shoptimax.de ***
Re: stream.url
Hi stefan, I used escape charaters and made it... It is not problem for a single file of 'solr apache' but it shows the same problem for the files like Wireless lan.ppt, Tom info.pdf. the curl i sent is:: curl http://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws%3Ffile=solrhttp://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws?file=solr %20%26%20apache.pdfliteral.id=schb5 Regards, satya
Re: stream.url
Hi, well, you'll have to write a routine which escapes all filenames before transmitting... wether in a shell, in Java, PHP, Javascript or whereever you are submitting your CURL calls. Here is a javascript example that helps with escaping: http://www.xs4all.nl/~jlpoutre/BoT/Javascript/Utils/endecode.html Cheers, Stefan Am 02.09.2010 10:07, schrieb satya swaroop: Hi stefan, I used escape charaters and made it... It is not problem for a single file of 'solrapache' but it shows the same problem for the files like Wireless lan.ppt, Tom info.pdf. the curl i sent is:: curl http://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws%3Ffile=solrhttp://localhost:8080/solr/update/extract?stream.url=http://remotehost:port/file_download.yaws?file=solr %20%26%20apache.pdfliteral.id=schb5 Regards, satya -- *** Stefan Moises Senior Softwareentwickler shoptimax GmbH Guntherstraße 45 a 90461 Nürnberg Amtsgericht Nürnberg HRB 21703 GF Friedrich Schreieck Tel.: 0911/25566-25 Fax: 0911/25566-29 moi...@shoptimax.de http://www.shoptimax.de ***
Re: stream.url
Hi, I made the curl from the shell(command prompt or terminal) with the escaping characters but the error is same when i saw in the remote system the request is not getting there Is there anything to be changed in config file inorder to enable the escaping characters for stream.url Did anybody try indexing files in remote system through stream.url, where the files name contain escape characters like ,space regards, satya
Re: stream.url problem
Hi all, I got the solution for my problem. I changed my port number and i kept the old one in the stream.url... so problem was that... thanks all Now i got another problem, it is when i send any requests to remote system for the files that have names with escape characters like ,space . For example= TomJerry.pdf i get a problem as Unexpected end of file from server... the request i sent is:: curl http://localhost:8080/solr/update/extract?stream.url=http://remotehost:8011/file_download.yaws?file=Wireless%20Lan.pdfliteral.id=su8 here file_download.yaws is a module that fetches the file and gives to solr. solr is able to index the files that doesnt contain the escape characters in the remote system.. example:: apache.txt, solr_apache.pdf the error i got is::: HTTP Status 500 - Unexpected end of file from server java.net.SocketException: Unexpected end of file from server at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1368) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1362) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1016) at org.apache.solr.common.util.ContentStreamBase$URLStream.getStream(ContentStreamBase.java:88) at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:161) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:57) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:133) at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:242) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1355) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:340) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) Caused by: java.net.SocketException: Unexpected end of file from server at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:769) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632) at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:766) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1072) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2173) at java.net.URLConnection.getContentType(URLConnection.java:485) at org.apache.solr.common.util.ContentStreamBase$URLStream.init(ContentStreamBase.java:81) at org.apache.solr.servlet.SolrRequestParsers.buildRequestFrom(SolrRequestParsers.java:138) at org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:117) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226) ... Regards, satya
stream.url problem
hi all, i am indexing the documents to solr that are in my system. now i need to index the files that are in remote system, i enabled the remote streaming to true in solrconfig.xml and when i use the stream.url it shows the error as connection refused and the detail of the error is::: when i sent the request in my browser as:: http://localhost:8080/solr/update/extract?stream.url=http://remotehost/home/san/Desktop/programming_erlang_armstrong.pdfliteral.id=schb2 i get the error as HTTP Status 500 - Connection refused java.net.ConnectException: Connection refused at sun.reflect.GeneratedConstructorAccessor11.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1368) at java.security.AccessController.doPrivileged(Native Method) at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1362) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1016) at org.apache.solr.common.util.ContentStreamBase$URLStream.getStream(ContentStreamBase.java:88) at org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(ExtractingDocumentLoader.java:161) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:237) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619) Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) at java.net.Socket.connect(Socket.java:525) at java.net.Socket.connect(Socket.java:475) at sun.net.NetworkClient.doConnect(NetworkClient.java:163) at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) at sun.net.www.http.HttpClient.openServer(HttpClient.java:529) at sun.net.www.http.HttpClient.init(HttpClient.java:233) at sun.net.www.http.HttpClient.New(HttpClient.java:306) at sun.net.www.http.HttpClient.New(HttpClient.java:323) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:860) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:801) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:726) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1049) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2173) at java.net.URLConnection.getContentType(URLConnection.java:485) at org.apache.solr.common.util.ContentStreamBase$URLStream.init(ContentStreamBase.java:81) at org.apache.solr.servlet.SolrRequestParsers.buildRequestFrom(SolrRequestParsers.java:136) at org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:116) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225) ... if any body know please help me with this regards, satya
Re: stream.url problem
hi all, i am indexing the documents to solr that are in my system. now i need to index the files that are in remote system, i enabled the remote streaming to true in solrconfig.xml and when i use the stream.url it shows the error as connection refused and the detail of the error is::: when i sent the request in my browser as:: http://localhost:8080/solr/update/extract?stream.url=http://remotehost/home/san/Desktop/programming_erlang_armstrong.pdfliteral.id=schb2 You probably use the wrong port. Try 8983 instead. /Tim
Re: stream.url problem
If the connector port number in your localhost is same as in other system then this error is probable..You can change port number in server.xml of your system or other system and make them different...If it is different only then one other probablity is remote access enabled or not... Rajani Maski 2010/8/17 Tim Terlegård tim.terleg...@gmail.com hi all, i am indexing the documents to solr that are in my system. now i need to index the files that are in remote system, i enabled the remote streaming to true in solrconfig.xml and when i use the stream.url it shows the error as connection refused and the detail of the error is::: when i sent the request in my browser as:: http://localhost:8080/solr/update/extract?stream.url=http://remotehost/home/san/Desktop/programming_erlang_armstrong.pdfliteral.id=schb2 You probably use the wrong port. Try 8983 instead. /Tim
Re: stream.url problem
Connection refused (in any context) almost always means that nothing is listening on the TCP port that you are trying to connect to. So either the process you are connecting to isn't running, or you are trying to connect to the wrong port. On Tue, Aug 17, 2010 at 6:18 AM, satya swaroop sswaro...@gmail.com wrote: hi all, i am indexing the documents to solr that are in my system. now i need to index the files that are in remote system, i enabled the remote streaming to true in solrconfig.xml and when i use the stream.url it shows the error as connection refused and the detail of the error is::: when i sent the request in my browser as:: http://localhost:8080/solr/update/extract?stream.url=http://remotehost/home/san/Desktop/programming_erlang_armstrong.pdfliteral.id=schb2 i get the error as HTTP Status 500 - Connection refused java.net.ConnectException: Connection refused at sun.reflect.GeneratedConstructorAccessor11.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at [snip] if any body know please help me with this regards, satya
stream.url Contention
I recently switched from posting a file (PDFs in this case) to the Extract handler, to using the Stream.URL parameter. I've noticed a huge amount of contention around opening URL connections: http-8080-Processor36 [BLOCKED] CPU time: 0:47 sun.net.www.protocol.file.Handler.openConnection(URL) java.net.URL.openConnection() sun.net.www.protocol.jar.JarURLConnection.init(URL, Handler) sun.net.www.protocol.jar.Handler.openConnection(URL) java.net.URL.openConnection() java.net.URL.openStream() java.lang.ClassLoader.getResourceAsStream(String) org.pdfbox.util.ResourceLoader.loadResource(String) org.pdfbox.util.ResourceLoader.loadProperties(String) org.pdfbox.util.PDFTextStripper.init() org.apache.tika.parser.pdf.PDF2XHTML.init(ContentHandler, Metadata) org.apache.tika.parser.pdf.PDF2XHTML.process(PDDocument, ContentHandler, Metadata) org.apache.tika.parser.pdf.PDFParser.parse(InputStream, ContentHandler, Metadata) org.apache.tika.parser.CompositeParser.parse(InputStream, ContentHandler, Metadata) org.apache.tika.parser.AutoDetectParser.parse(InputStream, ContentHandler, Metadata) org.apache.solr.handler.extraction.ExtractingDocumentLoader.load(SolrQueryRequest, SolrQueryResponse, ContentStream) org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(SolrQueryRequest, SolrQueryResponse) org.apache.solr.handler.RequestHandlerBase.handleRequest(SolrQueryRequest, SolrQueryResponse) org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(SolrQueryRequest, SolrQueryResponse) org.apache.solr.core.SolrCore.execute(SolrRequestHandler, SolrQueryRequest, SolrQueryResponse) org.apache.solr.servlet.SolrDispatchFilter.execute(HttpServletRequest, SolrRequestHandler, SolrQueryRequest, SolrQueryResponse) org.apache.solr.servlet.SolrDispatchFilter.doFilter(ServletRequest, ServletResponse, FilterChain) org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) org.apache.catalina.core.ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) org.apache.catalina.core.StandardWrapperValve.invoke(Request, Response) org.apache.catalina.core.StandardContextValve.invoke(Request, Response) org.apache.catalina.core.StandardHostValve.invoke(Request, Response) org.apache.catalina.valves.ErrorReportValve.invoke(Request, Response) org.apache.catalina.core.StandardEngineValve.invoke(Request, Response) org.apache.catalina.connector.CoyoteAdapter.service(Request, Response) org.apache.coyote.http11.Http11Processor.process(InputStream, OutputStream) org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(TcpConnection, Object[]) org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(Socket, TcpConnection, Object[]) org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(Object[]) org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run() java.lang.Thread.run() This seems to be a significant bottleneck, even when running only a handful of thread. Has anyone else run into this? Any ideas on how to reduce the blocking? Thanks, Gio.
