I figured instead of trying to index content, I'd simply issue a query via SolrJ. This seems related to my problem below. I create a CommonsHttpSolrServer instance in the manner already described and in a new method:
@Override public List<String> getNodeIdsForProductId(final String productId, final String partnerId) { final List<String> nodes = new ArrayList<String>(); final CommonsHttpSolrServer solrServer = (CommonsHttpSolrServer)getSolrServer(partnerId); final SolrQuery query = new SolrQuery(); query.setQuery("productId:" + productId); query.addField("nodeId"); try { final QueryResponse response = solrServer.query(query); final SolrDocumentList docs = response.getResults(); log.info(String.format("getNodeIdsForProductId - got %d nodes for productId: %s", docs.getNumFound(), productId)); for (SolrDocument doc : docs) { log.info(doc); } } catch (SolrServerException ex) { final String msg = String.format("Unable to query Solr server %s, for query: %s", solrServer.getBaseURL(), query); log.error(msg); throw new ServiceException(msg, ex); } return nodes; } When issuing the query I get: 2011-02-14 13:13:28 INFO solr.SolrProductIndexService - getSolrServer - Solr url: http://localhost:8080/solr/partner-tmo 2011-02-14 13:13:28 INFO solr.SolrProductIndexService - getSolrServer - construct server for url: http://localhost:8080/solr/partner-tmo 2011-02-14 13:13:28 ERROR solr.SolrProductIndexService - Unable to query Solr server http://localhost:8080/solr/partner-tmo, for query: q=productId%3Aproduct4&fl=nodeId ... Caused by: org.apache.solr.client.solrj.SolrServerException: org.apache.commons.httpclient.NoHttpResponseException: The server localhost failed to respond at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:484) ... Caused by: org.apache.commons.httpclient.NoHttpResponseException: The server localhost failed to respond at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1976) If I run this through the proxy again, I can see the request being made as: GET /solr/partner-tmo/select?q=productId%3Aproduct4&fl=nodeId&wt=xml&version=2.2 HTTP/1.1 User-Agent: Solr[org.apache.solr.client.solrj.impl.CommonsHttpSolrServer] 1.0 Host: localhost:8080 And I get no response from Solr. If instead I use this URL in Firefox: http://localhost:8080/solr/partner-tmo/select?q=productId%3Aproduct4&fl=nodeId&wt=xml&version=2.2 I get search results. What is it about SolrJ that is just not working out? What basic thing am I missing? Using Firefox here, or curl below, I can talk to Solr (running in Tomcat 6) just fine. But when going via SolrJ, I cannot update or query. All of this stuff is running on a single system. I guess I'll try a simpler app/unit test to see what happens... This is really a big problem for me. Any suggests are greatly appreciated. Thanks, Jeff On Feb 13, 2011, at 9:15 PM, Jeff Schmidt wrote: > Hello again: > > Back to the javabin iissue: > > On Feb 12, 2011, at 6:07 PM, Lance Norskog wrote: > >> --- But I'm unable to get SolrJ to work due to the 'javabin' version >> mismatch. I'm using the 1.4.1 version of SolrJ, but I always get an >> HTTP response code of 200, but the return entity is simply a null >> byte, which does not match the version number of 1 defined in Solr >> common. --- >> >> I've never seen this problem. At this point you are better off >> starting with 3.x instead of chasing this problem down. > > I'm now using the latest branch_3x built Solr and SolrJ. Other places I've > seen the message: > > Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 0) or > the data in not in 'javabin' format at > org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:99) at > org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41) > > > One was told to make sure the version of Solr and SolrJ are compatible, and > that the schema is valid. Unlike 1.4, I see 3.1 actually outputs the expected > and received version numbers, which is helpful. You can see the invalid > version of 0 is indicated which is the zero byte I receive in response. > > I have Solr running within Tomcat by following the wiki. I have the > conf/Catalina/localhost/solr.xml file set as: > > <?xml version="1.0" encoding="utf-8"?> > <Context > docBase="/usr/local/ingenuity/isec/solr/apache-solr-3.1-SNAPSHOT.war" > debug="0" crossContext="true"> > <Environment name="solr/home" type="java.lang.String" > > value="/Users/jas/535Consulting/Clients/Ingenuity/ProfServices/svn/trunk/ing/isec/src/main/solr/multicore" > override="true"/> > </Context> > > With that, I'm able to use my browser to index some content (DIH, curl etc.) > and issue queries, so it seems Solr is running okay in tomcat > (apache-tomcat-6.0.30). To index some Products, I have this simple method: > > @Override > public void addProducts(final Collection<Product> products, final > String indexName) { > > log.info(String.format("addProducts - indexing %d products to > Solr core: %s", > products.size(), indexName)); > > Assert.notNull(indexName); > > final Collection<SolrInputDocument> docs = new > ArrayList<SolrInputDocument>(); > for (Product product : products) { > > final SolrInputDocument doc = > createDocumentForProduct(product); > docs.add(doc); > log.info("addProduct: document to index: " + doc); > } > > final SolrServer solrServer = getSolrServer(indexName); > try { > solrServer.add(docs); > solrServer.commit(commitWaitFlush, commitWaitSearcher); > } catch (Exception ex) { > final String msg = String.format("Unable to add and > commit %d documents to core: %s", > products.size(), indexName); > log.error(msg); > throw new ServiceException(msg, ex); > } > } > > And I have: > > protected SolrServer getSolrServer(final String indexName) { > > final String url = solrServerBaseUrl + indexName; > log.info("getSolrServer - construct server for url: " + url); > try { > final CommonsHttpSolrServer solrServer = new > CommonsHttpSolrServer(solrServerBaseUrl + indexName); > //solrServer.setParser(new BinaryResponseParser()); > //solrServer.setParser(new XMLResponseParser()); > solrServer.setRequestWriter(new BinaryRequestWriter()); > return solrServer; > } catch (Exception ex) { > final String msg = String.format("Unable to create Solr > server for url: %s", url); > log.error(msg); > throw new ServiceException(msg, ex); > } > } > > Note that this is code for prototyping. :) > > As you can see, in getSolrServer() I'm trying various settings. > http://wiki.apache.org/solr/Solrj is tagged Solr 1.4, but I'm assuming it's > at least very similar in 3.1. For the core in question, solrconfig.xml does > have: > > <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" /> > <requestHandler name="/update/javabin" > class="solr.BinaryUpdateRequestHandler" /> > > I can see in the Solr log: > > Feb 13, 2011 3:35:14 PM org.apache.solr.core.RequestHandlers > initHandlersFromConfig > INFO: created /update/javabin: solr.BinaryUpdateRequestHandler > > Running this through the burp proxy to try to see what's going on, I can see > my application making the following request to Solr via SolrJ: > > ------------------------------------------ > POST /solr/partner-tmo/update/javabin?wt=javabin&version=2 HTTP/1.1 > User-Agent: Solr[org.apache.solr.client.solrj.impl.CommonsHttpSolrServer] 1.0 > Host: localhost:8090 > Content-Type: application/octet-stream > Content-Length: 543 > > Äà¶msÀà'delByIdà&delByQà$docs…Áà%boostÃà$name"idà#val-ING:afa|08520åÃæ&nodeIdç'ING:afaåÃæ)productIdç%08520åÃæ0nodeSourceIdTypeç"EGåÃæ,nodeSourceIdç#672åÃæ+productTypeç≠(chemical%shRNAåÃæ-description_tç? > This is the description for product > 08520åÃæ'brand_sç%FLUKAåÃæ%sku_sç(A980-852å…ÁåÃæ"idç-ING:afa|08530åÃæ&nodeIdç'ING:afaåÃæ)productIdç%08530åÃæ0nodeSourceIdTypeç"EGåÃæ,nodeSourceIdç#672åÃæ+productTypeç™(chemicalåÃæ-description_tç? > This is the description for product > 08530åÃæ'brand_sç%FLUKAåÃæ%sku_sç(A980-853å > ------------------------------------------ > > That looks pretty binary. In response I see: > > ------------------------------------------ > HTTP/1.0 200 OK > Content-type: application/octet-stream > Content-length: 1 > > > ------------------------------------------ > > Looking at the hex view, I can see the one byte of data is 0x00. > > My other approach was to go the XML route. So, to do this, I comment out the > setting of the request writer and go with the default, which the wiki says is > XML. Running this through the proxy I see: > > ------------------------------------------ > POST /solr/partner-tmo/update?wt=javabin&version=2 HTTP/1.1 > User-Agent: Solr[org.apache.solr.client.solrj.impl.CommonsHttpSolrServer] 1.0 > Host: localhost:8090 > Content-Type: text/xml; charset=utf-8 > Content-Length: 856 > > <add><doc boost="1.0"><field name="id">ING:afa|08520</field><field > name="nodeId">ING:afa</field><field name="productId">08520</field><field > name="nodeSourceIdType">EG</field><field > name="nodeSourceId">672</field><field > name="productType">chemical</field><field > name="productType">shRNA</field><field name="description_t">This is the > description for product 08520</field><field > name="brand_s">FLUKA</field><field name="sku_s">A980-852</field></doc><doc > boost="1.0"><field name="id">ING:afa|08530</field><field > name="nodeId">ING:afa</field><field name="productId">08530</field><field > name="nodeSourceIdType">EG</field><field > name="nodeSourceId">672</field><field > name="productType">chemical</field><field name="description_t">This is the > description for product 08530</field><field > name="brand_s">FLUKA</field><field name="sku_s">A980-853</field></doc></add> > ------------------------------------------ > > I can see it has specified the proper update handler in the URI and XML is > being uploaded. In response though, I get the exact same one as when going > binary, including the application/octet-stream content type. I end up with > the same javabin version mismatch stacktrace as well, even though I'm trying > to talk XML. But, the presence of wt=javabin&version=2 is not very > encouraging when going the default XML route. > > Just for grins, I added: > > solrServer.setParser(new XMLResponseParser()); > > And now the exception is: > > Caused by: > com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character (NULL, > unicode 0) encountered: not valid in any content| at [row,col > {unknown-source}]: [1,1] > at > com.ctc.wstx.sr.StreamScanner.constructNullCharException(StreamScanner.java:640) > > So, apparently javabin is out of the way, and now the zero byte returned is > mucking up the XML parser. According to proxy, the request is similar to the > previous one, but: > > POST /solr/partner-tmo/update?wt=xml&version=2.2 HTTP/1.1 > > So, great we are going the XML route, but the response is the same HTTP 200 > and a zero byte... > > So, I'm not sure what's going on. I can say though that I am not seeing any > log activity solr.2011-*.log once Solr has started and I attempt to issue > these requests. Maybe it's tomcat? But, if I go directly to Solr outside of > SolrJ and add some content to that same core, I do see log activity, and get > a valid response: > > [imac:solr/input-data/tmo-products] jas% curl --header "Content-type: > text/xml; charset=utf-8" --request POST -w "\nhttp code: %{http_code}\n" -d > @nodes-products.xml "http://localhost:8080/solr/partner-tmo/update" > <?xml version="1.0" encoding="UTF-8"?> > <response> > <lst name="responseHeader"><int name="status">0</int><int > name="QTime">11</int></lst> > </response> > > http code: 200 > [imac:solr/input-data/tmo-products] jas% > > Much better than a single null byte. > > Any idea of what is going on? Apologies for the long email, but I'm trying > to provide all the details. This problem must reside in something I'm doing. > I'm sure others are using SolrJ successfully. > > Thanks! > > Jeff > >> >> On Sat, Feb 12, 2011 at 1:37 PM, Jeff Schmidt <j...@535consulting.com> wrote: >>> Hello: >>> >>> I'm working on incorporating Solr into a SaaS based life sciences semantic >>> search project. This will be released in about six months. I'm trying to >>> determine which version of Solr makes the most sense. When going to the >>> Solr download page, there are 1.3.0, 1.4.0, and 1.4.1. I've been using >>> 1.4.1 while going through some examples in my Packt book ("Solr 1.4 >>> Enterprise Search Server"). >>> >>> But, I also see that Solr 3.1 and 4.0 are in the works. According to: >>> >>> >>> https://issues.apache.org/jira/browse/#selectedTab=com.atlassian.jira.plugin.system.project%3Aroadmap-panel >>> >>> there is a high degree of progress on both of those releases; including a >>> slew of bug fixes, new features, performance enhancements etc. Should I be >>> making use of one of the newer versions? The hierarchical faceting seems >>> like it could be quite useful. Are there any guesses on when either 3.1 or >>> 4.0 will be officially released? >>> >>> So far, 1.4.1 has been good. But I'm unable to get SolrJ to work due to the >>> 'javabin' version mismatch. I'm using the 1.4.1 version of SolrJ, but I >>> always get an HTTP response code of 200, but the return entity is simply a >>> null byte, which does not match the version number of 1 defined in Solr >>> common. Anyway, I can follow up on that issue if 1.4.1 is still the most >>> appropriate version to use these days. Otherwise, I'll try again with >>> whatever version you suggest. >>> >>> Thanks a lot! >>> >>> Jeff >>> -- >>> Jeff Schmidt >>> 535 Consulting >>> j...@535consulting.com >>> (650) 423-1068 >>> >>> >>> >>> >>> >>> >> >> >> >> -- >> Lance Norskog >> goks...@gmail.com > > -- > Jeff Schmidt > 535 Consulting > j...@535consulting.com > (650) 423-1068 > http://www.535consulting.com > > > > > > > -- Jeff Schmidt 535 Consulting j...@535consulting.com (650) 423-1068 http://www.535consulting.com