Re: Query regarding Solr-2242 patch for getting distinct facet counts.
In solr 1.4.1, for getting distinct facet terms count across shards, The piece of code added for getting count of distinct facet terms across distributed process is as followed: Class: facetcomponent.java Function: -- finishStage(ResponseBuilder rb) for (DistribFieldFacet dff : fi.facets.values()) { //just after this line of code else { // TODO: log error or throw exception? counts = dff.getLexSorted(); int namedistint = 0; namedistint=rb.req.getParams().getFieldInt(dff.getKey().toString(),FacetParams.FACET_NAMEDISTINCT,0); if (namedistint == 0) facet_fields.add(dff.getKey(), fieldCounts); if (namedistint == 1) facet_fields.add(numfacetTerms, counts.length); if (namedistint == 2) { NamedList resCount = new NamedList(); resCount.add(numfacetTerms, counts.length); resCount.add(counts, fieldCounts); facet_fields.add(dff.getKey(), resCount); } Is this flow correct ? I have worked with few test cases and it has worked fine. but i want to know if there are any bugs that can creep in here? (My concern is this piece of code should not effect the rest of logic) *Code flow with comments for reference:* Function : -- finishStage(ResponseBuilder rb) //in this for loop , for (DistribFieldFacet dff : fi.facets.values()) { //just after this line of code else { // TODO: log error or throw exception? counts = dff.getLexSorted(); int namedistint = 0; //default //get the value of facet.numterms from the input query namedistint=rb.req.getParams().getFieldInt(dff.getKey().toString(),FacetParams.FACET_NAMEDISTINCT,0); // based on the value for facet.numterms==0 or 1 or 2 , if conditions //Get only facet field counts if (namedistint == 0) { facet_fields.add(dff.getKey(), fieldCounts); } //get only distinct facet term count if (namedistint == 1) { facet_fields.add(numfacetTerms, counts.length); } //get facet field count and distinct term count. if (namedistint == 2) { NamedList resCount = new NamedList(); resCount.add(numfacetTerms, counts.length); resCount.add(counts, fieldCounts); facet_fields.add(dff.getKey(), resCount); } Regards, Rajani On Fri, May 27, 2011 at 1:14 PM, rajini maski rajinima...@gmail.com wrote: No such issues . Successfully integrated with 1.4.1 and it works across single index. for f.2.facet.numFacetTerms=1 parameter it will give the distinct count result for f.2.facet.numFacetTerms=2 parameter it will give counts as well as results for facets. But this is working only across single index not distributed process. The conditions you have added in simple facet.java- if namedistinct count ==int ( 0, 1 and 2 condtions).. Should it be added in distributed process function to enable it work across shards? Rajani On Fri, May 27, 2011 at 12:33 PM, Bill Bell billnb...@gmail.com wrote: I am pretty sure it does not yet support distributed shards.. But the patch was written for 4.0... So there might be issues with running it on 1.4.1. On 5/26/11 11:08 PM, rajini maski rajinima...@gmail.com wrote: The patch solr 2242 for getting count of distinct facet terms doesn't work for distributedProcess (https://issues.apache.org/jira/browse/SOLR-2242) The error log says HTTP ERROR 500 Problem accessing /solr/select. Reason: For input string: numFacetTerms java.lang.NumberFormatException: For input string: numFacetTerms at java.lang.NumberFormatException.forInputString(NumberFormatException.java: 48) at java.lang.Long.parseLong(Long.java:403) at java.lang.Long.parseLong(Long.java:461) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331) at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344) at org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(Fac etComponent.java:619) at org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponen t.java:265) at org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComp onent.java:235) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHa ndler.java:290) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas e.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java :338) at
Re: Query regarding Solr-2242 patch for getting distinct facet counts.
I am pretty sure it does not yet support distributed shards.. But the patch was written for 4.0... So there might be issues with running it on 1.4.1. On 5/26/11 11:08 PM, rajini maski rajinima...@gmail.com wrote: The patch solr 2242 for getting count of distinct facet terms doesn't work for distributedProcess (https://issues.apache.org/jira/browse/SOLR-2242) The error log says HTTP ERROR 500 Problem accessing /solr/select. Reason: For input string: numFacetTerms java.lang.NumberFormatException: For input string: numFacetTerms at java.lang.NumberFormatException.forInputString(NumberFormatException.java: 48) at java.lang.Long.parseLong(Long.java:403) at java.lang.Long.parseLong(Long.java:461) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331) at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344) at org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(Fac etComponent.java:619) at org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponen t.java:265) at org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComp onent.java:235) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHa ndler.java:290) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas e.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java :338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav a:241) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl er.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216 ) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnect ion.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:41 0) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:5 82) The query I passed : http://localhost:8983/solr/select?q=*:*facet=truefacet.field=2facet.fie ld=648facet.mincount=1facet.limit=-1f.2.facet.numFacetTerms=1rows=0sh ards=localhost:8983/solr,localhost:8985/solrtwo Anyone can suggest me the changes i need to make to enable the same funcionality for shards? When i do it across single core.. I get the correct results. I have applied the solr 2242 patch in solr1.4.1 Awaiting for reply Regards, Rajani
Re: Query regarding Solr-2242 patch for getting distinct facet counts.
No such issues . Successfully integrated with 1.4.1 and it works across single index. for f.2.facet.numFacetTerms=1 parameter it will give the distinct count result for f.2.facet.numFacetTerms=2 parameter it will give counts as well as results for facets. But this is working only across single index not distributed process. The conditions you have added in simple facet.java- if namedistinct count ==int ( 0, 1 and 2 condtions).. Should it be added in distributed process function to enable it work across shards? Rajani On Fri, May 27, 2011 at 12:33 PM, Bill Bell billnb...@gmail.com wrote: I am pretty sure it does not yet support distributed shards.. But the patch was written for 4.0... So there might be issues with running it on 1.4.1. On 5/26/11 11:08 PM, rajini maski rajinima...@gmail.com wrote: The patch solr 2242 for getting count of distinct facet terms doesn't work for distributedProcess (https://issues.apache.org/jira/browse/SOLR-2242) The error log says HTTP ERROR 500 Problem accessing /solr/select. Reason: For input string: numFacetTerms java.lang.NumberFormatException: For input string: numFacetTerms at java.lang.NumberFormatException.forInputString(NumberFormatException.java: 48) at java.lang.Long.parseLong(Long.java:403) at java.lang.Long.parseLong(Long.java:461) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331) at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344) at org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(Fac etComponent.java:619) at org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponen t.java:265) at org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComp onent.java:235) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHa ndler.java:290) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas e.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java :338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav a:241) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl er.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216 ) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnect ion.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:41 0) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:5 82) The query I passed : http://localhost:8983/solr/select?q=*:*facet=truefacet.field=2facet.fie ld=648facet.mincount=1facet.limit=-1f.2.facet.numFacetTerms=1rows=0sh ards=localhost:8983/solr,localhost:8985/solrtwo Anyone can suggest me the changes i need to make to enable the same funcionality for shards? When i do it across single core.. I get the correct results. I have applied the solr 2242 patch in solr1.4.1 Awaiting for reply Regards, Rajani
Query regarding Solr-2242 patch for getting distinct facet counts.
The patch solr 2242 for getting count of distinct facet terms doesn't work for distributedProcess (https://issues.apache.org/jira/browse/SOLR-2242) The error log says HTTP ERROR 500 Problem accessing /solr/select. Reason: For input string: numFacetTerms java.lang.NumberFormatException: For input string: numFacetTerms at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Long.parseLong(Long.java:403) at java.lang.Long.parseLong(Long.java:461) at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:331) at org.apache.solr.schema.TrieField.toInternal(TrieField.java:344) at org.apache.solr.handler.component.FacetComponent$DistribFieldFacet.add(FacetComponent.java:619) at org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponent.java:265) at org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:235) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:290) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) The query I passed : http://localhost:8983/solr/select?q=*:*facet=truefacet.field=2facet.field=648facet.mincount=1facet.limit=-1f.2.facet.numFacetTerms=1rows=0shards=localhost:8983/solr,localhost:8985/solrtwo Anyone can suggest me the changes i need to make to enable the same funcionality for shards? When i do it across single core.. I get the correct results. I have applied the solr 2242 patch in solr1.4.1 Awaiting for reply Regards, Rajani