NPE with faceting query on MoreLikeThis handler
Hi everyone, Recently I upgraded to solr 5.1.0. When trying to generate facets using the more like this handler, I now get a a NullPointerException. I never got this exception while using Solr 4.10.0 Details are below: Stack Trace: at org.apache.solr.request.SimpleFacets.getHeatmapCounts(SimpleFacets.java:1555) at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:284) at org.apache.solr.handler.MoreLikeThisHandler.handleRequestBody(MoreLikeThisHandler.java:233) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Query: qt=/mlt q=id:545dbb57b54c2403f286050e546dcdcab54cf2d074e5a2f7 mlt.mindf=5 mlt.mintf=1 mlt.minwl=3 mlt.boost=true fq=storeid:546dcdcab54cf2d074e5a2f7 mlt.fl=overview_mlt,abstract_mlt,description_mlt,company_profile_mlt,bio_mlt mlt.interestingTerms=details fl=conceptid,score sort=score desc start=0 rows=2 facet=true facet.field=tags facet.field=locations facet.mincount=1 facet.method=enum facet.limit=-1 facet.sort=count Schema.xml(relevant parts): field name=tags type=string indexed=true stored=true multiValued=true / field name=locations type=string indexed=true stored=true multiValued=true / dynamicField name=*_mlt stored=true indexed=true type=text_general termVectors=true multiValued=true / solrconfig.xml(relevant parts): requestHandler name=/mlt class=solr.MoreLikeThisHandler /requestHandler
Re: Faceting Query in Solr
It's working fine. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceting-Query-in-Solr-tp4104881p4105612.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceting Query in Solr
Hi i am using following two queries but i am not getting the out put as expected http://localhost:8080/solr/Testing/select?q=*%3A*wt=xmlindent=truefacet=truefacet.field=categoryId for the above query it is showing all the matched results as category1(Id-900)--10002 category2(Id-800)--5202 category3(Id-700)--1500 http://localhost:8080/solr/Testing/select?q=*:*wt=xmlindent=truefacet=truefacet.field=categoryIdfacet.query=categoryId:900 This has to show only results of categoryId 900 results i.e numfound must be 10002 http://localhost:8080/solr/Testing/select?q=*:*wt=xmlindent=truefacet=truefacet.field=categoryIdfacet.query=categoryId:900facet.query=categoryId:800 It has to show results of both categories i.e 10002+5202=total 15204 records. But it showing all the category records and if i use the fq=categoryId:800fq=categoryId:900 then it not showing any results. -- View this message in context: http://lucene.472066.n3.nabble.com/Faceting-Query-in-Solr-tp4104881p4105083.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceting Query in Solr
On Thu, Dec 5, 2013 at 1:34 PM, kumar pavan2...@gmail.com wrote: fq=categoryId:800fq=categoryId:900 what about fq=categoryId:(800 900) ? -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: Faceting Query in Solr
I used following url... http://localhost:8080/solr/Testing/select?q=*:*wt=xmlindent=truefacet=truefacet.field=categoryIdfq=categoryId:(900 800) but it is not showing any results -- View this message in context: http://lucene.472066.n3.nabble.com/Faceting-Query-in-Solr-tp4104881p4105106.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceting Query in Solr
It is working fine for fq=categoryId:(800 900) -- View this message in context: http://lucene.472066.n3.nabble.com/Faceting-Query-in-Solr-tp4104881p4105110.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceting Query in Solr
how about fq={!lucene q.op=OR}categoryId:(800 900)
On Thursday, December 5, 2013 1:26 PM, kumar pavan2...@gmail.com wrote:
I used following url...
http://localhost:8080/solr/Testing/select?q=*:*wt=xmlindent=truefacet=truefacet.field=categoryIdfq=categoryId:(900
800)
but it is not showing any results
--
View this message in context:
http://lucene.472066.n3.nabble.com/Faceting-Query-in-Solr-tp4104881p4105106.html
Sent from the Solr - User mailing list archive at Nabble.com.
Faceting Query in Solr
Hi, I indexed data into solr by using 5 categories. Each category is differentiated by categoryId. Now i have a situation that i need to show the results based on facets. Ex: []-category1 []-category2 []-category3 []-category4 []-category5 If the user checks the category1 it has to show the results based on categoryId-1 If the user checks 2 categories it has to show the results from two categories which the user checked If the user checks 3 categories it has to show the results from three categories and son on.like how many categories user checked i have to show results from checked categories My Schema is in the following way.. field name=id type=string indexed=true stored=true required=true multiValued=false / field name=categoryId type=int indexed=true stored=false required=true / field name=url type=string indexed=true stored=true required=true / field name=content type=string indexed=false stored=true multiValued=true required=true / Anyone help me how can i achieve this. Regards, Kumar -- View this message in context: http://lucene.472066.n3.nabble.com/Faceting-Query-in-Solr-tp4104881.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceting Query in Solr
The standard way of handling this kind of thing is with filter queries. For multi-select, you have to put in some javascript or something to make an OR clause when they check the boxes. So your query looks like fq=categoryID:(1 OR 2 OR 3) rather than fq=categoryID:1fq=categoryID:2fq=categoryID:3 Best, Erick On Wed, Dec 4, 2013 at 4:36 AM, kumar pavan2...@gmail.com wrote: Hi, I indexed data into solr by using 5 categories. Each category is differentiated by categoryId. Now i have a situation that i need to show the results based on facets. Ex: []-category1 []-category2 []-category3 []-category4 []-category5 If the user checks the category1 it has to show the results based on categoryId-1 If the user checks 2 categories it has to show the results from two categories which the user checked If the user checks 3 categories it has to show the results from three categories and son on.like how many categories user checked i have to show results from checked categories My Schema is in the following way.. field name=id type=string indexed=true stored=true required=true multiValued=false / field name=categoryId type=int indexed=true stored=false required=true / field name=url type=string indexed=true stored=true required=true / field name=content type=string indexed=false stored=true multiValued=true required=true / Anyone help me how can i achieve this. Regards, Kumar -- View this message in context: http://lucene.472066.n3.nabble.com/Faceting-Query-in-Solr-tp4104881.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: OutOfMemoryError | While Faceting Query
Hi Uwe, sorting should be well prepared. First rough check is fieldCache. You can see it with SolrAdmin Stats. The insanity_count there should be 0 (zero). Only sort on fields which are prepared for sorting and make sense to be sorted. Do only faceting on fields which make sense. I've seen systems with faceting on id, this is a no-go and doesn't make sense. The system pulls a lot of data from the index into memory which can lead to OOME. How to figure out what is killing your system? There is no general rule but what you can do is: - Start your test-system and make sure noone else is using it. - Start a monitor for your JVM running SOLR (e.g. jvisualvm). - use your search frontend and do searches, sorting, faceting of any combination possible and watch your heap memory if it has big jumps in memory heap. - Analyze your search log files and look for searches which have a very high Qtime. Repeat the searches with high Qtime and see if you get insanity_counts or heap memory jumps in JVM. Regards Bernd Am 06.12.2012 23:27, schrieb uwe72: Hi there, since i use a lot sorting and faceting i am getting very often an OutOfMemoryError. I have arround 6 million documents, index size arround 18GB and using tomcat with 1.8 GB max heap size. What can i do? What heap size is recommended in our case? Can i do other things in order to prevent OutOfMemoryError while using a lot of facets? Urgent, please help. Thanks, Uwe -- View this message in context: http://lucene.472066.n3.nabble.com/OutOfMemoryError-While-Faceting-Query-tp4024947.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: OutOfMemoryError | While Faceting Query
/solr-home/data/index/_3kj.frq)'='WiringDiagramSheetImpl.versionAsDate',long,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_LONG_PARSER=[J#2986832 entry#21 : 'NIOFSIndexInput(path=/home/connect/ConnectPORTAL/preview/solr-home/data/index/_3kj.frq)'='ModuleImpl.versionAsDate',long,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_LONG_PARSER=[J#2255175 entry#22 : 'NIOFSIndexInput(path=/home/connect/ConnectPORTAL/preview/solr-home/data/index/_192.frq)'='WiringDiagramSheetImpl.versionAsDate',long,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_LONG_PARSER=[J#19778817 entry#23 : 'NIOFSIndexInput(path=/home/connect/ConnectPORTAL/preview/solr-home/data/index/_192.frq)'='ModuleImpl.versionAsDate',long,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_LONG_PARSER=[J#32378546 insanity_count : 0 -- View this message in context: http://lucene.472066.n3.nabble.com/OutOfMemoryError-While-Faceting-Query-tp4024947p4025050.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Faceting Query
On Friday 11 February 2011 11:34 PM, Gora Mohanty wrote:
On Thu, Feb 10, 2011 at 12:21 PM, Isha Gargisha.g...@orkash.com wrote:
What is facet.pivot field? PLz explain with example
Does http://wiki.apache.org/solr/SimpleFacetParameters#facet.pivot not help?
Regards,
Gora
No, it is not showing any pivot results in my case
http://localhost:8984/solr/worldNews/select/?q=*%3A*version=2.2start=0rows=0indent=onfacet.pivot=category,country,KeyLocationfacet.pivot=country,categoryfacet=truefacet.field=categorywt=json
Output is:
{
responseHeader:{
status:0,
QTime:1,
params:{
facet:true,
indent:on,
start:0,
q:*:*,
facet.field:category,
wt:json,
facet.pivot:[category,country,KeyLocation,
country,category],
version:2.2,
rows:0}},
response:{numFound:6775,start:0,docs:[]
},
facet_counts:{
facet_queries:{},
facet_fields:{
category:[
Counterfeiting and Piracy ,2367,
Social Unrest ,2143,
Security Measures ,1064,
Fraud and Cheating,356,
Naxelites ,266,
Terrorism ,243,
Sex Crime ,232,
Shiv Sena ,76,
Major Crime ,23,
Drug Running and Organized Crime ,5]},
facet_dates:{}}}
Re: Faceting Query
Likely because it is a Solr 4.0 feature and you are using Solr 1.4.1.
That'd be my guess.
(Solr 4.0 is the latest greatest as yet unreleased version of Solr -
numbering scheme changed to fit with that of Lucene).
Upayavira
On Mon, 14 Feb 2011 16:05 +0530, Isha Garg isha.g...@orkash.com
wrote:
On Friday 11 February 2011 11:34 PM, Gora Mohanty wrote:
On Thu, Feb 10, 2011 at 12:21 PM, Isha Gargisha.g...@orkash.com wrote:
What is facet.pivot field? PLz explain with example
Does http://wiki.apache.org/solr/SimpleFacetParameters#facet.pivot not help?
Regards,
Gora
No, it is not showing any pivot results in my case
http://localhost:8984/solr/worldNews/select/?q=*%3A*version=2.2start=0rows=0indent=onfacet.pivot=category,country,KeyLocationfacet.pivot=country,categoryfacet=truefacet.field=categorywt=json
Output is:
{
responseHeader:{
status:0,
QTime:1,
params:{
facet:true,
indent:on,
start:0,
q:*:*,
facet.field:category,
wt:json,
facet.pivot:[category,country,KeyLocation,
country,category],
version:2.2,
rows:0}},
response:{numFound:6775,start:0,docs:[]
},
facet_counts:{
facet_queries:{},
facet_fields:{
category:[
Counterfeiting and Piracy ,2367,
Social Unrest ,2143,
Security Measures ,1064,
Fraud and Cheating,356,
Naxelites ,266,
Terrorism ,243,
Sex Crime ,232,
Shiv Sena ,76,
Major Crime ,23,
Drug Running and Organized Crime ,5]},
facet_dates:{}}}
---
Enterprise Search Consultant at Sourcesense UK,
Making Sense of Open Source
Re: Faceting Query
I am also working on same feature of solr 4.0
And I have doubt in the results am getting. I will post the cases here. If
anyone know why is it so,Please revert back...
I run a normal facet query with q parameter q=*:* and did facet=on
facet.field=stockfacet.filed=placefacet.field=quantityfacet.mincout=1
Results i got is-
facet_fields
stock
rice10/
bean10/
wheat10/
jowar10/
/stock
place
bangalore10/
Kolar10/
/place
quality
standard10/
high10/
/quality
/facet_fields
Now when I am doing this facet.pivot query with same q paramater (q= *:* )and
same data set ..
query - facet.pivot=stock,place,qualityfacet.mincout=1
Result I get is like this-
lst
rice
bangalore
high
/lst
lst
bean
bangalore
standard
/lst
lst
jowar
/lst
The point is .. Why I am not getting result hirearchy for wheat when it is
coming in the flat faceting above.
Awaiting reply
Regards,
Rajani Maski
On Mon, Feb 14, 2011 at 4:18 PM, rajini maski rajinima...@gmail.com wrote:
This feature works in SOLR 4.0 release.
You can follow this link for knowing how it works...
Click
herehttp://solr.pl/en/2010/10/25/hierarchical-faceting-pivot-facets-in-trunk/
Regards
Rajani Maski
On Mon, Feb 14, 2011 at 4:05 PM, Isha Garg isha.g...@orkash.com wrote:
On Friday 11 February 2011 11:34 PM, Gora Mohanty wrote:
On Thu, Feb 10, 2011 at 12:21 PM, Isha Gargisha.g...@orkash.com
wrote:
What is facet.pivot field? PLz explain with example
Does http://wiki.apache.org/solr/SimpleFacetParameters#facet.pivot not
help?
Regards,
Gora
No, it is not showing any pivot results in my case
http://localhost:8984/solr/worldNews/select/?q=*%3A*version=2.2start=0rows=0indent=onfacet.pivot=category,country,KeyLocationfacet.pivot=country,categoryfacet=truefacet.field=categorywt=json
Output is:
{
responseHeader:{
status:0,
QTime:1,
params:{
facet:true,
indent:on,
start:0,
q:*:*,
facet.field:category,
wt:json,
facet.pivot:[category,country,KeyLocation,
country,category],
version:2.2,
rows:0}},
response:{numFound:6775,start:0,docs:[]
},
facet_counts:{
facet_queries:{},
facet_fields:{
category:[
Counterfeiting and Piracy ,2367,
Social Unrest ,2143,
Security Measures ,1064,
Fraud and Cheating,356,
Naxelites ,266,
Terrorism ,243,
Sex Crime ,232,
Shiv Sena ,76,
Major Crime ,23,
Drug Running and Organized Crime ,5]},
facet_dates:{}}}
Re: Faceting Query
On Thu, Feb 10, 2011 at 12:00 PM, Isha Garg isha.g...@orkash.com wrote: Hi, What is the significance of copy field when used in faceting . plz explain with example. Not sure what you mean here. Could you provide details? Regards, Gora
Re: Faceting Query
On Thu, Feb 10, 2011 at 12:21 PM, Isha Garg isha.g...@orkash.com wrote: What is facet.pivot field? PLz explain with example Does http://wiki.apache.org/solr/SimpleFacetParameters#facet.pivot not help? Regards, Gora
Faceting Query
Hi, What is the significance of copy field when used in faceting . plz explain with example. Thanks! Isha
Faceting Query
What is facet.pivot field? PLz explain with example
