JSON facet API - can't reference 'val' field
Hi,
Is there any way to reference the 'val' value of a bucket in the JSON
facet API? I have a set of records that all have collection tags. I want
to facet on those collections and sort them by a function that uses the
number of search documents in that collection and the total number of
index documents in that collection. The first is just 'count', and the
second is 'docfreq(collections_sm,val)', but I can't seem to access the
'val' value. I can't get that value from the document fields as the
collection tag field is multivalued and some collections overlap. Any ideas?
Example query:
q=test={collections: { type: terms, field: collections_sm}}
Response:
..."facets":{
"count":78,
"collections":{
"buckets":[{
"val":"Ephemera",
"count":10},
{
"val":"Educational Ephemera",
"count":5}, ...
Query I want to get working:
q=test={
collections:{
type: terms,
field: collections_sm,
sort: {s: asc},
facet:{
s:"div(docfreq(collections_sm,val),count)"
}
}
}
Thanks,
Mel
Solr admin interface freezes on Chrome
Hi, Is anyone else experiencing problems using the admin interface on Chrome? It's been working for us for years, but suddenly this last week it's started freezing. It works fine until I run any query on a core, and then it'll return the data but then the page freezes and I can't scroll it. No obvious console errors. Works fine on Firefox, and I haven't made any changes to our Solr instance (v8.1.1) in a while. Thanks, -- Mel Mason Software Engineer Bodleian Digital Library Systems and Services Osney One Mon-Thurs 01865 280031 mel.ma...@bodleian.ox.ac.uk
JSON Facet doesn't allow date range facets
) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) Thanks, -- Mel Mason Software Engineer Bodleian Digital Library Systems and Services Osney One Mon-Thurs 01865 280031 mel.ma...@bodleian.ox.ac.uk
Re: JSON Facet doesn't allow date range facets
Unfortunately I need a date range field, e.g [2000-05-01 TO 2019-06-02]. DatePointFields can't represent that as far as I know. On 12/12/2019 11:40, Joel Bernstein wrote: There is a field type in the schema called pdate: This should work for you. The timeseries Streaming Expression uses the JSON facet API for range faceting and works really well. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 12, 2019 at 6:28 AM Mel Mason wrote: Hi, I'm trying to have a range facet on a field of type solr.DateRangeField. As far as I can tell, this isn't possible with JSONFacet, only with the old facet system - a quick google turns up several other people with the same problem. When using JSONFacet I get problems with this line of code: https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/search/facet/FacetRange.java#L255 It looks like during the change to JSONFacet the range facets have been restricted to only allow Trie or PointField fields. Is this likely to be fixed in future updates, or were there problems with using DateRangeFields? I could use the old parameter facet system, but there are features in JSONFacet I need. For completeness, the error I get: org.apache.solr.common.SolrException: Unable to range facet on field:date_dtr at org.apache.solr.search.facet.FacetRangeProcessor.getCalcForField(FacetRange.java:238) at org.apache.solr.search.facet.FacetRangeProcessor.(FacetRange.java:122) at org.apache.solr.search.facet.FacetRange.createFacetProcessor(FacetRange.java:65) at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:397) at org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:475) at org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:432) at org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64) at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:401) at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:146) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2566) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:756) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:542) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io .FillInterest.fillable
Re: JSON Facet doesn't allow date range facets
I need to store date ranges in the index. While DatePointField can be queried using date ranges, the actual value can only be a single date - it can't represent a range of dates as far as I know. On 12/12/2019 12:11, Joel Bernstein wrote: So something like this should work: [2000-05-01T00:00:01Z TO 2019-06-02T00:00:01Z] Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 12, 2019 at 7:08 AM Joel Bernstein wrote: With the DatePointField you can still do the range query, but I believe you'll need to specify the full ISO date string: 2000-05-01T01:01:01Z Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 12, 2019 at 6:46 AM Mel Mason wrote: Unfortunately I need a date range field, e.g [2000-05-01 TO 2019-06-02]. DatePointFields can't represent that as far as I know. On 12/12/2019 11:40, Joel Bernstein wrote: There is a field type in the schema called pdate: This should work for you. The timeseries Streaming Expression uses the JSON facet API for range faceting and works really well. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 12, 2019 at 6:28 AM Mel Mason wrote: Hi, I'm trying to have a range facet on a field of type solr.DateRangeField. As far as I can tell, this isn't possible with JSONFacet, only with the old facet system - a quick google turns up several other people with the same problem. When using JSONFacet I get problems with this line of code: https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/search/facet/FacetRange.java#L255 It looks like during the change to JSONFacet the range facets have been restricted to only allow Trie or PointField fields. Is this likely to be fixed in future updates, or were there problems with using DateRangeFields? I could use the old parameter facet system, but there are features in JSONFacet I need. For completeness, the error I get: org.apache.solr.common.SolrException: Unable to range facet on field:date_dtr at org.apache.solr.search.facet.FacetRangeProcessor.getCalcForField(FacetRange.java:238) at org.apache.solr.search.facet.FacetRangeProcessor.(FacetRange.java:122) at org.apache.solr.search.facet.FacetRange.createFacetProcessor(FacetRange.java:65) at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:397) at org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:475) at org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:432) at org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64) at org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:401) at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:146) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2566) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:756) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:542) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle
Re: support need in solr for min and max
Try looking at range JSON facets: https://lucene.apache.org/solr/guide/8_2/json-facet-api.html#range-facet. If you facet over the eventTimeStamp with a gap of 1 day, you should then be able to use a sub facet to return a min and max value (https://lucene.apache.org/solr/guide/8_2/json-facet-api.html#stat-facet-functions) for each day bucket. On 08/01/2020 11:07, Mohamed Azharuddin wrote: Hi team, We are migrating from mysql to apache solr since solr is fast in searching. Thank you. We had a scenario to *find 1) difference (max-min)* * 2) with group by date(timeStamp)* Given below is our mysql table : Untitled.png And mysql query is, */SELECT Date(eventTimeStamp), MAX(field) - MIN(field) AS Energy FROM PowerTable GROUP BY DATE(eventTimeStamp);/* will results, Untitled2.png So we have to calculate difference per day, where date column is in datetime format where we are using result grouping as */group=true=eventTimeStamp:[2019-12-11T00:00:00Z TO 2019-12-11T23:59:59Z]=eventTimeStamp:[2019-12-12T00:00:00Z TO 2019-12-12T23:59:59Z]/* Using Apache solr statistics option, we are able to calculate max and min for whole result, But we need max and min value per day basis. Untitled31.png When we try to get max and min value per day basis, we are able to fetch either min or max using following query. */=event1 desc or =event1 asc/* */ /* Untitled6.png But we need both min and max in single query. So kindly help us to go ahead. -- Regards, Azar@EJ
Re: Solr Admin Console hangs on Chrome
Good questions. I've been having similar problems for a while, for me, the UI in general is frozen, including the navigation buttons and query text boxes. Delay depends on the size of the json - if I do a request for 1000 rows with just 1 field each, it's a permanent 5s delay on scrolling and the text boxes and navigation buttons start working after a bit. If it's larger - e.g. 1 rows with 1 field each, or 10 rows with lots of large fields - then it just freezes indefinitely as far as I can tell. Smaller queries - e.g. 100 rows with 1 field each work fine. I don't see any errors in the console. The problems start around the time it starts chunking the json response, which may just be coincidence, but I think there are a few differences in how Chrome processes chunked responses. Although the response has a content-type of application/json set, which should avoid those problems. Firefox handles all of these queries without any problem. On 14/01/2020 09:01, Jan Høydahl wrote: How long delay do you see? Is it only for query panel or for the UI in general? A query for *:* is not necessarily a simple query, it depends on how many and large fields you have etc. Try a query with fl=id or fl=title and see if that helps. Jan 13. jan. 2020 kl. 22:29 skrev Webster Homer : I still see this issue with Chrome and the admin console. I am using Solr 7.3 In the Chrome console I see an error: "style.css:1 Failed to load resource: the server responded with a status of 404 (Not Found)" This used to work. It is unusably slow, even with a simple query like *:* -Original Message- From: Jan Høydahl Sent: Thursday, December 12, 2019 1:45 AM To: solr-user@lucene.apache.org Subject: Re: Solr Admin Console hangs on Chrome I have seen slowness when the result is a very large json but not for ordinary queries. How long delay do you see? Is it only for query panel or for the UI in general? Jan Høydahl 11. des. 2019 kl. 16:07 skrev Alexandre Rafalovitch : Check for popup and other tracker blockers. It is possible one of the resources has a similar name and triggers blocking. There was a thread in early October with a similar discussion, but apart from the blockers idea nothing else was discovered at the time. An easy way would be to create a new Chrome profile without any add-ons and try accessing Solr that way. This would differentiate "Chrome vs Firefox" and "Chrome vs Chrome plugins". Regards, Alex. On Wed, 11 Dec 2019 at 07:50, A Adel wrote: Hi - could you provide more details, such as Solr and browser network logs when using Chrome / other browsers? On Tue, Dec 10, 2019 at 5:48 PM Joel Bernstein wrote: Did a recent change to Chrome cause this? In Solr 8x, I'm not seeing slowness with Chrome on Mac. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Dec 10, 2019 at 8:26 PM SAGAR INGALE wrote: I am also facing the same issue for v6.4.0 On Wed, 11 Dec, 2019, 5:37 AM Joel Bernstein, wrote: What version of Solr? Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Dec 10, 2019 at 5:58 PM Arnold Bronley < arnoldbron...@gmail.com> wrote: I am also facing similar issue. I have also switched to other browsers to solve this issue. On Tue, Dec 10, 2019 at 2:22 PM Webster Homer < webster.ho...@milliporesigma.com> wrote: It seems like the Solr Admin console has become slow when you use it on the chrome browser. If I go to the query tab and execute a query, even the default *:* after that the browser window becomes very slow. I'm using chrome Version 78.0.3904.108 (Official Build) (64-bit) on Windows The work around is to use Firefox This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith. Click http://www.merckgroup.com/disclaimer to access the German, French, Spanish and Portuguese versions of this disclaimer. -- Sent from my iPhone This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this
