[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536621#comment-14536621 ] Yonik Seeley commented on SOLR-7214: Thanks, I opened SOLR-7518 for this. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[
https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534841#comment-14534841
]
Crawdaddy commented on SOLR-7214:
-
Ran into a bug in the Facet Module around shards tolerant behavior. If a shard
is down, a 500 error is thrown with an NPE. If we specify shards.tolerant=true
in our Solr config, I would expect partial responses in the faceting module as
well I think, right?
error:{
trace:java.lang.NullPointerException\n\tat
org.apache.solr.search.facet.FacetModule.handleResponses(FacetModule.java:172)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:355)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:829)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:446)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:368)\n\tat
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)\n\tat
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)\n\tat
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)\n\tat
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)\n\tat
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)\n\tat
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)\n\tat
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\n\tat
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\n\tat
java.lang.Thread.run(Thread.java:745)\n,
code:500}}
JSON Facet API
--
Key: SOLR-7214
URL: https://issues.apache.org/jira/browse/SOLR-7214
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Assignee: Yonik Seeley
Fix For: 5.1
Attachments: SOLR-7214.patch
Overview is here: http://yonik.com/json-facet-api/
The structured nature of nested sub-facets are more naturally expressed in a
nested structure like JSON rather than the flat structure that normal query
parameters provide.
Goals:
- First class JSON support
- Easier programmatic construction of complex nested facet commands
- Support a much more canonical response format that is easier for clients to
parse
- First class analytics support
- Support a cleaner way to do distributed faceting
- Support better integration with other search features
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523224#comment-14523224 ] Crawdaddy commented on SOLR-7214: - Wow, that was fast. Thanks Yonik. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[
https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522177#comment-14522177
]
Crawdaddy commented on SOLR-7214:
-
Yonik, I think I found a JSON faceting bug when sub-faceting a field on
unique(another_field). As part of the upgrade from HS to Solr 5.1, I wanted to
A/B test my queries between the two. I setup two identical 5-shard Solr
installs, 35M docs each - one running HS 0.09 and and the other Solr 5.1.
Issuing my facet query, I noticed that the unique counts were different between
the two.
This query, issued to my Solr 5.1 instance, demonstrates the inconsistency
between native facets and JSON facets (limits set low enough to repro the
issue):
rows=0q=John Lennonfq=keywords:[* TO
*]facet=truefacet.pivot=keywords,top_private_domain_sfacet.limit=10
json.facet={
keywords:{
terms:{
field:keywords,
limit: 2,
facet:{
unique_domains: 'unique(top_private_domain_s)'
}
}
}
}
A snippet of the results shows that the native facets return at least 10 unique
values (there are more) for the keyword Paul McCartney:
facet_pivot:{
keywords,top_private_domain_s:[{
field:keywords,
value:Paul McCartney,
count:602,
pivot:[{
field:top_private_domain_s,
value:taringa.net,
count:35},
{
field:top_private_domain_s,
value:dailymail.co.uk,
count:34},
{
field:top_private_domain_s,
value:beatlesbible.com,
count:33},
{
field:top_private_domain_s,
value:examiner.com,
count:22},
{
field:top_private_domain_s,
value:blogspot.com,
count:14},
{
field:top_private_domain_s,
value:musicradar.com,
count:13},
{
field:top_private_domain_s,
value:liverpoolecho.co.uk,
count:11},
{
field:top_private_domain_s,
value:rollingstone.com,
count:11},
{
field:top_private_domain_s,
value:about.com,
count:9},
{
field:top_private_domain_s,
value:answers.com,
count:8}]},
...
But the JSON facets say there's only 4 unique values:
facets:{
count:11859,
keywords:{
buckets:[{
val:Paul McCartney,
count:602,
unique_domains:4}]}}}
The results are correct when issuing the same search in Heliosearch:
facets:{
count:11859,
keywords:{
buckets:[{
val:Paul McCartney,
count:602,
unique_domains:228}]}}}
In all cases the doc count (602) is the same so I know it's hitting the same
documents.
Any advice you can offer as to whether you think this is a bug, or if the
behavior is intentionally different between the two systems, would be much
appreciated. If it is a bug but you think there's a workaround, that'd be
great to know too.
JSON Facet API
--
Key: SOLR-7214
URL: https://issues.apache.org/jira/browse/SOLR-7214
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Assignee: Yonik Seeley
Fix For: 5.1
Attachments: SOLR-7214.patch
Overview is here: http://yonik.com/json-facet-api/
The structured nature of nested sub-facets are more naturally expressed in a
nested structure like JSON rather than the flat structure that normal query
parameters provide.
Goals:
- First class JSON support
- Easier programmatic construction of complex nested facet commands
- Support a much more canonical response format that is easier for clients to
parse
- First class analytics support
- Support a cleaner way to do distributed faceting
- Support better integration with other search features
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522291#comment-14522291 ] Yonik Seeley commented on SOLR-7214: Hmmm, something weird is going on. Lots of code changed between lucene/solr 4 and 5, so it wasn't necessarily a straightforward port. You probably hit a bug I introduced. What's the field type of keywords (single or multiValued?) I assume top_private_domain_s is a standard single valued string. Also, what happens if you add distrib=false (a non-distributed request on a single shard)? JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522355#comment-14522355 ] Yonik Seeley commented on SOLR-7214: Ok, thanks for narrowing the issue down! JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522476#comment-14522476 ] Yonik Seeley commented on SOLR-7214: OK, I don't yet know what's behind it, but I have reproduced it. I'll open another issue. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522327#comment-14522327 ] Crawdaddy commented on SOLR-7214: - keywords is multi-valued and top_private_domain_s is a standard single valued string. Looks like it is a distrib problem - the numbers do look more realistic on a per-shard basis. Both HS and Solr 5.1 report the same per-shard numbers for Paul McCartney across my 5 shards: 70 + 76 + 90 + 78 + 48 = 362 I would assume then that 362 goes to the 228 number I saw above, once the list is uniq'd. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522374#comment-14522374 ] Crawdaddy commented on SOLR-7214: - Absolutely - let me know if there's anything else I can do Yonik. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14522481#comment-14522481 ] Yonik Seeley commented on SOLR-7214: Opened SOLR-7494 JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14505765#comment-14505765 ] Harry Yoo commented on SOLR-7214: - I have a same issue with Eric Falcao. When I try json facet on the TrieDateField, I've got ``` Unable to range facet on tried field of unexpected type at org.apache.solr.search.facet.FacetRangeProcessor.getRangeCountsIndexed(FacetRange.java:97) ``` It seems like xxxRangeEndpointCaculator is not implemented for date type. Tested on Solr 5.1. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14506253#comment-14506253 ] Yonik Seeley commented on SOLR-7214: Thanks for the report about range faceting on date fields... I've opened SOLR-7443 for this. Should be able to fix it soon. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484590#comment-14484590 ] Eric Falcao commented on SOLR-7214: --- Hi Yonik, loving this feature so far! One thing I'm seeing is that TrieIntField, TrieDateField and possibly others don't show any buckets when I attempt to facet on them using this new API. It doesn't throw an exception - I just see an empty array of buckets. I don't know if that's by design or not, but faceting on those field types does work using old facets. Our use case is to show a timeseries that shows count of docs per minute over some time range range. Hope that makes sense. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14484617#comment-14484617 ] Yonik Seeley commented on SOLR-7214: Hmmm, thanks for the report Eric, I'll look into it. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14480891#comment-14480891 ] Yonik Seeley commented on SOLR-7214: Thanks for the report, this should now be fixed in SOLR-7350 JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[
https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387176#comment-14387176
]
Crawdaddy commented on SOLR-7214:
-
I encountered a bug/incompatibility with JSON faceting on (multi-valued) fields
w/DocValues. Multi-valued in parentheses since I don't know if the bug is
exclusive to that or not. Issue seems similar to SOLR-6024.
My use case requires the Facet API/Analytics capabilities, and I both require
and desire DocValues due to the high cardinality of the values I store, and the
associated performance increase I get when faceting on them. Without
DocValues, I get the dreaded Too many values for UnInvertedField faceting on
field error.
Possible there's a quick fix you could propose, [~yo...@apache.org], that I
could back-port into Heliosearch until this stuff is available in Solr?
Example schema field:
field name=keywords type=string indexed=true stored=true
multiValued=true docValues=true/
Traditional Solr faceting on this field works:
[...]/select?rows=0q=toyotafacet=truefacet.field=keywords
JSON faceting returns Type mismatch: keywords was indexed as SORTED_SET:
[...]/select?rows=0q=toyotajson.facet={keywords:{terms:{field:keywords}}}
ERROR - 2015-03-30 10:52:05.806; org.apache.solr.common.SolrException;
org.apache.solr.common.SolrException: Type mismatch: keywords was indexed as
SORTED_SET
at org.apache.solr.search.facet.UnInvertedField.init(UnInvertedField.java:201)
at
org.apache.solr.search.facet.UnInvertedField.getUnInvertedField(UnInvertedField.java:964)
at
org.apache.solr.search.facet.FacetFieldProcessorUIF.findStartAndEndOrds(FacetField.java:463)
at
org.apache.solr.search.facet.FacetFieldProcessorFCBase.getFieldCacheCounts(FacetField.java:203)
at
org.apache.solr.search.facet.FacetFieldProcessorFCBase.process(FacetField.java:186)
at
org.apache.solr.search.facet.FacetProcessor.fillBucketSubs(FacetRequest.java:176)
at
org.apache.solr.search.facet.FacetProcessor.processSubs(FacetRequest.java:288)
at org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetRequest.java:266)
at org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:56)
at org.apache.solr.search.facet.FacetModule.process(FacetModule.java:87)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1966)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
JSON Facet API
--
Key: SOLR-7214
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384933#comment-14384933 ] ASF subversion and git services commented on SOLR-7214: --- Commit 1669712 from [~yo...@apache.org] in branch 'dev/trunk' [ https://svn.apache.org/r1669712 ] SOLR-7214: optimize multi-valued counts-only case JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14384941#comment-14384941 ] ASF subversion and git services commented on SOLR-7214: --- Commit 1669713 from [~yo...@apache.org] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1669713 ] SOLR-7214: optimize multi-valued counts-only case JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Assignee: Yonik Seeley Fix For: 5.1 Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377811#comment-14377811 ] Steve Molloy commented on SOLR-7214: I've created SOLR-7296 to deal with this multitude of implementations. Feel free to comment, contribute, insult, etc. :) JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[
https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379043#comment-14379043
]
Yonik Seeley commented on SOLR-7214:
bq. I'm thinking that while this new way of passing parameters to facetting is
good, we'll still need to support the old way to avoid any pains for users
currently doing it the old way.
Agree. I think we need to support the facet.field=myfield pretty much forever.
bq. If this new implementation supports all Solr needs, then let's simply have
a layer that can parse parameters into a JSON format that will be provided to
it.
Internally, there's not much JSON... for flexibility we parse it as soon as we
can and then it lives in SolrQueryRequest:
public MapString,Object getJSON();
So it's a map of string to Obect, where Object can be another Map, a List, or a
primitive such as Long, Double, Boolean, etc.
So a FacetFieldParser class looks at the generic MapString,Object and creates
a FacetField instance:
{code}
public class FacetField extends FacetRequest {
String field;
long offset;
long limit = 10;
long mincount = 1;
boolean missing;
boolean numBuckets;
String prefix;
String sortVariable;
SortDirection sortDirection;
FacetMethod method;
boolean allBuckets; // show cumulative stats across all buckets (this can
be different than non-bucketed stats across all docs because of multi-valued
docs)
[...]
{code}
The FacetField instance can create a FacetProcessor to actually do the faceting
work on an actual index.
For those familiar with how Queries work in Lucene/Solr:
{code}
FacetParser == QueryParser // FacetFieldParser, FacetRangeParser,
FacetQueryParser are subclasses
FacetRequest == Query// FacetField, FacetRange, FacetQuery are
subclasses
FacetProcessor == Scorer // FacetFieldProcessor, FacetRangeProcessor,
FacetQueryProcessor are subclasses
{code}
At the top level of a distributed search, a FaceRequest (like FacetField), can
create a FacetMerger to handle merging responses from shards.
bq. If this new implementation supports all Solr needs, then let's simply have
a layer that can parse parameters into a JSON format that will be provided to
it.
See the LegacyFacet class as an example of something that does conversions
between styles. It convers to the MapString,Object
bq. Also, it seems like the description of the FacetModule is Heliosearch
Faceting, is this intended?
Oversight. I just committed a rename.
We still need to handle SolrTestCaseHS as well (HS stands for HelioSearch).
Some of that class should prob just go back into SolrTestCaseJ4, but some of it
(the client stuff) might make sense somewhere else.
JSON Facet API
--
Key: SOLR-7214
URL: https://issues.apache.org/jira/browse/SOLR-7214
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Attachments: SOLR-7214.patch
Overview is here: http://yonik.com/json-facet-api/
The structured nature of nested sub-facets are more naturally expressed in a
nested structure like JSON rather than the flat structure that normal query
parameters provide.
Goals:
- First class JSON support
- Easier programmatic construction of complex nested facet commands
- Support a much more canonical response format that is easier for clients to
parse
- First class analytics support
- Support a cleaner way to do distributed faceting
- Support better integration with other search features
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378107#comment-14378107 ] Tomás Fernández Löbbe commented on SOLR-7214: - bq. let's simply have a layer that can parse parameters into a JSON format that will be provided to it When I read this Jira I thought this was the idea, it looks like it's much more than just the JSON API. I do think this is a great feature, but I agree that duplicating so much code (and current effort tracked in SOLR-6348) is not good. bq. I would prefer we resolve the user experience before we commit and release this. +1, Once this gets released it's going to be much more difficult to refactor and do changes that could break backward compatibility or stability. A random/unrelated thought, I think for patches like these (like 10k lines, including moved classes!) would be great candidates to use the review tool, it gets very difficult to review otherwise. Also, it seems like the description of the FacetModule is Heliosearch Faceting, is this intended? JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371443#comment-14371443 ] Grant Ingersoll commented on SOLR-7214: --- I should add, however, I think the hanging of off approach brings some interesting things to the table in terms of slicing and dicing things, but I admittedly haven't looked deeply at this new stuff. My main concern here isn't the implementation or any one approach, it's the we now have 2 approaches. That's not going to make for a good user experience. I would prefer we resolve the user experience before we commit and release this. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371421#comment-14371421 ] Grant Ingersoll commented on SOLR-7214: --- Yeah, I'm not a big fan of local params, so I'm all for a new approach to the API. We should work to consolidate and deprecate, while leveraging what we can under the hood. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371242#comment-14371242 ] Steve Molloy commented on SOLR-7214: I think the underlying implementations should be shared. While I agree that SimpleFacets got to the point of being anything but simple... I don't think having completely separate implementations will help at all. Haven't looked at the new code yet, but how hard would it be to roll in pivot stats (and everything under SOLR-6350) into this implementation? I'm thinking that while this new way of passing parameters to facetting is good, we'll still need to support the old way to avoid any pains for users currently doing it the old way. And this should be perfectly fine as after all, we're talking about how to pass parameters, not what to do about them. So, whatever underlying implementation is more solid, easier to maintain, evolve, etc. We should use that and have all functionality work with it. If this new implementation supports all Solr needs, then let's simply have a layer that can parse parameters into a JSON format that will be provided to it. If it's the other way around, let's parse the JSON into parameters for the facet processing. Either way, we should decouple the way to provide parameters from the actual processing, and we should have a single way of performing that processing for facets... JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371487#comment-14371487 ] Yonik Seeley commented on SOLR-7214: bq. it's the we now have 2 approaches. 3 approaches... you're forgetting the analytics component ;-) JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369993#comment-14369993 ] Yonik Seeley commented on SOLR-7214: bq. How does this all fit with the work many have been doing on stats, facets, etc? Good question. I added facet functions and sub facets (prior to the JSON API version) at the beginning of 2014. I hacked the functionality into SimpleFacets. That experience convinced me that we need more of a fresh start for faceting in general, so when I added JSON API support, I created a new faceting module. I think it will more cleanly support a lot of stuff we want to do with faceting / analytics and other search functionality in the future. Of course then SOLR-6351 was later created in the summer of 2014 that hangs stats off of pivots, which sort of duplicates some of the functionality I had been working on. Fair enough though, I had been doing my work in the heliosearch fork and Solr needed something. As far as APIs (and having more than one API), I've ran into a lot of people that wanted a more structured API that's easier to create programmatically. I'm also not sure if just continually expanding the use of localParams for this stuff is the way to go. So how would one merge these APIs? Then there's the analytics component, which is not yet distributed. It probably still has more functionality than either my stuff here, or the new pivot-stat stuff. But some have privately shared that they are not so keen on it's API. So yeah... it looks like we have at least 3 analytics solutions in various states of development. Not sure what to do about that. Suggestions welcome. We can look at sharing more implementation over time. I was going to do percentiles, but then I saw it in SOLR-6350 as well, so I'll prob hold off and see how much of that can be shared. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://yonik.com/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364060#comment-14364060 ] Grant Ingersoll commented on SOLR-7214: --- How does this all fit with the work many have been doing on stats, facets, etc? Is there a way we can merge these features/functionality such that users don't have completely separate APIs for this stuff? e.g. SOLR-6348 and its children? JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://heliosearch.org/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363910#comment-14363910 ] Crawdaddy commented on SOLR-7214: - Thank you, Yonik, for your generosity in bringing this back into Solr!! Hope to see this in a near-term 5.x release. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://heliosearch.org/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364063#comment-14364063 ] Timothy Potter commented on SOLR-7214: -- I added a TODO to the Ref guide page for this to be documented for the 5.1 release. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://heliosearch.org/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362730#comment-14362730 ] ASF subversion and git services commented on SOLR-7214: --- Commit 1666856 from [~yo...@apache.org] in branch 'dev/trunk' [ https://svn.apache.org/r1666856 ] SOLR-7214: JSON Facet API JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://heliosearch.org/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362776#comment-14362776 ] ASF subversion and git services commented on SOLR-7214: --- Commit 1666876 from [~yo...@apache.org] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1666876 ] SOLR-7214: JSON Facet API JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://heliosearch.org/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[ https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362164#comment-14362164 ] Yonik Seeley commented on SOLR-7214: Really high level implementation notes: FacetRequestParser takes in JSON and creates FacetRequest objects. FacetRequest objects create FacetProcessor objects on specific shards that are responsible for doing the faceting. FacetRequest objects can also create a FacetMerger, which is responsible for merging responses from multiple shards. To the degree possible, we push smarts down to the leaves rather than making them dumb. For example, instead of modifying things like offset or limit at the top level, the shards are aware that they are in a distributed search and know how to do that themselves. This is much less error prone as we can avoid modifying the original facet request. Things are designed to be fully pluggable as well. For example, someone can design their own aggregation and it has the right methods to handle both low level implementation as well as high level distributed search merging. JSON Facet API -- Key: SOLR-7214 URL: https://issues.apache.org/jira/browse/SOLR-7214 Project: Solr Issue Type: New Feature Reporter: Yonik Seeley Attachments: SOLR-7214.patch Overview is here: http://heliosearch.org/json-facet-api/ The structured nature of nested sub-facets are more naturally expressed in a nested structure like JSON rather than the flat structure that normal query parameters provide. Goals: - First class JSON support - Easier programmatic construction of complex nested facet commands - Support a much more canonical response format that is easier for clients to parse - First class analytics support - Support a cleaner way to do distributed faceting - Support better integration with other search features -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7214) JSON Facet API
[
https://issues.apache.org/jira/browse/SOLR-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362162#comment-14362162
]
Yonik Seeley commented on SOLR-7214:
Here's a message I sent to the Heliosearch forum last year:
{code}
Facet functions and subfacets (nested facets) have added a lot of
analytic power, but using separate query parameters for nested facets
has the downside of being very hard to construct / read for complex
nested facets. Specifying deeply nested facets with a naturally
nested structure (JSON) makes a lot of sense and can also make
programatic generation of requests easier.
The kitchen sink that is SimpleFacets has outlived it's usefulness, so
I'm developing a new faceting module for Heliosearch with the
following goals:
- first class JSON support
- support a much more canonical response format that is easier for
clients to parse
- first class analytics support
- support a cleaner way to do distributed faceting
- support better integration with other search features such as
grouping, joins, cross-core features
The JSON API:
Note that the JSON parser we use now supports comments, unquoted
simple strings, and single quoted strings, making the DSL much more
suited to hand-typing.
The top-level bucket is implicitly defined by the set of documents
matching the main query and filters (same as old faceting, but just
more explicit).
Given that we start out with a bucket, we can ask for stats at the top-level.
EXAMPLE:
json.facet={ x:'avg(price)', y:'unique(manufacturer)' }
RESPONSE: { count:17, x:37.5, y:7 }
// The top level facet bucket is just like any query facet and always
includes count
EXAMPLE: simple field facet
json.facet={genres:{terms:genre_field}}
// this is a short-form since no other params are desired
EXAMPLE: field facet with other params
json.facet={genres:{terms:{
field:genre_field,
offset:100,
limit:20,
mincount:5
}}
Notes - I switched to using terms for a field facet because of the
awkwardness of having field appear twice in a row (i.e.
mylabel:{field:{field:myfield, offset:... )
EXAMPLE: field facet with sub facets and stats
json.facet={genres:{terms:{
field:genre_field,
offset:100,
limit:20,
mincount:5,
facet : {// these facet commands will be done per-bucket of parent facet
x : 'avg(price)' , // a stat per-bucket
y : { query : 'popularity[5 TO 10]' } , // query sub-facet
z : { terms : manufacturer } // field/terms sub-facet
}
}}}
The output looks pretty much identical to the existing facet function
and sub-facet code:
http://heliosearch.org/solr-subfacets/
http://heliosearch.org/solr-facet-functions/
You can check out the tests so far too in TestJsonFacets.java
Implementation Notes:
- Agg (aggregations are currently a subclass of ValueSource and
piggyback off of the ability of users to plug in their own custom
value source parsers).
- a FacetRequestParser creates a FacetRequest, and then to execute
that request, a FacetProcessor is created.
- Much of the code is in a single file (FacetRequest.java), but this
is just temporary... it eases changes early on while things are in
flux.
Overall, I think this will end up striking a good balance between
readability, ad-hoc human generated requests, and programmatically
generated queries.
You can try it out (but it's early... faceting multivalued fields are
not supported yet).
For convenience, here's the example server you can try:
[dead link]
-Yonik
{code}
JSON Facet API
--
Key: SOLR-7214
URL: https://issues.apache.org/jira/browse/SOLR-7214
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Attachments: SOLR-7214.patch
Overview is here: http://heliosearch.org/json-facet-api/
The structured nature of nested sub-facets are more naturally expressed in a
nested structure like JSON rather than the flat structure that normal query
parameters provide.
Goals:
- First class JSON support
- Easier programmatic construction of complex nested facet commands
- Support a much more canonical response format that is easier for clients to
parse
- First class analytics support
- Support a cleaner way to do distributed faceting
- Support better integration with other search features
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
