[jira] [Commented] (SOLR-4736) Support group.mincount for Result Grouping
[
https://issues.apache.org/jira/browse/SOLR-4736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15542888#comment-15542888
]
Ryan Josal commented on SOLR-4736:
--
I just came across a use case where this would be useful, like facet.mincount
sometimes is. I'm not sure Type: Bug is the correct classification though.
> Support group.mincount for Result Grouping
> --
>
> Key: SOLR-4736
> URL: https://issues.apache.org/jira/browse/SOLR-4736
> Project: Solr
> Issue Type: Bug
> Components: search
>Affects Versions: 4.2
>Reporter: jefferyyuan
>Priority: Minor
> Labels: group, solr
> Fix For: 4.9, 6.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Result Grouping is a very useful feature: we can use it to find duplicate
> data in index, but it lacks of one feature-group.mincount.
> With group.mincount, we can specify that only groups that has equal or more
> than ${mincount} for the group field will be returned.
> Specially, we can use group.mincount=2 to only return duplicate data.
> Could we add this in future release? Thanks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2899) Add OpenNLP Analysis capabilities as a module
[ https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396928#comment-15396928 ] Ryan Josal commented on LUCENE-2899: Seconded, this is really useful stuff. > Add OpenNLP Analysis capabilities as a module > - > > Key: LUCENE-2899 > URL: https://issues.apache.org/jira/browse/LUCENE-2899 > Project: Lucene - Core > Issue Type: New Feature > Components: modules/analysis >Reporter: Grant Ingersoll >Assignee: Grant Ingersoll >Priority: Minor > Fix For: 4.9, 6.0 > > Attachments: LUCENE-2899-6.1.0.patch, LUCENE-2899-RJN.patch, > LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, LUCENE-2899.patch, > LUCENE-2899.patch, OpenNLPFilter.java, OpenNLPTokenizer.java > > > Now that OpenNLP is an ASF project and has a nice license, it would be nice > to have a submodule (under analysis) that exposed capabilities for it. Drew > Farris, Tom Morton and I have code that does: > * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it > would have to change slightly to buffer tokens) > * NamedEntity recognition as a TokenFilter > We are also planning a Tokenizer/TokenFilter that can put parts of speech as > either payloads (PartOfSpeechAttribute?) on a token or at the same position. > I'd propose it go under: > modules/analysis/opennlp -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8812) ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND
[
https://issues.apache.org/jira/browse/SOLR-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201525#comment-15201525
]
Ryan Josal commented on SOLR-8812:
--
On the topic of SOLR-2649, I just upgraded to 5.5 yesterday and SOLR-2649 broke
one of our test cases which was "hair ties -barbie" should return hair ties but
not barbie hair ties, and now it matches nothing. I assume this is intended,
but if not, maybe this ticket also addresses it?
> ExtendedDismaxQParser (edismax) ignores Boolean OR when q.op=AND
>
>
> Key: SOLR-8812
> URL: https://issues.apache.org/jira/browse/SOLR-8812
> Project: Solr
> Issue Type: Bug
> Components: query parsers
>Affects Versions: 5.5
>Reporter: Ryan Steinberg
>Priority: Blocker
> Fix For: 6.0, 5.5.1
>
> Attachments: SOLR-8812.patch
>
>
> The edismax parser ignores Boolean OR in queries when q.op=AND. This behavior
> is new to Solr 5.5.0 and an unexpected major change.
> Example:
> "q": "id:12345 OR zz",
> "defType": "edismax",
> "q.op": "AND",
> where "12345" is a known document ID and "zz" is a string NOT present
> in my data
> Version 5.5.0 produces zero results:
> "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+((id:12345
> DisjunctionMaxQuery((text:zz)))~2))/no_coord",
> "parsedquery_toString": "+((id:12345 (text:zz))~2)",
> "explain": {},
> "QParser": "ExtendedDismaxQParser"
> Version 5.4.0 produces one result as expected
> "rawquerystring": "id:12345 OR zz",
> "querystring": "id:12345 OR zz",
> "parsedquery": "(+(id:12345
> DisjunctionMaxQuery((text:zz/no_coord",
> "parsedquery_toString": "+(id:12345 (text:zz))"
> "explain": {},
> "QParser": "ExtendedDismaxQParser"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8175) Wordbreak spellchecker throws IOOBE with Occur.MUST term
[ https://issues.apache.org/jira/browse/SOLR-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032697#comment-15032697 ] Ryan Josal commented on SOLR-8175: -- Thanks for picking it up! > Wordbreak spellchecker throws IOOBE with Occur.MUST term > > > Key: SOLR-8175 > URL: https://issues.apache.org/jira/browse/SOLR-8175 > Project: Solr > Issue Type: Bug >Reporter: Ryan Josal >Assignee: James Dyer > Attachments: solr8175.patch > > > Using the WordBreakSolrSpellChecker, if a user queries for "+foo barbaz" and > "bar baz" is a suggestion for "barbaz", Solr will throw an > IndexOutOfBoundsException. As a result, a server driven by user queries > might throw a certain percentage of HTTP 500 responses as users hit this. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8175) Wordbreak spellchecker throws IOOBE with Occur.MUST term
[ https://issues.apache.org/jira/browse/SOLR-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974575#comment-14974575 ] Ryan Josal commented on SOLR-8175: -- BTW, this patch passes "ant test" and "ant precommit". > Wordbreak spellchecker throws IOOBE with Occur.MUST term > > > Key: SOLR-8175 > URL: https://issues.apache.org/jira/browse/SOLR-8175 > Project: Solr > Issue Type: Bug >Reporter: Ryan Josal > Attachments: solr8175.patch > > > Using the WordBreakSolrSpellChecker, if a user queries for "+foo barbaz" and > "bar baz" is a suggestion for "barbaz", Solr will throw an > IndexOutOfBoundsException. As a result, a server driven by user queries > might throw a certain percentage of HTTP 500 responses as users hit this. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-8175) Wordbreak spellchecker throws IOOBE with Occur.MUST term
Ryan Josal created SOLR-8175: Summary: Wordbreak spellchecker throws IOOBE with Occur.MUST term Key: SOLR-8175 URL: https://issues.apache.org/jira/browse/SOLR-8175 Project: Solr Issue Type: Bug Reporter: Ryan Josal Using the WordBreakSolrSpellChecker, if a user queries for "+foo barbaz" and "bar baz" is a suggestion for "barbaz", Solr will throw an IndexOutOfBoundsException. As a result, a server driven by user queries might throw a certain percentage of HTTP 500 responses as users hit this. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8175) Wordbreak spellchecker throws IOOBE with Occur.MUST term
[ https://issues.apache.org/jira/browse/SOLR-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Josal updated SOLR-8175: - Attachment: solr8175.patch Uploaded solr8175.patch, which includes an updated unit test which would throw the exception without the fix. Let me know what else I can do to help. > Wordbreak spellchecker throws IOOBE with Occur.MUST term > > > Key: SOLR-8175 > URL: https://issues.apache.org/jira/browse/SOLR-8175 > Project: Solr > Issue Type: Bug >Reporter: Ryan Josal > Attachments: solr8175.patch > > > Using the WordBreakSolrSpellChecker, if a user queries for "+foo barbaz" and > "bar baz" is a suggestion for "barbaz", Solr will throw an > IndexOutOfBoundsException. As a result, a server driven by user queries > might throw a certain percentage of HTTP 500 responses as users hit this. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6842) No way to limit the fields cached in memory and leads to OOM when there are thousand of fields (thousands)
[ https://issues.apache.org/jira/browse/LUCENE-6842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14963326#comment-14963326 ] Ryan Josal commented on LUCENE-6842: FWIW, I have seen an OOME due to a large number of fields before. I had an index with millions of documents and I wanted to experiment with ways to reduce full indexing time, and there was a way to reorganize the data that would reduce the number of docs by a factor of 2000 by adding 2000 fields (using a solr dynamic multivalued String with ~2-3 values on average). The idea was quickly canned after seeing an OOME during full indexing. This piece of data we may eventually move out of Lucene and into a datastore more suited like Cassandra. > No way to limit the fields cached in memory and leads to OOM when there are > thousand of fields (thousands) > -- > > Key: LUCENE-6842 > URL: https://issues.apache.org/jira/browse/LUCENE-6842 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 4.6.1 > Environment: Linux, openjdk 1.6.x >Reporter: Bala Kolla > Attachments: HistogramOfHeapUsage.png > > > I am opening this defect to get some guidance on how to handle a case of > server running out of memory and it seems like it's something to do how we > index. But want to know if there is anyway to reduce the impact of this on > memory usage before we look into the way of reducing the number of fields. > Basically we have many thousands of fields being indexed and it's causing a > large amount of memory being used (25GB) and eventually leading to > application to hang and force us to restart every few minutes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7953) BaseEditorialTransformer (elevated) doesn't work with LazyField
Ryan Josal created SOLR-7953: Summary: BaseEditorialTransformer (elevated) doesn't work with LazyField Key: SOLR-7953 URL: https://issues.apache.org/jira/browse/SOLR-7953 Project: Solr Issue Type: Bug Affects Versions: 5.2, 4.10.2 Reporter: Ryan Josal When using the QueryElevationComponent, using the [elevated] docTransformer doesn't always work. In the case the document is a LazyDocument, BaseEditorialTransformer#getKey will return the LazyField.toString() which is Object#toString() which of course isn't going to match any of the uniqueKeys. The fix is to change getKey to check instanceof IndexableField instead of just Field. I'm not sure the impact of this bug because I don't know how often LazyDocuments get used. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622449#comment-14622449
]
Ryan Josal commented on SOLR-6234:
--
It seems like the parser can find out if it's multivalued for itself by
checking the otherCore's Schema.
I think multivalued to field should be a separate ticket since it is not a
feature of the qparser
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Assignee: Timothy Potter
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch, SOLR-6234.patch,
otherHandler.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14622449#comment-14622449
]
Ryan Josal edited comment on SOLR-6234 at 7/10/15 3:22 PM:
---
It seems like the parser can find out if it's multivalued for itself by
checking the otherCore's Schema.
I think multivalued to field on the current core should be a separate ticket
since it is not a feature of the qparser, it's an improvement to JoinUtil. I
can have a patch for that later; I have it working.
was (Author: rjosal):
It seems like the parser can find out if it's multivalued for itself by
checking the otherCore's Schema.
I think multivalued to field should be a separate ticket since it is not a
feature of the qparser
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Assignee: Timothy Potter
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch, SOLR-6234.patch,
otherHandler.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621658#comment-14621658
]
Ryan Josal commented on SOLR-6234:
--
That makes a lot of sense David, I would prefer this as it would reduce
confusion, some similar code, and make implementation changes in the future
more flexible.
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Assignee: Timothy Potter
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch, SOLR-6234.patch,
otherHandler.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619385#comment-14619385
]
Ryan Josal commented on SOLR-6234:
--
That will be great Tim! For my personal use case, I added a feature to the
qparser that, after constructing the LocalSolrQueryRequest, would apply the
handler config (defaults, appends, invariants) from the otherCore (say
/select handler) to the request so that in a nutshell your other query could
be a simple string and it would follow whatever edismax rules you had
configured for the other core. That instead of the more strict db style lucene
query syntax. If there's interest I am happy to share the patch.
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Assignee: Timothy Potter
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619521#comment-14619521
]
Ryan Josal edited comment on SOLR-6234 at 7/8/15 10:47 PM:
---
I've attached otherHandler.patch, which is a patch on top of the existing
ScoreJoinQParserPlugin.java from the Dec 14 patch, which shows changes to add a
feature where the user can supply handler=/select (for example) in localparams,
to apply the config from the handler in the other core. You'll notice that the
request is built from localParams instead of params so that whatever your
current core params are don't override the other core configured params. If
this breaks something, it could be changed to only do that when the handler
localParam is present. This patch is quick and dirty because it uses
reflection to get the defaults,appends,invariants config from the
BaseRequestHandler class. Really, the access level of those variables should
change, or the method should be moved there.
Personally I'm using this feature to join my deals core with my products core
as the othercore, and I wanted it to do a regular search against the products
core. This approach could actually be used in the \!join qparser too. If this
patch is useful for somebody, great!
was (Author: rjosal):
I've attached otherHandler.patch, which is a patch on top of the existing
ScoreJoinQParserPlugin.java from the Dec 14 patch, which shows changes to add a
feature where the user can supply handler=/select (for example) in localparams,
to apply the config from the handler in the other core. You'll notice that the
request is built from localParams instead of params so that whatever your
current core params are don't override the other core configured params. If
this breaks something, it could be changed to only do that when the handler
localParam is present. This patch is quick and dirty because it uses
reflection to get the defaults,appends,invariants config from the
BaseRequestHandler class. Really, the access level of those variables should
change, or the method should be moved there.
Personally I'm using this feature to join my deals core with my products core
as the othercore, and I wanted it to do a regular search against the products
core. This approach could actually be used in the !join qparser too. If this
patch is useful for somebody, great!
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Assignee: Timothy Potter
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch, otherHandler.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Josal updated SOLR-6234:
-
Attachment: otherHandler.patch
I've attached otherHandler.patch, which is a patch on top of the existing
ScoreJoinQParserPlugin.java from the Dec 14 patch, which shows changes to add a
feature where the user can supply handler=/select (for example) in localparams,
to apply the config from the handler in the other core. You'll notice that the
request is built from localParams instead of params so that whatever your
current core params are don't override the other core configured params. If
this breaks something, it could be changed to only do that when the handler
localParam is present. This patch is quick and dirty because it uses
reflection to get the defaults,appends,invariants config from the
BaseRequestHandler class. Really, the access level of those variables should
change, or the method should be moved there.
Personally I'm using this feature to join my deals core with my products core
as the othercore, and I wanted it to do a regular search against the products
core. This approach could actually be used in the !join qparser too. If this
patch is useful for somebody, great!
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Assignee: Timothy Potter
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch, otherHandler.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7726) CodecFactory plugins are double informed of SolrCore
Ryan Josal created SOLR-7726: Summary: CodecFactory plugins are double informed of SolrCore Key: SOLR-7726 URL: https://issues.apache.org/jira/browse/SOLR-7726 Project: Solr Issue Type: Bug Affects Versions: 5.2.1, 5.0, 4.10 Reporter: Ryan Josal Priority: Minor During the SolrCore#initCodec initialization phase, it initializes the the CodecFactory by using ResourceLoader#newInstance, which will call inform(SolrCore), and then a few lines later it calls inform again. The SchemaCodecFactory, for example, does not break due to this because it has no effect to inform it multiple times. But any extensions or other CodecFactories may need to work around this by checking for a second inform call to prevent errors or just from running potentially complex init code twice. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586444#comment-14586444
]
Ryan Josal commented on SOLR-6234:
--
I've been using this for a few days now, and here are my comments:
*) it's more functionally useful to have the scores returned with this than the
!join parser.
*) basing it off JoinUtil is a clean solution that centralizes this type of
logic.
*) That being said, scorejoin has performed worse than !join in my tests,
usually almost twice as slow. It's not an exhaustive test by any means, just
an observation. So it doesn't seem fit to replace !join as it is, although it
does feel like this is the right direction to be going for joins.
*) JoinUtil does not currently support multivalue fields for the to index.
It chooses one of the matching values instead of applying the ScoreMode for all
matching documents. It's a fairly small change to TermsIncludingScoreQuery to
add this feature.
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586457#comment-14586457
]
Ryan Josal commented on SOLR-6234:
--
[~neerajlajpal], glad you got it working. There might be a caution with using
!scorejoin this way. The scores coming back from !scorejoin are from the query
to the fromIndex, and so they're not really related to the score coming from
the important_category matches. The matches from the scorejoin might have a
really high max score, while the matches from the important_category might have
a really low max_score, in a relative sense. Or vice versa. I think this use
case is workable but it's something to keep in mind.
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14583437#comment-14583437
]
Ryan Josal edited comment on SOLR-6234 at 6/12/15 2:15 PM:
---
[~neerajlajpal] Solr parses the value of q with a single parser, which will be
whatever your defType is. In order to do something like that you can use
nested queries:
{noformat}
important_category:322^4 _query_:{!scorejoin from=_root_ to=id score=avg
multiVals=false v=$jq}jq=(brandid:1398^4 OR brandid:237^4.5)
{noformat}
was (Author: rjosal):
[~neerajlajpal] Solr parses the value of q with a single parser, which will be
whatever your defType is. In order to do something like that you can use
nested queries:
important_category:322^4 _query_:{!scorejoin from=_root_ to=id score=avg
multiVals=false v=$jq}jq=(brandid:1398^4 OR brandid:237^4.5)
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14583437#comment-14583437
]
Ryan Josal commented on SOLR-6234:
--
[~neerajlajpal] Solr parses the value of q with a single parser, which will be
whatever your defType is. In order to do something like that you can use
nested queries:
important_category:322^4 _query_:{!scorejoin from=_root_ to=id score=avg
multiVals=false}
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14583437#comment-14583437
]
Ryan Josal edited comment on SOLR-6234 at 6/12/15 2:13 PM:
---
[~neerajlajpal] Solr parses the value of q with a single parser, which will be
whatever your defType is. In order to do something like that you can use
nested queries:
important_category:322^4 _query_:{!scorejoin from=_root_ to=id score=avg
multiVals=false v=$jq}jq=(brandid:1398^4 OR brandid:237^4.5)
was (Author: rjosal):
[~neerajlajpal] Solr parses the value of q with a single parser, which will be
whatever your defType is. In order to do something like that you can use
nested queries:
important_category:322^4 _query_:{!scorejoin from=_root_ to=id score=avg
multiVals=false}
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 5.3
Reporter: Mikhail Khludnev
Labels: features, patch, test
Fix For: 5.3
Attachments: SOLR-6234.patch, SOLR-6234.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6234) Scoring modes for query time join
[
https://issues.apache.org/jira/browse/SOLR-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14579608#comment-14579608
]
Ryan Josal commented on SOLR-6234:
--
This is awesome, the normal !join qparser is mainly only good for fq, but with
scoring, this is good as a q or subquery.
Scoring modes for query time join
--
Key: SOLR-6234
URL: https://issues.apache.org/jira/browse/SOLR-6234
Project: Solr
Issue Type: New Feature
Components: query parsers
Affects Versions: 4.10.3, Trunk
Reporter: Mikhail Khludnev
Labels: features, patch, test
Fix For: 5.0, Trunk
Attachments: SOLR-6234.patch, SOLR-6234.patch
it adds {{scorejoin}} query parser which calls Lucene's JoinUtil underneath.
It supports:
- {{score=none|avg|max|total}} local param (passed as ScoreMode to JoinUtil)
- {{score=none}} is *default*, eg if you *omit* this localparam
- supports {{b=100}} param to pass {{Query.setBoost()}}.
- {{multiVals=true|false}} is introduced
- there is a test coverage for cross core join case.
- so far it joins string and multivalue string fields (Sorted, SortedSet,
Binary), but not Numerics DVs. follow-up LUCENE-5868
-there was a bug in cross core join, however there is a workaround for it-
it's fixed in Dec'14 patch.
Note: the development of this patch was sponsored by an anonymous contributor
and approved for release under Apache License.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7152) Cannot override default rq with no rankquery
Ryan Josal created SOLR-7152:
Summary: Cannot override default rq with no rankquery
Key: SOLR-7152
URL: https://issues.apache.org/jira/browse/SOLR-7152
Project: Solr
Issue Type: Bug
Affects Versions: 4.10.2
Reporter: Ryan Josal
Priority: Minor
In your update handler, if you put a default parameter for rq=, you cannot
override it with no rankquery. You can try passing rq=, which will override
the default value to an empty string, but the QueryComponent will complain that
it isn't a valid RankQuery. Looks like a simple fix to not only check for null
but also isEmpty.
Possible workaround for now:
Check for empty rq string in the prepare method of an earlier search component
and remove it. This isn't a workable solution though if your rankquery param
uses a {!switch ...} that can set to empty in a default case for example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3015) QParserPlugins can not be SolrCoreAware
[
https://issues.apache.org/jira/browse/SOLR-3015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14146798#comment-14146798
]
Ryan Josal commented on SOLR-3015:
--
I ran into a problem with this today. I have a QParserPlugin that would like
to get some info from the index at startup, but without having it
SolrCoreAware, I can't getSolrConfig().getDataDir(). Having run into the
circular init problem before in the past, I understand the motive to reduce
risk of problems. On a related note, I have a SearchComponent that wants to
(but isn't allow to) be ResourceLoaderAware so it can load some data file from
the config dir. Interestingly, QParserPlugin CAN be ResourceLoaderAware. At
least with that one I can put the instanceDir in my config because that
property is automatically populated, but I can't work around the data.dir
problem that way because it is left blank. My personal feeling is that it
would be great if these artificial limitations could be safely lifted somehow.
QParserPlugins can not be SolrCoreAware
---
Key: SOLR-3015
URL: https://issues.apache.org/jira/browse/SOLR-3015
Project: Solr
Issue Type: New Feature
Reporter: Karl Wright
Labels: closehook, qparserplugin, solrcoreaware
Fix For: 3.6, 4.0-ALPHA
Attachments: SOLR-3015.patch
QParserPlugin cannot be made SolrCoreAware:
{code}
[junit] org.apache.solr.common.SolrException: Invalid 'Aware' object:
org.apache.solr.mcf.ManifoldCFQParserPlugin@18941f7 --
org.apache.solr.util.plugin.SolrCoreAware must be an instance of:
[org.apache.solr.request.SolrRequestHandler]
[org.apache.solr.response.QueryResponseWriter]
[org.apache.solr.handler.component.SearchComponent]
[org.apache.solr.update.processor.UpdateRequestProcessorFactory]
[org.apache.solr.handler.component.ShardHandlerFactory]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6396) Allow the name of core.properties file used in discovery to be specified by an environment variable
[ https://issues.apache.org/jira/browse/SOLR-6396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105662#comment-14105662 ] Ryan Josal commented on SOLR-6396: -- Confirmed it does work the way Hoss suggests; using property expansion in core.properties meets the need. Allow the name of core.properties file used in discovery to be specified by an environment variable --- Key: SOLR-6396 URL: https://issues.apache.org/jira/browse/SOLR-6396 Project: Solr Issue Type: Improvement Affects Versions: 4.9, 5.0 Reporter: Erick Erickson Assignee: Erick Erickson Priority: Minor This was brought up on the user's list. I haven't thought this through, but it seems reasonable. This has some interesting implications in the core rename case, i.e. The unloaded props file will have the different name as well. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3083) all JMX Mbeans are identified as java.lang.String even if numeric
[
https://issues.apache.org/jira/browse/SOLR-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534434#comment-13534434
]
Ryan Josal commented on SOLR-3083:
--
I applied this patch to Solr 3.6 - I noticed it missed the hitratio mbean
attribute from [Fast]LRUCache. Looks like that attribute is actually
calculated into a String and so would have to be fixed by itself. I'm not sure
if this is still present in the latest builds though.
all JMX Mbeans are identified as java.lang.String even if numeric
-
Key: SOLR-3083
URL: https://issues.apache.org/jira/browse/SOLR-3083
Project: Solr
Issue Type: Bug
Components: SearchComponents - other
Affects Versions: 4.0-ALPHA
Reporter: Tagged Siteops
Assignee: Ryan McKinley
Priority: Minor
Labels: interoperability,, jmx, monitoring, statistics
Fix For: 4.0-ALPHA
Attachments: Screen shot 2012-01-31 at 2.19.06 PM.png,
SOLR-3083-Made-JMX-Mbean-information-use-numeric-typ.patch
When trying to monitor our Solr instances with JMXtrans
(https://github.com/lookfirst/jmxtrans), we discovered that all the Mbeans
were identifying themselves as java.lang.String. This breaks the
GangliaWriter in JMXtrans because the software decides it's a string instead
of a numeric values and you can't graph a string. Other software we are
monitoring with JMXtrans (e.g. Kafka) correctly identify their numeric values
as numbers (doubles in this case).
For example, avgTimePerRequest in Solr is identified as a string which makes
no sense. This behavior makes Solr less interoperable with the rest of the
Java ecosystem.
The problem code is in JmxMonitoredMap.java
NamedList dynamicStats = infoBean.getStatistics();
if (dynamicStats != null) {
for (int i = 0; i dynamicStats.size(); i++) {
String name = dynamicStats.getName(i);
if (!staticStats.contains(name))
attrInfoList.add(new MBeanAttributeInfo(dynamicStats.getName(i),
String.class.getName(), null, true, false, false));
}
I will try to attach a screenshot.
Thanks for Solr! We use the heck out of it, it's awesome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3866) CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496352#comment-13496352 ] Ryan Josal commented on SOLR-3866: -- The use case I (and it appears the author of the instigating thread for this issue: http://osdir.com/ml/solr-user.lucene.apache.org/2012-09/msg00893.html) have for CoreAdmin SWAP is to support a backup core (on deck core) that I can rebuild the index from scratch on. CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud - Key: SOLR-3866 URL: https://issues.apache.org/jira/browse/SOLR-3866 Project: Solr Issue Type: Bug Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0 Reporter: Hoss Man We need to define what the expected behavior of using the CoreAdminHandler's SWAP and RENAME commands is if you are running in SolrCloud mode. At the moment, it seems to introduce a disconnect between which collection the SolrCore thinks it's a part of (and what appears in the persisted solr.xml) vs what collection ZooKeeper thinks the SolrCore(s) that were swaped/renamed are a part of. We should either fix this, or document it if it as actually consistent and intentional for low level controls, or disable commands like SWAP/RENAME in SolrCloud mode https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201209.mbox/%3CCALB4QrP2GZAwLeAiy%3DfcmOLYbc5r0i9Tp1DquyPS8mMJPwCgnw%40mail.gmail.com%3E -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
