[jira] [Commented] (SOLR-9909) Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory
[ https://issues.apache.org/jira/browse/SOLR-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081134#comment-17081134 ] David Smiley commented on SOLR-9909: +1 I don't love the name "SolrjNamedThreadFactory" for the 'j' in there but whatever. > Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory > > > Key: SOLR-9909 > URL: https://issues.apache.org/jira/browse/SOLR-9909 > Project: Solr > Issue Type: Task >Reporter: Shalin Shekhar Mangar >Priority: Trivial > Fix For: 6.7, 7.0 > > Attachments: SOLR-9909-01.patch, SOLR-9909-02.patch, > SOLR-9909-03.patch, SOLR-9909.patch > > > DefaultSolrThreadFactory and SolrjNamedThreadFactory have exactly the same > code. Let's remove one of them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14345) Error messages are not properly propagated with non-default response parsers
[
https://issues.apache.org/jira/browse/SOLR-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081132#comment-17081132
]
Lucene/Solr QA commented on SOLR-14345:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 46m
11s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
41s{color} | {color:green} solrj in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-14345 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12999547/SOLR-14345.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 527e6516604 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/733/testReport/ |
| modules | C: solr/core solr/solrj solr/test-framework U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/733/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Error messages are not properly propagated with non-default response parsers
>
>
> Key: SOLR-14345
> URL: https://issues.apache.org/jira/browse/SOLR-14345
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-14345.patch, SOLR-14345.patch, SOLR-14345.patch,
> SOLR-14345.patch
>
>
> Default {{ResponsParseer}} is {{BinaryResponseParser}}. when non-default
> response parser is specified in the request then, the error message is
> propagated to user. This happens in solrCloud mode.
> I came across this problem when working on adding some test which uses
> {{SolrTestCaseHS}} but similar problem exists with SolrJ client
> Also, same problem exists in both HttpSolrClient and Http2SolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-9909) Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory
[ https://issues.apache.org/jira/browse/SOLR-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081131#comment-17081131 ] Shalin Shekhar Mangar commented on SOLR-9909: - Patch updated to master. > Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory > > > Key: SOLR-9909 > URL: https://issues.apache.org/jira/browse/SOLR-9909 > Project: Solr > Issue Type: Task >Reporter: Shalin Shekhar Mangar >Priority: Trivial > Fix For: 6.7, 7.0 > > Attachments: SOLR-9909-01.patch, SOLR-9909-02.patch, > SOLR-9909-03.patch, SOLR-9909.patch > > > DefaultSolrThreadFactory and SolrjNamedThreadFactory have exactly the same > code. Let's remove one of them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-9909) Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory
[ https://issues.apache.org/jira/browse/SOLR-9909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-9909: Attachment: SOLR-9909.patch > Nuke one of DefaultSolrThreadFactory and SolrjNamedThreadFactory > > > Key: SOLR-9909 > URL: https://issues.apache.org/jira/browse/SOLR-9909 > Project: Solr > Issue Type: Task >Reporter: Shalin Shekhar Mangar >Priority: Trivial > Fix For: 6.7, 7.0 > > Attachments: SOLR-9909-01.patch, SOLR-9909-02.patch, > SOLR-9909-03.patch, SOLR-9909.patch > > > DefaultSolrThreadFactory and SolrjNamedThreadFactory have exactly the same > code. Let's remove one of them. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14402) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search
[ https://issues.apache.org/jira/browse/SOLR-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-14402: - Fix Version/s: 8.6 master (9.0) Resolution: Fixed Status: Resolved (was: Patch Available) > Avoid creating new exceptions for every request made to > MDCAwareThreadPoolExecutor by distributed search > > > Key: SOLR-14402 > URL: https://issues.apache.org/jira/browse/SOLR-14402 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 7.4 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Fix For: master (9.0), 8.6 > > Attachments: SOLR-14402.patch > > > SOLR-11880 tried to do the same and it succeeded for update shard handler but > the implementation was wrong for http shard handler because the executor > created during construction is overwritten in the init() method. The commit > for SOLR-11880 is at https://github.com/apache/lucene-solr/commit/5a47ed4/ > Thanks [~caomanhdat] for spotting this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14402) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search
[ https://issues.apache.org/jira/browse/SOLR-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081120#comment-17081120 ] ASF subversion and git services commented on SOLR-14402: Commit f68c498e4fb891efbd598ea2007985334c731baf in lucene-solr's branch refs/heads/branch_8x from Shalin Shekhar Mangar [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f68c498 ] SOLR-14402: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search. This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which fixed distributed updates but not distributed search. (cherry picked from commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760) > Avoid creating new exceptions for every request made to > MDCAwareThreadPoolExecutor by distributed search > > > Key: SOLR-14402 > URL: https://issues.apache.org/jira/browse/SOLR-14402 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 7.4 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Attachments: SOLR-14402.patch > > > SOLR-11880 tried to do the same and it succeeded for update shard handler but > the implementation was wrong for http shard handler because the executor > created during construction is overwritten in the init() method. The commit > for SOLR-11880 is at https://github.com/apache/lucene-solr/commit/5a47ed4/ > Thanks [~caomanhdat] for spotting this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11880) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search and update operations
[
https://issues.apache.org/jira/browse/SOLR-11880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081121#comment-17081121
]
ASF subversion and git services commented on SOLR-11880:
Commit f68c498e4fb891efbd598ea2007985334c731baf in lucene-solr's branch
refs/heads/branch_8x from Shalin Shekhar Mangar
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f68c498 ]
SOLR-14402: Avoid creating new exceptions for every request made to
MDCAwareThreadPoolExecutor by distributed search.
This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which
fixed distributed updates but not distributed search.
(cherry picked from commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760)
> Avoid creating new exceptions for every request made to
> MDCAwareThreadPoolExecutor by distributed search and update operations
> --
>
> Key: SOLR-11880
> URL: https://issues.apache.org/jira/browse/SOLR-11880
> Project: Solr
> Issue Type: Bug
>Reporter: Varun Thacker
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: 7.4, 8.0
>
> Attachments: SOLR-11880.patch
>
>
> MDCAwareThreadPoolExecutor has this line in it's{{{execute}} method
>
> {code:java}
> final Exception submitterStackTrace = new Exception("Submitter stack
> trace");{code}
> This means that every call via the a thread pool will create this exception,
> and only when it sees an error will it be used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11880) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search and update operations
[
https://issues.apache.org/jira/browse/SOLR-11880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081119#comment-17081119
]
ASF subversion and git services commented on SOLR-11880:
Commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760 in lucene-solr's branch
refs/heads/master from Shalin Shekhar Mangar
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d52c102 ]
SOLR-14402: Avoid creating new exceptions for every request made to
MDCAwareThreadPoolExecutor by distributed search.
This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which
fixed distributed updates but not distributed search.
> Avoid creating new exceptions for every request made to
> MDCAwareThreadPoolExecutor by distributed search and update operations
> --
>
> Key: SOLR-11880
> URL: https://issues.apache.org/jira/browse/SOLR-11880
> Project: Solr
> Issue Type: Bug
>Reporter: Varun Thacker
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: 7.4, 8.0
>
> Attachments: SOLR-11880.patch
>
>
> MDCAwareThreadPoolExecutor has this line in it's{{{execute}} method
>
> {code:java}
> final Exception submitterStackTrace = new Exception("Submitter stack
> trace");{code}
> This means that every call via the a thread pool will create this exception,
> and only when it sees an error will it be used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14402) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search
[ https://issues.apache.org/jira/browse/SOLR-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081118#comment-17081118 ] ASF subversion and git services commented on SOLR-14402: Commit d52c1021e543b8cc2dd7b8d1d181e3dba160a760 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d52c102 ] SOLR-14402: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search. This is a fix for incomplete optimization made by SOLR-11880 in Solr 7.4 which fixed distributed updates but not distributed search. > Avoid creating new exceptions for every request made to > MDCAwareThreadPoolExecutor by distributed search > > > Key: SOLR-14402 > URL: https://issues.apache.org/jira/browse/SOLR-14402 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 7.4 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Attachments: SOLR-14402.patch > > > SOLR-11880 tried to do the same and it succeeded for update shard handler but > the implementation was wrong for http shard handler because the executor > created during construction is overwritten in the init() method. The commit > for SOLR-11880 is at https://github.com/apache/lucene-solr/commit/5a47ed4/ > Thanks [~caomanhdat] for spotting this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14402) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search
[
https://issues.apache.org/jira/browse/SOLR-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081113#comment-17081113
]
Lucene/Solr QA commented on SOLR-14402:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 46m
52s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-14402 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12999534/SOLR-14402.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 527e6516604 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/732/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/732/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Avoid creating new exceptions for every request made to
> MDCAwareThreadPoolExecutor by distributed search
>
>
> Key: SOLR-14402
> URL: https://issues.apache.org/jira/browse/SOLR-14402
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
>Affects Versions: 7.4
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Attachments: SOLR-14402.patch
>
>
> SOLR-11880 tried to do the same and it succeeded for update shard handler but
> the implementation was wrong for http shard handler because the executor
> created during construction is overwritten in the init() method. The commit
> for SOLR-11880 is at https://github.com/apache/lucene-solr/commit/5a47ed4/
> Thanks [~caomanhdat] for spotting this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9315) redefine (Classic & Standard) QueryParser semantics to be consistent: prioritize prefix op > infix op > default op
[
https://issues.apache.org/jira/browse/LUCENE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated LUCENE-9315:
-
Summary: redefine (Classic & Standard) QueryParser semantics to be
consistent: prioritize prefix op > infix op > default op (was: redfine (Classi
& Standard) QueryParser semantics to be consistent: prioritize prefix op >
infix op > default op)
> redefine (Classic & Standard) QueryParser semantics to be consistent:
> prioritize prefix op > infix op > default op
> --
>
> Key: LUCENE-9315
> URL: https://issues.apache.org/jira/browse/LUCENE-9315
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/queryparser
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: LUCENE-9315.patch
>
>
> For as long as I can remember, the way QueryParser deals with the "infix"
> operators {{AND}} & {{OR}} hasn't made much sense unless they are used
> consistently to express pure boolean logic (ie: always explicitly specified,
> and never more then 2 clauses to a query). As soon as you have query strings
> where a BooleanQuery has more then 2 clauses, or you have query strings that
> mix {{AND}} & {{OR}} with the "prefix" {{+}} & {{-|NOT}} operators, or query
> strings where not every clause has an operator, or (absolutely the most
> confusing) you mix the types of operators _and_ change the QueryParser
> "default op" from {{OR}} to {{AND}} the behavior just becomes inpossible to
> make sense of for new users - and hard to explain/justify. (It's not
> precedence based, it's not left to right, it's just ... weird.)
> The problem is so confusing to new users, that I wrote a blog post almost 10
> years ago (?!?) trying to convince people that using {{AND}} & {{OR}} was a
> terrible idea unless they were used only in strict boolean expressions)...
> [https://lucidworks.com/post/why-not-and-or-and-not/]
> ...and yet it still regularly comes up as a point of confusion.
> A lot this weird behavior seems to be historical artifact of how
> {{QueryParserBase.addClauses()}} works - a method whose basic semantics
> haven't really changed since Lucne 1.0.1, back before the introductiong of
> {{QueryParser.setDefaultOperator()}}. Some of those early choices seemed to
> be predicated on the idea that {{AND}} should take "precedence" (i use that
> term loosely) over {{OR}} as it parses clauses left to right, purely becuase
> {{OR}} was the "default" assumption (and had - and stll has - no
> corrisponding "prefix" operator). As functionality in QueryParser has grown,
> a lot of the assumptions made in the code and the resulting parse behavior
> really make no sense to users, particularly in "non trivial" query strings.
> In many cases, parse behavior that can seem "intentional" to new users, even
> for input where every clause is impacted by an explicit {{AND}} or {{OR}}
> operators, can suddenly be flipped on it's head when the "default operator"
> is changed (ex: "{{X AND Y OR Z}}"), or if the only the order of "clauses" in
> the string changes (ex: previous example vs "{{Z OR Y AND X}}") even though
> it's clear from other queries that there is no strict precedence of operators.
>
> The "root" of the problem, as I see it, is that
> {{QueryParserBase.addClauses()}} allows {{AND}} & {{OR}} to modify the
> {{Occur}} property of the previously parsed {{BooleanClause}} depending on
> _if_ that {{BooleanClause.getOccur()}} value matches the "default operator"
> for the parser, w/o any considerationg to _why_ that that {{getOccur()}}
> value matches the "default operator" - ie: did it actually come from the
> "default" or was it explicitly set by something in the query string? (ie: a
> prior infix operator)
>
> I propose that starting with Lucene 9.0, we redefine the semantics in
> {{QueryParserBase}} such that:
> * "Prefix" operators ({{+}} | {{-}} | {{NOT}}) always take precedence (over
> any "Infix" operator or QueryParser default) in setting the {{Occur}} value
> of the clause they prefix.
> * "Infix" operators ({{AND}} | {{OR}}) are evaluated left to right and used
> to set the {{Occur}} value of the clauses adjacent to them (that do not
> already have a {{Occur}} value set by a "Pefix" operator)
> * the {{QueryParser.getDefaultOperator()}} is only used to set the {{Occur}}
> value of any clause that did not get an {{Occur}} value assigned by either a
> prefix or (prior) infix operator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9315) redfine (Classi & Standard) QueryParser semantics to be consistent: prioritize prefix op > infix op > default op
[
https://issues.apache.org/jira/browse/LUCENE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081105#comment-17081105
]
David Smiley commented on LUCENE-9315:
--
+1 and you rock!
> redfine (Classi & Standard) QueryParser semantics to be consistent:
> prioritize prefix op > infix op > default op
>
>
> Key: LUCENE-9315
> URL: https://issues.apache.org/jira/browse/LUCENE-9315
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/queryparser
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: LUCENE-9315.patch
>
>
> For as long as I can remember, the way QueryParser deals with the "infix"
> operators {{AND}} & {{OR}} hasn't made much sense unless they are used
> consistently to express pure boolean logic (ie: always explicitly specified,
> and never more then 2 clauses to a query). As soon as you have query strings
> where a BooleanQuery has more then 2 clauses, or you have query strings that
> mix {{AND}} & {{OR}} with the "prefix" {{+}} & {{-|NOT}} operators, or query
> strings where not every clause has an operator, or (absolutely the most
> confusing) you mix the types of operators _and_ change the QueryParser
> "default op" from {{OR}} to {{AND}} the behavior just becomes inpossible to
> make sense of for new users - and hard to explain/justify. (It's not
> precedence based, it's not left to right, it's just ... weird.)
> The problem is so confusing to new users, that I wrote a blog post almost 10
> years ago (?!?) trying to convince people that using {{AND}} & {{OR}} was a
> terrible idea unless they were used only in strict boolean expressions)...
> [https://lucidworks.com/post/why-not-and-or-and-not/]
> ...and yet it still regularly comes up as a point of confusion.
> A lot this weird behavior seems to be historical artifact of how
> {{QueryParserBase.addClauses()}} works - a method whose basic semantics
> haven't really changed since Lucne 1.0.1, back before the introductiong of
> {{QueryParser.setDefaultOperator()}}. Some of those early choices seemed to
> be predicated on the idea that {{AND}} should take "precedence" (i use that
> term loosely) over {{OR}} as it parses clauses left to right, purely becuase
> {{OR}} was the "default" assumption (and had - and stll has - no
> corrisponding "prefix" operator). As functionality in QueryParser has grown,
> a lot of the assumptions made in the code and the resulting parse behavior
> really make no sense to users, particularly in "non trivial" query strings.
> In many cases, parse behavior that can seem "intentional" to new users, even
> for input where every clause is impacted by an explicit {{AND}} or {{OR}}
> operators, can suddenly be flipped on it's head when the "default operator"
> is changed (ex: "{{X AND Y OR Z}}"), or if the only the order of "clauses" in
> the string changes (ex: previous example vs "{{Z OR Y AND X}}") even though
> it's clear from other queries that there is no strict precedence of operators.
>
> The "root" of the problem, as I see it, is that
> {{QueryParserBase.addClauses()}} allows {{AND}} & {{OR}} to modify the
> {{Occur}} property of the previously parsed {{BooleanClause}} depending on
> _if_ that {{BooleanClause.getOccur()}} value matches the "default operator"
> for the parser, w/o any considerationg to _why_ that that {{getOccur()}}
> value matches the "default operator" - ie: did it actually come from the
> "default" or was it explicitly set by something in the query string? (ie: a
> prior infix operator)
>
> I propose that starting with Lucene 9.0, we redefine the semantics in
> {{QueryParserBase}} such that:
> * "Prefix" operators ({{+}} | {{-}} | {{NOT}}) always take precedence (over
> any "Infix" operator or QueryParser default) in setting the {{Occur}} value
> of the clause they prefix.
> * "Infix" operators ({{AND}} | {{OR}}) are evaluated left to right and used
> to set the {{Occur}} value of the clauses adjacent to them (that do not
> already have a {{Occur}} value set by a "Pefix" operator)
> * the {{QueryParser.getDefaultOperator()}} is only used to set the {{Occur}}
> value of any clause that did not get an {{Occur}} value assigned by either a
> prefix or (prior) infix operator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] andyvuong opened a new pull request #1424: Enable shared store via system property only
andyvuong opened a new pull request #1424: Enable shared store via system property only URL: https://github.com/apache/lucene-solr/pull/1424 Removing a previous [need](https://github.com/apache/lucene-solr/pull/1223) to configure shared storage feature via solr.xml and only requiring it to be enabled via passed system properties. Pass in -DsharedStoreEnabled=true to the solr binary or set env variable SHARED_STORE_ENABLED. Note sys props passed to the binary will take precedence. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14382) delegating search component
[
https://issues.apache.org/jira/browse/SOLR-14382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081057#comment-17081057
]
David Smiley commented on SOLR-14382:
-
Ehh; I'm kinda skeptical on the value it brings to add to Solr-core. At least
what you propose is simple / small. I know it may be a hollow suggestion to
suggest it as an external plugin because we don't yet have a repository for it
to be discoverable but that's where I'm leaning.
> delegating search component
> ---
>
> Key: SOLR-14382
> URL: https://issues.apache.org/jira/browse/SOLR-14382
> Project: Solr
> Issue Type: New Feature
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> An out-of-the-box search component that supports the configuration and
> selective use of custom search components.
> Illustrative solrconfig.xml snippet:
> {code:java}
> class="org.apache.solr.handler.component.QueryComponent"/>
>
>
> class="org.apache.solr.handler.component.DelegatingSearchComponent">
> queryDefault
>
> queryFoo
> queryBar
>
> QUERY
>
> {code}
> Illustrative queries:
> * for {{/select?q=\*:\*}} the default case i.e. QueryComponent is used
> * for {{/select?q=\*:\*=true}} the FooQueryComponent is used
> * for {{/select?q=\*:\*=true}} the BarQueryComponent is used
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9315) redfine (Classi & Standard) QueryParser semantics to be consistent: prioritize prefix op > infix op > default op
[
https://issues.apache.org/jira/browse/LUCENE-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter updated LUCENE-9315:
---
Attachment: LUCENE-9315.patch
Status: Open (was: Open)
I'm attaching a "strawman" patch that demonstrates an implementation of these
changes. Here's a table showing some non-trivial queries, and how they are
parsed in both the "Current" QueryParserBase implemantion, as well as the
"Proposed" implemenation with my patch...
||Input|| ||Current default=OR||Current default=AND|| ||Proposed
default=OR||Proposed default=AND||
|X AND Y OR Z| |{color:#172b4d}+x +y z{color}|{color:#172b4d}+x y z{color}| |+x
+y z|+x +y z|
|Z OR Y AND X| |z +y +x|z +y +x| |z y +x|z y +x|
|X Y OR Z| |x y z|+x y z| |x y z|+x y z|
|X Y AND Z| |x +y +z|+x +y +z| |x +y +z|+x +y +z|
|+Y OR Z| |+y z|y z| |+y z|+y z|
|+Y AND Z| |+y +z|+y +z| |+y +z|+y +z|
|Z OR +Y| |z +y|z y| |z +y|z +y|
|Z AND +Y| |+z +y|+z +y| |+z +y|+z +y|
Two things to note:
* This patch has a lot of nocommits around the fact that it "breaks"
{{BooleanClause}} to remove the requirement that {{Occur}} must be non-null --
this is purely to keep the {{QueryParserBase.java}} porrtions of hte change
simple and focusd on the _logical_ changes being made.
** If we decide to proceed with this idea, then the "correct" fix is to stop
using {{BooleanClause}} directly in the various {{QueryParserBase}} protected
methods, and instead introduce some new, equivilent, data structure for
modeling "A Query Clause with _optional_ Occur metadata"
* this change breaks only one existing test:
{{QueryParserTestBase.testPrecedence()}} -- which nominally tests that {{"A AND
B OR C AND D"}} parses equivilently to {{"+A +B +C +D"}}, but this test only
currently passes whhen {{QueryParser.getDefaultOperator()}} returns {{OR}} ...
in the case of {{QueryParser.setDefaultOperator(AND)}} the {{" OR "}} in that
input is effecitvely ignored.
** Wih the patch, the test is updated to consistently expect this (effectively
"fully qualified") input string to be parsed as {{"+A +B C +D"}} (ie: left to
right) regardless of what the default operator is.
** BUT: These test changes break
{{oal.queryparser.flexible.standard.TestStandardQP.testPrecedence}} ... IIUC
this is testing that the flexible {{StandardSyntaxParser}} is equivilent to
Classic {{QueryParser}} -- I haven't dug into this much, but i think if we want
to move forward with these semantic changes to {{QueryParser}} it just means
some equivilent changes to the grammer and/or node builders for
{{StandardSyntaxParser}} (or maybe just {{BooleanQuery2ModifierNodeProcessor}}
?)
What do folks think of persuing this change, knowing it's a back compat break?
> redfine (Classi & Standard) QueryParser semantics to be consistent:
> prioritize prefix op > infix op > default op
>
>
> Key: LUCENE-9315
> URL: https://issues.apache.org/jira/browse/LUCENE-9315
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/queryparser
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: LUCENE-9315.patch
>
>
> For as long as I can remember, the way QueryParser deals with the "infix"
> operators {{AND}} & {{OR}} hasn't made much sense unless they are used
> consistently to express pure boolean logic (ie: always explicitly specified,
> and never more then 2 clauses to a query). As soon as you have query strings
> where a BooleanQuery has more then 2 clauses, or you have query strings that
> mix {{AND}} & {{OR}} with the "prefix" {{+}} & {{-|NOT}} operators, or query
> strings where not every clause has an operator, or (absolutely the most
> confusing) you mix the types of operators _and_ change the QueryParser
> "default op" from {{OR}} to {{AND}} the behavior just becomes inpossible to
> make sense of for new users - and hard to explain/justify. (It's not
> precedence based, it's not left to right, it's just ... weird.)
> The problem is so confusing to new users, that I wrote a blog post almost 10
> years ago (?!?) trying to convince people that using {{AND}} & {{OR}} was a
> terrible idea unless they were used only in strict boolean expressions)...
> [https://lucidworks.com/post/why-not-and-or-and-not/]
> ...and yet it still regularly comes up as a point of confusion.
> A lot this weird behavior seems to be historical artifact of how
> {{QueryParserBase.addClauses()}} works - a method whose basic semantics
> haven't really changed since Lucne 1.0.1, back before the introductiong of
> {{QueryParser.setDefaultOperator()}}. Some of those early choices seemed to
> be predicated on the idea that {{AND}} should take "precedence" (i use that
> term loosely) over {{OR}} as it parses
[jira] [Created] (LUCENE-9315) redfine (Classi & Standard) QueryParser semantics to be consistent: prioritize prefix op > infix op > default op
Chris M. Hostetter created LUCENE-9315:
--
Summary: redfine (Classi & Standard) QueryParser semantics to be
consistent: prioritize prefix op > infix op > default op
Key: LUCENE-9315
URL: https://issues.apache.org/jira/browse/LUCENE-9315
Project: Lucene - Core
Issue Type: Improvement
Components: core/queryparser
Reporter: Chris M. Hostetter
For as long as I can remember, the way QueryParser deals with the "infix"
operators {{AND}} & {{OR}} hasn't made much sense unless they are used
consistently to express pure boolean logic (ie: always explicitly specified,
and never more then 2 clauses to a query). As soon as you have query strings
where a BooleanQuery has more then 2 clauses, or you have query strings that
mix {{AND}} & {{OR}} with the "prefix" {{+}} & {{-|NOT}} operators, or query
strings where not every clause has an operator, or (absolutely the most
confusing) you mix the types of operators _and_ change the QueryParser "default
op" from {{OR}} to {{AND}} the behavior just becomes inpossible to make sense
of for new users - and hard to explain/justify. (It's not precedence based,
it's not left to right, it's just ... weird.)
The problem is so confusing to new users, that I wrote a blog post almost 10
years ago (?!?) trying to convince people that using {{AND}} & {{OR}} was a
terrible idea unless they were used only in strict boolean expressions)...
[https://lucidworks.com/post/why-not-and-or-and-not/]
...and yet it still regularly comes up as a point of confusion.
A lot this weird behavior seems to be historical artifact of how
{{QueryParserBase.addClauses()}} works - a method whose basic semantics haven't
really changed since Lucne 1.0.1, back before the introductiong of
{{QueryParser.setDefaultOperator()}}. Some of those early choices seemed to be
predicated on the idea that {{AND}} should take "precedence" (i use that term
loosely) over {{OR}} as it parses clauses left to right, purely becuase {{OR}}
was the "default" assumption (and had - and stll has - no corrisponding
"prefix" operator). As functionality in QueryParser has grown, a lot of the
assumptions made in the code and the resulting parse behavior really make no
sense to users, particularly in "non trivial" query strings. In many cases,
parse behavior that can seem "intentional" to new users, even for input where
every clause is impacted by an explicit {{AND}} or {{OR}} operators, can
suddenly be flipped on it's head when the "default operator" is changed (ex:
"{{X AND Y OR Z}}"), or if the only the order of "clauses" in the string
changes (ex: previous example vs "{{Z OR Y AND X}}") even though it's clear
from other queries that there is no strict precedence of operators.
The "root" of the problem, as I see it, is that
{{QueryParserBase.addClauses()}} allows {{AND}} & {{OR}} to modify the
{{Occur}} property of the previously parsed {{BooleanClause}} depending on _if_
that {{BooleanClause.getOccur()}} value matches the "default operator" for the
parser, w/o any considerationg to _why_ that that {{getOccur()}} value matches
the "default operator" - ie: did it actually come from the "default" or was it
explicitly set by something in the query string? (ie: a prior infix operator)
I propose that starting with Lucene 9.0, we redefine the semantics in
{{QueryParserBase}} such that:
* "Prefix" operators ({{+}} | {{-}} | {{NOT}}) always take precedence (over
any "Infix" operator or QueryParser default) in setting the {{Occur}} value of
the clause they prefix.
* "Infix" operators ({{AND}} | {{OR}}) are evaluated left to right and used to
set the {{Occur}} value of the clauses adjacent to them (that do not already
have a {{Occur}} value set by a "Pefix" operator)
* the {{QueryParser.getDefaultOperator()}} is only used to set the {{Occur}}
value of any clause that did not get an {{Occur}} value assigned by either a
prefix or (prior) infix operator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-12301) Umbrella issue: paramaterize logging calls in Solr, use consistent naming conventions for the logger
[
https://issues.apache.org/jira/browse/SOLR-12301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson updated SOLR-12301:
--
Description:
See the discussion at SOLR-12286 for a lot of background, but the short form is
that logging calls of the form
log.info("somehting" + "something");
and
log.info("soemthing {}", object.someFunction());
where someFunction includes toString()
generate useless garbage/work even when the message is not printed.
log.info("somehting {}", "something");
and
log.info("soemthing {}", object);
do not. The first form is something of a relic, and there are even some uses of
the second.
This will entail a LOT of changes, but almost all secretarial. I'll take it in
chunks.
was:
See the discussion at SOLR-12286 for a lot of background, but the short form is
that logging calls of the form
log.info("somehting" + "something");
and
log.info("soemthing {}", object.toString());
generate useless garbage even when logging at a more restrictive level (say
WARN), whereas
log.info("somehting {}", "something");
and
log.infl("soemthing {}", object);
do not. The first form is something of a relic, and there are even some uses of
the second.
So, let's tackle subsets of the source code as subordinate JIRAs
> Umbrella issue: paramaterize logging calls in Solr, use consistent naming
> conventions for the logger
>
>
> Key: SOLR-12301
> URL: https://issues.apache.org/jira/browse/SOLR-12301
> Project: Solr
> Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> See the discussion at SOLR-12286 for a lot of background, but the short form
> is that logging calls of the form
> log.info("somehting" + "something");
> and
> log.info("soemthing {}", object.someFunction());
> where someFunction includes toString()
> generate useless garbage/work even when the message is not printed.
> log.info("somehting {}", "something");
> and
> log.info("soemthing {}", object);
> do not. The first form is something of a relic, and there are even some uses
> of the second.
> This will entail a LOT of changes, but almost all secretarial. I'll take it
> in chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6980) Collection deletion, creation and shared configsets
[ https://issues.apache.org/jira/browse/SOLR-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-6980. -- Resolution: Won't Fix Thinking about this, I think it's a bad idea to remove configsets. We routinely ask people to delete a collection and reindex, say to add docValues=true or when upgrading a major version. One way people do this is to delete the collection and start over. If we deleted the configset at that point they'd lose their old schema. The configsets api will allow these to be cleaned up without having to use zkCli or the like so I think it's best to not do anything. If anyone wants to pick this up, feel free to reopen. > Collection deletion, creation and shared configsets > --- > > Key: SOLR-6980 > URL: https://issues.apache.org/jira/browse/SOLR-6980 > Project: Solr > Issue Type: Improvement >Reporter: Grant Ingersoll >Assignee: Erick Erickson >Priority: Major > > If a configset is not being shared and a collection is deleted, I think we > should also delete the configset, or at least the copy of it for that > collection. > You can see the ill effects of this by doing: > # create a data-driven schema collection > # index some content to it, thus materializing an actual schema > # delete the collection > # create a new collection w/ the same name > # observe that the new collection has the old materialized schema -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12963) change default for 'uninvertible' to 'false' (dependent on new schema 'version')
[
https://issues.apache.org/jira/browse/SOLR-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080997#comment-17080997
]
Erick Erickson commented on SOLR-12963:
---
Random thought here. Defaulting to false will surely surprise people, however
it would just be a schema change w/o any requirement to re-index. So while
perhaps a surprise, it's not that onerous to change the schema and reload the
collection.
> change default for 'uninvertible' to 'false' (dependent on new schema
> 'version')
>
>
> Key: SOLR-12963
> URL: https://issues.apache.org/jira/browse/SOLR-12963
> Project: Solr
> Issue Type: Sub-task
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-12963.patch
>
>
> We should consider changing the default behavior of the {{uninvertible}}
> field option to be dependnt on the schema {{version}} property, such that
> moving forward the fields/fieldtypes will default to {{uninvertible ==
> false}} unless an explicit {{uninvertible=true}} is specified by the user.
> There are a lot of considerations regarding the existing behavior of
> functionality (like faceting) when the (effective) value of {{uninvertible}}
> is false because we move forward with changing this in a way that could
> suprise/confuse new users or existing users w/ long heald expectations that
> certain behavior would just "work" w/o understanding that was because of
> FieldCache/uninversion.
> See parent issue for more background/discussion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14388) Remove references to rule-based replica placement in the ref guide
[ https://issues.apache.org/jira/browse/SOLR-14388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-14388. --- Resolution: Won't Fix Cassandra's comments make sense to me... > Remove references to rule-based replica placement in the ref guide > -- > > Key: SOLR-14388 > URL: https://issues.apache.org/jira/browse/SOLR-14388 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation, SolrCloud >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > There are lots of references to rule-based replica placement in the ref > guide, but it's being deprecated (although that effort seems to be stalled). > We should at least direct people to using a Policy before the rule-based > placement code is removed. > I'll put up a patch shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13132) Improve JSON "terms" facet performance when sorted by relatedness
[
https://issues.apache.org/jira/browse/SOLR-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080937#comment-17080937
]
Michael Gibney commented on SOLR-13132:
---
Yesterday and just now, pushed changes that I think address all the points
you've brought up.
# Removed the "XXX temporary!" comment. This related code is still important,
so the comment was misleading
# "{{CountSlotAccFactory}} interface seems like dead code" – it's providing a
hook that will be used e.g. with the introduction of caching. Could indeed be
removed at this point, but I've left it in for now since there was a lot of
other more substantial refactoring and I wanted to make sure I didn't
inadvertently do something that would make this harder to reintroduce later.
# ReadOnlyCountSlotAcc is now an interface, not an abstract class w/ wrapper
implementation. Good point :)
# {{FilterCtStruct}} and {{SweepCountAccStruct}} were indeed essentially
redundant. Removed the former.
# Regarding "brittleness" all great suggestions. I basically tried to follow
what you outlined as the "'cleaner' approach"; for now instead of adding
{{registerSweepingAccs(...)}} directly to {{SlotAcc}} I initially left it as a
new interface (processors checking by {{instanceof}}). Honestly I did this b/c
I missed that you suggested adding the new method directly to {{SlotAcc}}.
Perfectly happy to dispense with the special interface/instanceof checks and
put it directly in {{SlotAcc}} of course!
# Acted on advice re: explicitness of semantics for requesting sweep
collection. Changed "disable_sweep_collection" (default "false" + confusing
heuristics) to "sweep_collection", defaulting to "true" and acted upon only if
supported by all components. (TODO: I'm pretty sure I haven't properly gotten
the final chosen method into the facet debug output yet).
# I _think_ I acted on all the advice about the weirdness of returning new
arrays, placing the "base" collector in the final array position, etc. That did
make things nicer I think. (related topic: it really is important to keep track
of which is the base domain, b/c we're sweeping (for purposes of count
collection) over a union domain, only some of which correspond to the base
domain, which is the only domain for which subsequent collection may be
necessary in the event that {{collectAcc!=null}}).
The main other change that followed as a consequence of this work is that I
think it's necessary (if we may now replace {{collectAcc}} with null when full
domain collection can be accomplished via sweep count collection only) to
explicitly separate the read-access to full-domain SlotAccs so that output can
be handled appropriately. Formerly, collectAcc served for both collection _and_
read-access for such SlotAccs, but the 1:1 correspondence that made that work
is no longer a valid assumption; and can't use the other existing references
that get read for output ({{accs}}, {{otherAccs}}, {{deferredAggs}}, etc.) b/c
they're handled quite differently (on a 1-slot, 1-bucket-at-a-time a la carte
way). I could be missing something here, but in any event that's what explains
the introduction of the new {{FacetFieldProcessor.fullDomainAccs}} field.
I have not yet incorporated the testing patch that you provided above, but I
wanted to post what I have now, since I _think_ it's fairly complete (pending
additional testing!).
> Improve JSON "terms" facet performance when sorted by relatedness
> --
>
> Key: SOLR-13132
> URL: https://issues.apache.org/jira/browse/SOLR-13132
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 7.4, master (9.0)
>Reporter: Michael Gibney
>Priority: Major
> Attachments: SOLR-13132-with-cache-01.patch,
> SOLR-13132-with-cache.patch, SOLR-13132.patch, SOLR-13132_testSweep.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When sorting buckets by {{relatedness}}, JSON "terms" facet must calculate
> {{relatedness}} for every term.
> The current implementation uses a standard uninverted approach (either
> {{docValues}} or {{UnInvertedField}}) to get facet counts over the domain
> base docSet, and then uses that initial pass as a pre-filter for a
> second-pass, inverted approach of fetching docSets for each relevant term
> (i.e., {{count > minCount}}?) and calculating intersection size of those sets
> with the domain base docSet.
> Over high-cardinality fields, the overhead of per-term docSet creation and
> set intersection operations increases request latency to the point where
> relatedness sort may not be usable in practice (for my use case, even after
> applying the patch for SOLR-13108, for a field with ~220k unique terms per
> core, QTime for high-cardinality domain
[jira] [Commented] (LUCENE-9298) Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared
[
https://issues.apache.org/jira/browse/LUCENE-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080894#comment-17080894
]
ASF subversion and git services commented on LUCENE-9298:
-
Commit 527e6516604e0f7ded72d8ac6f7b6b79884b62c5 in lucene-solr's branch
refs/heads/master from Nhat Nguyen
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=527e651 ]
LUCENE-9298: Fix TestBufferedUpdates
This test failed on Elastic CI because we did not add any term in the
loop. This commit ensures that we always add at least one docId, term
and query in the test.
> Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are
> cleared
>
>
> Key: LUCENE-9298
> URL: https://issues.apache.org/jira/browse/LUCENE-9298
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.5
>Reporter: yubinglei
>Assignee: Simon Willnauer
>Priority: Minor
> Labels: easyfix, newbie, pull-request-available
> Fix For: master (9.0), 8.6
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> As mentioned in this
> [https://github.com/apache/lucene-solr/pull/513#discussion_r399131681]
> the method clearDeletedDocIds in BufferedUpdates.java has a bug, it can't
> reset bytesUsed correctly
> {code:java}
> void clearDeletedDocIds() {
> deleteDocIDs.clear();
> bytesUsed.addAndGet(-deleteDocIDs.size() *
> BufferedUpdates.BYTES_PER_DEL_DOCID);
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9298) Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared
[
https://issues.apache.org/jira/browse/LUCENE-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080893#comment-17080893
]
ASF subversion and git services commented on LUCENE-9298:
-
Commit 1eef78d5f89eac07fca23df9a35cf016f420112d in lucene-solr's branch
refs/heads/branch_8x from Nhat Nguyen
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1eef78d ]
LUCENE-9298: Fix TestBufferedUpdates
This test failed on Elastic CI because we did not add any term in the
loop. This commit ensures that we always add at least one docId, term
and query in the test.
> Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are
> cleared
>
>
> Key: LUCENE-9298
> URL: https://issues.apache.org/jira/browse/LUCENE-9298
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.5
>Reporter: yubinglei
>Assignee: Simon Willnauer
>Priority: Minor
> Labels: easyfix, newbie, pull-request-available
> Fix For: master (9.0), 8.6
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> As mentioned in this
> [https://github.com/apache/lucene-solr/pull/513#discussion_r399131681]
> the method clearDeletedDocIds in BufferedUpdates.java has a bug, it can't
> reset bytesUsed correctly
> {code:java}
> void clearDeletedDocIds() {
> deleteDocIDs.clear();
> bytesUsed.addAndGet(-deleteDocIDs.size() *
> BufferedUpdates.BYTES_PER_DEL_DOCID);
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9285) Transient test failure in TestAddIndexes.testAddIndexesWithRollback
[
https://issues.apache.org/jira/browse/LUCENE-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless resolved LUCENE-9285.
Resolution: Fixed
> Transient test failure in TestAddIndexes.testAddIndexesWithRollback
> ---
>
> Key: LUCENE-9285
> URL: https://issues.apache.org/jira/browse/LUCENE-9285
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Michael McCandless
>Priority: Major
>
> Alas does not seem to reproduce for me:
> {noformat}
> org.apache.lucene.index.TestAddIndexes > test suite's output saved to
> /home/mike/src/lfd/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.inde\
> x.TestAddIndexes.txt, copied below:
> > java.nio.file.NoSuchFileException:
> _5j_Lucene85FieldsIndexfile_pointers_15.tmp
> > at
> __randomizedtesting.SeedInfo.seed([8511E1AF630BC270:63360E40C7A4A90E]:0)
> > at
> org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
> > at
> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:602)
> > at
> org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
> > at
> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
> > at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
> > at
> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:449)
> > at
> org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2334)
> > at
> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2275)
> > at
> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2268)
> > at
> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:974)
> > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> > at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> > at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
>
[jira] [Commented] (LUCENE-9285) Transient test failure in TestAddIndexes.testAddIndexesWithRollback
[
https://issues.apache.org/jira/browse/LUCENE-9285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080645#comment-17080645
]
Michael McCandless commented on LUCENE-9285:
I suspect [~simonw] just fixed this with this commit:
https://github.com/apache/lucene-solr/commit/e376582e25d02f85b415eb1c0456f3fdc800fc84
> Transient test failure in TestAddIndexes.testAddIndexesWithRollback
> ---
>
> Key: LUCENE-9285
> URL: https://issues.apache.org/jira/browse/LUCENE-9285
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Michael McCandless
>Priority: Major
>
> Alas does not seem to reproduce for me:
> {noformat}
> org.apache.lucene.index.TestAddIndexes > test suite's output saved to
> /home/mike/src/lfd/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.inde\
> x.TestAddIndexes.txt, copied below:
> > java.nio.file.NoSuchFileException:
> _5j_Lucene85FieldsIndexfile_pointers_15.tmp
> > at
> __randomizedtesting.SeedInfo.seed([8511E1AF630BC270:63360E40C7A4A90E]:0)
> > at
> org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
> > at
> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:602)
> > at
> org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
> > at
> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
> > at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
> > at
> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:449)
> > at
> org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2334)
> > at
> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2275)
> > at
> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2268)
> > at
> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:974)
> > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> > at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> > at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> > at
>
[jira] [Commented] (LUCENE-9314) Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when matching a single document
[ https://issues.apache.org/jira/browse/LUCENE-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080640#comment-17080640 ] Alan Woodward commented on LUCENE-9314: --- Good catch! I think a simpler fix would be to check the length of the array in `DocumentBatch.of(Analyzer, Document...)` and if it contains only one doc then return an `SingletonDocumentBatch`. Could you add a test as well? > Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when > matching a single document > --- > > Key: LUCENE-9314 > URL: https://issues.apache.org/jira/browse/LUCENE-9314 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/other >Affects Versions: 8.2, 8.3, 8.4, 8.5 >Reporter: Pierre-Luc Perron >Priority: Minor > Labels: easyfix, performance, pull-request-available > Attachments: LUCENE-9314.patch, LUCENE-9314.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Lucene monitor function, [match single > document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276], > wraps a single document into a array of documents. Hence, it always calls > the function, [match many > documents|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L256], > which builds a > [MultiDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L56] > rather than a > [SingletonDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L46]. > The former uses ByteBuffersDirectory while later uses MemoryIndex. > As per documentation, > [MemoryIndex|https://lucene.apache.org/core/8_2_0/memory/index.html] is a > high-performance single-document main memory Apache Lucene fulltext search > index. Hence, Lucene monitor should use it when matching a single document. > The patch routes [match single > document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276] > to a SingletonDocumentBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9314) Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when matching a single document
[ https://issues.apache.org/jira/browse/LUCENE-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Luc Perron updated LUCENE-9314: -- Attachment: LUCENE-9314.patch > Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when > matching a single document > --- > > Key: LUCENE-9314 > URL: https://issues.apache.org/jira/browse/LUCENE-9314 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/other >Affects Versions: 8.2, 8.3, 8.4, 8.5 >Reporter: Pierre-Luc Perron >Priority: Minor > Labels: easyfix, performance, pull-request-available > Attachments: LUCENE-9314.patch, LUCENE-9314.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Lucene monitor function, [match single > document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276], > wraps a single document into a array of documents. Hence, it always calls > the function, [match many > documents|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L256], > which builds a > [MultiDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L56] > rather than a > [SingletonDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L46]. > The former uses ByteBuffersDirectory while later uses MemoryIndex. > As per documentation, > [MemoryIndex|https://lucene.apache.org/core/8_2_0/memory/index.html] is a > high-performance single-document main memory Apache Lucene fulltext search > index. Hence, Lucene monitor should use it when matching a single document. > The patch routes [match single > document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276] > to a SingletonDocumentBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly
mikemccand commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly URL: https://github.com/apache/lucene-solr/pull/1397#issuecomment-612095846 ++! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly
s1monw commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly URL: https://github.com/apache/lucene-solr/pull/1397#issuecomment-612064206 @mikemccand I will go ahead and merge this! thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9314) Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when matching a single document
[ https://issues.apache.org/jira/browse/LUCENE-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Luc Perron updated LUCENE-9314: -- Description: Lucene monitor function, [match single document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276], wraps a single document into a array of documents. Hence, it always calls the function, [match many documents|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L256], which builds a [MultiDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L56] rather than a [SingletonDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L46]. The former uses ByteBuffersDirectory while later uses MemoryIndex. As per documentation, [MemoryIndex|https://lucene.apache.org/core/8_2_0/memory/index.html] is a high-performance single-document main memory Apache Lucene fulltext search index. Hence, Lucene monitor should use it when matching a single document. The patch routes [match single document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276] to a SingletonDocumentBatch. was: Lucene monitor function, [match single document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276], wraps a single document into a array of documents. Hence, it always calls the function, [match many documents|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L256], which build a [MultiDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L56] rather than a [SingletonDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L46]. The former uses ByteBuffersDirectory while later uses MemoryIndex. As per documentation, [MemoryIndex|https://lucene.apache.org/core/8_2_0/memory/index.html] is a high-performance single-document main memory Apache Lucene fulltext search index. Hence, Lucene monitor should use it when matching a single document. The patch routes [match single document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276] to a SingletonDocumentBatch. > Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when > matching a single document > --- > > Key: LUCENE-9314 > URL: https://issues.apache.org/jira/browse/LUCENE-9314 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/other >Affects Versions: 8.2, 8.3, 8.4, 8.5 >Reporter: Pierre-Luc Perron >Priority: Minor > Labels: easyfix, performance, pull-request-available > Attachments: LUCENE-9314.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Lucene monitor function, [match single > document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276], > wraps a single document into a array of documents. Hence, it always calls > the function, [match many > documents|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L256], > which builds a > [MultiDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L56] > rather than a > [SingletonDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L46]. > The former uses ByteBuffersDirectory while later uses MemoryIndex. > As per documentation, > [MemoryIndex|https://lucene.apache.org/core/8_2_0/memory/index.html] is a > high-performance single-document main memory Apache Lucene fulltext search > index. Hence, Lucene monitor should use it when matching a single document. > The patch routes
[GitHub] [lucene-solr] plperron opened a new pull request #1423: LUCENE-9314: Match a document with MemoryIndex on LuceneMonitor
plperron opened a new pull request #1423: LUCENE-9314: Match a document with MemoryIndex on LuceneMonitor URL: https://github.com/apache/lucene-solr/pull/1423 see [LUCENE-9314](https://issues.apache.org/jira/browse/LUCENE-9314) # Description Lucene monitor function, match single document, wraps a single document into a array of documents. Hence, it always calls the function, match many documents, which build a MultiDocumentBatch rather than a SingletonDocumentBatch. The former uses ByteBuffersDirectory while later uses MemoryIndex. As per documentation, MemoryIndex is a high-performance single-document main memory Apache Lucene fulltext search index. Hence, Lucene monitor should use it when matching a single document. # Solution The solution is to call DocumentBatch.of with a single document rather than an array of document. # Tests The feature was already there but seems to have been overlook when moving the original project [Luwak](https://github.com/flaxsearch/luwak) to Lucene. The tests still pass. # Checklist Please review the following and check all that apply: - [X] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [X] I have created a Jira issue and added the issue ID to my pull request title. - [X] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [X] I have developed this patch against the `master` branch. - [X] I have run `ant precommit` and the appropriate test suite. - [X] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly
mikemccand commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly URL: https://github.com/apache/lucene-solr/pull/1397#issuecomment-612056038 > I assume this time the limited resource are the hands using the keyboard :) Indeed! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9314) Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when matching a single document
[ https://issues.apache.org/jira/browse/LUCENE-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre-Luc Perron updated LUCENE-9314: -- Attachment: LUCENE-9314.patch > Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when > matching a single document > --- > > Key: LUCENE-9314 > URL: https://issues.apache.org/jira/browse/LUCENE-9314 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/other >Affects Versions: 8.2, 8.3, 8.4, 8.5 >Reporter: Pierre-Luc Perron >Priority: Minor > Labels: easyfix, performance, pull-request-available > Attachments: LUCENE-9314.patch > > > Lucene monitor function, [match single > document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276], > wraps a single document into a array of documents. Hence, it always calls > the function, [match many > documents|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L256], > which build a > [MultiDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L56] > rather than a > [SingletonDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L46]. > The former uses ByteBuffersDirectory while later uses MemoryIndex. > As per documentation, > [MemoryIndex|https://lucene.apache.org/core/8_2_0/memory/index.html] is a > high-performance single-document main memory Apache Lucene fulltext search > index. Hence, Lucene monitor should use it when matching a single document. > The patch routes [match single > document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276] > to a SingletonDocumentBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly
mikemccand commented on issue #1397: LUCENE-9304: Refactor DWPTPool to pool DWPT directly URL: https://github.com/apache/lucene-solr/pull/1397#issuecomment-612055874 OK indexing throughput is noisy, but net/net I think there's no perf impact. This is `wikimediumall` from `luceneutil` on 128 core box, JDK 11: ``` /l/logs/simon1.log:Indexer: 223.42653129383794 GB/hour plain text /l/logs/simon2.log:Indexer: 264.0324599263973 GB/hour plain text /l/logs/simon3.log:Indexer: 215.7334958611665 GB/hour plain text /l/logs/trunk1.log:Indexer: 209.16003685553957 GB/hour plain text /l/logs/trunk2.log:Indexer: 210.4946120847491 GB/hour plain text /l/logs/trunk3.log:Indexer: 231.340039364 GB/hour plain text ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9314) Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when matching a single document
Pierre-Luc Perron created LUCENE-9314: - Summary: Lucene monitor module uses ByteBuffersDirectory rather than MemoryIndex when matching a single document Key: LUCENE-9314 URL: https://issues.apache.org/jira/browse/LUCENE-9314 Project: Lucene - Core Issue Type: Improvement Components: modules/other Affects Versions: 8.5, 8.4, 8.3, 8.2 Reporter: Pierre-Luc Perron Lucene monitor function, [match single document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276], wraps a single document into a array of documents. Hence, it always calls the function, [match many documents|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L256], which build a [MultiDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L56] rather than a [SingletonDocumentBatch|https://github.com/apache/lucene-solr/blob/71d335ff688982cef10a648c914623c81ced/lucene/monitor/src/java/org/apache/lucene/monitor/DocumentBatch.java#L46]. The former uses ByteBuffersDirectory while later uses MemoryIndex. As per documentation, [MemoryIndex|https://lucene.apache.org/core/8_2_0/memory/index.html] is a high-performance single-document main memory Apache Lucene fulltext search index. Hence, Lucene monitor should use it when matching a single document. The patch routes [match single document|https://github.com/apache/lucene-solr/blob/e376582e25d02f85b415eb1c0456f3fdc800fc84/lucene/monitor/src/java/org/apache/lucene/monitor/Monitor.java#L276] to a SingletonDocumentBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080525#comment-17080525
]
Dustin Haver commented on LUCENE-9221:
--
[~jeremynovey] Got it. Thanks for the clarification. [link
title|http://example.com] !Screen Shot 2020-04-10 at 8.29.32 AM.png!
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, image-2020-04-10-07-04-00-267.png,
> image.png, lucene-invert-a.png, zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dustin Haver updated LUCENE-9221:
-
Attachment: Screen Shot 2020-04-10 at 8.29.32 AM.png
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, Screen Shot 2020-04-10 at 8.29.32
> AM.png, image-2020-04-10-07-04-00-267.png, image.png, lucene-invert-a.png,
> zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14345) Error messages are not properly propagated with non-default response parsers
[
https://issues.apache.org/jira/browse/SOLR-14345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-14345:
Attachment: SOLR-14345.patch
> Error messages are not properly propagated with non-default response parsers
>
>
> Key: SOLR-14345
> URL: https://issues.apache.org/jira/browse/SOLR-14345
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Munendra S N
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-14345.patch, SOLR-14345.patch, SOLR-14345.patch,
> SOLR-14345.patch
>
>
> Default {{ResponsParseer}} is {{BinaryResponseParser}}. when non-default
> response parser is specified in the request then, the error message is
> propagated to user. This happens in solrCloud mode.
> I came across this problem when working on adding some test which uses
> {{SolrTestCaseHS}} but similar problem exists with SolrJ client
> Also, same problem exists in both HttpSolrClient and Http2SolrClient
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-11775) json.facet can use inconsistent Long/Integer for "count" depending on shard count
[
https://issues.apache.org/jira/browse/SOLR-11775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Munendra S N updated SOLR-11775:
Fix Version/s: master (9.0)
Resolution: Fixed
Status: Resolved (was: Patch Available)
> json.facet can use inconsistent Long/Integer for "count" depending on shard
> count
> -
>
> Key: SOLR-11775
> URL: https://issues.apache.org/jira/browse/SOLR-11775
> Project: Solr
> Issue Type: Bug
> Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Fix For: master (9.0)
>
> Attachments: SOLR-11775.patch, SOLR-11775.patch
>
>
> (NOTE: I noticed this while working on a test for {{type: range}} but it's
> possible other facet types may be affected as well)
> When dealing with a single core request -- either standalone or a collection
> with only one shard -- json.facet seems to use "Integer" objects to return
> the "count" of facet buckets, however if the shard count is increased then
> the end client gets a "Long" object for the "count"
> (This isn't noticable when using {{wt=json}} but can be very problematic when
> trying to write client code using {{wt=xml}} or SolrJ
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-11775) json.facet can use inconsistent Long/Integer for "count" depending on shard count
[
https://issues.apache.org/jira/browse/SOLR-11775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080506#comment-17080506
]
ASF subversion and git services commented on SOLR-11775:
Commit 36b280bd0a21952ea54c7567f037eb48dc93205a in lucene-solr's branch
refs/heads/master from Munendra S N
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=36b280b ]
SOLR-11775: return long val for facet count in json facet
* Long value is returned for any count related to json facets
irrespective of number of shards
> json.facet can use inconsistent Long/Integer for "count" depending on shard
> count
> -
>
> Key: SOLR-11775
> URL: https://issues.apache.org/jira/browse/SOLR-11775
> Project: Solr
> Issue Type: Bug
> Components: Facet Module
>Reporter: Chris M. Hostetter
>Assignee: Munendra S N
>Priority: Major
> Attachments: SOLR-11775.patch, SOLR-11775.patch
>
>
> (NOTE: I noticed this while working on a test for {{type: range}} but it's
> possible other facet types may be affected as well)
> When dealing with a single core request -- either standalone or a collection
> with only one shard -- json.facet seems to use "Integer" objects to return
> the "count" of facet buckets, however if the shard count is increased then
> the end client gets a "Long" object for the "count"
> (This isn't noticable when using {{wt=json}} but can be very problematic when
> trying to write client code using {{wt=xml}} or SolrJ
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14365) CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values
[
https://issues.apache.org/jira/browse/SOLR-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080492#comment-17080492
]
ASF subversion and git services commented on SOLR-14365:
Commit 71d335ff688982cef10a648c914623c81ced in lucene-solr's branch
refs/heads/master from Cao Manh Dat
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=71d335f ]
SOLR-14365: Automatically grow size of groupHeadValues
> CollapsingQParser - Avoiding always allocate int[] and float[] with size
> equals to number of unique values
> --
>
> Key: SOLR-14365
> URL: https://issues.apache.org/jira/browse/SOLR-14365
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.4.1
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-14365.patch
>
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Since Collapsing is a PostFilter, documents reach Collapsing must match with
> all filters and queries, so the number of documents Collapsing need to
> collect/compute score is a small fraction of the total number documents in
> the index. So why do we need to always consume the memory (for int[] and
> float[] array) for all unique values of the collapsed field? If the number of
> unique values of the collapsed field found in the documents that match
> queries and filters is 300 then we only need int[] and float[] array with
> size of 300 and not 1.2 million in size. However, we don't know which value
> of the collapsed field will show up in the results so we cannot use a smaller
> array.
> The easy fix for this problem is using as much as we need by using IntIntMap
> and IntFloatMap that hold primitives and are much more space efficient than
> the Java HashMap. These maps can be slower (10x or 20x) than plain int[] and
> float[] if matched documents is large (almost all documents matched queries
> and other filters). But our belief is that does not happen that frequently
> (how frequently do we run collapsing on the entire index?).
> For this issue I propose adding 2 methods for collapsing which is
> * array : which is current implementation
> * hash : which is new approach and will be default method
> later we can add another method {{smart}} which is automatically pick method
> based on comparision between {{number of docs matched queries and filters}}
> and {{number of unique values of the field}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080491#comment-17080491
]
Jeremy Novey commented on LUCENE-9221:
--
Thanks, [~DHAVER0313]. Apologies if I wasn't being clear about the bottom
triangle. I was thinking more 'A' shaped (for Apache). Example attached.
!lucene-invert-a.png!
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, image-2020-04-10-07-04-00-267.png,
> image.png, lucene-invert-a.png, zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremy Novey updated LUCENE-9221:
-
Attachment: lucene-invert-a.png
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, image-2020-04-10-07-04-00-267.png,
> image.png, lucene-invert-a.png, zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-11901) Improve how logging collects class name information
[
https://issues.apache.org/jira/browse/SOLR-11901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson resolved SOLR-11901.
---
Fix Version/s: 7.4
Resolution: Done
The problem statement is not accurate.
%c (lowercase) does _not_ create a Throwable
%C (uppercase) _does_ generate a Throwable and is the expensive one, and we
don't have any of those any more in the Pattern Layouts.
> Improve how logging collects class name information
> ---
>
> Key: SOLR-11901
> URL: https://issues.apache.org/jira/browse/SOLR-11901
> Project: Solr
> Issue Type: Improvement
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Minor
> Fix For: 7.4
>
>
> The log4j.properties that we ship with Solr has this Pattern
> {code}
> %d{-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica}
> %X{core}] %c{1.} %m%n
> {code}
> The {{%c}} collects class name information ( more on this
> http://logging.apache.org/log4j/2.x/manual/async.html#Location ) which
> creates a throwable (
> https://github.com/apache/log4j/blob/trunk/src/main/java/org/apache/log4j/spi/LoggingEvent.java#L253
> ) and it can be expensive
> Here is the stack trace excerpt from the JFR capture which lead to this issue
> {code}
> org.apache.log4j.spi.LoggingEvent.getLocationInformation()
> org.apache.log4j.helpers.PatternParser$ClassNamePatternConverter.getFullyQualifiedName(LoggingEvent)
> org.apache.log4j.helpers.PatternParser$NamedPatternConverter.convert(LoggingEvent)
> org.apache.log4j.helpers.PatternConverter.format(StringBuffer, LoggingEvent)
> org.apache.log4j.PatternLayout.format(LoggingEvent)
> org.apache.log4j.WriterAppender.subAppend(LoggingEvent)
> org.apache.log4j.RollingFileAppender.subAppend(LoggingEvent)
> ...
> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.finish()
> 214,658 32.42 0
> {code}
> We could remove capturing the class name information from the default config
> but ideally capturing the classname is useful. So if we can find a way that
> doesn't need to create a throwable then it's ideal.
> Here is an interesting read :
> https://shipilev.net/blog/2014/exceptional-performance/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080467#comment-17080467
]
Dustin Haver commented on LUCENE-9221:
--
[~jeremynovey] [~cpoerschke] Thanks for the feedback. I added the triangle at
the bottom of the feather and added the original green and a darker blue green
to add contrast.
!image-2020-04-10-07-04-00-267.png!
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, image-2020-04-10-07-04-00-267.png,
> image.png, zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9221) Lucene Logo Contest
[
https://issues.apache.org/jira/browse/LUCENE-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dustin Haver updated LUCENE-9221:
-
Attachment: image-2020-04-10-07-04-00-267.png
> Lucene Logo Contest
> ---
>
> Key: LUCENE-9221
> URL: https://issues.apache.org/jira/browse/LUCENE-9221
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Ryan Ernst
>Priority: Trivial
> Attachments: LuceneLogo.png, image-2020-04-10-07-04-00-267.png,
> image.png, zabetak-1-7.pdf
>
>
> The Lucene logo has served the project well for almost 20 years. However, it
> does sometimes show its age and misses modern nice-to-haves like invertable
> or grayscale variants.
>
> The PMC would like to have a contest to replace the current logo. This issue
> will serve as the submission mechanism for that contest. When the submission
> deadline closes, a community poll will be used to guide the PMC in the
> decision of which logo to choose. Keeping the current logo will be a possible
> outcome of this decision, if a majority likes the current logo more than any
> other proposal.
>
> The logo should adhere to the guidelines set forth by Apache for project
> logos ([https://www.apache.org/foundation/marks/pmcs#graphics]), specifically
> that the full project name, "Apache Lucene", must appear in the logo
> (although the word "Apache" may be in a smaller font than "Lucene").
>
> The contest will last approximately one month. The submission deadline is
> -*Monday, March 16, 2020*- *Monday, April 6, 2020*. Submissions should be
> attached in a single zip or tar archive, with the filename of the form
> {{[user]-[proposal number].[extension]}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] shalinmangar commented on a change in pull request #1417: SOLR-12847: Auto-create a policy rule that corresponds to maxShardsPerNode
shalinmangar commented on a change in pull request #1417: SOLR-12847: Auto-create a policy rule that corresponds to maxShardsPerNode URL: https://github.com/apache/lucene-solr/pull/1417#discussion_r406737685 ## File path: solr/solrj/src/java/org/apache/solr/common/params/CollectionAdminParams.java ## @@ -127,4 +127,8 @@ /** Option to follow aliases when deciding the target of a collection admin command. */ String FOLLOW_ALIASES = "followAliases"; + + /** Prefix for automatically created config elements. */ + String AUTO_PREFIX = ".auto_"; Review comment: We use a suffix ".AUTOCREATED" for configsets, maybe we can use the same here? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] shalinmangar commented on a change in pull request #1417: SOLR-12847: Auto-create a policy rule that corresponds to maxShardsPerNode
shalinmangar commented on a change in pull request #1417: SOLR-12847:
Auto-create a policy rule that corresponds to maxShardsPerNode
URL: https://github.com/apache/lucene-solr/pull/1417#discussion_r406736277
##
File path:
solr/core/src/java/org/apache/solr/cloud/api/collections/CreateCollectionCmd.java
##
@@ -407,14 +466,81 @@ public void call(ClusterState clusterState, ZkNodeProps
message, NamedList resul
.assignPullReplicas(numPullReplicas)
.onNodes(nodeList)
.build();
- Assign.AssignStrategyFactory assignStrategyFactory = new
Assign.AssignStrategyFactory(cloudManager);
+ // if Strategy.POLICY is in use AND maxShardsPerNode was specified then
for back-compat it needs to
+ // be translated into a rule (see SOLR-12847).
+ // Modify the autoscaling rules as needed BEFORE actually calling
assign, setting configToRestore as
+ // a side-effect to restore the original config if the creation fails
+ if ((strategy == Assign.AssignStrategyFactory.Strategy.POLICY) &&
+ (maxReplicasPerNode != null && maxReplicasPerNode !=
Integer.MAX_VALUE)) {
+maybeAddMaxReplicasRule(cloudManager, maxReplicasPerNode,
docCollection, configToRestore);
Review comment:
I would have preferred to do this outside of this method instead of adding a
side effect here. That way the addition of the clause as well as its cleanup
can be in one place.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] shalinmangar commented on a change in pull request #1417: SOLR-12847: Auto-create a policy rule that corresponds to maxShardsPerNode
shalinmangar commented on a change in pull request #1417: SOLR-12847:
Auto-create a policy rule that corresponds to maxShardsPerNode
URL: https://github.com/apache/lucene-solr/pull/1417#discussion_r406735984
##
File path:
solr/core/src/java/org/apache/solr/cloud/api/collections/CreateCollectionCmd.java
##
@@ -407,14 +466,81 @@ public void call(ClusterState clusterState, ZkNodeProps
message, NamedList resul
.assignPullReplicas(numPullReplicas)
.onNodes(nodeList)
.build();
- Assign.AssignStrategyFactory assignStrategyFactory = new
Assign.AssignStrategyFactory(cloudManager);
+ // if Strategy.POLICY is in use AND maxShardsPerNode was specified then
for back-compat it needs to
+ // be translated into a rule (see SOLR-12847).
+ // Modify the autoscaling rules as needed BEFORE actually calling
assign, setting configToRestore as
+ // a side-effect to restore the original config if the creation fails
+ if ((strategy == Assign.AssignStrategyFactory.Strategy.POLICY) &&
+ (maxReplicasPerNode != null && maxReplicasPerNode !=
Integer.MAX_VALUE)) {
+maybeAddMaxReplicasRule(cloudManager, maxReplicasPerNode,
docCollection, configToRestore);
+ }
Assign.AssignStrategy assignStrategy =
assignStrategyFactory.create(clusterState, docCollection);
replicaPositions = assignStrategy.assign(cloudManager, assignRequest);
sessionWrapper.set(PolicyHelper.getLastSessionWrapper(true));
}
return replicaPositions;
}
+ /**
+ * Optionally add a policy that implements 'maxShardsPerNode' and set the
collection to use this policy.
+ * @param cloudManager SolrCloudManager
+ * @param maxReplicasPerNode max number of replicas per node
+ * @param docCollection collection (its properties may be modified as a
side-effect)
+ * @param configToRestore if AutoScalingConfig has been changed as a result
of this method the original
+ *value is set here in order to restore it in case
of failures
+ */
+ public static void maybeAddMaxReplicasRule(SolrCloudManager cloudManager,
int maxReplicasPerNode, DocCollection docCollection,
+
AtomicReference configToRestore) throws IOException,
InterruptedException {
+AutoScalingConfig initialConfig =
cloudManager.getDistribStateManager().getAutoScalingConfig();
+Policy policy = initialConfig.getPolicy();
+String policyName = docCollection.getPolicyName();
+if (policyName == null) {
+ policyName = CollectionAdminParams.AUTO_PREFIX + docCollection.getName();
+ docCollection.getProperties().put(Policy.POLICY, policyName);
+}
+Map> policies = policy.getPolicies();
+List clauses = policies.get(policyName);
+maxReplicasPerNode++; // only < supported
+Clause c = Clause.create("{'replica': '<" + maxReplicasPerNode +"' ,
'shard': '#ANY', 'node': '#ANY'}");
+boolean alreadyExists = false;
+if (clauses != null) {
+ for (Clause clause : clauses) {
+if (clause.equals(c)) {
+ alreadyExists = true;
+ break;
+}
+if (clause.getReplica() != null) {
+ throw new Assign.AssignmentException("Both an existing policy=" +
policyName + " and " +
Review comment:
This block will always execute if were to add a default policy like the one
on SOLR-12845. We should test if there is a similar clause in the policy where
similar means that only the value of N in `replica < N` is different than
maxShardsPerNode
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] shalinmangar commented on a change in pull request #1417: SOLR-12847: Auto-create a policy rule that corresponds to maxShardsPerNode
shalinmangar commented on a change in pull request #1417: SOLR-12847: Auto-create a policy rule that corresponds to maxShardsPerNode URL: https://github.com/apache/lucene-solr/pull/1417#discussion_r406736732 ## File path: solr/core/src/java/org/apache/solr/cloud/api/collections/DeleteCollectionCmd.java ## @@ -159,6 +165,36 @@ public void call(ClusterState state, ZkNodeProps message, NamedList results) thr }); } + // remove auto-created policy if this is the only collection that uses it Review comment: Maybe we should disallow people from using these auto created policies in the first place? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9309) IW#addIndices(CodecReader) might delete files concurrently to IW#rollback
[
https://issues.apache.org/jira/browse/LUCENE-9309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer resolved LUCENE-9309.
-
Fix Version/s: 8.6
master (9.0)
Lucene Fields: New,Patch Available (was: New)
Assignee: Simon Willnauer
Resolution: Fixed
> IW#addIndices(CodecReader) might delete files concurrently to IW#rollback
> -
>
> Key: LUCENE-9309
> URL: https://issues.apache.org/jira/browse/LUCENE-9309
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Assignee: Simon Willnauer
>Priority: Major
> Fix For: master (9.0), 8.6
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> During work on LUCENE-9304 [~mikemccand] ran into a failure:
> {noformat}
> org.apache.lucene.index.TestAddIndexes > test suite's output saved to
> /home/mike/src/simon/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.index.TestAddIndexes.txt,
> copied below:
>> java.nio.file.NoSuchFileException:
> _gt_Lucene85FieldsIndex-doc_ids_6u.tmp
>> at
> __randomizedtesting.SeedInfo.seed([4760FA81FBD4B2CE:A147156E5F7BD9B0]:0)
>> at
> org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
>> at
> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
>> at
> org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
>> at
> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
>> at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
>> at
> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:449)
>> at
> org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2334)
>> at
> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2275)
>> at
> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2268)
>> at
> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:974)
> 2> NOTE: reproduce with: ant test -Dtestcase=TestAddIndexes
> -Dtests.method=testAddIndexesWithRollback -Dtests.seed=4760FA81FBD4B2CE
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=fr-GP -Dtests.t\
> imezone=Asia/Tbilisi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> 2> NOTE: test params are: codec=Asserting(Lucene84):
> {c=PostingsFormat(name=LuceneFixedGap),
> id=PostingsFormat(name=LuceneFixedGap),
> f1=PostingsFormat(name=LuceneFixedGap), f2=BlockTreeOrds(blocksize=128)\
> , version=BlockTreeOrds(blocksize=128), content=FST50},
> docValues:{dv=DocValuesFormat(name=Lucene80),
> soft_delete=DocValuesFormat(name=Lucene80),
> doc=DocValuesFormat(name=Lucene80), id=DocValuesFormat(name=\
> Asserting), content=DocValuesFormat(name=Asserting),
> doc2d=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=982,
> maxMBSortInHeap=5.837219998050092,
> sim=Asserting(org.apache.lucene.search.similarities.As\
> sertingSimilarity@6ce38471), locale=fr-GP, timezone=Asia/Tbilisi
> {noformat}
> While this unfortunately doesn't reproduce it's likely a bug that exists for
> quite some time but never showed up until LUCENE-9147 which uses a temporary
> output. That's fine but with IW#addIndices(CodecReader...) not registering
> the merge it does in the IW we never wait for the merge to finish while
> rollback and if that merge finishes concurrently it will also remove these
> .tmp files.
> There are many ways to fix this and I can work on a patch, but hey do we
> really need to be able to add indices while we index and do that on an open
> and live IW or can it be a tool on top of it?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9298) Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared
[
https://issues.apache.org/jira/browse/LUCENE-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer resolved LUCENE-9298.
-
Fix Version/s: 8.6
master (9.0)
Lucene Fields: New,Patch Available (was: New)
Assignee: Simon Willnauer
Resolution: Fixed
> Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are
> cleared
>
>
> Key: LUCENE-9298
> URL: https://issues.apache.org/jira/browse/LUCENE-9298
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.5
>Reporter: yubinglei
>Assignee: Simon Willnauer
>Priority: Minor
> Labels: easyfix, newbie, pull-request-available
> Fix For: master (9.0), 8.6
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> As mentioned in this
> [https://github.com/apache/lucene-solr/pull/513#discussion_r399131681]
> the method clearDeletedDocIds in BufferedUpdates.java has a bug, it can't
> reset bytesUsed correctly
> {code:java}
> void clearDeletedDocIds() {
> deleteDocIDs.clear();
> bytesUsed.addAndGet(-deleteDocIDs.size() *
> BufferedUpdates.BYTES_PER_DEL_DOCID);
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9309) IW#addIndices(CodecReader) might delete files concurrently to IW#rollback
[
https://issues.apache.org/jira/browse/LUCENE-9309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080415#comment-17080415
]
ASF subversion and git services commented on LUCENE-9309:
-
Commit 29c7f07de2b5ccb65840ed11987d4b7750f1f070 in lucene-solr's branch
refs/heads/branch_8x from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=29c7f07 ]
LUCENE-9309: Wait for #addIndexes merges when aborting merges (#1418)
The SegmentMerger usage in IW#addIndexes(CodecReader...) might make changes
to the Directory while the IW tries to clean-up files on rollback. This
causes issues like FileNotFoundExceptions when IDF tries to remove temp files.
This changes adds a waiting mechanism to the abortMerges method that, in
addition
to the running merges, also waits for merges in addIndices(CodecReader...)
> IW#addIndices(CodecReader) might delete files concurrently to IW#rollback
> -
>
> Key: LUCENE-9309
> URL: https://issues.apache.org/jira/browse/LUCENE-9309
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> During work on LUCENE-9304 [~mikemccand] ran into a failure:
> {noformat}
> org.apache.lucene.index.TestAddIndexes > test suite's output saved to
> /home/mike/src/simon/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.index.TestAddIndexes.txt,
> copied below:
>> java.nio.file.NoSuchFileException:
> _gt_Lucene85FieldsIndex-doc_ids_6u.tmp
>> at
> __randomizedtesting.SeedInfo.seed([4760FA81FBD4B2CE:A147156E5F7BD9B0]:0)
>> at
> org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
>> at
> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
>> at
> org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
>> at
> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
>> at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
>> at
> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:449)
>> at
> org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2334)
>> at
> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2275)
>> at
> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2268)
>> at
> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:974)
> 2> NOTE: reproduce with: ant test -Dtestcase=TestAddIndexes
> -Dtests.method=testAddIndexesWithRollback -Dtests.seed=4760FA81FBD4B2CE
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=fr-GP -Dtests.t\
> imezone=Asia/Tbilisi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> 2> NOTE: test params are: codec=Asserting(Lucene84):
> {c=PostingsFormat(name=LuceneFixedGap),
> id=PostingsFormat(name=LuceneFixedGap),
> f1=PostingsFormat(name=LuceneFixedGap), f2=BlockTreeOrds(blocksize=128)\
> , version=BlockTreeOrds(blocksize=128), content=FST50},
> docValues:{dv=DocValuesFormat(name=Lucene80),
> soft_delete=DocValuesFormat(name=Lucene80),
> doc=DocValuesFormat(name=Lucene80), id=DocValuesFormat(name=\
> Asserting), content=DocValuesFormat(name=Asserting),
> doc2d=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=982,
> maxMBSortInHeap=5.837219998050092,
> sim=Asserting(org.apache.lucene.search.similarities.As\
> sertingSimilarity@6ce38471), locale=fr-GP, timezone=Asia/Tbilisi
> {noformat}
> While this unfortunately doesn't reproduce it's likely a bug that exists for
> quite some time but never showed up until LUCENE-9147 which uses a temporary
> output. That's fine but with IW#addIndices(CodecReader...) not registering
> the merge it does in the IW we never wait for the merge to finish while
> rollback and if that merge finishes concurrently it will also remove these
> .tmp files.
> There are many ways to fix this and I can work on a patch, but hey do we
> really need to be able to add indices while we index and do that on an open
> and live IW or can it be a tool on top of it?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9298) Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared
[
https://issues.apache.org/jira/browse/LUCENE-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080414#comment-17080414
]
ASF subversion and git services commented on LUCENE-9298:
-
Commit e0d8ad813c06df030dd87ba8b7f8e598b90de2fb in lucene-solr's branch
refs/heads/branch_8x from YuBinglei
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e0d8ad8 ]
LUCENE-9298: Improve RAM accounting in BufferedUpdates when deleted doc IDs and
terms are cleared (#1389)
> Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are
> cleared
>
>
> Key: LUCENE-9298
> URL: https://issues.apache.org/jira/browse/LUCENE-9298
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.5
>Reporter: yubinglei
>Priority: Minor
> Labels: easyfix, newbie, pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> As mentioned in this
> [https://github.com/apache/lucene-solr/pull/513#discussion_r399131681]
> the method clearDeletedDocIds in BufferedUpdates.java has a bug, it can't
> reset bytesUsed correctly
> {code:java}
> void clearDeletedDocIds() {
> deleteDocIDs.clear();
> bytesUsed.addAndGet(-deleteDocIDs.size() *
> BufferedUpdates.BYTES_PER_DEL_DOCID);
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9309) IW#addIndices(CodecReader) might delete files concurrently to IW#rollback
[
https://issues.apache.org/jira/browse/LUCENE-9309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080411#comment-17080411
]
ASF subversion and git services commented on LUCENE-9309:
-
Commit e376582e25d02f85b415eb1c0456f3fdc800fc84 in lucene-solr's branch
refs/heads/master from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e376582 ]
LUCENE-9309: Wait for #addIndexes merges when aborting merges (#1418)
The SegmentMerger usage in IW#addIndexes(CodecReader...) might make changes
to the Directory while the IW tries to clean-up files on rollback. This
causes issues like FileNotFoundExceptions when IDF tries to remove temp files.
This changes adds a waiting mechanism to the abortMerges method that, in
addition
to the running merges, also waits for merges in addIndices(CodecReader...)
> IW#addIndices(CodecReader) might delete files concurrently to IW#rollback
> -
>
> Key: LUCENE-9309
> URL: https://issues.apache.org/jira/browse/LUCENE-9309
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> During work on LUCENE-9304 [~mikemccand] ran into a failure:
> {noformat}
> org.apache.lucene.index.TestAddIndexes > test suite's output saved to
> /home/mike/src/simon/lucene/core/build/test-results/test/outputs/OUTPUT-org.apache.lucene.index.TestAddIndexes.txt,
> copied below:
>> java.nio.file.NoSuchFileException:
> _gt_Lucene85FieldsIndex-doc_ids_6u.tmp
>> at
> __randomizedtesting.SeedInfo.seed([4760FA81FBD4B2CE:A147156E5F7BD9B0]:0)
>> at
> org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:148)
>> at
> org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607)
>> at
> org.apache.lucene.store.LockValidatingDirectoryWrapper.deleteFile(LockValidatingDirectoryWrapper.java:38)
>> at
> org.apache.lucene.index.IndexFileDeleter.deleteFile(IndexFileDeleter.java:696)
>> at
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:690)
>> at
> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:449)
>> at
> org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2334)
>> at
> org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2275)
>> at
> org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2268)
>> at
> org.apache.lucene.index.TestAddIndexes.testAddIndexesWithRollback(TestAddIndexes.java:974)
> 2> NOTE: reproduce with: ant test -Dtestcase=TestAddIndexes
> -Dtests.method=testAddIndexesWithRollback -Dtests.seed=4760FA81FBD4B2CE
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=fr-GP -Dtests.t\
> imezone=Asia/Tbilisi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
> 2> NOTE: test params are: codec=Asserting(Lucene84):
> {c=PostingsFormat(name=LuceneFixedGap),
> id=PostingsFormat(name=LuceneFixedGap),
> f1=PostingsFormat(name=LuceneFixedGap), f2=BlockTreeOrds(blocksize=128)\
> , version=BlockTreeOrds(blocksize=128), content=FST50},
> docValues:{dv=DocValuesFormat(name=Lucene80),
> soft_delete=DocValuesFormat(name=Lucene80),
> doc=DocValuesFormat(name=Lucene80), id=DocValuesFormat(name=\
> Asserting), content=DocValuesFormat(name=Asserting),
> doc2d=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=982,
> maxMBSortInHeap=5.837219998050092,
> sim=Asserting(org.apache.lucene.search.similarities.As\
> sertingSimilarity@6ce38471), locale=fr-GP, timezone=Asia/Tbilisi
> {noformat}
> While this unfortunately doesn't reproduce it's likely a bug that exists for
> quite some time but never showed up until LUCENE-9147 which uses a temporary
> output. That's fine but with IW#addIndices(CodecReader...) not registering
> the merge it does in the IW we never wait for the merge to finish while
> rollback and if that merge finishes concurrently it will also remove these
> .tmp files.
> There are many ways to fix this and I can work on a patch, but hey do we
> really need to be able to add indices while we index and do that on an open
> and live IW or can it be a tool on top of it?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw merged pull request #1418: LUCENE-9309: Wait for #addIndexes merges when aborting merges
s1monw merged pull request #1418: LUCENE-9309: Wait for #addIndexes merges when aborting merges URL: https://github.com/apache/lucene-solr/pull/1418 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14402) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search
[ https://issues.apache.org/jira/browse/SOLR-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-14402: - Status: Patch Available (was: Open) > Avoid creating new exceptions for every request made to > MDCAwareThreadPoolExecutor by distributed search > > > Key: SOLR-14402 > URL: https://issues.apache.org/jira/browse/SOLR-14402 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 7.4 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Attachments: SOLR-14402.patch > > > SOLR-11880 tried to do the same and it succeeded for update shard handler but > the implementation was wrong for http shard handler because the executor > created during construction is overwritten in the init() method. The commit > for SOLR-11880 is at https://github.com/apache/lucene-solr/commit/5a47ed4/ > Thanks [~caomanhdat] for spotting this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14402) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search
[ https://issues.apache.org/jira/browse/SOLR-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080406#comment-17080406 ] Shalin Shekhar Mangar commented on SOLR-14402: -- Here's a simple patch that sets enableSubmitterStackTrace to false while creating the executor inside HttpShardHandlerFactory's init method. It removes the executor that was initialized in the class attribute because it is overwritten in init anyway. The patch also fixes ZkControllerTest and OverseerTest which were using HttpShardHandlerFactory without calling init first. > Avoid creating new exceptions for every request made to > MDCAwareThreadPoolExecutor by distributed search > > > Key: SOLR-14402 > URL: https://issues.apache.org/jira/browse/SOLR-14402 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 7.4 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Attachments: SOLR-14402.patch > > > SOLR-11880 tried to do the same and it succeeded for update shard handler but > the implementation was wrong for http shard handler because the executor > created during construction is overwritten in the init() method. The commit > for SOLR-11880 is at https://github.com/apache/lucene-solr/commit/5a47ed4/ > Thanks [~caomanhdat] for spotting this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14402) Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed search
[ https://issues.apache.org/jira/browse/SOLR-14402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-14402: - Attachment: SOLR-14402.patch > Avoid creating new exceptions for every request made to > MDCAwareThreadPoolExecutor by distributed search > > > Key: SOLR-14402 > URL: https://issues.apache.org/jira/browse/SOLR-14402 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: 7.4 >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Minor > Attachments: SOLR-14402.patch > > > SOLR-11880 tried to do the same and it succeeded for update shard handler but > the implementation was wrong for http shard handler because the executor > created during construction is overwritten in the init() method. The commit > for SOLR-11880 is at https://github.com/apache/lucene-solr/commit/5a47ed4/ > Thanks [~caomanhdat] for spotting this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9298) Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared
[
https://issues.apache.org/jira/browse/LUCENE-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080404#comment-17080404
]
ASF subversion and git services commented on LUCENE-9298:
-
Commit 2935186c5b807394f25ea7a6d42ee3d7dbef5e43 in lucene-solr's branch
refs/heads/master from YuBinglei
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2935186 ]
LUCENE-9298: Improve RAM accounting in BufferedUpdates when deleted doc IDs and
terms are cleared (#1389)
> Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are
> cleared
>
>
> Key: LUCENE-9298
> URL: https://issues.apache.org/jira/browse/LUCENE-9298
> Project: Lucene - Core
> Issue Type: Bug
>Affects Versions: 8.5
>Reporter: yubinglei
>Priority: Minor
> Labels: easyfix, newbie, pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> As mentioned in this
> [https://github.com/apache/lucene-solr/pull/513#discussion_r399131681]
> the method clearDeletedDocIds in BufferedUpdates.java has a bug, it can't
> reset bytesUsed correctly
> {code:java}
> void clearDeletedDocIds() {
> deleteDocIDs.clear();
> bytesUsed.addAndGet(-deleteDocIDs.size() *
> BufferedUpdates.BYTES_PER_DEL_DOCID);
> }
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw merged pull request #1389: LUCENE-9298: Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared
s1monw merged pull request #1389: LUCENE-9298: Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared URL: https://github.com/apache/lucene-solr/pull/1389 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw commented on issue #1389: LUCENE-9298: Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared
s1monw commented on issue #1389: LUCENE-9298: Improve RAM accounting in BufferedUpdates when deleted doc IDs and terms are cleared URL: https://github.com/apache/lucene-solr/pull/1389#issuecomment-611976552 thanks!! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14365) CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values
[
https://issues.apache.org/jira/browse/SOLR-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080401#comment-17080401
]
Shalin Shekhar Mangar commented on SOLR-14365:
--
I just saw this test failure on master which seems related and is reproducible:
{code}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestRandomCollapseQParserPlugin
-Dtests.method=testRandomCollpaseWithSort -Dtests.seed=20C0F4D7CBA81876
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=lv
-Dtests.timezone=America/St_Johns -Dtests.asserts=true
-Dtests.file.encoding=ANSI_X3.4-1968
[junit4] FAILURE 7.30s J4 |
TestRandomCollapseQParserPlugin.testRandomCollpaseWithSort <<<
[junit4]> Throwable #1: java.lang.AssertionError: collapseKey too big --
need to grow array?
[junit4]>at
__randomizedtesting.SeedInfo.seed([20C0F4D7CBA81876:257D871EE0002B85]:0)
[junit4]>at
org.apache.solr.search.CollapsingQParserPlugin$SortFieldsCompare.setGroupValues(CollapsingQParserPlugin.java:2702)
[junit4]>at
org.apache.solr.search.CollapsingQParserPlugin$IntSortSpecStrategy.collapse(CollapsingQParserPlugin.java:2544)
[junit4]>at
org.apache.solr.search.CollapsingQParserPlugin$IntFieldValueCollector.collect(CollapsingQParserPlugin.java:1223)
[junit4]>at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:254)
[junit4]>at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:205)
[junit4]>at
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
[junit4]>at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:739)
[junit4]>at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:526)
[junit4]>at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:202)
[junit4]>at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1651)
[junit4]>at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1469)
[junit4]>at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
[junit4]>at
org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1487)
[junit4]>at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:399)
[junit4]>at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:328)
[junit4]>at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:209)
[junit4]>at
org.apache.solr.core.SolrCore.execute(SolrCore.java:2565)
[junit4]>at
org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:227)
[junit4]>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)
[junit4]>at
org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1003)
[junit4]>at
org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1018)
[junit4]>at
org.apache.solr.search.TestRandomCollapseQParserPlugin.testRandomCollpaseWithSort(TestRandomCollapseQParserPlugin.java:158)
[junit4]>at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{code}
> CollapsingQParser - Avoiding always allocate int[] and float[] with size
> equals to number of unique values
> --
>
> Key: SOLR-14365
> URL: https://issues.apache.org/jira/browse/SOLR-14365
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.4.1
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-14365.patch
>
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Since Collapsing is a PostFilter, documents reach Collapsing must match with
> all filters and queries, so the number of documents Collapsing need to
> collect/compute score is a small fraction of the total number documents in
> the index. So why do we need to always consume the memory (for int[] and
> float[] array) for all unique values of the collapsed field? If the number of
> unique values of the collapsed field found in the documents that match
> queries and filters is 300 then we only need int[] and float[] array with
> size of 300 and not 1.2 million in size. However, we don't know which value
> of the collapsed field will show up in the results so we cannot use a smaller
> array.
> The easy fix for this problem is
[jira] [Created] (SOLR-14404) CoreContainer level custom requesthandlers
Noble Paul created SOLR-14404:
-
Summary: CoreContainer level custom requesthandlers
Key: SOLR-14404
URL: https://issues.apache.org/jira/browse/SOLR-14404
Project: Solr
Issue Type: New Feature
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul
Assignee: Noble Paul
caveats:
* The class should implement {{org.apache.solr.api.Api}}. Which means only V2
APIs are supported
* The path should have prefix {{/api/cluster}} or {{/api/node}}. It should not
conflict with any of the exiting handlers
add a handler
{code}
curl -X POST -H 'Content-type:application/json' --data-binary '
{
"add-requesthandler": {
"name":"myhandler", "class": "full.ClassName"
}
}' http://localhost:8983/api/cluster
{code}
remove a handler
{code}
curl -X POST -H 'Content-type:application/json' --data-binary '
{
"remove-requesthandler": "myhandler"
}' http://localhost:8983/api/cluster
{code}
The configuration will be stored in the {{clusterprops.json}}
as
{code}
{
"requestHandler" : {
"myhandler" : {"class": "full.ClassName" }
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13662) Package manager CLI
[ https://issues.apache.org/jira/browse/SOLR-13662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-13662: -- Labels: packagemanager (was: ) > Package manager CLI > --- > > Key: SOLR-13662 > URL: https://issues.apache.org/jira/browse/SOLR-13662 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Ishan Chattopadhyaya >Priority: Major > Labels: packagemanager > Fix For: 8.4 > > Attachments: plugin-cli.png > > Time Spent: 19h 10m > Remaining Estimate: 0h > > Design details and usage details are here: > https://docs.google.com/document/d/15b3m3i3NFDKbhkhX_BN0MgvPGZaBj34TKNF2-UNC3U8/edit?ts=5d86a8ad# -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14212) " plugins with packages" seems incomplete
[ https://issues.apache.org/jira/browse/SOLR-14212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-14212: -- Labels: packagemanager (was: ) > " plugins with packages" seems incomplete > -- > > Key: SOLR-14212 > URL: https://issues.apache.org/jira/browse/SOLR-14212 > Project: Solr > Issue Type: Sub-task >Affects Versions: 8.5 >Reporter: Christine Poerschke >Priority: Major > Labels: packagemanager > Attachments: SOLR-14212.patch > > > SOLR-14125 follow-on, details to follow. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13661) A package management system for Solr
[ https://issues.apache.org/jira/browse/SOLR-13661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-13661: -- Labels: package packagemanager (was: package) > A package management system for Solr > > > Key: SOLR-13661 > URL: https://issues.apache.org/jira/browse/SOLR-13661 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Assignee: Ishan Chattopadhyaya >Priority: Major > Labels: package, packagemanager > Fix For: 8.4 > > Attachments: plugin-usage.png, repos.png > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Here's the design doc: > https://docs.google.com/document/d/15b3m3i3NFDKbhkhX_BN0MgvPGZaBj34TKNF2-UNC3U8/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14151) Make schema components load from packages
[
https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-14151:
--
Labels: packagemanager (was: )
> Make schema components load from packages
> -
>
> Key: SOLR-14151
> URL: https://issues.apache.org/jira/browse/SOLR-14151
> Project: Solr
> Issue Type: Sub-task
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Labels: packagemanager
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Example:
> {code:xml}
>
>
>
>generateNumberParts="0" catenateWords="0"
> catenateNumbers="0" catenateAll="0"/>
>
>
>
>
> {code}
> * When a package is updated, the entire {{IndexSchema}} object is refreshed,
> but the SolrCore object is not reloaded
> * Any component can be prefixed with the package name
> * The semantics of loading plugins remain the same as that of the components
> in {{solrconfig.xml}}
> * Plugins can be registered using schema API
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14365) CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values
[
https://issues.apache.org/jira/browse/SOLR-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080339#comment-17080339
]
ASF subversion and git services commented on SOLR-14365:
Commit adbd714b37d794e9aa7615e61c431e42162c1d3c in lucene-solr's branch
refs/heads/master from Cao Manh Dat
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=adbd714 ]
SOLR-14365: CollapsingQParser - Avoiding always allocate int[] and float[] with
size equals to number of unique values (WIP) (#1395)
> CollapsingQParser - Avoiding always allocate int[] and float[] with size
> equals to number of unique values
> --
>
> Key: SOLR-14365
> URL: https://issues.apache.org/jira/browse/SOLR-14365
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.4.1
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-14365.patch
>
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Since Collapsing is a PostFilter, documents reach Collapsing must match with
> all filters and queries, so the number of documents Collapsing need to
> collect/compute score is a small fraction of the total number documents in
> the index. So why do we need to always consume the memory (for int[] and
> float[] array) for all unique values of the collapsed field? If the number of
> unique values of the collapsed field found in the documents that match
> queries and filters is 300 then we only need int[] and float[] array with
> size of 300 and not 1.2 million in size. However, we don't know which value
> of the collapsed field will show up in the results so we cannot use a smaller
> array.
> The easy fix for this problem is using as much as we need by using IntIntMap
> and IntFloatMap that hold primitives and are much more space efficient than
> the Java HashMap. These maps can be slower (10x or 20x) than plain int[] and
> float[] if matched documents is large (almost all documents matched queries
> and other filters). But our belief is that does not happen that frequently
> (how frequently do we run collapsing on the entire index?).
> For this issue I propose adding 2 methods for collapsing which is
> * array : which is current implementation
> * hash : which is new approach and will be default method
> later we can add another method {{smart}} which is automatically pick method
> based on comparision between {{number of docs matched queries and filters}}
> and {{number of unique values of the field}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] CaoManhDat merged pull request #1395: SOLR-14365: CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values (WIP)
CaoManhDat merged pull request #1395: SOLR-14365: CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values (WIP) URL: https://github.com/apache/lucene-solr/pull/1395 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14365) CollapsingQParser - Avoiding always allocate int[] and float[] with size equals to number of unique values
[
https://issues.apache.org/jira/browse/SOLR-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080323#comment-17080323
]
ASF subversion and git services commented on SOLR-14365:
Commit 0e148071042056587bd934796bce3c27da81f922 in lucene-solr's branch
refs/heads/jira/SOLR-14365 from Cao Manh Dat
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=0e14807 ]
Merge branch 'master' into jira/SOLR-14365
> CollapsingQParser - Avoiding always allocate int[] and float[] with size
> equals to number of unique values
> --
>
> Key: SOLR-14365
> URL: https://issues.apache.org/jira/browse/SOLR-14365
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 8.4.1
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-14365.patch
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Since Collapsing is a PostFilter, documents reach Collapsing must match with
> all filters and queries, so the number of documents Collapsing need to
> collect/compute score is a small fraction of the total number documents in
> the index. So why do we need to always consume the memory (for int[] and
> float[] array) for all unique values of the collapsed field? If the number of
> unique values of the collapsed field found in the documents that match
> queries and filters is 300 then we only need int[] and float[] array with
> size of 300 and not 1.2 million in size. However, we don't know which value
> of the collapsed field will show up in the results so we cannot use a smaller
> array.
> The easy fix for this problem is using as much as we need by using IntIntMap
> and IntFloatMap that hold primitives and are much more space efficient than
> the Java HashMap. These maps can be slower (10x or 20x) than plain int[] and
> float[] if matched documents is large (almost all documents matched queries
> and other filters). But our belief is that does not happen that frequently
> (how frequently do we run collapsing on the entire index?).
> For this issue I propose adding 2 methods for collapsing which is
> * array : which is current implementation
> * hash : which is new approach and will be default method
> later we can add another method {{smart}} which is automatically pick method
> based on comparision between {{number of docs matched queries and filters}}
> and {{number of unique values of the field}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9310) IntelliJ attempts to resolve provider property in jar manifest configuration and fails during project import
[ https://issues.apache.org/jira/browse/LUCENE-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080318#comment-17080318 ] ASF subversion and git services commented on LUCENE-9310: - Commit dbb4be1ca93607c2555fe8b2b2cb3318be582edb in lucene-solr's branch refs/heads/jira/SOLR-14365 from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=dbb4be1 ] LUCENE-9310: workaround for IntelliJ gradle import > IntelliJ attempts to resolve provider property in jar manifest configuration > and fails during project import > > > Key: LUCENE-9310 > URL: https://issues.apache.org/jira/browse/LUCENE-9310 > Project: Lucene - Core > Issue Type: Task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Major > Fix For: master (9.0) > > > It shouldn't be the case but it is. I don't know why. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9286) FST arc.copyOf clones BitTables and this can lead to excessive memory use
[ https://issues.apache.org/jira/browse/LUCENE-9286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080320#comment-17080320 ] ASF subversion and git services commented on LUCENE-9286: - Commit 6bba35a7096ef2d4023ba9339e9cc30268d520ce in lucene-solr's branch refs/heads/jira/SOLR-14365 from Bruno Roustant [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6bba35a ] LUCENE-9286: FST.Arc.BitTable reads directly FST bytes. Arc is lightweight again and FSTEnum traversal faster. > FST arc.copyOf clones BitTables and this can lead to excessive memory use > - > > Key: LUCENE-9286 > URL: https://issues.apache.org/jira/browse/LUCENE-9286 > Project: Lucene - Core > Issue Type: Bug >Affects Versions: 8.5 >Reporter: Dawid Weiss >Assignee: Bruno Roustant >Priority: Major > Fix For: 8.6 > > Attachments: screen-[1].png > > Time Spent: 1h 20m > Remaining Estimate: 0h > > I see a dramatic increase in the amount of memory required for construction > of (arguably large) automata. It currently OOMs with 8GB of memory consumed > for bit tables. I am pretty sure this didn't require so much memory before > (the automaton is ~50MB after construction). > Something bad happened in between. Thoughts, [~broustant], [~sokolov]? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9278) Make javadoc folder structure follow Gradle project path
[ https://issues.apache.org/jira/browse/LUCENE-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080319#comment-17080319 ] ASF subversion and git services commented on LUCENE-9278: - Commit 4f92cd414c4da6ac6163ff4101b0e07fb94fd067 in lucene-solr's branch refs/heads/jira/SOLR-14365 from Tomoko Uchida [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4f92cd4 ] LUCENE-9278: Use -linkoffline instead of relative paths to make links to other projects (#1388) > Make javadoc folder structure follow Gradle project path > > > Key: LUCENE-9278 > URL: https://issues.apache.org/jira/browse/LUCENE-9278 > Project: Lucene - Core > Issue Type: Task > Components: general/build >Reporter: Tomoko Uchida >Assignee: Tomoko Uchida >Priority: Major > Fix For: master (9.0) > > Time Spent: 6h 20m > Remaining Estimate: 0h > > Current javadoc folder structure is derived from Ant project name. e.g.: > [https://lucene.apache.org/core/8_4_1/analyzers-icu/index.html] > [https://lucene.apache.org/solr/8_4_1/solr-solrj/index.html] > For Gradle build, it should also follow gradle project structure (path) > instead of ant one, to keep things simple to manage [1]. Hence, it will look > like this: > [https://lucene.apache.org/core/9_0_0/analysis/icu/index.html] > [https://lucene.apache.org/solr/9_0_0/solr/solrj/index.html] > [1] The change was suggested at the conversation between Dawid Weiss and I on > a github pr: [https://github.com/apache/lucene-solr/pull/1304] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9077) Gradle build
[ https://issues.apache.org/jira/browse/LUCENE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080321#comment-17080321 ] ASF subversion and git services commented on LUCENE-9077: - Commit c7cac5749a76be20c4ade6a71c7eea862b593e06 in lucene-solr's branch refs/heads/jira/SOLR-14365 from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c7cac57 ] LUCENE-9077: make git always keep .gradle files with LF EOLs. > Gradle build > > > Key: LUCENE-9077 > URL: https://issues.apache.org/jira/browse/LUCENE-9077 > Project: Lucene - Core > Issue Type: Task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Major > Fix For: master (9.0) > > Attachments: LUCENE-9077-javadoc-locale-en-US.patch > > Time Spent: 2.5h > Remaining Estimate: 0h > > This task focuses on providing gradle-based build equivalent for Lucene and > Solr (on master branch). See notes below on why this respin is needed. > The code lives on *gradle-master* branch. It is kept with sync with *master*. > Try running the following to see an overview of helper guides concerning > typical workflow, testing and ant-migration helpers: > gradlew :help > A list of items that needs to be added or requires work. If you'd like to > work on any of these, please add your name to the list. Once you have a > patch/ pull request let me (dweiss) know - I'll try to coordinate the merges. > * (/) Apply forbiddenAPIs > * (/) Generate hardware-aware gradle defaults for parallelism (count of > workers and test JVMs). > * (/) Fail the build if --tests filter is applied and no tests execute > during the entire build (this allows for an empty set of filtered tests at > single project level). > * (/) Port other settings and randomizations from common-build.xml > * (/) Configure security policy/ sandboxing for tests. > * (/) test's console output on -Ptests.verbose=true > * (/) add a :helpDeps explanation to how the dependency system works > (palantir plugin, lockfile) and how to retrieve structured information about > current dependencies of a given module (in a tree-like output). > * (/) jar checksums, jar checksum computation and validation. This should be > done without intermediate folders (directly on dependency sets). > * (/) verify min. JVM version and exact gradle version on build startup to > minimize odd build side-effects > * (/) Repro-line for failed tests/ runs. > * (/) add a top-level README note about building with gradle (and the > required JVM). > * (/) add an equivalent of 'validate-source-patterns' > (check-source-patterns.groovy) to precommit. > * (/) add an equivalent of 'rat-sources' to precommit. > * (/) add an equivalent of 'check-example-lucene-match-version' (solr only) > to precommit. > * (/) javadoc compilation > Hard-to-implement stuff already investigated: > * (/) (done) -*Printing console output of failed tests.* There doesn't seem > to be any way to do this in a reasonably efficient way. There are onOutput > listeners but they're slow to operate and solr tests emit *tons* of output so > it's an overkill.- > * (!) (LUCENE-9120) *Tests working with security-debug logs or other > JVM-early log output*. Gradle's test runner works by redirecting Java's > stdout/ syserr so this just won't work. Perhaps we can spin the ant-based > test runner for such corner-cases. > Of lesser importance: > * Add an equivalent of 'documentation-lint" to precommit. > * (/) Do not require files to be committed before running precommit. (staged > files are fine). > * (/) add rendering of javadocs (gradlew javadoc) > * Attach javadocs to maven publications. > * Add test 'beasting' (rerunning the same suite multiple times). I'm afraid > it'll be difficult to run it sensibly because gradle doesn't offer cwd > separation for the forked test runners. > * if you diff solr packaged distribution against ant-created distribution > there are minor differences in library versions and some JARs are excluded/ > moved around. I didn't try to force these as everything seems to work (tests, > etc.) – perhaps these differences should be fixed in the ant build instead. > * (/) identify and port various "regenerate" tasks from ant builds (javacc, > precompiled automata, etc.) > * Fill in POM details in gradle/defaults-maven.gradle so that they reflect > the previous content better (dependencies aside). > * Add any IDE integration layers that should be added (I use IntelliJ and it > imports the project out of the box, without the need for any special tuning). > ** Remove idea task and just import from the gradle model? One less thing to > maintain. > * Add Solr packaging for docs/* (see TODO in packaging/build.gradle; > currently XSLT...) > * I didn't bother adding Solr
[jira] [Commented] (LUCENE-9311) IntelliJ import attempts to compile solr-ref-guide tools/ and fails
[ https://issues.apache.org/jira/browse/LUCENE-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080322#comment-17080322 ] ASF subversion and git services commented on LUCENE-9311: - Commit 2437f3f56c99e08de567d44f346e4555b867a855 in lucene-solr's branch refs/heads/jira/SOLR-14365 from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2437f3f ] LUCENE-9311: detect intellij reimport and modify sourceset to exclude solr-ref-guide/tools (#1422) > IntelliJ import attempts to compile solr-ref-guide tools/ and fails > --- > > Key: LUCENE-9311 > URL: https://issues.apache.org/jira/browse/LUCENE-9311 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Major > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > > This used to work but now doesn't. Don't know why (we exclude customized ant > tasks but IntelliJ doesn't seem to pick this up). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14376) Optimize SolrIndexSearcher.getDocSet and getProcessedFilter for empty fq
[ https://issues.apache.org/jira/browse/SOLR-14376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080316#comment-17080316 ] ASF subversion and git services commented on SOLR-14376: Commit 5bfbdc5325aa331549e9700734042c9b582fc3fc in lucene-solr's branch refs/heads/jira/SOLR-14365 from David Smiley [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5bfbdc5 ] SOLR-14376: optimize SolrIndexSearcher.getDocSet when matches everything * getProcessedFilter now returns null filter if it's all docs more reliably * getProcessedFilter now documented clearly as an internal method * getDocSet detects all-docs and exits early with getLiveDocs * small refactoring to getDocSetBits/makeDocSetBits Closes #1399 > Optimize SolrIndexSearcher.getDocSet and getProcessedFilter for empty fq > > > Key: SOLR-14376 > URL: https://issues.apache.org/jira/browse/SOLR-14376 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: search >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Fix For: 8.6 > > Time Spent: 20m > Remaining Estimate: 0h > > If either SolrIndexSearcher.getDocSet or getProcessedFilter are called with > an null/empty query list, we should be able to short-circuit this with a > getLiveDocSet response. Today getDocSet builds up a new DocSet, bit by bit > :-/ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9266) ant nightly-smoke fails due to presence of build.gradle
[ https://issues.apache.org/jira/browse/LUCENE-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080317#comment-17080317 ] ASF subversion and git services commented on LUCENE-9266: - Commit 793a3becfbde09d1d48c6aa2d189ea677fca7ac2 in lucene-solr's branch refs/heads/jira/SOLR-14365 from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=793a3be ] LUCENE-9266: correct windows gradle wrapper download script - wrong placement of the quote. > ant nightly-smoke fails due to presence of build.gradle > --- > > Key: LUCENE-9266 > URL: https://issues.apache.org/jira/browse/LUCENE-9266 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: master (9.0) > > Time Spent: 6.5h > Remaining Estimate: 0h > > Seen on Jenkins - > [https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1617/console] > > Reproduced locally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
