[jira] [Resolved] (LUCENE-9531) Consolidate duplicated generated classes CharStream and FastCharStream
[ https://issues.apache.org/jira/browse/LUCENE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss resolved LUCENE-9531. - Fix Version/s: master (9.0) Resolution: Fixed > Consolidate duplicated generated classes CharStream and FastCharStream > -- > > Key: LUCENE-9531 > URL: https://issues.apache.org/jira/browse/LUCENE-9531 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > Fix For: master (9.0) > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss merged pull request #1886: LUCENE-9531: Consolidate duplicated generated classes CharStream and FastCharStream
dweiss merged pull request #1886: URL: https://github.com/apache/lucene-solr/pull/1886 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9531) Consolidate duplicated generated classes CharStream and FastCharStream
[ https://issues.apache.org/jira/browse/LUCENE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198164#comment-17198164 ] ASF subversion and git services commented on LUCENE-9531: - Commit 5ec2bac91c5ed75719e4b6b574f2e92d77a353ba in lucene-solr's branch refs/heads/master from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5ec2bac ] LUCENE-9531: Consolidate duplicated generated classes CharStream and FastCharStream (#1886) > Consolidate duplicated generated classes CharStream and FastCharStream > -- > > Key: LUCENE-9531 > URL: https://issues.apache.org/jira/browse/LUCENE-9531 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14824) Simplify set of Ref Guide build parameters
[ https://issues.apache.org/jira/browse/SOLR-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198162#comment-17198162 ] Dawid Weiss commented on SOLR-14824: bq. The usecase here is that on "branch_9_7" regardless of whether 'baseVersion' is 9.7.0, or 9.7.1, etc... (making the default "version" 9.7.0-SNAPSHOT, 9.7.1-SNAPSHOT, etc..), we want to be able to "build" the "9.7" ref-guide, and by default the builds should be "-DRAFT" but optionally a committer should be able to indicate that it's "non-DRAFT" so they can (re-)publish it to the site. Right. This is relevant to the discussion I had with Uwe about having a "release" commit where the version would be actually fixed (not provided on command-line during release). I accept his argument that this increases chances of accidentally building a "release" build by a third-party (although I disagree this is harmful - if somebody does it, so be it?). In the above case I think it'd be consistent with the "release procedure" to have the same argument passed on command line. If somebody wishes to release a non-draft (non-snapshot?) version of the guide, they would use the same syntax as when compiling release artifacts. Think this way: even if somebody doesn't know about the ref guide and runs top-level "assemble everything in version X" then the guide would be compiled with an appropriate non-version status. Does this make sense? > Simplify set of Ref Guide build parameters > -- > > Key: SOLR-14824 > URL: https://issues.apache.org/jira/browse/SOLR-14824 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build, documentation >Reporter: Cassandra Targett >Assignee: Chris M. Hostetter >Priority: Minor > Attachments: SOLR-14824.patch, SOLR-14824.patch, SOLR-14824.patch > > > While trying to solve LUCENE-9495, I thought it might be a good idea to try > to simplify the set of variables and properties used during the Ref Guide > build. > There are 3 areas to work on: > 1. Remove the "barebones-html" build. With Gradle the build is self-contained > and {{gradlew check}} and {{gradle precommit}} could just build the full docs > and check them. > 2. Remove some properties that only existed for a hypothetical need related > to the now-removed PDF. > 3. Change remaining properties to be defined directly in build.gradle instead > of relying on ant properties functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14875) Make SolrEventListeners load from packages
[ https://issues.apache.org/jira/browse/SOLR-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul resolved SOLR-14875. --- Fix Version/s: 8.7 Resolution: Fixed > Make SolrEventListeners load from packages > -- > > Key: SOLR-14875 > URL: https://issues.apache.org/jira/browse/SOLR-14875 > Project: Solr > Issue Type: Sub-task > Components: packages >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Fix For: 8.7 > > Time Spent: 20m > Remaining Estimate: 0h > > This is a hot reloaded plugin (does not result in a core reload) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9523) Speedup query shapes for geometries that generate multiple points
[ https://issues.apache.org/jira/browse/LUCENE-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ignacio Vera resolved LUCENE-9523. -- Fix Version/s: 8.7 Assignee: Ignacio Vera Resolution: Fixed > Speedup query shapes for geometries that generate multiple points > - > > Key: LUCENE-9523 > URL: https://issues.apache.org/jira/browse/LUCENE-9523 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Assignee: Ignacio Vera >Priority: Major > Fix For: 8.7 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > When indexing lines or polygons into a shape field, we generally index > multiple points for the same document. When querying, we typically visited a > subset of this points. In many cases when visiting points from the same > document, we already know the relationship and therefore computing the > relationship between that point and the query shape is not necessary. > When using dense visitors(eg. visitors backed by a FixedBitSet), we can check > if the relationship is already known and therefore skip that point. > Finally for intersects relationship we normally use sparse visitors but I > wonder in the case where the number of points >> number of docs, we should > use a dense visitor so we can skip points from documents where we know it > intersects. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9523) Speedup query shapes for geometries that generate multiple points
[ https://issues.apache.org/jira/browse/LUCENE-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198130#comment-17198130 ] ASF subversion and git services commented on LUCENE-9523: - Commit 47215d4a858c5959ea5c745249057db356f2cf16 in lucene-solr's branch refs/heads/branch_8x from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=47215d4 ] LUCENE-9523: Speed up query shapes for geometries that generate multiple points (#1866) In query shapes over shape fields, skip points while traversing the BKD tree when the relationship with the document is already known > Speedup query shapes for geometries that generate multiple points > - > > Key: LUCENE-9523 > URL: https://issues.apache.org/jira/browse/LUCENE-9523 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > When indexing lines or polygons into a shape field, we generally index > multiple points for the same document. When querying, we typically visited a > subset of this points. In many cases when visiting points from the same > document, we already know the relationship and therefore computing the > relationship between that point and the query shape is not necessary. > When using dense visitors(eg. visitors backed by a FixedBitSet), we can check > if the relationship is already known and therefore skip that point. > Finally for intersects relationship we normally use sparse visitors but I > wonder in the case where the number of points >> number of docs, we should > use a dense visitor so we can skip points from documents where we know it > intersects. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9523) Speedup query shapes for geometries that generate multiple points
[ https://issues.apache.org/jira/browse/LUCENE-9523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198128#comment-17198128 ] ASF subversion and git services commented on LUCENE-9523: - Commit fbf8e4f04481a14617618d05a54ba925842c7058 in lucene-solr's branch refs/heads/master from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=fbf8e4f ] LUCENE-9523: Speed up query shapes for geometries that generate multiple points (#1866) In query shapes over shape fields, skip points while traversing the BKD tree when the relationship with the document is already known > Speedup query shapes for geometries that generate multiple points > - > > Key: LUCENE-9523 > URL: https://issues.apache.org/jira/browse/LUCENE-9523 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Ignacio Vera >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > When indexing lines or polygons into a shape field, we generally index > multiple points for the same document. When querying, we typically visited a > subset of this points. In many cases when visiting points from the same > document, we already know the relationship and therefore computing the > relationship between that point and the query shape is not necessary. > When using dense visitors(eg. visitors backed by a FixedBitSet), we can check > if the relationship is already known and therefore skip that point. > Finally for intersects relationship we normally use sparse visitors but I > wonder in the case where the number of points >> number of docs, we should > use a dense visitor so we can skip points from documents where we know it > intersects. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] iverase merged pull request #1866: LUCENE-9523: Speed up query shapes for geometries that generate multiple points
iverase merged pull request #1866: URL: https://github.com/apache/lucene-solr/pull/1866 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198061#comment-17198061 ] Noble Paul edited comment on SOLR-14151 at 9/18/20, 2:18 AM: - {quote}You keep patching and patching, reverting and reverting the reverts {quote} The reverts are done to an accidental commit I did. not to master Do you think I'm just randomly adding things and making changes? I run the tests in my machine and I beast them.I commit changes when it is not failing. The bug is still there and it has been there for ages. We know what the failure is . It is a red herring. In fact it does not affect any real user was (Author: noble.paul): {quote}You keep patching and patching, reverting and reverting the reverts {quote} Do you think I'm just randomly adding things and making changes? I run the tests in my machine and I beast them.I commit changes when it is not failing. The bug is still there and it has been there for ages. We know what the failure is . It is a red herring. In fact it does not affect any real user > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14875) Make SolrEventListeners load from packages
[ https://issues.apache.org/jira/browse/SOLR-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198075#comment-17198075 ] ASF subversion and git services commented on SOLR-14875: Commit ee0a374bb8389282341e49cda195d5630c81e8e5 in lucene-solr's branch refs/heads/master from Noble Paul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ee0a374 ] SOLR-14875: Make SolrEventListeners load from packages (#1887) > Make SolrEventListeners load from packages > -- > > Key: SOLR-14875 > URL: https://issues.apache.org/jira/browse/SOLR-14875 > Project: Solr > Issue Type: Sub-task > Components: packages >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This is a hot reloaded plugin (does not result in a core reload) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14875) Make SolrEventListeners load from packages
[ https://issues.apache.org/jira/browse/SOLR-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198074#comment-17198074 ] ASF subversion and git services commented on SOLR-14875: Commit ee0a374bb8389282341e49cda195d5630c81e8e5 in lucene-solr's branch refs/heads/master from Noble Paul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ee0a374 ] SOLR-14875: Make SolrEventListeners load from packages (#1887) > Make SolrEventListeners load from packages > -- > > Key: SOLR-14875 > URL: https://issues.apache.org/jira/browse/SOLR-14875 > Project: Solr > Issue Type: Sub-task > Components: packages >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This is a hot reloaded plugin (does not result in a core reload) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14875) Make SolrEventListeners load from packages
[ https://issues.apache.org/jira/browse/SOLR-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198073#comment-17198073 ] ASF subversion and git services commented on SOLR-14875: Commit ee0a374bb8389282341e49cda195d5630c81e8e5 in lucene-solr's branch refs/heads/master from Noble Paul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ee0a374 ] SOLR-14875: Make SolrEventListeners load from packages (#1887) > Make SolrEventListeners load from packages > -- > > Key: SOLR-14875 > URL: https://issues.apache.org/jira/browse/SOLR-14875 > Project: Solr > Issue Type: Sub-task > Components: packages >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This is a hot reloaded plugin (does not result in a core reload) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14875) Make SolrEventListeners load from packages
[ https://issues.apache.org/jira/browse/SOLR-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198070#comment-17198070 ] ASF subversion and git services commented on SOLR-14875: Commit ee0a374bb8389282341e49cda195d5630c81e8e5 in lucene-solr's branch refs/heads/master from Noble Paul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ee0a374 ] SOLR-14875: Make SolrEventListeners load from packages (#1887) > Make SolrEventListeners load from packages > -- > > Key: SOLR-14875 > URL: https://issues.apache.org/jira/browse/SOLR-14875 > Project: Solr > Issue Type: Sub-task > Components: packages >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This is a hot reloaded plugin (does not result in a core reload) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul merged pull request #1887: SOLR-14875: Make SolrEventListeners load from packages
noblepaul merged pull request #1887: URL: https://github.com/apache/lucene-solr/pull/1887 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198061#comment-17198061 ] Noble Paul edited comment on SOLR-14151 at 9/18/20, 2:02 AM: - {quote}You keep patching and patching, reverting and reverting the reverts {quote} Do you think I'm just randomly adding things and making changes? I run the tests in my machine and I beast them.I commit changes when it is not failing. The bug is still there and it has been there for ages. We know what the failure is . It is a red herring. In fact it does not affect any real user was (Author: noble.paul): {quote}. You keep patching and patching, reverting and reverting the reverts,quote} Do you think I'm just randomly adding things and making changes? I run the tests in my machine and I beast them.I commit changes when it is not failing. The bug is still there and it has been there for ages. We know what the failure is . It is a red herring. In fact it does not affect any real user > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198061#comment-17198061 ] Noble Paul commented on SOLR-14151: --- {quote}. You keep patching and patching, reverting and reverting the reverts,quote} Do you think I'm just randomly adding things and making changes? I run the tests in my machine and I beast them.I commit changes when it is not failing. The bug is still there and it has been there for ages. We know what the failure is . It is a red herring. In fact it does not affect any real user > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on pull request #1881: Upgrade zookeeper version to 3.6.2 to use recent version of netty
ErickErickson commented on pull request #1881: URL: https://github.com/apache/lucene-solr/pull/1881#issuecomment-694532744 Hmmm, All the jars for netty that ship with Solr are already 4.1.50.Final, it's an explicit dependency in versions.props. Of course if you run an external ZK, you can download any version you choose. That said, there's no reason _not_ to upgrade ZK, I just doubt getting 4.1.50 is germane. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198010#comment-17198010 ] Ishan Chattopadhyaya commented on SOLR-14151: - There is a real bug in Solr. Tests are failing due to that bug (not because of the feature added here). If the failures are so disconcerting, let us disable the tests and feel happy about life. > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14859) [* TO *] queries on DateRange fields miss results
[ https://issues.apache.org/jira/browse/SOLR-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197996#comment-17197996 ] Chris M. Hostetter commented on SOLR-14859: --- bq. Working on tests now Check out BadIndexSchemaTest for examples of testing that we "fail" expectedly on fields/types with bad options bq. ... checkSchemaField returns void but could return a new SchemaField that the FieldType creates based on whatever changes it wants to make. Except that in the current lifecycle, FT.checkSchemaField is called from the SchemaField constructor -- so like i said: we'd either have to seriously change the lifecycle/API of SchemaField, or make the constructor override it's own props by the output of checkSchemaField ... but i think we're in agreement we don't wnat to go down the road : ) > [* TO *] queries on DateRange fields miss results > - > > Key: SOLR-14859 > URL: https://issues.apache.org/jira/browse/SOLR-14859 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: query parsers >Affects Versions: 8.5 >Reporter: Jason Gerlowski >Assignee: Jason Gerlowski >Priority: Major > Attachments: SOLR-14859.patch, SOLR-14859.patch, SOLR-14859.patch, > query-debug.png, reproduce.sh, schema.png > > > "exists" queries ({{[* TO *]}}) on DateRange fields return 0 results > regardless of docs in the index with values in that field. > The issue appears to be that the query is converted into a > {{NormsFieldExistsQuery}}, even though DateRangeField uses omitNorms=true by > default. Probably introduced by SOLR-11746's changes to these optimizable > range queries. > I've attached a script to reproduce the issue (tested on Solr 8.6.2) and > screenshots showing showing schema and query-parsing info for the > reproduction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197975#comment-17197975 ] Tomas Eduardo Fernandez Lobbe commented on SOLR-14151: -- Noble, when you submit code that breaks the build you are supposed to either fix immediately or revert until you can stabilize and then merge again. This is not something new, it's common sense, and it's what everyone else does. This has been failing for more than a week, accounting for close to 60% of the failures. You keep patching and patching, reverting and reverting the reverts, [without clear understanding of the fix|https://github.com/apache/lucene-solr/pull/1815#issuecomment-687092388]. If it's some other bug that this is hitting, you should still revert until that other bug is fixed, it's that simple. bq. There are 3 options lets do #3 until #1 is done. I'm fine with #2 too, but that would be a backwards incompatible change, so should be master only. > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-14859) [* TO *] queries on DateRange fields miss results
[ https://issues.apache.org/jira/browse/SOLR-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197966#comment-17197966 ] Jason Gerlowski edited comment on SOLR-14859 at 9/17/20, 9:16 PM: -- Thanks for the quick review Hoss - I made the changes you suggested and it looks good with manual tests. Working on tests now - it looks like it broke something in DateRangeFieldTest so still tracking that down... bq. Or we could change the api of checkSchemaField so that can return the 'final' properties the SchemaField should use I thought about this a bit too - {{checkSchemaField}} returns void but could return a new SchemaField that the FieldType creates based on whatever changes it wants to make. But agreed that anything like that is overkill for what we want here. was (Author: gerlowskija): Thanks for the quick review Hoss - I made the changes you suggested and it looks good with manual tests. Working on tests now - it looks like it broke something in DateRangeFieldTest so still tracking that down... > [* TO *] queries on DateRange fields miss results > - > > Key: SOLR-14859 > URL: https://issues.apache.org/jira/browse/SOLR-14859 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: query parsers >Affects Versions: 8.5 >Reporter: Jason Gerlowski >Assignee: Jason Gerlowski >Priority: Major > Attachments: SOLR-14859.patch, SOLR-14859.patch, SOLR-14859.patch, > query-debug.png, reproduce.sh, schema.png > > > "exists" queries ({{[* TO *]}}) on DateRange fields return 0 results > regardless of docs in the index with values in that field. > The issue appears to be that the query is converted into a > {{NormsFieldExistsQuery}}, even though DateRangeField uses omitNorms=true by > default. Probably introduced by SOLR-11746's changes to these optimizable > range queries. > I've attached a script to reproduce the issue (tested on Solr 8.6.2) and > screenshots showing showing schema and query-parsing info for the > reproduction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14859) [* TO *] queries on DateRange fields miss results
[ https://issues.apache.org/jira/browse/SOLR-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gerlowski updated SOLR-14859: --- Attachment: SOLR-14859.patch Status: Open (was: Open) Thanks for the quick review Hoss - I made the changes you suggested and it looks good with manual tests. Working on tests now - it looks like it broke something in DateRangeFieldTest so still tracking that down... > [* TO *] queries on DateRange fields miss results > - > > Key: SOLR-14859 > URL: https://issues.apache.org/jira/browse/SOLR-14859 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: query parsers >Affects Versions: 8.5 >Reporter: Jason Gerlowski >Assignee: Jason Gerlowski >Priority: Major > Attachments: SOLR-14859.patch, SOLR-14859.patch, SOLR-14859.patch, > query-debug.png, reproduce.sh, schema.png > > > "exists" queries ({{[* TO *]}}) on DateRange fields return 0 results > regardless of docs in the index with values in that field. > The issue appears to be that the query is converted into a > {{NormsFieldExistsQuery}}, even though DateRangeField uses omitNorms=true by > default. Probably introduced by SOLR-11746's changes to these optimizable > range queries. > I've attached a script to reproduce the issue (tested on Solr 8.6.2) and > screenshots showing showing schema and query-parsing info for the > reproduction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] epugh commented on pull request #1869: SOLR-14866 autowidth tables in ref guide
epugh commented on pull request #1869: URL: https://github.com/apache/lucene-solr/pull/1869#issuecomment-694501316 Okay, so I think I'm done, but would love one more set of eyes! I did tweak the `asciidoc-syntax.adoc` and put in the autowidth approach. What we had listed actually wasn't ever used in the docs ;-) Please let me know what you think. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9530) Clean up javacc generation script (gradle)
[ https://issues.apache.org/jira/browse/LUCENE-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197951#comment-17197951 ] ASF subversion and git services commented on LUCENE-9530: - Commit 4f344cb0d4087171f59cc3c22265237bbeec50a4 in lucene-solr's branch refs/heads/SOLR-14866 from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4f344cb ] LUCENE-9530: cleaned up javacc gradle generation scripts. (#1883) * LUCENE-9530: cleaned up gradle javacc generation/ tweaks script so that it's consistent across runs. Removed ant remnants. > Clean up javacc generation script (gradle) > -- > > Key: LUCENE-9530 > URL: https://issues.apache.org/jira/browse/LUCENE-9530 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14613) Provide a clean API for pluggable replica assignment implementations
[ https://issues.apache.org/jira/browse/SOLR-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197956#comment-17197956 ] ASF subversion and git services commented on SOLR-14613: Commit dbba48b3e56be541572f170a9fae95d039236c1a in lucene-solr's branch refs/heads/SOLR-14866 from Ilan Ginzburg [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=dbba48b ] SOLR-14613: use set-placement-plugin for both setting and unsetting plugin config > Provide a clean API for pluggable replica assignment implementations > > > Key: SOLR-14613 > URL: https://issues.apache.org/jira/browse/SOLR-14613 > Project: Solr > Issue Type: Improvement > Components: AutoScaling >Reporter: Andrzej Bialecki >Assignee: Ilan Ginzburg >Priority: Major > Time Spent: 41h > Remaining Estimate: 0h > > As described in SIP-8 the current autoscaling Policy implementation has > several limitations that make it difficult to use for very large clusters and > very large collections. SIP-8 also mentions the possible migration path by > providing alternative implementations of the placement strategies that are > less complex but more efficient in these very large environments. > We should review the existing APIs that the current autoscaling engine uses > ({{SolrCloudManager}} , {{AssignStrategy}} , {{Suggester}} and related > interfaces) to see if they provide a sufficient and minimal API for plugging > in alternative autoscaling placement strategies, and if necessary refactor > the existing APIs. > Since these APIs are internal it should be possible to do this without > breaking back-compat. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197953#comment-17197953 ] ASF subversion and git services commented on SOLR-14151: Commit 515608a087f81c13ad22efba72e48fc64d9d5ae7 in lucene-solr's branch refs/heads/SOLR-14866 from noblepaul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=515608a ] SOLR-14151: fixed the classloading issue > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197958#comment-17197958 ] ASF subversion and git services commented on LUCENE-9529: - Commit 33f728007853ee53c71f259206734bfd9155593d in lucene-solr's branch refs/heads/SOLR-14866 from Adrien Grand [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=33f7280 ] LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks. (#1882) The problem of tracking dirtiness via numbers of chunks is that larger chunks make stored fields readers more likely to be considered dirty, so I'm trying to work around it by tracking numbers of docs instead. > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Fix For: 8.7 > > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14792) Remove VelocityResponseWriter from Solr 9
[ https://issues.apache.org/jira/browse/SOLR-14792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197955#comment-17197955 ] ASF subversion and git services commented on SOLR-14792: Commit 2364a7aded9f04137eeb8520d69ffe97f7caef66 in lucene-solr's branch refs/heads/SOLR-14866 from Erik Hatcher [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2364a7a ] SOLR-14792: Remove VelocityResponseWriter > Remove VelocityResponseWriter from Solr 9 > - > > Key: SOLR-14792 > URL: https://issues.apache.org/jira/browse/SOLR-14792 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 9 >Reporter: Erik Hatcher >Priority: Blocker > Time Spent: 1h > Remaining Estimate: 0h > > VelocityResponseWriter was deprecated in SOLR-14065. It can now be removed > from 9's code branch. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197950#comment-17197950 ] ASF subversion and git services commented on SOLR-14151: Commit cbb1659640cd51be8b403eda8399c527af1c848e in lucene-solr's branch refs/heads/SOLR-14866 from noblepaul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=cbb1659 ] Revert "Revert "SOLR-14151: Bug fixes (#1815)"" This reverts commit 27a14fe48139019a4c09ba072751e093fc5cb5f1. Undoing accidental commit > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9530) Clean up javacc generation script (gradle)
[ https://issues.apache.org/jira/browse/LUCENE-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197952#comment-17197952 ] ASF subversion and git services commented on LUCENE-9530: - Commit 4f344cb0d4087171f59cc3c22265237bbeec50a4 in lucene-solr's branch refs/heads/SOLR-14866 from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4f344cb ] LUCENE-9530: cleaned up javacc gradle generation scripts. (#1883) * LUCENE-9530: cleaned up gradle javacc generation/ tweaks script so that it's consistent across runs. Removed ant remnants. > Clean up javacc generation script (gradle) > -- > > Key: LUCENE-9530 > URL: https://issues.apache.org/jira/browse/LUCENE-9530 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9527) Upgrade javacc to 7.0.4
[ https://issues.apache.org/jira/browse/LUCENE-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197954#comment-17197954 ] ASF subversion and git services commented on LUCENE-9527: - Commit 6c9d7adf7985e4b895a80e561f1a3bb5820126a8 in lucene-solr's branch refs/heads/SOLR-14866 from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6c9d7ad ] LUCENE-9527: upgrade javacc to 7.0.4 (#1884) > Upgrade javacc to 7.0.4 > --- > > Key: LUCENE-9527 > URL: https://issues.apache.org/jira/browse/LUCENE-9527 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14871) Use Annotations for v2 APIs in/cluster path
[ https://issues.apache.org/jira/browse/SOLR-14871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197949#comment-17197949 ] ASF subversion and git services commented on SOLR-14871: Commit 5bc7fb286182eeca4e4d2c9ff9dfb98f6a399125 in lucene-solr's branch refs/heads/SOLR-14866 from noblepaul [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5bc7fb2 ] SOLR-14871: remove unused test > Use Annotations for v2 APIs in/cluster path > --- > > Key: SOLR-14871 > URL: https://issues.apache.org/jira/browse/SOLR-14871 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > We have custom json specs for each of these APIs. With the annotation > framework , it can be made simple and readable and we can eliminate a lot of > code -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14824) Simplify set of Ref Guide build parameters
[ https://issues.apache.org/jira/browse/SOLR-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris M. Hostetter updated SOLR-14824: -- Attachment: SOLR-14824.patch Status: Open (was: Open) {quote}This looks weird (and comment is inconsistent with code). {quote} Ah, yeah – that comment was (obtusely) trying to explain that for the "X.Y" ref guide we always link to the javadocs of the "X.Y.0" release. I've beefed up the comments on the version & path variables to make it more clear their purpose & "style" of values each one contains. {quote}If this draft status is related to version then it could be automatically detected (version.contains("SNAPSHOT")). {quote} That wouldn't work because by default 'version' (in the top level build.gradle file) always contains the SNAPSHOT suffix by default (unless 'version.release' or 'version.suffix' is overriden on the command line.) The usecase here is that on "branch_9_7" regardless of whether 'baseVersion' is 9.7.0, or 9.7.1, etc... (making the default "version" 9.7.0-SNAPSHOT, 9.7.1-SNAPSHOT, etc..), we want to be able to "build" the "9.7" ref-guide, and by default the builds should be "\-DRAFT" but optionally a committer should be able to indicate that it's "non-DRAFT" so they can (re-)publish it to the site. I suppose instead of using {{\-PsolrGuideDraft=false}} we could document that the way to build an "non-DRAFT" copy of the guide is to use {{\-Dversion.suffix=''}} ? ... but the approach in the patch seems easier for the actual usecase/workflow/process of building & publishing the ref-guide. (If/When we get to the point of wanting to build & publishing the ref-guide automatically on every release, then it might make sense to drive the 'DRAFT" behavior based on wether 'version' has a suffix on it – but let's cross that bridge when we come to it) {quote}For this issue I was mainly looking at the solrRootPath property that's obviously not a relative URL ... {quote} Oh, sure – yeah. I fixed that to be based on {{project(':solr')}} {quote}I think the changes to build.gradle that remove the asciidocAttrs section are going to cause some unexpected consequences. There are a couple parameters defined there that aren't in _config.yml.template, specifically: {quote} Those attributes have never affected the jekyll build: they've only ever been used when gradle/ant ran asciidoctor directly for the bare-bones (and PDF) builds. The ant build.xml file actually had a comment pointing out this distinction, and I'm pretty sure the reason for that choice was because of how jekyll uses asciidoctor slightly differently (for things like TOC), and they were only needed by PDF renderer? "attribute-missing" and "icons" are actually hard coded in {{_config.yml.template}} (with those same values) but only in the "asciidoctor > attributes" section, not in the "solr-attributes" YAML reference, so they can't mistakenly be used as variables inside of content or in liquid templates... {noformat} attributes: <<: *solr-attributes-ref attribute-missing: "warn" icons: "font" source-highlighter: "rouge" rouge-theme: "thankful-eyes" stem: {noformat} ...if you think we should add the other 4 we can do that, but we should take care and review the output carefully since they may (probably?) change the look/feel of the content from what we currently get on master/8x since they haven't been used in the jekyll builds up to now. {quote}I couldn't get the patch to apply so can't check these but they're worth a look. {quote} there were conflicts introduced by the removal of velocity ... the patching i'm attaching now should apply clean. > Simplify set of Ref Guide build parameters > -- > > Key: SOLR-14824 > URL: https://issues.apache.org/jira/browse/SOLR-14824 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build, documentation >Reporter: Cassandra Targett >Assignee: Chris M. Hostetter >Priority: Minor > Attachments: SOLR-14824.patch, SOLR-14824.patch, SOLR-14824.patch > > > While trying to solve LUCENE-9495, I thought it might be a good idea to try > to simplify the set of variables and properties used during the Ref Guide > build. > There are 3 areas to work on: > 1. Remove the "barebones-html" build. With Gradle the build is self-contained > and {{gradlew check}} and {{gradle precommit}} could just build the full docs > and check them. > 2. Remove some properties that only existed for a hypothetical need related > to the now-removed PDF. > 3. Change remaining properties to be defined directly in build.gradle instead > of relying on ant properties functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005) ---
[jira] [Commented] (LUCENE-9530) Clean up javacc generation script (gradle)
[ https://issues.apache.org/jira/browse/LUCENE-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197934#comment-17197934 ] Dawid Weiss commented on LUCENE-9530: - [~erickerickson] you may want to check out the javacc.gradle now - I think I did a fairly good job at moving from ant regexps to more, ahem, groovy approach. :) > Clean up javacc generation script (gradle) > -- > > Key: LUCENE-9530 > URL: https://issues.apache.org/jira/browse/LUCENE-9530 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1877: SOLR-13181: param macro expansion could throw
dsmiley commented on a change in pull request #1877: URL: https://github.com/apache/lucene-solr/pull/1877#discussion_r490480292 ## File path: solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java ## @@ -136,14 +136,15 @@ private String _expand(String val) { } else if (idx < 0) { if (sb == null) return val; -sb.append(val.substring(start)); +sb.append(val, start, val.length()); return sb.toString(); } // found unescaped "${" + final int matchedStart = idx; idx += macroStart.length(); Review comment: Can now remove idx manipulation until later when idx & start are changed in one line ```suggestion ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12891) Injection Dangers in Streaming Expressions
[ https://issues.apache.org/jira/browse/SOLR-12891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197901#comment-17197901 ] David Smiley commented on SOLR-12891: - I appreciate the attention to detail on identifying this problem, and the peer review here, but respectfully, IMO the committed solution of modifying MacroExpander to check for one special parameter is a total hack. It feels weird/yucky that user input is subject to parameter substitution -- "q" param or whatever else you concoct into other parameters. I'm not sure if the worst that could happen is merely lots of confusion (e.g. why does my attempt to search for code snippets not work when this syntax is used), or wether it might have sinister effects in some systems. I'd instead rather see macro expansion simply not work on parameters sent into the request, except maybe an allow-list that would include the "json" parameter and maybe one or two others, at most. I could also imagine a local-param for a query to explicitly opt-in. Parameters specified in configuration (request handler or params.json) would always be eligible for macro-expansion. WDYT? I realize I'm proposing a breaking change that would only be acceptable for a major release (e.g. 9.0). > Injection Dangers in Streaming Expressions > -- > > Key: SOLR-12891 > URL: https://issues.apache.org/jira/browse/SOLR-12891 > Project: Solr > Issue Type: Bug > Components: streaming expressions >Affects Versions: 7.5, 8.0 >Reporter: Gus Heck >Priority: Minor > Labels: security > Fix For: 8.1, master (9.0) > > Attachments: SOLR-12891.patch, SOLR-12891.patch, SOLR-12891.patch, > SOLR-12891.patch, SOLR12819example.java > > > I just spent some time fiddling with streaming expressions for fun, reading > Erick Erickson's blog > ([https://lucidworks.com/2017/12/06/streaming-expressions-in-solrj/)] and the > example given in the ref guide > ([https://lucene.apache.org/solr/guide/7_5/streaming-expressions.html#streaming-requests-and-responses)] > and it occurred to me that we are recommending string concatenation into an > expression language with the power to harm the server, or other network > services visible from the server. I'm starting this Jira as a security issue > to avoid creating a public impression of insecurity, feel free to undo that > if I have guessed wrong. I haven't developed an exploit example, but it would > go something like this: > # Some portion of an expression is built including user supplied data using > the techniques we're recommending in the ref guide > # Malicious user constructs input data that breaks out of the expression > (SOLR-10894 is relevant here), probably somewhere inside a let() expression > where one could simply define an additional variable taking the value of a > malicious expression... > # update() expression is executed to add/overwrite data, jdbc() makes a JDBC > connection to a database visible to the server, or the malicious expression > executes some very expensive expression for DOS effect. > Technically this is of course the fault of the end user who allowed unchecked > input into programmatic execution, but when I think about how to check the > input I realize that the only way to be sure is to construct for myself a > notion of exactly how the parser behaves and then determine what needs to be > escaped. To do this I need to dig into the expression parser code... > How to escape input is also already unclear as shown by SOLR-10894 > There's another important wrinkle that would easily be missed by someone > trying to construct their own escaping/protection system relating to > parameter substitution as discussed here: SOLR-8458 > I think the solution to this is that SolrJ API should be enhanced to provide > an escaping utility at a minimum and possibly a "prepared expression" similar > to SQL prepared statements and call this issue to attention in the ref guide > once these tools are available... > Additionally, templating features might be a useful addition to help folks > manage large expressions and facilitate re-use of patterns... such templating > should also have this issue in mind when/if they are added. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14824) Simplify set of Ref Guide build parameters
[ https://issues.apache.org/jira/browse/SOLR-14824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197900#comment-17197900 ] Cassandra Targett commented on SOLR-14824: -- Sorry for delay looking at the patch...I'm a little distracted this week. I think the changes to {{build.gradle}} that remove the asciidocAttrs section are going to cause some unexpected consequences. There are a couple parameters defined there that aren't in {{_config.yml.template}}, specifically: {code} -'attribute-missing' : 'warn', -'icons' : 'font', -'icon-set' : 'fa', -'figure-caption!' : '', -'idprefix' : '', -'idseparator' : '-', {code} I couldn't get the patch to apply so can't check these but they're worth a look. > Simplify set of Ref Guide build parameters > -- > > Key: SOLR-14824 > URL: https://issues.apache.org/jira/browse/SOLR-14824 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: Build, documentation >Reporter: Cassandra Targett >Assignee: Chris M. Hostetter >Priority: Minor > Attachments: SOLR-14824.patch, SOLR-14824.patch > > > While trying to solve LUCENE-9495, I thought it might be a good idea to try > to simplify the set of variables and properties used during the Ref Guide > build. > There are 3 areas to work on: > 1. Remove the "barebones-html" build. With Gradle the build is self-contained > and {{gradlew check}} and {{gradle precommit}} could just build the full docs > and check them. > 2. Remove some properties that only existed for a hypothetical need related > to the now-removed PDF. > 3. Change remaining properties to be defined directly in build.gradle instead > of relying on ant properties functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand resolved LUCENE-9529. -- Fix Version/s: 8.7 Resolution: Fixed > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Fix For: 8.7 > > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14859) [* TO *] queries on DateRange fields miss results
[ https://issues.apache.org/jira/browse/SOLR-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197877#comment-17197877 ] Chris M. Hostetter commented on SOLR-14859: --- {noformat} +// NOCOMMIT - I'm only setting here the things that are directly specified in PrefixTreeStrategy.FIELD_TYPE, but +// there are other values there I'm not checking (docValues, etc.). Should I check _everything_ here? {noformat} I think no matter what we do, it's going to be crufty just because abstraction / separation of concerns that exist, and how there's not a very clear delineation between the "args" that get handled by the 'lucene.FieldType' (inside the Strategy) vs the args that solr/AbstractSpatialFieldType handles (multiValued, stored, etc...). ... i just didn't realize how crufty until i saw your patch and really thought about it : ) But maybe instead of the list of "if" statements, a simpler/cleaner way to approach this might be... * declare a static Map of solrArgName -> "hardcoded default that can't be changed" based on what is in {{PrefixTreeStrategy.FIELD_TYPE}}, ignoring the things like "stored" that AbstractSpatialFieldType handles directly. something like... {code:java} private static final Map hardcodedDefaults = new HashMap<>(); static { final lucene.FieldType ft = PrefixTreeStrategy.FIELD_TYPE; final IndexOptions opts = ft.indexOptions(); hardcodedDefaults.put("omitNorms", ft.omitNorms().toString()); hardcodedDefaults.put("termVectors", ft.storeTermVectors()); hardcodedDefaults.put("omitTermFreqAndPositions",opts.equals(DOCS)); // etc... } {code} * in setArgs, loop over the entries in 'hardcodedDefaults' WARNing if the key is found in 'args ** NOTE: i'm (now) suggesting that we log a warning even if the arg is set to the same value as the hardcoded default – because we can WARN "FieldType {} does not allow {} to be specified in schema, hardcoded behavior is {}={}" so people who just so happen to be specifying the "default" of foo=bar in their schema aren't completely suprised when they try foo=yak and discover setting 'foo' has never actaully been supported. * then "putAll" of hardcodedDefaults to args. {quote}The patch uses checkSchemaField to check the field declaration. Unlike setArgs which lets us modify the fieldType-properties, checkSchemaField doesn't have a great way to modify the SchemaField (afaict), so I had to settle for throwing SolrException's there. {quote} {quote}I don't love the inconsistency between the two bullet points above - it seems wrong to warn-and-correct bad settings but throw exceptions when those same bad settings exist on the . Should we throw errors on both cases? Or if we'd prefer to warn-and-correct on both cases, is there a way to modify the SchemaField to make corrections from within checkSchemaField? {quote} Hmmm, yeah ... SchemaFields are really ment to be immutable, so we'd have to add something like 'setProperties(int)' to SchemaField to be able to allow checkSchemaField to modify it but i don't relaly like the that idea. Or we could change the api of checkSchemaField so that can return the 'final' properties the SchemaField should use? ... i hate that idea less; but changing the SchemaField / FieldType apis like that kind of feels like a "disproportionate" response to the problem at hand. *Odds are, most people aren't going out of their way to set weird options on these fieldType/field declarations – the main problem is really that the "defaults" solr "assumes" are wrong at the Lucene level. Fixing those SchemaField properties so things like the query parser work properly is what's really important. warning/failing if the user specifies something that's going to be ignored anyway is secondary.* So i think you're probably right, and we should probably throw an exception in both cases Odds are most people will never see those failure "on upgrade" because they probably haven't mucked with the field/fieldType settings for these FieldTypes. and if they do we're doing them a favor by preventing weird ass erors like the one that prompted this Jira In which case, going back to my suggestion above: we should probably WARN if any of the "hardcodedDefault" attributes are specified _even using the same value as the one that's "hardcoded"_ but FAIL if the value is specified *AND* differs from the hardcoded default ... right? > [* TO *] queries on DateRange fields miss results > - > > Key: SOLR-14859 > URL: https://issues.apache.org/jira/browse/SOLR-14859 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: query parsers >Affects Versions: 8.5 >Reporter: Jason Gerlowski >Assignee: Jason Gerlowski >
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1877: SOLR-13181: param macro expansion could throw
dsmiley commented on a change in pull request #1877: URL: https://github.com/apache/lucene-solr/pull/1877#discussion_r490438268 ## File path: solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java ## @@ -154,14 +155,14 @@ else if (idx < 0) { } if (matchedStart > 0) { -sb.append(val.substring(start, matchedStart)); +sb.append(val, start, matchedStart); } // update "start" to be at the end of ${...} - start = rbrace + 1; + idx = start = rbrace + 1; - // String inbetween = val.substring(idx, rbrace); - StrParser parser = new StrParser(val, idx, rbrace); + // String in-between = val.substring(idx, rbrace); Review comment: ```suggestion // String in-between braces ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #1877: SOLR-13181: param macro expansion could throw
dsmiley commented on a change in pull request #1877: URL: https://github.com/apache/lucene-solr/pull/1877#discussion_r490430100 ## File path: solr/core/src/java/org/apache/solr/request/macro/MacroExpander.java ## @@ -188,7 +189,7 @@ else if (failOnMissingParams) { } catch (SyntaxError syntaxError) { // append the part we would have skipped -sb.append( val.substring(matchedStart, start) ); +sb.append(val, matchedStart, start); Review comment: IntelliJ pointed this out to me :-) It didn't know about the val.substring(start) (implied val.length()) so I did those manually. Yes, it'd be nice if everywhere we could do this in one go. Feel free to file an issue to do so if you have time. IntelliJ can make quick work of that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197844#comment-17197844 ] ASF subversion and git services commented on LUCENE-9529: - Commit 830bd186a8d72ce6cc96f2856c269ef02e98d3c5 in lucene-solr's branch refs/heads/branch_8x from Adrien Grand [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=830bd18 ] LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks. (#1882) The problem of tracking dirtiness via numbers of chunks is that larger chunks make stored fields readers more likely to be considered dirty, so I'm trying to work around it by tracking numbers of docs instead. > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197831#comment-17197831 ] ASF subversion and git services commented on LUCENE-9529: - Commit 33f728007853ee53c71f259206734bfd9155593d in lucene-solr's branch refs/heads/master from Adrien Grand [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=33f7280 ] LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks. (#1882) The problem of tracking dirtiness via numbers of chunks is that larger chunks make stored fields readers more likely to be considered dirty, so I'm trying to work around it by tracking numbers of docs instead. > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197843#comment-17197843 ] ASF subversion and git services commented on LUCENE-9529: - Commit 33f728007853ee53c71f259206734bfd9155593d in lucene-solr's branch refs/heads/master from Adrien Grand [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=33f7280 ] LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks. (#1882) The problem of tracking dirtiness via numbers of chunks is that larger chunks make stored fields readers more likely to be considered dirty, so I'm trying to work around it by tracking numbers of docs instead. > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197829#comment-17197829 ] ASF subversion and git services commented on LUCENE-9529: - Commit 33f728007853ee53c71f259206734bfd9155593d in lucene-solr's branch refs/heads/master from Adrien Grand [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=33f7280 ] LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks. (#1882) The problem of tracking dirtiness via numbers of chunks is that larger chunks make stored fields readers more likely to be considered dirty, so I'm trying to work around it by tracking numbers of docs instead. > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197820#comment-17197820 ] ASF subversion and git services commented on LUCENE-9529: - Commit 33f728007853ee53c71f259206734bfd9155593d in lucene-solr's branch refs/heads/master from Adrien Grand [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=33f7280 ] LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks. (#1882) The problem of tracking dirtiness via numbers of chunks is that larger chunks make stored fields readers more likely to be considered dirty, so I'm trying to work around it by tracking numbers of docs instead. > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9529) Larger stored fields block sizes mean we're more likely to disable optimized bulk merging
[ https://issues.apache.org/jira/browse/LUCENE-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197816#comment-17197816 ] ASF subversion and git services commented on LUCENE-9529: - Commit 33f728007853ee53c71f259206734bfd9155593d in lucene-solr's branch refs/heads/master from Adrien Grand [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=33f7280 ] LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks. (#1882) The problem of tracking dirtiness via numbers of chunks is that larger chunks make stored fields readers more likely to be considered dirty, so I'm trying to work around it by tracking numbers of docs instead. > Larger stored fields block sizes mean we're more likely to disable optimized > bulk merging > - > > Key: LUCENE-9529 > URL: https://issues.apache.org/jira/browse/LUCENE-9529 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Whenever possible when merging stored fields, Lucene tries to copy the > compressed data instead of decompressing the source segment to then > re-compressing in the destination segment. A problem with this approach is > that if some blocks are incomplete (typically the last block of a segment) > then it remains incomplete in the destination segment too, and if we do it > for too long we end up with a bad compression ratio. So Lucene keeps track of > these incomplete blocks, and makes sure to keep a ratio of incomplete blocks > below 1%. > But as we increased the block size, it has become more likely to have a high > ratio of incomplete blocks. E.g. if you have a segment with 1MB of stored > fields, with 16kB blocks like before, you have 63 complete blocks and 1 > incomplete block, or 1.6%. But now with ~512kB blocks, you have one complete > block and 1 incomplete block, ie. 50%. > I'm not sure how to fix it or even whether it should be fixed but wanted to > open an issue to track this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz merged pull request #1882: LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks.
jpountz merged pull request #1882: URL: https://github.com/apache/lucene-solr/pull/1882 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz merged pull request #1888: Further tune Lucene87StoredFieldsFormat for small documents.
jpountz merged pull request #1888: URL: https://github.com/apache/lucene-solr/pull/1888 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14859) [* TO *] queries on DateRange fields miss results
[ https://issues.apache.org/jira/browse/SOLR-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gerlowski updated SOLR-14859: --- Attachment: SOLR-14859.patch Status: Open (was: Open) Thanks for helping lay out an approach here guys. I've attached a patch which tries to follow that. No tests yet, but I'll add those once we agree where we want to log vs throw errors. Some notes: * The added logic lives in AbstractSpatialPrefixTreeFieldType so that it fires for all field types which use PrefixTreeStrategy, not just DateRangeField. * The patch uses {{setArgs}} to check the fieldType declaration. There we log warnings on any explicit fieldType options that contradict the hardcoded FieldType, and we set property defaults that match the FieldType. * The patch uses {{checkSchemaField}} to check the field declaration. Unlike {{setArgs}} which lets us modify the fieldType-properties, {{checkSchemaField}} doesn't have a great way to modify the SchemaField (afaict), so I had to settle for throwing SolrException's there. * I don't love the inconsistency between the two bullet points above - it seems wrong to warn-and-correct bad {{}} settings but throw exceptions when those same bad settings exist on the {{}}. Should we throw errors on both cases? Or if we'd prefer to warn-and-correct on both cases, is there a way to modify the SchemaField to make corrections from within checkSchemaField? * There's a couple NOCOMMIT comments where I'd appreciate feedback from any reviewers. Pending any feedback I'll aim to make the fieldType and field checking consistent by having them both throw SolrExceptions on any explicit settings that contradict the hardcoded FieldType. I'm happy to standardize the other way too though if anyone has suggestions on how to modify the SchemaField from within {{checkSchemaFields}} > [* TO *] queries on DateRange fields miss results > - > > Key: SOLR-14859 > URL: https://issues.apache.org/jira/browse/SOLR-14859 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: query parsers >Affects Versions: 8.5 >Reporter: Jason Gerlowski >Assignee: Jason Gerlowski >Priority: Major > Attachments: SOLR-14859.patch, SOLR-14859.patch, query-debug.png, > reproduce.sh, schema.png > > > "exists" queries ({{[* TO *]}}) on DateRange fields return 0 results > regardless of docs in the index with values in that field. > The issue appears to be that the query is converted into a > {{NormsFieldExistsQuery}}, even though DateRangeField uses omitNorms=true by > default. Probably introduced by SOLR-11746's changes to these optimizable > range queries. > I've attached a script to reproduce the issue (tested on Solr 8.6.2) and > screenshots showing showing schema and query-parsing info for the > reproduction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on pull request #1882: LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks.
rmuir commented on pull request #1882: URL: https://github.com/apache/lucene-solr/pull/1882#issuecomment-694339576 +1 We can always try to improve this thing later in subsequent issues. This still keeps a hard safety net on the number of dirty chunks, so I can't see it introducing any new worst-case behavior This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on pull request #1888: Further tune Lucene87StoredFieldsFormat for small documents.
rmuir commented on pull request #1888: URL: https://github.com/apache/lucene-solr/pull/1888#issuecomment-694337503 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9449) Skip non-competitive documents when sort by _doc with search after
[ https://issues.apache.org/jira/browse/LUCENE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197774#comment-17197774 ] ASF subversion and git services commented on LUCENE-9449: - Commit 754216821001772a0890d7245765e2ffb3f917f2 in lucene-solr's branch refs/heads/branch_8x from Mayya Sharipova [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7542168 ] LUCENE-9449 Skip docs with _doc sort and "after" (#1725) (#1856) - Enhance DocComparator to provide an iterator over competitive documents when searching with "after". This iterator can quickly position on the desired "after" document skipping all documents and segments before "after". - Redesign numeric comparators to move to separate package. Backport for #LUCENE-9449 > Skip non-competitive documents when sort by _doc with search after > -- > > Key: LUCENE-9449 > URL: https://issues.apache.org/jira/browse/LUCENE-9449 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Mayya Sharipova >Priority: Minor > Time Spent: 5h 20m > Remaining Estimate: 0h > > Enhance DocComparator to provide an iterator over competitive documents when > search ing with "after" FieldDoc. > This iterator can quickly position on the desired "after" document, and skip > all documents or even segments that contain documents before "after" > This is especially efficient when "after" is high. > > Related to LUCENE-9280 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova merged pull request #1856: LUCENE-9449 Skip docs with _doc sort and "after" (#1725)
mayya-sharipova merged pull request #1856: URL: https://github.com/apache/lucene-solr/pull/1856 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9449) Skip non-competitive documents when sort by _doc with search after
[ https://issues.apache.org/jira/browse/LUCENE-9449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197773#comment-17197773 ] ASF subversion and git services commented on LUCENE-9449: - Commit 754216821001772a0890d7245765e2ffb3f917f2 in lucene-solr's branch refs/heads/branch_8x from Mayya Sharipova [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7542168 ] LUCENE-9449 Skip docs with _doc sort and "after" (#1725) (#1856) - Enhance DocComparator to provide an iterator over competitive documents when searching with "after". This iterator can quickly position on the desired "after" document skipping all documents and segments before "after". - Redesign numeric comparators to move to separate package. Backport for #LUCENE-9449 > Skip non-competitive documents when sort by _doc with search after > -- > > Key: LUCENE-9449 > URL: https://issues.apache.org/jira/browse/LUCENE-9449 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Mayya Sharipova >Priority: Minor > Time Spent: 5h 20m > Remaining Estimate: 0h > > Enhance DocComparator to provide an iterator over competitive documents when > search ing with "after" FieldDoc. > This iterator can quickly position on the desired "after" document, and skip > all documents or even segments that contain documents before "after" > This is especially efficient when "after" is high. > > Related to LUCENE-9280 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova commented on a change in pull request #1856: LUCENE-9449 Skip docs with _doc sort and "after" (#1725)
mayya-sharipova commented on a change in pull request #1856: URL: https://github.com/apache/lucene-solr/pull/1856#discussion_r490355700 ## File path: lucene/CHANGES.txt ## @@ -59,6 +65,12 @@ Optimizations * LUCENE-9373: FunctionMatchQuery now accepts a "matchCost" optimization hint. (Maxim Glazkov, David Smiley) +* LUCENE-9449: Enhance DocComparator to provide an iterator over competitive + documents when searching with "after". This iterator can quickly position + on the desired "after" document skipping all documents and segments before + "after". Also redesign numeric comparators to provide skipping functionality + by default. (Mayya Sharipova, Jim Ferenczi) Review comment: Addressed in the last commit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on pull request #1758: URL: https://github.com/apache/lucene-solr/pull/1758#issuecomment-694280735 Would love to see as part of this PR an example of an event listener doing something (remotely) useful, to get a better feel of the whole process. That sample code could be removed later when "real" code doing event listening is introduced. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-14656) Deprecate current autoscaling framework, remove from master
[ https://issues.apache.org/jira/browse/SOLR-14656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya reassigned SOLR-14656: --- Assignee: (was: Ishan Chattopadhyaya) > Deprecate current autoscaling framework, remove from master > --- > > Key: SOLR-14656 > URL: https://issues.apache.org/jira/browse/SOLR-14656 > Project: Solr > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.7 > > Attachments: Screenshot from 2020-07-18 07-49-01.png > > Time Spent: 3h > Remaining Estimate: 0h > > The autoscaling framework is being re-designed in SOLR-14613 (SIP: > https://cwiki.apache.org/confluence/display/SOLR/SIP-8+Autoscaling+policy+engine+V2). > The current autoscaling framework is very inefficient, improperly designed > and too bloated and doesn't receive the level of support we aspire to provide > for all components that we ship. > This issue is to deprecate current autoscaling framework in 8x, so we can > focus on the new autoscaling framework afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758: URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490294923 ## File path: solr/core/src/java/org/apache/solr/cluster/events/ClusterPropertiesChangedEvent.java ## @@ -0,0 +1,35 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.cluster.events; + +import java.util.Map; + +/** + * Event generated when {@link org.apache.solr.common.cloud.ZkStateReader#CLUSTER_PROPS} is modified. + */ +public interface ClusterPropertiesChangedEvent extends ClusterEvent { + + @Override + default EventType getType() { +return EventType.CLUSTER_PROPERTIES_CHANGED; + } + + Map getOldClusterProperties(); Review comment: Is it really the role of the event to provide the old vs new properties? I'd prefer the event to provide the change only, let the subscriber fetch the new state if it wants to assemble the before/after state (and possibly the listener is keeping its own copy of what the old state is, and that old state is not necessarily equal to the old state as known by the change event implementation). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758: URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490288384 ## File path: solr/core/src/java/org/apache/solr/cluster/events/ClusterEventProducer.java ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.cluster.events; + +import org.apache.solr.cloud.ClusterSingleton; + +import java.util.Collections; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.ConcurrentHashMap; + +/** + * Component that produces {@link ClusterEvent} instances. + */ +public interface ClusterEventProducer extends ClusterSingleton { + + String PLUGIN_NAME = "clusterEventProducer"; + + /** + * Returns a modifiable map of event types and listeners to process events + * of a given type. + */ + Map> getEventListeners(); Review comment: What about using an `EnumMap` here instead? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758: URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490285588 ## File path: solr/core/src/java/org/apache/solr/cluster/events/ClusterEventListener.java ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.cluster.events; + +import java.util.Set; + +/** + * Components that want to be notified of cluster-wide events should use this. + * + * XXX should this work only for ClusterSingleton-s? some types of events may be + * XXX difficult (or pointless) to propagate to every node. + */ +public interface ClusterEventListener { + + /** + * The types of events that this listener can process. + */ + Set getEventTypes(); Review comment: Similarly, it might be useful to unregister a listener on some event types but not all... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758: URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490284824 ## File path: solr/core/src/java/org/apache/solr/cluster/events/ClusterEventListener.java ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.cluster.events; + +import java.util.Set; + +/** + * Components that want to be notified of cluster-wide events should use this. + * + * XXX should this work only for ClusterSingleton-s? some types of events may be + * XXX difficult (or pointless) to propagate to every node. + */ +public interface ClusterEventListener { + + /** + * The types of events that this listener can process. + */ + Set getEventTypes(); Review comment: Why have event types listened to as a listener attribute rather than as a parameter passed upon registration? Even types part of the listener make it harder to reuse listeners. For example a listener that logs the event and that can be registered wherever needed (this model would force subclassing it each time). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc commented on a change in pull request #1758: SOLR-14749: Provide a clean API for cluster-level event processing, Initial draft.
murblanc commented on a change in pull request #1758: URL: https://github.com/apache/lucene-solr/pull/1758#discussion_r490279251 ## File path: solr/core/src/java/org/apache/solr/cluster/events/CollectionsAddedEvent.java ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.cluster.events; + +import java.util.Collection; + +/** + * Event generated when some collections have been added. + */ +public interface CollectionsAddedEvent extends ClusterEvent { + + @Override + default EventType getType() { +return EventType.COLLECTIONS_ADDED; + } + + Collection getCollectionNames(); Review comment: Shall we move this to an iterator model? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on pull request #1881: Upgrade zookeeper version to 3.6.2 to use recent version of netty
ErickErickson commented on pull request #1881: URL: https://github.com/apache/lucene-solr/pull/1881#issuecomment-694260257 Does this have an associated Solr JIRA? I couldn't find one... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197714#comment-17197714 ] Noble Paul commented on SOLR-14151: --- This will go into 8.7 > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14151) Make schema components load from packages
[ https://issues.apache.org/jira/browse/SOLR-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197712#comment-17197712 ] Daniel Worley commented on SOLR-14151: -- [~noble.paul] Just tested and looks good! Will this make it into 8.7 or is that to be determined? Thanks! > Make schema components load from packages > - > > Key: SOLR-14151 > URL: https://issues.apache.org/jira/browse/SOLR-14151 > Project: Solr > Issue Type: Sub-task >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > Labels: packagemanager > Fix For: 8.7 > > Time Spent: 12h 50m > Remaining Estimate: 0h > > Example: > {code:xml} > > > >generateNumberParts="0" catenateWords="0" > catenateNumbers="0" catenateAll="0"/> > > > > > {code} > * When a package is updated, the entire {{IndexSchema}} object is refreshed, > but the SolrCore object is not reloaded > * Any component can be prefixed with the package name > * The semantics of loading plugins remain the same as that of the components > in {{solrconfig.xml}} > * Plugins can be registered using schema API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz opened a new pull request #1888: Further tune Lucene87StoredFieldsFormat for small documents.
jpountz opened a new pull request #1888: URL: https://github.com/apache/lucene-solr/pull/1888 The increase of the maximum number of chunks per doc done in previous issues was mostly random. I'd like to provide users with a similar trade-off with what the old versions of BEST_SPEED and BEST_COMPRESSION used to do. So since BEST_SPEED used to compress at most 128 docs at once, I think we should roughly make it 128*10 now. I made it 1024 to account for the fact that there is a preset dict as well that need decompressing. And similarly BEST_COMPRESSION used to allow 4x more docs than BEST_SPEED, so I made it 4096. With such larger numbers of docs per chunk, the decoding of metadata became a bottleneck for stored field access so I made it a bit faster by doing bulk decoding of the packed longs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] noblepaul opened a new pull request #1887: SOLR-14875: Make SolrEventListeners load from packages
noblepaul opened a new pull request #1887: URL: https://github.com/apache/lucene-solr/pull/1887 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14875) Make SolrEventListeners load from packages
[ https://issues.apache.org/jira/browse/SOLR-14875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-14875: -- Description: This is a hot reloaded plugin (does not result in a core reload) > Make SolrEventListeners load from packages > -- > > Key: SOLR-14875 > URL: https://issues.apache.org/jira/browse/SOLR-14875 > Project: Solr > Issue Type: Sub-task > Components: packages >Reporter: Noble Paul >Assignee: Noble Paul >Priority: Major > > This is a hot reloaded plugin (does not result in a core reload) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14875) Make SolrEventListeners load from packages
Noble Paul created SOLR-14875: - Summary: Make SolrEventListeners load from packages Key: SOLR-14875 URL: https://issues.apache.org/jira/browse/SOLR-14875 Project: Solr Issue Type: Sub-task Components: packages Reporter: Noble Paul Assignee: Noble Paul -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-13071) Add JWT Auth support in bin/solr
[ https://issues.apache.org/jira/browse/SOLR-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-13071: --- Description: Once SOLR-12121 gets in, we should add support to {{bin/solr}} start scripts so they can authenticate with Solr using a JWT token. A preferred way would perhaps be through {{solr.in.sh}} and add new {noformat} SOLR_AUTH_TYPE=token SOLR_AUTHENTICATION_OPTS=-DjwtToken= {noformat} A disadvantage with this method is that the user needs to know how to obtain the token, and the token needs to be long-lived. A more sophisticated way would be a {{bin/solr auth login}} command that opens a browser window with the IDP login screen and saves the short-lived access token and optionally refresh token, in the file system. was: Once SOLR-12121 gets in, we should add support to {{bin/solr}} start scripts so they can authenticate with Solr using a JWT token. A preferred way would perhaps be through {{solr.in.sh}} and add new {noformat} SOLR_AUTH_TYPE=token SOLR_AUTHENTICATION_OPTS=-DjwtToken={noformat} > Add JWT Auth support in bin/solr > > > Key: SOLR-13071 > URL: https://issues.apache.org/jira/browse/SOLR-13071 > Project: Solr > Issue Type: Improvement > Components: scripts and tools >Reporter: Jan Høydahl >Priority: Major > > Once SOLR-12121 gets in, we should add support to {{bin/solr}} start scripts > so they can authenticate with Solr using a JWT token. A preferred way would > perhaps be through {{solr.in.sh}} and add new > {noformat} > SOLR_AUTH_TYPE=token > SOLR_AUTHENTICATION_OPTS=-DjwtToken= > {noformat} > A disadvantage with this method is that the user needs to know how to obtain > the token, and the token needs to be long-lived. A more sophisticated way > would be a {{bin/solr auth login}} command that opens a browser window with > the IDP login screen and saves the short-lived access token and optionally > refresh token, in the file system. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] silvestrelosada commented on a change in pull request #1419: SOLR-14397: Vector Search in Solr
silvestrelosada commented on a change in pull request #1419: URL: https://github.com/apache/lucene-solr/pull/1419#discussion_r490229291 ## File path: solr/core/src/java/org/apache/solr/schema/DenseVectorField.java ## @@ -0,0 +1,198 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.solr.schema; + +import java.io.IOException; +import java.lang.invoke.MethodHandles; +import java.nio.ByteBuffer; +import java.nio.FloatBuffer; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import org.apache.lucene.document.StoredField; +import org.apache.lucene.index.IndexableField; +import org.apache.lucene.queries.function.ValueSource; +import org.apache.lucene.queries.function.valuesource.vectors.FloatVectorFieldSource; +import org.apache.lucene.queries.function.valuesource.vectors.FloatVectorFieldSource.Encoding; +import org.apache.lucene.search.SortField; +import org.apache.lucene.util.BytesRef; +import org.apache.lucene.util.BytesRefBuilder; +import org.apache.solr.common.params.MapSolrParams; +import org.apache.solr.common.params.SolrParams; +import org.apache.solr.response.TextResponseWriter; +import org.apache.solr.search.QParser; +import org.apache.solr.uninverting.UninvertingReader.Type; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import static org.apache.lucene.queries.function.valuesource.vectors.FloatVectorFieldSource.rawStringToVector; +import static org.apache.lucene.queries.function.valuesource.vectors.FloatVectorFieldSource.vectorToRawString; + +public class DenseVectorField extends FieldType { + + private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); + public static final String DIMENSIONS_PARAM = "dimensions"; + public static final int DEFAULT_DIMENSIONS = -1; //Don't enforce. + public static int dimensions; //-1 = don't validate + public static final String ENCODING = "encoding"; Review comment: I think it shouldnt be static, if you declare several fields on schema, the same encoder will be applied to all of them This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14613) Provide a clean API for pluggable replica assignment implementations
[ https://issues.apache.org/jira/browse/SOLR-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197672#comment-17197672 ] ASF subversion and git services commented on SOLR-14613: Commit dbba48b3e56be541572f170a9fae95d039236c1a in lucene-solr's branch refs/heads/master from Ilan Ginzburg [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=dbba48b ] SOLR-14613: use set-placement-plugin for both setting and unsetting plugin config > Provide a clean API for pluggable replica assignment implementations > > > Key: SOLR-14613 > URL: https://issues.apache.org/jira/browse/SOLR-14613 > Project: Solr > Issue Type: Improvement > Components: AutoScaling >Reporter: Andrzej Bialecki >Assignee: Ilan Ginzburg >Priority: Major > Time Spent: 41h > Remaining Estimate: 0h > > As described in SIP-8 the current autoscaling Policy implementation has > several limitations that make it difficult to use for very large clusters and > very large collections. SIP-8 also mentions the possible migration path by > providing alternative implementations of the placement strategies that are > less complex but more efficient in these very large environments. > We should review the existing APIs that the current autoscaling engine uses > ({{SolrCloudManager}} , {{AssignStrategy}} , {{Suggester}} and related > interfaces) to see if they provide a sufficient and minimal API for plugging > in alternative autoscaling placement strategies, and if necessary refactor > the existing APIs. > Since these APIs are internal it should be possible to do this without > breaking back-compat. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc merged pull request #1885: SOLR-14613: use set-placement-plugin for both setting and unsetting plugin config
murblanc merged pull request #1885: URL: https://github.com/apache/lucene-solr/pull/1885 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14792) Remove VelocityResponseWriter from Solr 9
[ https://issues.apache.org/jira/browse/SOLR-14792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197659#comment-17197659 ] ASF subversion and git services commented on SOLR-14792: Commit 2364a7aded9f04137eeb8520d69ffe97f7caef66 in lucene-solr's branch refs/heads/master from Erik Hatcher [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2364a7a ] SOLR-14792: Remove VelocityResponseWriter > Remove VelocityResponseWriter from Solr 9 > - > > Key: SOLR-14792 > URL: https://issues.apache.org/jira/browse/SOLR-14792 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 9 >Reporter: Erik Hatcher >Priority: Blocker > Time Spent: 1h > Remaining Estimate: 0h > > VelocityResponseWriter was deprecated in SOLR-14065. It can now be removed > from 9's code branch. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9528) Clean up obsolete and commented-out cruft from StandardSyntaxParser.jj
[ https://issues.apache.org/jira/browse/LUCENE-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197650#comment-17197650 ] Erick Erickson commented on LUCENE-9528: Theoretically I suppose some kind of versioning makes sense. OTOH, how many problems have we really had because it's not versioned? My guess is that it's not really worth the effort at this point, but I could be persuaded either way. > Clean up obsolete and commented-out cruft from StandardSyntaxParser.jj > -- > > Key: LUCENE-9528 > URL: https://issues.apache.org/jira/browse/LUCENE-9528 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The indentation in that file is crazy. So are micro-optimizations which make > reading the syntax parser much more difficult than it actually is. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (LUCENE-9528) Clean up obsolete and commented-out cruft from StandardSyntaxParser.jj
[ https://issues.apache.org/jira/browse/LUCENE-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss reassigned LUCENE-9528: --- Assignee: Dawid Weiss > Clean up obsolete and commented-out cruft from StandardSyntaxParser.jj > -- > > Key: LUCENE-9528 > URL: https://issues.apache.org/jira/browse/LUCENE-9528 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The indentation in that file is crazy. So are micro-optimizations which make > reading the syntax parser much more difficult than it actually is. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-14873) The SOLR 8 AND JDK Crashed by update document
[ https://issues.apache.org/jira/browse/SOLR-14873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-14873. --- Resolution: Won't Fix Please raise questions like this on the user's list, we try to reserve JIRAs for known bugs/enhancements rather than usage questions. The JIRA system is not a support portal. See: http://lucene.apache.org/solr/community.html#mailing-lists-irc there are links to both Lucene and Solr mailing lists there. A _lot_ more people will see your question on that list and may be able to help more quickly. If it's determined that this really is a code issue or enhancement to Lucene or Solr and not a configuration/usage problem, we can raise a new JIRA or reopen this one. This is _highly_ likely to be a corrupt JVM and/or operating system and/or configuration issue. So first I'd try this on a stand-alone Solr, then an isolated Docker etc. > The SOLR 8 AND JDK Crashed by update document > - > > Key: SOLR-14873 > URL: https://issues.apache.org/jira/browse/SOLR-14873 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: update, UpdateRequestProcessors >Affects Versions: 8.5.2, 8.6.2 > Environment: Server 1: > * OS: CentOS 7 > * CPU: 24 core > * Mem: 64G > Workstation: > * OS: Ubuntu 20.04 (vmware) > * Mem: 32G(guest 16G) > * CPU: 2 Core > Docker: 19.03 > docker-compose: 1.26.2 > >Reporter: Leo >Priority: Blocker > Attachments: hs_err_pid13.log > > > We tried to upgrade the current search engine to 8.6, but the system crashed > while testing indexing documents.This happens every time a large file is > indexing, the system crashes when the document is more than 5M. > We deploy SOLR based on cloud mode and run it based on Docker. Each instance > allocates 8G of memory. > This problem also exists in SOLR 8.5. > I have uploaded the report generated by the system to the attachment. > Please help me fix it. Thanks! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9531) Consolidate duplicated generated classes CharStream and FastCharStream
[ https://issues.apache.org/jira/browse/LUCENE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197648#comment-17197648 ] Dawid Weiss commented on LUCENE-9531: - I think it's ready. This does affect downstream API consumers but porting should be trivial (changed import) and the benefits of having one implementation to worry about seem to outweigh the cons. > Consolidate duplicated generated classes CharStream and FastCharStream > -- > > Key: LUCENE-9531 > URL: https://issues.apache.org/jira/browse/LUCENE-9531 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (SOLR-14874) Add API support to add/edit issuers for JWT Auth
Jan Høydahl created SOLR-14874: -- Summary: Add API support to add/edit issuers for JWT Auth Key: SOLR-14874 URL: https://issues.apache.org/jira/browse/SOLR-14874 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: Authentication Reporter: Jan Høydahl In https://lucene.apache.org/solr/guide/8_4/jwt-authentication-plugin.html#editing-jwt-authentication-plugin-configuration we document that: bq. There is currently no support for adding multiple token issuers though REST API This issue will take care of adding such support. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-6152) Pre-populating values into search parameters on the query page of solr admin
[ https://issues.apache.org/jira/browse/SOLR-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197644#comment-17197644 ] Jakob Furrer commented on SOLR-6152: New patch. The latest observations by [~janhoy] have all been addressed. > Pre-populating values into search parameters on the query page of solr admin > > > Key: SOLR-6152 > URL: https://issues.apache.org/jira/browse/SOLR-6152 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Affects Versions: 4.3.1 >Reporter: Dmitry Kan >Assignee: Jan Høydahl >Priority: Major > Attachments: SOLR-6152.patch, SOLR-6152.patch, SOLR-6152.patch, > SOLR-6152.patch, SOLR-6152.patch, copy_url_to_clipboard.png, > copy_url_to_clipboard_v2.png, > prefilling_and_extending_the_multivalue_parameter_fq.png, > prepoluate_query_parameters_query_page.bmp > > > In some use cases, it is highly desirable to be able to pre-populate the > query page of solr admin with specific values. > In particular use case of mine, the solr admin user must pass a date range > value without which the query would fail. > It isn't easy to remember the value format for non-solr experts, so I would > like to have a way of hooking that value "example" into the query page. > See the screenshot attached, where I have inserted the fq parameter with date > range into the Raw Query Parameters. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-6152) Pre-populating values into search parameters on the query page of solr admin
[ https://issues.apache.org/jira/browse/SOLR-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Furrer updated SOLR-6152: --- Attachment: SOLR-6152.patch > Pre-populating values into search parameters on the query page of solr admin > > > Key: SOLR-6152 > URL: https://issues.apache.org/jira/browse/SOLR-6152 > Project: Solr > Issue Type: Improvement > Components: Admin UI >Affects Versions: 4.3.1 >Reporter: Dmitry Kan >Assignee: Jan Høydahl >Priority: Major > Attachments: SOLR-6152.patch, SOLR-6152.patch, SOLR-6152.patch, > SOLR-6152.patch, SOLR-6152.patch, copy_url_to_clipboard.png, > copy_url_to_clipboard_v2.png, > prefilling_and_extending_the_multivalue_parameter_fq.png, > prepoluate_query_parameters_query_page.bmp > > > In some use cases, it is highly desirable to be able to pre-populate the > query page of solr admin with specific values. > In particular use case of mine, the solr admin user must pass a date range > value without which the query would fail. > It isn't easy to remember the value format for non-solr experts, so I would > like to have a way of hooking that value "example" into the query page. > See the screenshot attached, where I have inserted the fq parameter with date > range into the Raw Query Parameters. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss opened a new pull request #1886: LUCENE-9531: Consolidate duplicated generated classes CharStream and FastCharStream
dweiss opened a new pull request #1886: URL: https://github.com/apache/lucene-solr/pull/1886 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] jpountz commented on pull request #1882: LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks.
jpountz commented on pull request #1882: URL: https://github.com/apache/lucene-solr/pull/1882#issuecomment-694177685 Agreed, I just pushed a change to take care of term vectors as well that I had locally. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] rmuir commented on pull request #1882: LUCENE-9529: Track dirtiness of stored fields via a number of docs, not chunks.
rmuir commented on pull request #1882: URL: https://github.com/apache/lucene-solr/pull/1882#issuecomment-694177068 we may want to apply new logic to the CompressingTermVectorsWriter as well for consistency? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Assigned] (SOLR-14656) Deprecate current autoscaling framework, remove from master
[ https://issues.apache.org/jira/browse/SOLR-14656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya reassigned SOLR-14656: --- Assignee: Ishan Chattopadhyaya > Deprecate current autoscaling framework, remove from master > --- > > Key: SOLR-14656 > URL: https://issues.apache.org/jira/browse/SOLR-14656 > Project: Solr > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Attachments: Screenshot from 2020-07-18 07-49-01.png > > Time Spent: 3h > Remaining Estimate: 0h > > The autoscaling framework is being re-designed in SOLR-14613 (SIP: > https://cwiki.apache.org/confluence/display/SOLR/SIP-8+Autoscaling+policy+engine+V2). > The current autoscaling framework is very inefficient, improperly designed > and too bloated and doesn't receive the level of support we aspire to provide > for all components that we ship. > This issue is to deprecate current autoscaling framework in 8x, so we can > focus on the new autoscaling framework afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9531) Consolidate duplicated generated classes CharStream and FastCharStream
[ https://issues.apache.org/jira/browse/LUCENE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197639#comment-17197639 ] Dawid Weiss commented on LUCENE-9531: - There are four nearly-exact copies of CharStream (javacc-generated interface) and FastCharStream (Lucene's implementation). This duplication is unnecessary. We can just point at the right implementation/ interface during generation. > Consolidate duplicated generated classes CharStream and FastCharStream > -- > > Key: LUCENE-9531 > URL: https://issues.apache.org/jira/browse/LUCENE-9531 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9531) Consolidate duplicated generated classes CharStream and FastCharStream
Dawid Weiss created LUCENE-9531: --- Summary: Consolidate duplicated generated classes CharStream and FastCharStream Key: LUCENE-9531 URL: https://issues.apache.org/jira/browse/LUCENE-9531 Project: Lucene - Core Issue Type: Improvement Reporter: Dawid Weiss Assignee: Dawid Weiss -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14656) Deprecate current autoscaling framework, remove from master
[ https://issues.apache.org/jira/browse/SOLR-14656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated SOLR-14656: Fix Version/s: 8.7 > Deprecate current autoscaling framework, remove from master > --- > > Key: SOLR-14656 > URL: https://issues.apache.org/jira/browse/SOLR-14656 > Project: Solr > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.7 > > Attachments: Screenshot from 2020-07-18 07-49-01.png > > Time Spent: 3h > Remaining Estimate: 0h > > The autoscaling framework is being re-designed in SOLR-14613 (SIP: > https://cwiki.apache.org/confluence/display/SOLR/SIP-8+Autoscaling+policy+engine+V2). > The current autoscaling framework is very inefficient, improperly designed > and too bloated and doesn't receive the level of support we aspire to provide > for all components that we ship. > This issue is to deprecate current autoscaling framework in 8x, so we can > focus on the new autoscaling framework afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9531) Consolidate duplicated generated classes CharStream and FastCharStream
[ https://issues.apache.org/jira/browse/LUCENE-9531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-9531: Parent: LUCENE-9526 Issue Type: Sub-task (was: Improvement) > Consolidate duplicated generated classes CharStream and FastCharStream > -- > > Key: LUCENE-9531 > URL: https://issues.apache.org/jira/browse/LUCENE-9531 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14656) Deprecate current autoscaling framework, remove from master
[ https://issues.apache.org/jira/browse/SOLR-14656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197638#comment-17197638 ] Ishan Chattopadhyaya commented on SOLR-14656: - Thanks [~ab], would you like to add the notices? > Deprecate current autoscaling framework, remove from master > --- > > Key: SOLR-14656 > URL: https://issues.apache.org/jira/browse/SOLR-14656 > Project: Solr > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Fix For: 8.7 > > Attachments: Screenshot from 2020-07-18 07-49-01.png > > Time Spent: 3h > Remaining Estimate: 0h > > The autoscaling framework is being re-designed in SOLR-14613 (SIP: > https://cwiki.apache.org/confluence/display/SOLR/SIP-8+Autoscaling+policy+engine+V2). > The current autoscaling framework is very inefficient, improperly designed > and too bloated and doesn't receive the level of support we aspire to provide > for all components that we ship. > This issue is to deprecate current autoscaling framework in 8x, so we can > focus on the new autoscaling framework afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9527) Upgrade javacc to 7.0.4
[ https://issues.apache.org/jira/browse/LUCENE-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197635#comment-17197635 ] ASF subversion and git services commented on LUCENE-9527: - Commit 6c9d7adf7985e4b895a80e561f1a3bb5820126a8 in lucene-solr's branch refs/heads/master from Dawid Weiss [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6c9d7ad ] LUCENE-9527: upgrade javacc to 7.0.4 (#1884) > Upgrade javacc to 7.0.4 > --- > > Key: LUCENE-9527 > URL: https://issues.apache.org/jira/browse/LUCENE-9527 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14656) Deprecate current autoscaling framework, remove from master
[ https://issues.apache.org/jira/browse/SOLR-14656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya updated SOLR-14656: Priority: Blocker (was: Major) > Deprecate current autoscaling framework, remove from master > --- > > Key: SOLR-14656 > URL: https://issues.apache.org/jira/browse/SOLR-14656 > Project: Solr > Issue Type: Improvement >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Blocker > Attachments: Screenshot from 2020-07-18 07-49-01.png > > Time Spent: 3h > Remaining Estimate: 0h > > The autoscaling framework is being re-designed in SOLR-14613 (SIP: > https://cwiki.apache.org/confluence/display/SOLR/SIP-8+Autoscaling+policy+engine+V2). > The current autoscaling framework is very inefficient, improperly designed > and too bloated and doesn't receive the level of support we aspire to provide > for all components that we ship. > This issue is to deprecate current autoscaling framework in 8x, so we can > focus on the new autoscaling framework afresh. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9527) Upgrade javacc to 7.0.4
[ https://issues.apache.org/jira/browse/LUCENE-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss resolved LUCENE-9527. - Fix Version/s: master (9.0) Assignee: Dawid Weiss Resolution: Fixed > Upgrade javacc to 7.0.4 > --- > > Key: LUCENE-9527 > URL: https://issues.apache.org/jira/browse/LUCENE-9527 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: master (9.0) > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss merged pull request #1884: LUCENE-9527: upgrade javacc to 7.0.4
dweiss merged pull request #1884: URL: https://github.com/apache/lucene-solr/pull/1884 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] murblanc opened a new pull request #1885: SOLR-14613: use set-placement-plugin for both setting and unsetting plugin config
murblanc opened a new pull request #1885: URL: https://github.com/apache/lucene-solr/pull/1885 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss opened a new pull request #1884: LUCENE-9527: upgrade javacc to 7.0.4
dweiss opened a new pull request #1884: URL: https://github.com/apache/lucene-solr/pull/1884 # Description Please provide a short description of the changes you're making with this pull request. # Solution Please provide a short description of the approach taken to implement your solution. # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [ ] I have developed this patch against the `master` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9527) Upgrade javacc to 7.0.4
[ https://issues.apache.org/jira/browse/LUCENE-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-9527: Summary: Upgrade javacc to 7.0.4 (was: Upgrade javacc to 7.0.5) > Upgrade javacc to 7.0.4 > --- > > Key: LUCENE-9527 > URL: https://issues.apache.org/jira/browse/LUCENE-9527 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9527) Upgrade javacc to 7.0.4
[ https://issues.apache.org/jira/browse/LUCENE-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197600#comment-17197600 ] Dawid Weiss commented on LUCENE-9527: - Reverting to 7.0.4. Any javacc between 7.0.5 and 7.0.9 is broken. See this issue: https://github.com/javacc/javacc/issues/183 > Upgrade javacc to 7.0.4 > --- > > Key: LUCENE-9527 > URL: https://issues.apache.org/jira/browse/LUCENE-9527 > Project: Lucene - Core > Issue Type: Sub-task >Reporter: Dawid Weiss >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org