[GitHub] [lucene-solr] muse-dev[bot] commented on a change in pull request #2276: Improve backwards compatibility tests for sorted indexes.

[GitBox]

muse-dev[bot] commented on a change in pull request #2276:
URL: https://github.com/apache/lucene-solr/pull/2276#discussion_r567189689



##
File path: 
lucene/backward-codecs/src/test/org/apache/lucene/backward_index/TestBackwardsCompatibility.java
##
@@ -1079,8 +1069,7 @@ public void searchIndex(Directory dir, String oldName, 
int minIndexMajorVersion)

Review comment:
   *NULL_DEREFERENCE:*  object `liveDocs` last assigned on line 1064 could 
be null and is dereferenced at line 1067.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] jtibshirani opened a new pull request #2276: Improve backwards compatibility tests for sorted indexes.

[GitBox]

jtibshirani opened a new pull request #2276:
URL: https://github.com/apache/lucene-solr/pull/2276


   This PR also cleans up some old checks that only applied to pre-6.0 indices.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-7642) Should launching Solr in cloud mode using a ZooKeeper chroot create the chroot znode if it doesn't exist?

[Gus Heck (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275457#comment-17275457
 ] 

Gus Heck commented on SOLR-7642:


ch does indeed mean change, but it's a reference to the unix chroot operation 
(https://en.wikipedia.org/wiki/Chroot). I think it should be createZkChRoot for 
consistency both with other documentation and with similar concepts at the OS 
level. 

For those familiar with chroot elsewhere it's read as "create zk chroot" 
meaning isolating the zk stuff to it's own sub-tree and preventing upward 
access. One could argue for not capitalizing the R, but I think we do 
capitalize elsewhere so best to be consistent.

> Should launching Solr in cloud mode using a ZooKeeper chroot create the 
> chroot znode if it doesn't exist?
> -
>
> Key: SOLR-7642
> URL: https://issues.apache.org/jira/browse/SOLR-7642
> Project: Solr
>  Issue Type: Improvement
>Reporter: Timothy Potter
>Priority: Minor
> Attachments: SOLR-7642.patch, SOLR-7642.patch, SOLR-7642.patch, 
> SOLR-7642.patch, SOLR-7642_tag_7.5.0.patch, 
> SOLR-7642_tag_7.5.0_proposition.patch
>
>
> If you launch Solr for the first time in cloud mode using a ZooKeeper 
> connection string that includes a chroot leads to the following 
> initialization error:
> {code}
> ERROR - 2015-06-05 17:15:50.410; [   ] org.apache.solr.common.SolrException; 
> null:org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified 
> in ZkHost but the znode doesn't exist. localhost:2181/lan
> at 
> org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:339)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:140)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:110)
> at 
> org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
> at 
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at 
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> {code}
> The work-around for this is to use the scripts/cloud-scripts/zkcli.sh script 
> to create the chroot znode (bootstrap action does this).
> I'm wondering if we shouldn't just create the znode if it doesn't exist? Or 
> is that some violation of using a chroot?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13209) NullPointerException from call in org.apache.solr.search.SolrIndexSearcher.getDocSet

[Cader Hancock (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275425#comment-17275425
 ] 

Cader Hancock edited comment on SOLR-13209 at 1/29/21, 11:41 PM:
-

I'm going to be tottaly honest, I have no idea what I'm doing. This is my first 
FOSS PR and first time with a _*real*_ codebase.  I ran a quick ripgrep regex 
through the repo to try to figure out where I should start looking to place the 
throw statement(s). 
{code:java}
rg -Ul 
'(?s)SolrIndexSearcher.*?\n.*?getDocSet|getDocSet.*?\n.*?SolrIndexSearcher' ./
{code}
Which gave me 
{code:java}
./solr/core/src/test/org/apache/solr/search/SolrIndexSearcherTest.java
./solr/core/src/test/org/apache/solr/search/TestSearchPerf.java
./solr/core/src/test/org/apache/solr/search/TestFiltering.java
./solr/core/src/java/org/apache/solr/util/SolrPluginUtils.java
./solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java
./solr/core/src/java/org/apache/solr/search/JoinQuery.java
./solr/core/src/java/org/apache/solr/search/Grouping.java
./solr/core/src/java/org/apache/solr/search/GraphTermsQParserPlugin.java
./solr/core/src/java/org/apache/solr/search/DocSetUtil.java
./solr/core/src/java/org/apache/solr/search/join/GraphQuery.java
./solr/core/src/java/org/apache/solr/search/join/CrossCollectionJoinQuery.java
./solr/core/src/java/org/apache/solr/search/grouping/CommandHandler.java
./solr/core/src/java/org/apache/solr/search/grouping/distributed/command/QueryCommand.java
./solr/core/src/java/org/apache/solr/search/facet/UnInvertedField.java
./solr/core/src/java/org/apache/solr/search/facet/FacetFieldProcessorByEnumTermsStream.java
./solr/core/src/java/org/apache/solr/search/facet/FacetProcessor.java
./solr/core/src/java/org/apache/solr/request/SimpleFacets.java
./solr/core/src/java/org/apache/solr/query/SolrRangeQuery.java
./solr/core/src/java/org/apache/solr/query/FilterQuery.java
./solr/core/src/java/org/apache/solr/handler/component/TermsComponent.java
./solr/core/src/java/org/apache/solr/handler/component/SpellCheckComponent.java
./solr/core/src/java/org/apache/solr/handler/component/PivotFacetProcessor.java
./solr/core/src/java/org/apache/solr/handler/component/PhrasesIdentificationComponent.java
./solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
./solr/core/src/java/org/apache/solr/handler/component/StatsField.java
./solr/core/src/java/org/apache/solr/handler/tagger/TaggerRequestHandler.java
./solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
./solr/contrib/analytics/src/java/org/apache/solr/handler/AnalyticsHandler.java
{code}
Do you have a reccomendation for where I should start digging?


was (Author: cader.hancock):
I'm going to be tottaly honest, I have no idea what I'm doing. This is my first 
FOSS PR and first time with a _*real*_ codebase.  I ran a quick ripgrep regex 
through the repo to try to figure out where I should start looking to place the 
throw statement(s). 
{code:java}
// rg -Ul 
'(?s)SolrIndexSearcher.*?\n.*?getDocSet|getDocSet.*?\n.*?SolrIndexSearcher' ./
{code}
Which gave me 
{code:java}
// ./solr/core/src/test/org/apache/solr/search/SolrIndexSearcherTest.java
./solr/core/src/test/org/apache/solr/search/TestSearchPerf.java
./solr/core/src/test/org/apache/solr/search/TestFiltering.java
./solr/core/src/java/org/apache/solr/util/SolrPluginUtils.java
./solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java
./solr/core/src/java/org/apache/solr/search/JoinQuery.java
./solr/core/src/java/org/apache/solr/search/Grouping.java
./solr/core/src/java/org/apache/solr/search/GraphTermsQParserPlugin.java
./solr/core/src/java/org/apache/solr/search/DocSetUtil.java
./solr/core/src/java/org/apache/solr/search/join/GraphQuery.java
./solr/core/src/java/org/apache/solr/search/join/CrossCollectionJoinQuery.java
./solr/core/src/java/org/apache/solr/search/grouping/CommandHandler.java
./solr/core/src/java/org/apache/solr/search/grouping/distributed/command/QueryCommand.java
./solr/core/src/java/org/apache/solr/search/facet/UnInvertedField.java
./solr/core/src/java/org/apache/solr/search/facet/FacetFieldProcessorByEnumTermsStream.java
./solr/core/src/java/org/apache/solr/search/facet/FacetProcessor.java
./solr/core/src/java/org/apache/solr/request/SimpleFacets.java
./solr/core/src/java/org/apache/solr/query/SolrRangeQuery.java
./solr/core/src/java/org/apache/solr/query/FilterQuery.java
./solr/core/src/java/org/apache/solr/handler/component/TermsComponent.java
./solr/core/src/java/org/apache/solr/handler/component/SpellCheckComponent.java
./solr/core/src/java/org/apache/solr/handler/component/PivotFacetProcessor.java
./solr/core/src/java/org/apache/solr/handler/component/PhrasesIdentificationComponent.java
./solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
./solr/core/src/java/org/apache/solr/handler/component/StatsField.java
./solr/co

[jira] [Commented] (SOLR-14978) Run OOM Killer Script in Foreground Solr

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275427#comment-17275427
 ] 

ASF subversion and git services commented on SOLR-14978:


Commit 1b0dc0dc59143c68ebe1126a1efc92374887f66d in lucene-solr's branch 
refs/heads/reference_impl from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1b0dc0d ]

SOLR-14978 OOM Killer in Foreground (#2055)

Combine Docker and bin/solr OOM handling scripts, move OOM handling to 
foreground Solr as well.

Co-authored-by: Houston Putman 


> Run OOM Killer Script in Foreground Solr
> 
>
> Key: SOLR-14978
> URL: https://issues.apache.org/jira/browse/SOLR-14978
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
>  Labels: operability
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> As discussed in SOLR-8145 before the advent of Docker containers, Solr did 
> not usually run unattended and in the foreground so there was not much reason 
> to run the OOM killer script in that mode. However, not it makes sense to 
> handle in foreground as well.
> We should also consider making it easier to configure heap dumps as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14978) Run OOM Killer Script in Foreground Solr

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275426#comment-17275426
 ] 

ASF subversion and git services commented on SOLR-14978:


Commit 1b0dc0dc59143c68ebe1126a1efc92374887f66d in lucene-solr's branch 
refs/heads/reference_impl_dev from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=1b0dc0d ]

SOLR-14978 OOM Killer in Foreground (#2055)

Combine Docker and bin/solr OOM handling scripts, move OOM handling to 
foreground Solr as well.

Co-authored-by: Houston Putman 


> Run OOM Killer Script in Foreground Solr
> 
>
> Key: SOLR-14978
> URL: https://issues.apache.org/jira/browse/SOLR-14978
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
>  Labels: operability
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> As discussed in SOLR-8145 before the advent of Docker containers, Solr did 
> not usually run unattended and in the foreground so there was not much reason 
> to run the OOM killer script in that mode. However, not it makes sense to 
> handle in foreground as well.
> We should also consider making it easier to configure heap dumps as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13209) NullPointerException from call in org.apache.solr.search.SolrIndexSearcher.getDocSet

[Cader Hancock (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275425#comment-17275425
 ] 

Cader Hancock commented on SOLR-13209:
--

I'm going to be tottaly honest, I have no idea what I'm doing. This is my first 
FOSS PR and first time with a _*real*_ codebase.  I ran a quick ripgrep regex 
through the repo to try to figure out where I should start looking to place the 
throw statement(s). 
{code:java}
// rg -Ul 
'(?s)SolrIndexSearcher.*?\n.*?getDocSet|getDocSet.*?\n.*?SolrIndexSearcher' ./
{code}
Which gave me 
{code:java}
// ./solr/core/src/test/org/apache/solr/search/SolrIndexSearcherTest.java
./solr/core/src/test/org/apache/solr/search/TestSearchPerf.java
./solr/core/src/test/org/apache/solr/search/TestFiltering.java
./solr/core/src/java/org/apache/solr/util/SolrPluginUtils.java
./solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java
./solr/core/src/java/org/apache/solr/search/JoinQuery.java
./solr/core/src/java/org/apache/solr/search/Grouping.java
./solr/core/src/java/org/apache/solr/search/GraphTermsQParserPlugin.java
./solr/core/src/java/org/apache/solr/search/DocSetUtil.java
./solr/core/src/java/org/apache/solr/search/join/GraphQuery.java
./solr/core/src/java/org/apache/solr/search/join/CrossCollectionJoinQuery.java
./solr/core/src/java/org/apache/solr/search/grouping/CommandHandler.java
./solr/core/src/java/org/apache/solr/search/grouping/distributed/command/QueryCommand.java
./solr/core/src/java/org/apache/solr/search/facet/UnInvertedField.java
./solr/core/src/java/org/apache/solr/search/facet/FacetFieldProcessorByEnumTermsStream.java
./solr/core/src/java/org/apache/solr/search/facet/FacetProcessor.java
./solr/core/src/java/org/apache/solr/request/SimpleFacets.java
./solr/core/src/java/org/apache/solr/query/SolrRangeQuery.java
./solr/core/src/java/org/apache/solr/query/FilterQuery.java
./solr/core/src/java/org/apache/solr/handler/component/TermsComponent.java
./solr/core/src/java/org/apache/solr/handler/component/SpellCheckComponent.java
./solr/core/src/java/org/apache/solr/handler/component/PivotFacetProcessor.java
./solr/core/src/java/org/apache/solr/handler/component/PhrasesIdentificationComponent.java
./solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java
./solr/core/src/java/org/apache/solr/handler/component/StatsField.java
./solr/core/src/java/org/apache/solr/handler/tagger/TaggerRequestHandler.java
./solr/contrib/ltr/src/java/org/apache/solr/ltr/feature/SolrFeature.java
./solr/contrib/analytics/src/java/org/apache/solr/handler/AnalyticsHandler.java
{code}
Do you have a reccomendation for where I should start digging?

> NullPointerException from call in 
> org.apache.solr.search.SolrIndexSearcher.getDocSet
> 
>
> Key: SOLR-13209
> URL: https://issues.apache.org/jira/browse/SOLR-13209
> Project: Solr
>  Issue Type: Bug
>Affects Versions: master (9.0)
> Environment: h1. Steps to reproduce
> * Use a Linux machine.
> * Build commit {{ea2c8ba}} of Solr as described in the section below.
> * Build the films collection as described below.
> * Start the server using the command {{./bin/solr start -f -p 8983 -s 
> /tmp/home}}
> * Request the URL given in the bug description.
> h1. Compiling the server
> {noformat}
> git clone https://github.com/apache/lucene-solr
> cd lucene-solr
> git checkout ea2c8ba
> ant compile
> cd solr
> ant server
> {noformat}
> h1. Building the collection and reproducing the bug
> We followed [Exercise 
> 2|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html#exercise-2] from 
> the [Solr 
> Tutorial|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html].
> {noformat}
> mkdir -p /tmp/home
> echo '' > 
> /tmp/home/solr.xml
> {noformat}
> In one terminal start a Solr instance in foreground:
> {noformat}
> ./bin/solr start -f -p 8983 -s /tmp/home
> {noformat}
> In another terminal, create a collection of movies, with no shards and no 
> replication, and initialize it:
> {noformat}
> bin/solr create -c films
> curl -X POST -H 'Content-type:application/json' --data-binary '{"add-field": 
> {"name":"name", "type":"text_general", "multiValued":false, "stored":true}}' 
> http://localhost:8983/solr/films/schema
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '{"add-copy-field" : {"source":"*","dest":"_text_"}}' 
> http://localhost:8983/solr/films/schema
> ./bin/post -c films example/films/films.json
> curl -v “URL_BUG”
> {noformat}
> Please check the issue description below to find the “URL_BUG” that will 
> allow you to reproduce the issue reported.
>Reporter: Cesar Rodriguez
>Priority: Minor
>  Labels: diffblue, newdev
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Requesting the following URL causes Solr to 

[jira] [Commented] (LUCENE-9686) TestDirectIODirectory#testFloatsUnderflow can fail assertion

[Julie Tibshirani (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275421#comment-17275421
 ] 

Julie Tibshirani commented on LUCENE-9686:
--

Thanks [~zacharymorn] for looking into this. It'd be great if to have someone 
review who's familiar with the original PR -- tagging [~dweiss] and 
[~uschindler] so they're aware.

> TestDirectIODirectory#testFloatsUnderflow can fail assertion
> 
>
> Key: LUCENE-9686
> URL: https://issues.apache.org/jira/browse/LUCENE-9686
> Project: Lucene - Core
>  Issue Type: Test
>Reporter: Julie Tibshirani
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Reproduction line:
> {code}
> ./gradlew test --tests TestDirectIODirectory.testFloatsUnderflow 
> -Dtests.seed=73B56EAB13269C91 -Dtests.slow=true -Dtests.badapples=true 
> -Dtests.locale=haw-US -Dtests.timezone=America/Inuvik -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
> {code}
> I didn't have the chance to look deeply, but it seems like the wrong 
> exception type is being thrown:
> {code:java}
>  junit.framework.AssertionFailedError: Unexpected exception type, expected 
> EOFException but got java.nio.BufferUnderflowException
> at 
> __randomizedtesting.SeedInfo.seed([73B56EAB13269C91:1FD75ACA1CD83E9C]:0)
> at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2895)
> at 
> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2876)
> at 
> org.apache.lucene.store.BaseDirectoryTestCase.testFloatsUnderflow(BaseDirectoryTestCase.java:291)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15123) bin/solr tools all have different styles of describing command opitons

[David Eric Pugh (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275419#comment-17275419
 ] 

David Eric Pugh commented on SOLR-15123:


[~ctargett]I'd love your review on the associated Pull Request.

> bin/solr tools all have different styles of describing command opitons
> --
>
> Key: SOLR-15123
> URL: https://issues.apache.org/jira/browse/SOLR-15123
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCLI
>Affects Versions: 8.7
>Reporter: David Eric Pugh
>Assignee: David Eric Pugh
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The various Tools exposed by the SolrCLI each have a different style to the 
> way the options are described.   It would be nice bit of polish if they all 
> had similar style descriptions.  This issue is to go through and make sure 
> they follow a consistent pattern, like all being complete sentences.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] epugh opened a new pull request #2275: SOLR-15123: Make all Tool option descriptions follow the same general pattern.

[GitBox]

epugh opened a new pull request #2275:
URL: https://github.com/apache/lucene-solr/pull/2275


   # Description
   different people have written different SolrCLI tools over time, and it 
shows in the user help.
   
   
   # Solution
   Copy edit all the tool options.
   
   # Tests
   
   Just docs.   In fact, it appears tools don't have tests.  ;-(
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-15123) bin/solr tools all have different styles of describing command opitons

[David Eric Pugh (Jira)]

David Eric Pugh created SOLR-15123:
--

 Summary: bin/solr tools all have different styles of describing 
command opitons
 Key: SOLR-15123
 URL: https://issues.apache.org/jira/browse/SOLR-15123
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrCLI
Affects Versions: 8.7
Reporter: David Eric Pugh
Assignee: David Eric Pugh


The various Tools exposed by the SolrCLI each have a different style to the way 
the options are described.   It would be nice bit of polish if they all had 
similar style descriptions.  This issue is to go through and make sure they 
follow a consistent pattern, like all being complete sentences.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9705) Move all codec formats to the o.a.l.codecs.Lucene90 package

[Mike Drob (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275415#comment-17275415
 ] 

Mike Drob commented on LUCENE-9705:
---

Right, I get that there are new codecs all the time. What I'm trying to 
understand is that if we do this now, and then for Lucene 9.1 we come up with 
some new improvement that changes one of the index formats does that mean that 
*all* of the formats have to be moved again? I thought it was nice having old 
formats easily distinguishable, so that a developer can look and see more 
easily what's new and what's not.

At the end of the day, I trust you to do the right thing, please don't take 
this as me trying to hold up the improvements. I want to make sure that if 
we're addressing technical debt, we're actually addressing it and not replacing 
it with other (potentially prettier?) technical debt.

> Move all codec formats to the o.a.l.codecs.Lucene90 package
> ---
>
> Key: LUCENE-9705
> URL: https://issues.apache.org/jira/browse/LUCENE-9705
> Project: Lucene - Core
>  Issue Type: Wish
>Reporter: Ignacio Vera
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Current formats are distributed in different packages, prefixed with the 
> Lucene version they were created. With the upcoming release of Lucene 9.0, it 
> would be nice to move all those formats to just the o.a.l.codecs.Lucene90 
> package (and of course moving the current ones to the backwards-codecs).
> This issue would actually facilitate moving the directory API to little 
> endian (LUCENE-9047) as the only codecs that would need to handle backwards 
> compatibility will be the codecs in backwards codecs.
> In addition, it can help formalising the use of internal versions vs format 
> versioning ( LUCENE-9616)
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] jtibshirani opened a new pull request #2274: LUCENE-9705: Create Lucene90LiveDocsFormat

[GitBox]

jtibshirani opened a new pull request #2274:
URL: https://github.com/apache/lucene-solr/pull/2274


   For now this is just a copy of Lucene50LiveDocsFormat. The existing
   Lucene50LiveDocsFormat was moved to backwards-codecs.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] jtibshirani commented on a change in pull request #2268: LUCENE-9705: Move Lucene50CompoundFormat to Lucene90CompoundFormat

[GitBox]

jtibshirani commented on a change in pull request #2268:
URL: https://github.com/apache/lucene-solr/pull/2268#discussion_r567116775



##
File path: 
lucene/backward-codecs/src/java/org/apache/lucene/backward_codecs/lucene50/Lucene50CompoundFormat.java
##
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.backward_codecs.lucene50;
+
+import java.io.IOException;
+import org.apache.lucene.codecs.CodecUtil;
+import org.apache.lucene.codecs.CompoundDirectory;
+import org.apache.lucene.codecs.CompoundFormat;
+import org.apache.lucene.index.SegmentInfo;
+import org.apache.lucene.store.DataOutput;
+import org.apache.lucene.store.Directory;
+import org.apache.lucene.store.IOContext;
+
+/**
+ * Lucene 5.0 compound file format
+ *
+ * Files:
+ *
+ * 
+ *   .cfs: An optional "virtual" file consisting of all the 
other index files for
+ *   systems that frequently run out of file handles.
+ *   .cfe: The "virtual" compound file's entry table holding 
all entries in the
+ *   corresponding .cfs file.
+ * 
+ *
+ * Description:
+ *
+ * 
+ *   Compound (.cfs) --> Header, FileData FileCount, Footer
+ *   Compound Entry Table (.cfe) --> Header, FileCount,  FileCount
+ *   Header --> {@link CodecUtil#writeIndexHeader IndexHeader}
+ *   FileCount --> {@link DataOutput#writeVInt VInt}
+ *   DataOffset,DataLength,Checksum --> {@link DataOutput#writeLong 
UInt64}
+ *   FileName --> {@link DataOutput#writeString String}
+ *   FileData --> raw file data
+ *   Footer --> {@link CodecUtil#writeFooter CodecFooter}
+ * 
+ *
+ * Notes:
+ *
+ * 
+ *   FileCount indicates how many files are contained in this compound 
file. The entry table
+ *   that follows has that many entries.
+ *   Each directory entry contains a long pointer to the start of this 
file's data section, the
+ *   files length, and a String with that file's name.
+ * 
+ */
+public final class Lucene50CompoundFormat extends CompoundFormat {
+
+  /** Extension of compound file */
+  static final String DATA_EXTENSION = "cfs";
+  /** Extension of compound file entries */
+  static final String ENTRIES_EXTENSION = "cfe";
+
+  static final String DATA_CODEC = "Lucene50CompoundData";
+  static final String ENTRY_CODEC = "Lucene50CompoundEntries";
+  static final int VERSION_START = 0;

Review comment:
   Do we still want to keep this internal versioning? Or could we simplify 
and remove it entirely?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] cammiemw commented on pull request #2097: LUCENE-9537

[GitBox]

cammiemw commented on pull request #2097:
URL: https://github.com/apache/lucene-solr/pull/2097#issuecomment-770074489


   Hi @mikemccand!  The smoothingScore method requires a docId because the most 
important implementation of smoothingScore is in TermScorer, which uses the 
postingsEnum to get the docId.  However, in the case of the smoothing score, 
there will be no posting for the term in the document.  Let me know if you 
think a different way of getting the docId would work better.
   
   Thanks so much for all your help and time throughout this process!  I am 
very excited about this change and hope that I can continue to add more.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] mikemccand commented on pull request #2097: LUCENE-9537

[GitBox]

mikemccand commented on pull request #2097:
URL: https://github.com/apache/lucene-solr/pull/2097#issuecomment-770069329


   Thanks @cammiemw!
   
   One thing I noticed after I pushed was the new `Scorable.smothingScore(int 
docId)` method: why does this take the `int docId` parameter?  Shouldn't it 
take no parameters, like `score()`, and always return smoothing score for the 
"current" `docId` instead?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15122) ClusterEventProducerTest.testEvents is unstable

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275380#comment-17275380
 ] 

ASF subversion and git services commented on SOLR-15122:


Commit 5cca4645177e68dcfd47641d7b7afa172137ee21 in lucene-solr's branch 
refs/heads/master from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5cca464 ]

SOLR-15122 Debug Logging


> ClusterEventProducerTest.testEvents is unstable
> ---
>
> Key: SOLR-15122
> URL: https://issues.apache.org/jira/browse/SOLR-15122
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Priority: Major
>
> This test looks to be unstable according to Jenkins since about Nov 5. I just 
> started seeing occasional failures locally when running the whole suite but 
> cannot reproduce when running in isolation.
> https://lists.apache.org/thread.html/rf0c16b257bc3236ea414be51451806352b55f15d4949f4fd54a3b71a%40%3Cbuilds.lucene.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9713) Did we lose the UTF-8 BOM detection in lucene/CHANGES.txt in gradle migration?

[Michael McCandless (Jira)]

Michael McCandless created LUCENE-9713:
--

 Summary: Did we lose the UTF-8 BOM detection in lucene/CHANGES.txt 
in gradle migration?
 Key: LUCENE-9713
 URL: https://issues.apache.org/jira/browse/LUCENE-9713
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Michael McCandless


While backporting LUCENE-9537, {{ant precommit}} failed in 8.x due to UTF-8 BOM 
header in {{lucene/CHANGES.txt}}.

I checked on mainline and the BOM was also present there, but {{gradle 
precommit}} seemed not to notice.

Did we lose that in the gradle migration?  Or, did we decide UTF-8 BOM is 
OK/wanted?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9537) Add Indri Search Engine Functionality to Lucene

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275378#comment-17275378
 ] 

ASF subversion and git services commented on LUCENE-9537:
-

Commit a6125fb938553fe2cd8cd035b8ffb329a7a1a288 in lucene-solr's branch 
refs/heads/branch_8x from cammiemw
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a6125fb ]

LUCENE-9537: Add initial Indri search engine functionality to Lucene


> Add Indri Search Engine Functionality to Lucene
> ---
>
> Key: LUCENE-9537
> URL: https://issues.apache.org/jira/browse/LUCENE-9537
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Cameron VandenBerg
>Priority: Major
>  Labels: patch
> Attachments: LUCENE-9537.patch, LUCENE-INDRI.patch
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> Indri ([http://lemurproject.org/indri.php]) is an academic search engine 
> developed by The University of Massachusetts and Carnegie Mellon University.  
> The major difference between Lucene and Indri is that Indri will give a 
> document a "smoothing score" to a document that does not contain the search 
> term, which has improved the search ranking accuracy in our experiments.  I 
> have created an Indri patch, which adds the search code needed to implement 
> the Indri AND logic as well as Indri's implementation of Dirichlet Smoothing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15122) ClusterEventProducerTest.testEvents is unstable

[Mike Drob (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275377#comment-17275377
 ] 

Mike Drob commented on SOLR-15122:
--

[~ab] - looks like this has been flaky since is was introduced in SOLR-14749.

> ClusterEventProducerTest.testEvents is unstable
> ---
>
> Key: SOLR-15122
> URL: https://issues.apache.org/jira/browse/SOLR-15122
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Priority: Major
>
> This test looks to be unstable according to Jenkins since about Nov 5. I just 
> started seeing occasional failures locally when running the whole suite but 
> cannot reproduce when running in isolation.
> https://lists.apache.org/thread.html/rf0c16b257bc3236ea414be51451806352b55f15d4949f4fd54a3b71a%40%3Cbuilds.lucene.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9537) Add Indri Search Engine Functionality to Lucene

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275376#comment-17275376
 ] 

ASF subversion and git services commented on LUCENE-9537:
-

Commit 4d839225b147a2c4d1a6d6363957b249d54bdfb4 in lucene-solr's branch 
refs/heads/master from Michael McCandless
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4d83922 ]

LUCENE-9537: move to 8.9 section in CHANGES.txt; make it consistent with 8.x's 
CHANGES.txt; remove the leading UTF-8 BOM


> Add Indri Search Engine Functionality to Lucene
> ---
>
> Key: LUCENE-9537
> URL: https://issues.apache.org/jira/browse/LUCENE-9537
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Cameron VandenBerg
>Priority: Major
>  Labels: patch
> Attachments: LUCENE-9537.patch, LUCENE-INDRI.patch
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> Indri ([http://lemurproject.org/indri.php]) is an academic search engine 
> developed by The University of Massachusetts and Carnegie Mellon University.  
> The major difference between Lucene and Indri is that Indri will give a 
> document a "smoothing score" to a document that does not contain the search 
> term, which has improved the search ranking accuracy in our experiments.  I 
> have created an Indri patch, which adds the search code needed to implement 
> the Indri AND logic as well as Indri's implementation of Dirichlet Smoothing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9712) UnifiedHighlighter, optimize WEIGHT_MATCHES when many fields

[David Smiley (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275373#comment-17275373
 ] 

David Smiley commented on LUCENE-9712:
--

I suspect the culprit is in this method: 
{{org.apache.lucene.search.uhighlight.FieldOffsetStrategy#createOffsetsEnumsWeightMatcher
 }}

-- which will compute the Matches from the Weight – but this will be called 
once per field.  I suspect most of the cost is in re-computing this over and 
over again.  If we can assume there is no "getFieldMatcher" (i.e. assume 
{{hl.requireFieldMatch=true}}), and that offset source is from the actual index 
(no re-analysis), then the leafReader could be the same across fields, and thus 
the Matches would be re-usable.  But how to re-use it across fields?  There's 
no clear place nearby since this part of the code is very field-centric.  
UHComponents is immutable; that could be changed to hold some Map.  Or, I was 
thinking maybe the Query could be wrapped with an impl that has a Weight that 
caches its Matches result for a given leafReader docId pair. H.

This kind of highlights a structural challenge in the UH in which it is very 
field centric, and thus it's not clear where to share info across fields of the 
same doc.  Above I qualified some ideas that would only work for an index based 
offset source (in postings), but it'd suck not to handle re-analysis, which is 
popular.  Again, if there was a more document centric approach, then the 
underling MemoryIndex could be built across fields, which would then enable 
re-use of Matches since MI's leafReader would be the same.

> UnifiedHighlighter, optimize WEIGHT_MATCHES when many fields
> 
>
> Key: LUCENE-9712
> URL: https://issues.apache.org/jira/browse/LUCENE-9712
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Reporter: David Smiley
>Priority: Major
>
> A user reported that highlighting many fields per document in WEIGHT_MATCHES 
> mode is quite slow:   
> [https://lists.apache.org/thread.html/r152c74a884b5ff72f3d530fc452bb0865cc7f24ca35ccf7d1d1e4952%40%3Csolr-user.lucene.apache.org%3E]
> The query is a DisjunctionMax over many fields – basically the ones being 
> highlighted.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14253) Remove Sleeps from OverseerCollectionMessageHandler

[Mike Drob (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275360#comment-17275360
 ] 

Mike Drob commented on SOLR-14253:
--

Rebased the changes onto current code, and picked up a few more similar 
improvements from Mark's branch.

> Remove Sleeps from OverseerCollectionMessageHandler
> ---
>
> Key: SOLR-14253
> URL: https://issues.apache.org/jira/browse/SOLR-14253
> Project: Solr
>  Issue Type: Bug
>  Components: Server
>Reporter: Mike Drob
>Priority: Major
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> From the conversations with Mark Miller a few months back - there are a lot 
> of places in the server code where we have hard sleeps instead of relying on 
> notifications and watchers to handle state.
> I will begin to tackle these one at a time, starting with 
> OverseerCollectionMessageHandler.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-7642) Should launching Solr in cloud mode using a ZooKeeper chroot create the chroot znode if it doesn't exist?

[Isabelle Giguere (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275359#comment-17275359
 ] 

Isabelle Giguere commented on SOLR-7642:


[~janhoy], [~gus] : I don't see the "Enable Patch Review" button
I think I did see that button previously, but now, I can't.  The documentation 
doesn't mention that the button would only be visible to Solr committers : 
https://cwiki.apache.org/confluence/display/SOLR/HowToContribute#HowToContribute-Contributingyourwork
Can one of you enable the patch review, if you feel it's appropriate ? 
Thanks.

> Should launching Solr in cloud mode using a ZooKeeper chroot create the 
> chroot znode if it doesn't exist?
> -
>
> Key: SOLR-7642
> URL: https://issues.apache.org/jira/browse/SOLR-7642
> Project: Solr
>  Issue Type: Improvement
>Reporter: Timothy Potter
>Priority: Minor
> Attachments: SOLR-7642.patch, SOLR-7642.patch, SOLR-7642.patch, 
> SOLR-7642.patch, SOLR-7642_tag_7.5.0.patch, 
> SOLR-7642_tag_7.5.0_proposition.patch
>
>
> If you launch Solr for the first time in cloud mode using a ZooKeeper 
> connection string that includes a chroot leads to the following 
> initialization error:
> {code}
> ERROR - 2015-06-05 17:15:50.410; [   ] org.apache.solr.common.SolrException; 
> null:org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified 
> in ZkHost but the znode doesn't exist. localhost:2181/lan
> at 
> org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:339)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:140)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:110)
> at 
> org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
> at 
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at 
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> {code}
> The work-around for this is to use the scripts/cloud-scripts/zkcli.sh script 
> to create the chroot znode (bootstrap action does this).
> I'm wondering if we shouldn't just create the znode if it doesn't exist? Or 
> is that some violation of using a chroot?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9694) New tool for creating a deterministic index

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275358#comment-17275358
 ] 

ASF subversion and git services commented on LUCENE-9694:
-

Commit cac5c2a4b2bf0bded94a8d25effb21cca0566a52 in lucene-solr's branch 
refs/heads/master from Michael McCandless
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=cac5c2a ]

LUCENE-9694: make new DocumentSelector interface public so it is usable outside 
of its own package


> New tool for creating a deterministic index
> ---
>
> Key: LUCENE-9694
> URL: https://issues.apache.org/jira/browse/LUCENE-9694
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: general/tools
>Reporter: Haoyu Zhai
>Priority: Minor
> Fix For: master (9.0), 8.9
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Lucene's index is segmented, and sometimes number of segments and documents 
> arrangement greatly impact performance.
> Given a stable index sort, our team create a tool that records document 
> arrangement (called index map) of an index and rearrange another index 
> (consists of same documents) into the same structure (segment num, and 
> documents included in each segment).
> This tool could be also used in lucene benchmarks for a faster deterministic 
> index construction (if I understand correctly lucene benchmark is using a 
> single thread manner to achieve this).
>  
> We've already had some discussion in email
> [https://markmail.org/message/lbtdntclpnocmfuf]
> And I've implemented the first method, using {{IndexWriter.addIndexes}} and a 
> customized {{FilteredCodecReader}} to achieve the goal. The index 
> construction time is about 25min and time executing this tool is about 10min.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] mikemccand commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

mikemccand commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r567085518



##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -353,12 +349,104 @@ public FacetLabel getPath(int ordinal) throws 
IOException {
 }
 
 synchronized (categoryCache) {
-  categoryCache.put(catIDInteger, ret);
+  categoryCache.put(ordinal, ret);
 }
 
 return ret;
   }
 
+  private FacetLabel getPathFromCache(int ordinal) {
+// TODO: can we use an int-based hash impl, such as IntToObjectMap,

Review comment:
   Oooh that is a great idea, and low-hanging fruit, and would greatly 
reduce the RAM usage for this cache.
   
   I think `DirectoryTaxonomyWriter` also has such a cache that we could change 
to a native map.
   
   Could you open a spinoff issue?

##
File path: lucene/CHANGES.txt
##
@@ -11,6 +11,8 @@ New Features
 
 * LUCENE-9004: Approximate nearest vector search via NSW graphs
 
+* LUCENE-9476: DirectoryTaxonomyReader now provides a getBulkPath API (Gautam 
Worah)

Review comment:
   Can you expand a bit more?  E.g.:
   
   ```
   LUCENE-9476: add new getBulkPath API to DirectoryTaxonomyReader to more 
efficiently retrieve FacetLabels for multiple facet ordinals at once
   ```

##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -353,12 +349,104 @@ public FacetLabel getPath(int ordinal) throws 
IOException {
 }
 
 synchronized (categoryCache) {
-  categoryCache.put(catIDInteger, ret);
+  categoryCache.put(ordinal, ret);
 }
 
 return ret;
   }
 
+  private FacetLabel getPathFromCache(int ordinal) {
+// TODO: can we use an int-based hash impl, such as IntToObjectMap,
+// wrapped as LRU?
+synchronized (categoryCache) {
+  return categoryCache.get(ordinal);
+}
+  }
+
+  private void isOrdinalInIndexReaderRange(int ordinal, int indexReaderMaxDoc)
+  throws IllegalArgumentException {
+if (ordinal < 0 || ordinal >= indexReaderMaxDoc) {
+  throw new IllegalArgumentException(
+  "ordinal "
+  + ordinal
+  + " is out of the range of the indexReader "
+  + indexReader.toString());
+}
+  }
+
+  /**
+   * Returns an array of FacetLabels for a given array of ordinals.
+   *
+   * This API is generally faster than iteratively calling {@link
+   * 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyReader#getPath} 
over an array of
+   * ordinals.
+   *
+   * This API is only available for Lucene indexes created with 8.7+ codec 
because it uses
+   * BinaryDocValues instead of StoredFields. Use the {@link
+   * 
org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyReader#getPath} 
method for indices
+   * created with codec version older than 8.7.
+   *
+   * @param ordinals Array of ordinals that are assigned to categories 
inserted into the taxonomy
+   * index
+   * @throws IOException if the taxonomy index is created using the older 
StoredFields based codec.
+   */
+  public FacetLabel[] getBulkPath(int... ordinals) throws IOException {
+ensureOpen();
+
+FacetLabel[] bulkPath = new FacetLabel[ordinals.length];
+// remember the original positions of ordinals before they are sorted
+IntIntScatterMap originalPosition = new IntIntScatterMap();
+int indexReaderMaxDoc = indexReader.maxDoc();
+for (int i = 0; i < ordinals.length; i++) {
+  // check whether the ordinal is valid before accessing the cache
+  isOrdinalInIndexReaderRange(ordinals[i], indexReaderMaxDoc);
+  // check the cache before trying to find it in the index
+  FacetLabel ordinalPath = getPathFromCache(ordinals[i]);
+  if (ordinalPath != null) {
+bulkPath[i] = ordinalPath;
+  }
+  originalPosition.put(ordinals[i], i);
+}
+
+Arrays.sort(ordinals);
+int readerIndex;
+int leafReaderMaxDoc = 0;
+int leafReaderDocBase = 0;
+LeafReader leafReader;
+LeafReaderContext leafReaderContext = null;
+BinaryDocValues values = null;
+
+for (int ord : ordinals) {
+  if (bulkPath[originalPosition.get(ord)] == null) {
+if (values == null || ord > leafReaderMaxDoc) {
+
+  readerIndex = ReaderUtil.subIndex(ord, indexReader.leaves());
+  leafReaderContext = indexReader.leaves().get(readerIndex);
+  leafReader = leafReaderContext.reader();
+  leafReaderMaxDoc = leafReader.maxDoc();
+  leafReaderDocBase = leafReaderContext.docBase;
+  values = leafReader.getBinaryDocValues(Consts.FULL);
+
+  // this check is only needed once to confirm that the index uses 
BinaryDocValues
+  boolean success = values.advanceExact(ord - leafReaderDocBase);
+  if (success == false) {
+throw new IOException(
+"the taxonomy inde

[jira] [Created] (LUCENE-9712) UnifiedHighlighter, optimize WEIGHT_MATCHES when many fields

[David Smiley (Jira)]

David Smiley created LUCENE-9712:


 Summary: UnifiedHighlighter, optimize WEIGHT_MATCHES when many 
fields
 Key: LUCENE-9712
 URL: https://issues.apache.org/jira/browse/LUCENE-9712
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/highlighter
Reporter: David Smiley


A user reported that highlighting many fields per document in WEIGHT_MATCHES 
mode is quite slow:   
[https://lists.apache.org/thread.html/r152c74a884b5ff72f3d530fc452bb0865cc7f24ca35ccf7d1d1e4952%40%3Csolr-user.lucene.apache.org%3E]

The query is a DisjunctionMax over many fields – basically the ones being 
highlighted.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] markrmiller commented on pull request #2263: SOLR-14978 OOM Killer in Foreground (#2055)

[GitBox]

markrmiller commented on pull request #2263:
URL: https://github.com/apache/lucene-solr/pull/2263#issuecomment-770041322


   Sorry @madrob, didn't mean to get those wip commits in the dev branch, they 
were meant to go another branch for testing.
   
   Can you update this? Sorry about the hassle.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14949) Ability to customize docker image name/base image

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275341#comment-17275341
 ] 

ASF subversion and git services commented on SOLR-14949:


Commit 53af6fcc9d7fc4e02a8288ffcc189a80b1880cc0 in lucene-solr's branch 
refs/heads/reference_impl from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=53af6fc ]

SOLR-14949: Adding githubUrl option for docker build. (#2074)



> Ability to customize docker image name/base image
> -
>
> Key: SOLR-14949
> URL: https://issues.apache.org/jira/browse/SOLR-14949
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> The current docker build will generate an image with the name 
> {{apache/solr:}}. If users want to build custom images and push them 
> to their own docker orgs, then this should be more customizable.
> The following inputs should be customizable in the first pass at least:
>  * Docker Image Repo - default "apache/solr"
>  * Docker Image Tag - default to the project version
>  * Docker Image Name (This will set the entire thing, overriding the previous 
> two options) - Defaults to ":"
>  * Base Docker Image (This is the docker image that Solr Builds itself on top 
> of) - Defaults to "openjdk:11-jre-slim"
>  * Github URL ("github.com" or a mirror for github releases. This allows for 
> building the solr docker image behind a firewall that does not have access to 
> github.com)
> All will be optional.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15075) Remove the Docker Gradle plugin dependency and solr/docker/package module

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275343#comment-17275343
 ] 

ASF subversion and git services commented on SOLR-15075:


Commit 9be59f9ed43638a95cc00422d063ebfbf736479a in lucene-solr's branch 
refs/heads/reference_impl from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9be59f9 ]

SOLR-15075: Solr docker gradle improvements (#2197)

* Removed docker plugin from gradle builds.
* Removed package docker image.
* Tasks now have correct inputs/outputs/dependencies.
* Move gradle help text to docker folder.
* Reduce duplicated Docker layer by doing file removal and chmod in another 
stage.

Co-authored-by: David Smiley 


> Remove the Docker Gradle plugin dependency and solr/docker/package module
> -
>
> Key: SOLR-15075
> URL: https://issues.apache.org/jira/browse/SOLR-15075
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker, Gradle
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Currently Solr uses the [gradle docker 
> plugin|https://github.com/palantir/gradle-docker] to manage building Solr 
> docker images. When migrating the docker build into Solr, using the plugin 
> was the path of least resistance and allowed us to migrate without having to 
> think a lot about the gradle part.
> Now that the docker image building works, it may be beneficial to migrate 
> away from the docker plugin, so that we can have better control over build 
> process. The steps are simple enough anyways, so we wouldn't be sacrificing 
> much to have more flexibility.
> Given the discussion in the community over release vs local docker image 
> builds, there does not seem to be a desire to have separate builds for them. 
> Therefore we no longer need the solr/docker/package module. Instead the 
> solr/docker module will pull in the solr/packaging output and directly put it 
> into the docker image.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15001) Docker: require init_var_solr.sh; don't init in Dockerfile

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275342#comment-17275342
 ] 

ASF subversion and git services commented on SOLR-15001:


Commit 17b983ea300537bde4cbd25aaa3902c42fd31ffa in lucene-solr's branch 
refs/heads/reference_impl from David Smiley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=17b983e ]

SOLR-15001 Docker: require init_var_solr.sh (#2083)

The Dockerfile should not initialize /var/solr's contents because this is 
confusing and redundant with init_var_solr.sh.
No need for init_var_solr.sh to echo what it does; VERBOSE can be used to 
accomplish that.

Separate CHANGES.md for Docker and contrib modules.


> Docker: require init_var_solr.sh; don't init in Dockerfile
> --
>
> Key: SOLR-15001
> URL: https://issues.apache.org/jira/browse/SOLR-15001
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> I propose removing initialization of /var/solr from the Dockerfile, thus 
> leaving only init_var_solr to do this. The fact that it's in two places means 
> that the image has two solr.xml, two zoo.cfg, two log4j2.xml. This 
> initialization itself must be maintained twice. That leads to confusion (it 
> did with my colleagues and I) about which copy is going to be used. Imagine 
> you are basing your company Solr Dockerfile on top of this one (i.e. official 
> is the FROM) and need to do modifications. Do you modify 
> /opt/solr/server/solr/solr.xml? Surprise surprise, sometimes it is copied to 
> /var/solr/data/ by the init_var_solr script but _sometimes_ it isn't because 
> the Dockerfile here will do it, thus ignoring the customizations made to 
> solr.xml in the next image layer.
> After making this change, our wonderful tests exposed that solr-demo wasn't 
> invoking init_var_solr.
> THIS ISSUE IS COPIED FROM https://github.com/docker-solr/docker-solr/pull/354



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14947) Gradle docker build task should output image id or tag

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275336#comment-17275336
 ] 

ASF subversion and git services commented on SOLR-14947:


Commit d135f2c916865fcef88a1028ea6f37d4ce643e3c in lucene-solr's branch 
refs/heads/reference_impl from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d135f2c ]

SOLR-14947: Print out image info after gradle docker task. (#2007)



> Gradle docker build task should output image id or tag
> --
>
> Key: SOLR-14947
> URL: https://issues.apache.org/jira/browse/SOLR-14947
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Affects Versions: master (9.0)
>Reporter: Mike Drob
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> When calling {{./gradlew docker}} it build the image, but I have no idea what 
> it's called so that I can run it.
> When I run with {{--info}} I get the requested output, but also a lot of 
> other stuff.
> {code}
> Successfully built 02d86c5c7357
> Successfully tagged apache/solr:9.0.0-SNAPSHOT
> :solr:docker:docker (Thread[Execution worker for ':' Thread 11,5,main]) 
> completed. Took 0.172 secs.
> {code}
> We could put some of that in a {{doLast}} block, similar to 
> https://github.com/apache/lucene-solr/blob/master/gradle/releasing.gradle#L51



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14949) Ability to customize docker image name/base image

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275340#comment-17275340
 ] 

ASF subversion and git services commented on SOLR-14949:


Commit a649da35a571d4802f8b4cd41684abcb093669ff in lucene-solr's branch 
refs/heads/reference_impl from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a649da3 ]

SOLR-14949: Ability to customize Solr Docker build (#2020)

Also added a gradlew helpDocker page.


> Ability to customize docker image name/base image
> -
>
> Key: SOLR-14949
> URL: https://issues.apache.org/jira/browse/SOLR-14949
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> The current docker build will generate an image with the name 
> {{apache/solr:}}. If users want to build custom images and push them 
> to their own docker orgs, then this should be more customizable.
> The following inputs should be customizable in the first pass at least:
>  * Docker Image Repo - default "apache/solr"
>  * Docker Image Tag - default to the project version
>  * Docker Image Name (This will set the entire thing, overriding the previous 
> two options) - Defaults to ":"
>  * Base Docker Image (This is the docker image that Solr Builds itself on top 
> of) - Defaults to "openjdk:11-jre-slim"
>  * Github URL ("github.com" or a mirror for github releases. This allows for 
> building the solr docker image behind a firewall that does not have access to 
> github.com)
> All will be optional.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14955) Add env var options for the Prometheus Exporter bin scripts

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275338#comment-17275338
 ] 

ASF subversion and git services commented on SOLR-14955:


Commit 240dad4f0e7d9f725a01aab6d4c4f9f9f4c09480 in lucene-solr's branch 
refs/heads/reference_impl from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=240dad4 ]

SOLR-14955: Add env var options to Prometheus Export scripts. (#2038)


> Add env var options for the Prometheus Exporter bin scripts
> ---
>
> Key: SOLR-14955
> URL: https://issues.apache.org/jira/browse/SOLR-14955
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - prometheus-exporter
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: 8.8
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The prometheus exporter bin scripts get the job done, but are quite lean and 
> don't provide a large amount of help in setting up the exporter.
> A list of things that could be improved:
>  * Allowing users to pass [command line 
> options|https://lucene.apache.org/solr/guide/8_6/monitoring-solr-with-prometheus-and-grafana.html#command-line-parameters]
>  through environment variables
>  * Support the ZK ACL environment variables
>  * Similar memory options to solr/bin/solr
> These are just a few ideas, but a little work would go a long way here.
> Previous discussion:
>  * [docker-solr#307|https://github.com/docker-solr/docker-solr/issues/307]
>  * [docker-solr#224|https://github.com/docker-solr/docker-solr/pull/224]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14972) Change default port of prometheus exporter

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275339#comment-17275339
 ] 

ASF subversion and git services commented on SOLR-14972:


Commit df17d924da446d967f4af4ae9c5c7a544cc1d825 in lucene-solr's branch 
refs/heads/reference_impl from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=df17d92 ]

SOLR-14972: Change default port of prometheus exporter to 8989 (#2046)



> Change default port of prometheus exporter
> --
>
> Key: SOLR-14972
> URL: https://issues.apache.org/jira/browse/SOLR-14972
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - prometheus-exporter
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The default port of prometheus exporter is 9983, which is exactly the same 
> port as the embedded Zookeeper (8983+1000), which would prevent someone from 
> testing both on their laptop with default settings.
> Use e.g. 8989 instead



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15075) Remove the Docker Gradle plugin dependency and solr/docker/package module

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275344#comment-17275344
 ] 

ASF subversion and git services commented on SOLR-15075:


Commit 7ecdea2bd87276b62488b66967f4923142a587db in lucene-solr's branch 
refs/heads/reference_impl from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7ecdea2 ]

SOLR-15075: Remove docker gradle artifact

Gradle is currently picking up the dockerBuild task in "assemble",
because of the build artifact.


> Remove the Docker Gradle plugin dependency and solr/docker/package module
> -
>
> Key: SOLR-15075
> URL: https://issues.apache.org/jira/browse/SOLR-15075
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker, Gradle
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Currently Solr uses the [gradle docker 
> plugin|https://github.com/palantir/gradle-docker] to manage building Solr 
> docker images. When migrating the docker build into Solr, using the plugin 
> was the path of least resistance and allowed us to migrate without having to 
> think a lot about the gradle part.
> Now that the docker image building works, it may be beneficial to migrate 
> away from the docker plugin, so that we can have better control over build 
> process. The steps are simple enough anyways, so we wouldn't be sacrificing 
> much to have more flexibility.
> Given the discussion in the community over release vs local docker image 
> builds, there does not seem to be a desire to have separate builds for them. 
> Therefore we no longer need the solr/docker/package module. Instead the 
> solr/docker module will pull in the solr/packaging output and directly put it 
> into the docker image.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14957) Add Prometheus Exporter bin to PATH in docker image

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275337#comment-17275337
 ] 

ASF subversion and git services commented on SOLR-14957:


Commit 284cafe9485a7bef01a6138b7f917d3af7ab3305 in lucene-solr's branch 
refs/heads/reference_impl from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=284cafe ]

SOLR-14957: Add Prometheus Exporter to docker PATH. Fix classpath issues. 
(#2017)


> Add Prometheus Exporter bin to PATH in docker image
> ---
>
> Key: SOLR-14957
> URL: https://issues.apache.org/jira/browse/SOLR-14957
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - prometheus-exporter, Docker
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently the Solr docker image contains the prometheus exporter script 
> {{bin/solr-exporter}}, and this script is executable. Therefore the 
> prometheus exporter _can_ be run via the Solr docker image.
> However the user has to know the location of this script when trying to run 
> the prometheus-exporter. It would be best if the {{prometheus-exporter/bin}} 
> were added to the PATH, so that the prometheus exporter could be run through 
> the following:
> {code:bash}
> docker run solr solr-exporter -b http://host.docker.internal:8983/solr
> {code}
> Discussion at 
> [docker-solr#307|https://github.com/docker-solr/docker-solr/issues/307]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15075) Remove the Docker Gradle plugin dependency and solr/docker/package module

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275335#comment-17275335
 ] 

ASF subversion and git services commented on SOLR-15075:


Commit 7ecdea2bd87276b62488b66967f4923142a587db in lucene-solr's branch 
refs/heads/reference_impl_dev from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7ecdea2 ]

SOLR-15075: Remove docker gradle artifact

Gradle is currently picking up the dockerBuild task in "assemble",
because of the build artifact.


> Remove the Docker Gradle plugin dependency and solr/docker/package module
> -
>
> Key: SOLR-15075
> URL: https://issues.apache.org/jira/browse/SOLR-15075
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker, Gradle
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Currently Solr uses the [gradle docker 
> plugin|https://github.com/palantir/gradle-docker] to manage building Solr 
> docker images. When migrating the docker build into Solr, using the plugin 
> was the path of least resistance and allowed us to migrate without having to 
> think a lot about the gradle part.
> Now that the docker image building works, it may be beneficial to migrate 
> away from the docker plugin, so that we can have better control over build 
> process. The steps are simple enough anyways, so we wouldn't be sacrificing 
> much to have more flexibility.
> Given the discussion in the community over release vs local docker image 
> builds, there does not seem to be a desire to have separate builds for them. 
> Therefore we no longer need the solr/docker/package module. Instead the 
> solr/docker module will pull in the solr/packaging output and directly put it 
> into the docker image.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15075) Remove the Docker Gradle plugin dependency and solr/docker/package module

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275334#comment-17275334
 ] 

ASF subversion and git services commented on SOLR-15075:


Commit 9be59f9ed43638a95cc00422d063ebfbf736479a in lucene-solr's branch 
refs/heads/reference_impl_dev from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9be59f9 ]

SOLR-15075: Solr docker gradle improvements (#2197)

* Removed docker plugin from gradle builds.
* Removed package docker image.
* Tasks now have correct inputs/outputs/dependencies.
* Move gradle help text to docker folder.
* Reduce duplicated Docker layer by doing file removal and chmod in another 
stage.

Co-authored-by: David Smiley 


> Remove the Docker Gradle plugin dependency and solr/docker/package module
> -
>
> Key: SOLR-15075
> URL: https://issues.apache.org/jira/browse/SOLR-15075
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker, Gradle
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Currently Solr uses the [gradle docker 
> plugin|https://github.com/palantir/gradle-docker] to manage building Solr 
> docker images. When migrating the docker build into Solr, using the plugin 
> was the path of least resistance and allowed us to migrate without having to 
> think a lot about the gradle part.
> Now that the docker image building works, it may be beneficial to migrate 
> away from the docker plugin, so that we can have better control over build 
> process. The steps are simple enough anyways, so we wouldn't be sacrificing 
> much to have more flexibility.
> Given the discussion in the community over release vs local docker image 
> builds, there does not seem to be a desire to have separate builds for them. 
> Therefore we no longer need the solr/docker/package module. Instead the 
> solr/docker module will pull in the solr/packaging output and directly put it 
> into the docker image.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14972) Change default port of prometheus exporter

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275328#comment-17275328
 ] 

ASF subversion and git services commented on SOLR-14972:


Commit df17d924da446d967f4af4ae9c5c7a544cc1d825 in lucene-solr's branch 
refs/heads/reference_impl_dev from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=df17d92 ]

SOLR-14972: Change default port of prometheus exporter to 8989 (#2046)



> Change default port of prometheus exporter
> --
>
> Key: SOLR-14972
> URL: https://issues.apache.org/jira/browse/SOLR-14972
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - prometheus-exporter
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The default port of prometheus exporter is 9983, which is exactly the same 
> port as the embedded Zookeeper (8983+1000), which would prevent someone from 
> testing both on their laptop with default settings.
> Use e.g. 8989 instead



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14955) Add env var options for the Prometheus Exporter bin scripts

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275327#comment-17275327
 ] 

ASF subversion and git services commented on SOLR-14955:


Commit 240dad4f0e7d9f725a01aab6d4c4f9f9f4c09480 in lucene-solr's branch 
refs/heads/reference_impl_dev from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=240dad4 ]

SOLR-14955: Add env var options to Prometheus Export scripts. (#2038)


> Add env var options for the Prometheus Exporter bin scripts
> ---
>
> Key: SOLR-14955
> URL: https://issues.apache.org/jira/browse/SOLR-14955
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - prometheus-exporter
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: 8.8
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> The prometheus exporter bin scripts get the job done, but are quite lean and 
> don't provide a large amount of help in setting up the exporter.
> A list of things that could be improved:
>  * Allowing users to pass [command line 
> options|https://lucene.apache.org/solr/guide/8_6/monitoring-solr-with-prometheus-and-grafana.html#command-line-parameters]
>  through environment variables
>  * Support the ZK ACL environment variables
>  * Similar memory options to solr/bin/solr
> These are just a few ideas, but a little work would go a long way here.
> Previous discussion:
>  * [docker-solr#307|https://github.com/docker-solr/docker-solr/issues/307]
>  * [docker-solr#224|https://github.com/docker-solr/docker-solr/pull/224]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15001) Docker: require init_var_solr.sh; don't init in Dockerfile

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275333#comment-17275333
 ] 

ASF subversion and git services commented on SOLR-15001:


Commit 17b983ea300537bde4cbd25aaa3902c42fd31ffa in lucene-solr's branch 
refs/heads/reference_impl_dev from David Smiley
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=17b983e ]

SOLR-15001 Docker: require init_var_solr.sh (#2083)

The Dockerfile should not initialize /var/solr's contents because this is 
confusing and redundant with init_var_solr.sh.
No need for init_var_solr.sh to echo what it does; VERBOSE can be used to 
accomplish that.

Separate CHANGES.md for Docker and contrib modules.


> Docker: require init_var_solr.sh; don't init in Dockerfile
> --
>
> Key: SOLR-15001
> URL: https://issues.apache.org/jira/browse/SOLR-15001
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> I propose removing initialization of /var/solr from the Dockerfile, thus 
> leaving only init_var_solr to do this. The fact that it's in two places means 
> that the image has two solr.xml, two zoo.cfg, two log4j2.xml. This 
> initialization itself must be maintained twice. That leads to confusion (it 
> did with my colleagues and I) about which copy is going to be used. Imagine 
> you are basing your company Solr Dockerfile on top of this one (i.e. official 
> is the FROM) and need to do modifications. Do you modify 
> /opt/solr/server/solr/solr.xml? Surprise surprise, sometimes it is copied to 
> /var/solr/data/ by the init_var_solr script but _sometimes_ it isn't because 
> the Dockerfile here will do it, thus ignoring the customizations made to 
> solr.xml in the next image layer.
> After making this change, our wonderful tests exposed that solr-demo wasn't 
> invoking init_var_solr.
> THIS ISSUE IS COPIED FROM https://github.com/docker-solr/docker-solr/pull/354



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14949) Ability to customize docker image name/base image

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275331#comment-17275331
 ] 

ASF subversion and git services commented on SOLR-14949:


Commit 53af6fcc9d7fc4e02a8288ffcc189a80b1880cc0 in lucene-solr's branch 
refs/heads/reference_impl_dev from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=53af6fc ]

SOLR-14949: Adding githubUrl option for docker build. (#2074)



> Ability to customize docker image name/base image
> -
>
> Key: SOLR-14949
> URL: https://issues.apache.org/jira/browse/SOLR-14949
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> The current docker build will generate an image with the name 
> {{apache/solr:}}. If users want to build custom images and push them 
> to their own docker orgs, then this should be more customizable.
> The following inputs should be customizable in the first pass at least:
>  * Docker Image Repo - default "apache/solr"
>  * Docker Image Tag - default to the project version
>  * Docker Image Name (This will set the entire thing, overriding the previous 
> two options) - Defaults to ":"
>  * Base Docker Image (This is the docker image that Solr Builds itself on top 
> of) - Defaults to "openjdk:11-jre-slim"
>  * Github URL ("github.com" or a mirror for github releases. This allows for 
> building the solr docker image behind a firewall that does not have access to 
> github.com)
> All will be optional.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14947) Gradle docker build task should output image id or tag

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275325#comment-17275325
 ] 

ASF subversion and git services commented on SOLR-14947:


Commit d135f2c916865fcef88a1028ea6f37d4ce643e3c in lucene-solr's branch 
refs/heads/reference_impl_dev from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d135f2c ]

SOLR-14947: Print out image info after gradle docker task. (#2007)



> Gradle docker build task should output image id or tag
> --
>
> Key: SOLR-14947
> URL: https://issues.apache.org/jira/browse/SOLR-14947
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Build
>Affects Versions: master (9.0)
>Reporter: Mike Drob
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> When calling {{./gradlew docker}} it build the image, but I have no idea what 
> it's called so that I can run it.
> When I run with {{--info}} I get the requested output, but also a lot of 
> other stuff.
> {code}
> Successfully built 02d86c5c7357
> Successfully tagged apache/solr:9.0.0-SNAPSHOT
> :solr:docker:docker (Thread[Execution worker for ':' Thread 11,5,main]) 
> completed. Took 0.172 secs.
> {code}
> We could put some of that in a {{doLast}} block, similar to 
> https://github.com/apache/lucene-solr/blob/master/gradle/releasing.gradle#L51



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14949) Ability to customize docker image name/base image

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275329#comment-17275329
 ] 

ASF subversion and git services commented on SOLR-14949:


Commit a649da35a571d4802f8b4cd41684abcb093669ff in lucene-solr's branch 
refs/heads/reference_impl_dev from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a649da3 ]

SOLR-14949: Ability to customize Solr Docker build (#2020)

Also added a gradlew helpDocker page.


> Ability to customize docker image name/base image
> -
>
> Key: SOLR-14949
> URL: https://issues.apache.org/jira/browse/SOLR-14949
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Docker
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> The current docker build will generate an image with the name 
> {{apache/solr:}}. If users want to build custom images and push them 
> to their own docker orgs, then this should be more customizable.
> The following inputs should be customizable in the first pass at least:
>  * Docker Image Repo - default "apache/solr"
>  * Docker Image Tag - default to the project version
>  * Docker Image Name (This will set the entire thing, overriding the previous 
> two options) - Defaults to ":"
>  * Base Docker Image (This is the docker image that Solr Builds itself on top 
> of) - Defaults to "openjdk:11-jre-slim"
>  * Github URL ("github.com" or a mirror for github releases. This allows for 
> building the solr docker image behind a firewall that does not have access to 
> github.com)
> All will be optional.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14957) Add Prometheus Exporter bin to PATH in docker image

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275326#comment-17275326
 ] 

ASF subversion and git services commented on SOLR-14957:


Commit 284cafe9485a7bef01a6138b7f917d3af7ab3305 in lucene-solr's branch 
refs/heads/reference_impl_dev from Houston Putman
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=284cafe ]

SOLR-14957: Add Prometheus Exporter to docker PATH. Fix classpath issues. 
(#2017)


> Add Prometheus Exporter bin to PATH in docker image
> ---
>
> Key: SOLR-14957
> URL: https://issues.apache.org/jira/browse/SOLR-14957
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - prometheus-exporter, Docker
>Affects Versions: master (9.0)
>Reporter: Houston Putman
>Assignee: Houston Putman
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Currently the Solr docker image contains the prometheus exporter script 
> {{bin/solr-exporter}}, and this script is executable. Therefore the 
> prometheus exporter _can_ be run via the Solr docker image.
> However the user has to know the location of this script when trying to run 
> the prometheus-exporter. It would be best if the {{prometheus-exporter/bin}} 
> were added to the PATH, so that the prometheus exporter could be run through 
> the following:
> {code:bash}
> docker run solr solr-exporter -b http://host.docker.internal:8983/solr
> {code}
> Discussion at 
> [docker-solr#307|https://github.com/docker-solr/docker-solr/issues/307]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9711) "gradle precommit" no longer catches accidental package-private APIs?

[Michael McCandless (Jira)]

Michael McCandless created LUCENE-9711:
--

 Summary: "gradle precommit" no longer catches accidental 
package-private APIs?
 Key: LUCENE-9711
 URL: https://issues.apache.org/jira/browse/LUCENE-9711
 Project: Lucene - Core
  Issue Type: Bug
Affects Versions: master (9.0)
Reporter: Michael McCandless


While backporting the new exciting deterministic multi-segment indexing tool, 
LUCENE-9694, {{ant precommit}} failed, because the new {{DocumentSelector}} API 
was accidentally package private, missing its {{public}} modifier.

[Aside: I sometimes feel we should not put unit tests in the same package as 
the APIs they are testing.  We of course do this to make testing internal, 
package-private state, possible/easier.  But it then leads to API bugs like 
this, where we fail to make an API public when it should be.]

Anyways, luckily, the old crazy hacky Python javadoc linter in 8.x caught this 
issue, and I fixed it on backport, and will shortly fix it in mainline as well. 
 But {{gradle precommit}} on mainline failed to catch it, I think?

Is this a known regression in our {{gradle}} migration?  Do we have plans to 
somehow recover it?  It clearly sometimes catches important API bugs!  And this 
is not the first time it's caught such bugs...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-15122) ClusterEventProducerTest.testEvents is unstable

[Mike Drob (Jira)]

Mike Drob created SOLR-15122:


 Summary: ClusterEventProducerTest.testEvents is unstable
 Key: SOLR-15122
 URL: https://issues.apache.org/jira/browse/SOLR-15122
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Tests
Reporter: Mike Drob


This test looks to be unstable according to Jenkins since about Nov 5. I just 
started seeing occasional failures locally when running the whole suite but 
cannot reproduce when running in isolation.

https://lists.apache.org/thread.html/rf0c16b257bc3236ea414be51451806352b55f15d4949f4fd54a3b71a%40%3Cbuilds.lucene.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9694) New tool for creating a deterministic index

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275312#comment-17275312
 ] 

ASF subversion and git services commented on LUCENE-9694:
-

Commit 6f913a2bc7cdac73056290b8a8212c53ec9b4561 in lucene-solr's branch 
refs/heads/branch_8x from Michael McCandless
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6f913a2 ]

LUCENE-9694: fix other precommit issues, including unused import, but most 
importantly that DocumentSelector was accidentally package private, making it 
difficult to use


> New tool for creating a deterministic index
> ---
>
> Key: LUCENE-9694
> URL: https://issues.apache.org/jira/browse/LUCENE-9694
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: general/tools
>Reporter: Haoyu Zhai
>Priority: Minor
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Lucene's index is segmented, and sometimes number of segments and documents 
> arrangement greatly impact performance.
> Given a stable index sort, our team create a tool that records document 
> arrangement (called index map) of an index and rearrange another index 
> (consists of same documents) into the same structure (segment num, and 
> documents included in each segment).
> This tool could be also used in lucene benchmarks for a faster deterministic 
> index construction (if I understand correctly lucene benchmark is using a 
> single thread manner to achieve this).
>  
> We've already had some discussion in email
> [https://markmail.org/message/lbtdntclpnocmfuf]
> And I've implemented the first method, using {{IndexWriter.addIndexes}} and a 
> customized {{FilteredCodecReader}} to achieve the goal. The index 
> construction time is about 25min and time executing this tool is about 10min.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-9694) New tool for creating a deterministic index

[Michael McCandless (Jira)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-9694.

Fix Version/s: 8.9
   master (9.0)
   Resolution: Fixed

Thank you [~zhai7631]!

> New tool for creating a deterministic index
> ---
>
> Key: LUCENE-9694
> URL: https://issues.apache.org/jira/browse/LUCENE-9694
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: general/tools
>Reporter: Haoyu Zhai
>Priority: Minor
> Fix For: master (9.0), 8.9
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Lucene's index is segmented, and sometimes number of segments and documents 
> arrangement greatly impact performance.
> Given a stable index sort, our team create a tool that records document 
> arrangement (called index map) of an index and rearrange another index 
> (consists of same documents) into the same structure (segment num, and 
> documents included in each segment).
> This tool could be also used in lucene benchmarks for a faster deterministic 
> index construction (if I understand correctly lucene benchmark is using a 
> single thread manner to achieve this).
>  
> We've already had some discussion in email
> [https://markmail.org/message/lbtdntclpnocmfuf]
> And I've implemented the first method, using {{IndexWriter.addIndexes}} and a 
> customized {{FilteredCodecReader}} to achieve the goal. The index 
> construction time is about 25min and time executing this tool is about 10min.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9694) New tool for creating a deterministic index

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275311#comment-17275311
 ] 

ASF subversion and git services commented on LUCENE-9694:
-

Commit cb82502e9f6699b60fecd56247bcebe3081293c8 in lucene-solr's branch 
refs/heads/branch_8x from Michael McCandless
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=cb82502 ]

LUCENE-9694: fixes after backporting to compile/run under JDK 8 for Lucene 8.x


> New tool for creating a deterministic index
> ---
>
> Key: LUCENE-9694
> URL: https://issues.apache.org/jira/browse/LUCENE-9694
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: general/tools
>Reporter: Haoyu Zhai
>Priority: Minor
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Lucene's index is segmented, and sometimes number of segments and documents 
> arrangement greatly impact performance.
> Given a stable index sort, our team create a tool that records document 
> arrangement (called index map) of an index and rearrange another index 
> (consists of same documents) into the same structure (segment num, and 
> documents included in each segment).
> This tool could be also used in lucene benchmarks for a faster deterministic 
> index construction (if I understand correctly lucene benchmark is using a 
> single thread manner to achieve this).
>  
> We've already had some discussion in email
> [https://markmail.org/message/lbtdntclpnocmfuf]
> And I've implemented the first method, using {{IndexWriter.addIndexes}} and a 
> customized {{FilteredCodecReader}} to achieve the goal. The index 
> construction time is about 25min and time executing this tool is about 10min.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9694) New tool for creating a deterministic index

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275310#comment-17275310
 ] 

ASF subversion and git services commented on LUCENE-9694:
-

Commit 3da86cb5b368c7325ac89be709b17f9c65eeafdf in lucene-solr's branch 
refs/heads/branch_8x from Patrick Zhai
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3da86cb ]

LUCENE-9694: New tool for creating a deterministic index (#2246)



> New tool for creating a deterministic index
> ---
>
> Key: LUCENE-9694
> URL: https://issues.apache.org/jira/browse/LUCENE-9694
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: general/tools
>Reporter: Haoyu Zhai
>Priority: Minor
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Lucene's index is segmented, and sometimes number of segments and documents 
> arrangement greatly impact performance.
> Given a stable index sort, our team create a tool that records document 
> arrangement (called index map) of an index and rearrange another index 
> (consists of same documents) into the same structure (segment num, and 
> documents included in each segment).
> This tool could be also used in lucene benchmarks for a faster deterministic 
> index construction (if I understand correctly lucene benchmark is using a 
> single thread manner to achieve this).
>  
> We've already had some discussion in email
> [https://markmail.org/message/lbtdntclpnocmfuf]
> And I've implemented the first method, using {{IndexWriter.addIndexes}} and a 
> customized {{FilteredCodecReader}} to achieve the goal. The index 
> construction time is about 25min and time executing this tool is about 10min.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-10321) Unified highlighter returns empty fields when using glob

[David Smiley (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-10321:

Issue Type: Improvement  (was: Bug)
  Priority: Major  (was: Minor)

> Unified highlighter returns empty fields when using glob
> 
>
> Key: SOLR-10321
> URL: https://issues.apache.org/jira/browse/SOLR-10321
> Project: Solr
>  Issue Type: Improvement
>  Components: highlighter
>Affects Versions: 6.4.2
>Reporter: Markus Jelsma
>Priority: Major
> Fix For: 7.0
>
>
> {code}
> q=lama&hl.method=unified&hl.fl=content_*
> {code}
> returns:
> {code}
>name="http://www.nu.nl/weekend/3771311/dalai-lama-inspireert-westen.html";>
> 
> 
>   Nobelprijs Voorafgaand aan zijn bezoek aan Nederland is de dalai 
> lama in Noorwegen om te vieren dat 25 jaar geleden de 
> Nobelprijs voor de Vrede aan hem werd toegekend. Anders dan in Nederland 
> wordt de dalai lama niet ontvangen in het Noorse 
> parlement. 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>   
> {code}
> FastVector and original do not emit: 
> {code}
> 
> 
> 
> 
> 
> 
> 
> 
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12663) Highlighting is not working with docValues only String field

[David Smiley (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved SOLR-12663.
-
Fix Version/s: 8.5
   Resolution: Fixed

Implemented via SOLR-14194 in 8.5 for the original highlighter (though not the 
Unified one).

> Highlighting is not working with docValues only String field
> 
>
> Key: SOLR-12663
> URL: https://issues.apache.org/jira/browse/SOLR-12663
> Project: Solr
>  Issue Type: Bug
>  Components: highlighter
>Affects Versions: 7.2.1
>Reporter: Karthik Ramachandran
>Priority: Major
> Fix For: 8.5
>
>
> Highlighting is not working with docValues only String field.  Need to check 
> SortableTextField as mentioned by [~erickerickson]
> Schema:
>  
>    id
>    
>     required="true"/>
>     stored="true"/>
>     stored="false"/>
>    
>  
> Data:
>  [\\{"id":1,"name":"Testing line 1"},\\{"id":2,"name":"Testing line 
> 2"},\\{"id":3,"name":"Testing line 3"}]
> Query:
>  
> [http://localhost:8983/solr/test/select?q=Testing*&df=name&hl=true&hl.fl=name,name1]
> Response:
>  {"response":{"numFound":3,"start":0,"docs":[
> {"id":"1","name":"Testing line 1","name1":"Testing line 1"}
> ,\{"id":"2","name":"Testing line 2","name1":"Testing line 
> 2"},\{"id":"3","name":"Testing line 3","name1":"Testing line 
> 3"}]},"highlighting":{"1":
> {"name":["Testing line 1"]}
> ,"2":{"name":["Testing line 2"]},"3":{"name":["Testing line 
> 3"]}}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr-operator] sepulworld commented on pull request #206: Remove the SolrCollection and SolrCollectionAlias CRDs

[GitBox]

sepulworld commented on pull request #206:
URL: 
https://github.com/apache/lucene-solr-operator/pull/206#issuecomment-770020782


   Updated local_tutorial.md here: 
https://github.com/apache/lucene-solr-operator/pull/207 if you wanted to review 
and merge separately



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr-operator] sepulworld opened a new pull request #207: docs: removing step in local tutorial for collection creation

[GitBox]

sepulworld opened a new pull request #207:
URL: https://github.com/apache/lucene-solr-operator/pull/207


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9537) Add Indri Search Engine Functionality to Lucene

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275289#comment-17275289
 ] 

ASF subversion and git services commented on LUCENE-9537:
-

Commit 9cc5c9b7983cf181280e18819f8516dc16ccf1ef in lucene-solr's branch 
refs/heads/master from cammiemw
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9cc5c9b ]

LUCENE-9537: Add initial Indri search engine functionality to Lucene



> Add Indri Search Engine Functionality to Lucene
> ---
>
> Key: LUCENE-9537
> URL: https://issues.apache.org/jira/browse/LUCENE-9537
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Cameron VandenBerg
>Priority: Major
>  Labels: patch
> Attachments: LUCENE-9537.patch, LUCENE-INDRI.patch
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> Indri ([http://lemurproject.org/indri.php]) is an academic search engine 
> developed by The University of Massachusetts and Carnegie Mellon University.  
> The major difference between Lucene and Indri is that Indri will give a 
> document a "smoothing score" to a document that does not contain the search 
> term, which has improved the search ranking accuracy in our experiments.  I 
> have created an Indri patch, which adds the search code needed to implement 
> the Indri AND logic as well as Indri's implementation of Dirichlet Smoothing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-8319) NPE when creating pivot

[Isabelle Giguere (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Isabelle Giguere updated SOLR-8319:
---
Attachment: (was: SOLR-7642.patch)

> NPE when creating pivot
> ---
>
> Key: SOLR-8319
> URL: https://issues.apache.org/jira/browse/SOLR-8319
> Project: Solr
>  Issue Type: Bug
>Reporter: Neil Ireson
>Priority: Major
> Attachments: SOLR-8319.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I get a NPE, the trace is shown at the end.
> The problem seems to be this line in the getSubset method:
>   Query query = ft.getFieldQuery(null, field, pivotValue);
> Which takes a value from the index and then analyses it to create a query. I 
> believe the problem is that when my analysis process is applied twice it 
> results in a null query. OK this might be seen as my issue because of dodgy 
> analysis, I thought it might be because I have the wrong order with 
> LengthFilterFactory before EnglishPossessiveFilterFactory and 
> KStemFilterFactory, i.e.:
> 
> 
>  
> So that "cat's" -> "cat" -> "", however any filter order I tried still 
> resulted in a NPE, and perhaps there is a viable case where parsing a term 
> twice results in a null query.
> The thing is I don't see why when the query term comes from the index it has 
> to undergo any analysis. If the term is from the index can it not simply be 
> created using a TermQuery, which I would imagine would also be faster. I 
> altered the "getFieldQuery" line above to the following and that has fixed my 
> NPE issue.
>   Query query = new TermQuery(new Term(field.getName(), pivotValue));
> So far this hasn't caused any other issues but perhaps that is due to my use 
> of Solr, rather than actually fixing an issue. 
> o.a.s.c.SolrCore java.lang.NullPointerException
> at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at 
> org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:91)
> at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:130)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:1296)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.getSubset(PivotFacetProcessor.java:375)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:305)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:228)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:170)
> at 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:262)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at 
> org.eclipse.jetty.server.HttpCo

[jira] [Updated] (SOLR-8319) NPE when creating pivot

[Isabelle Giguere (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Isabelle Giguere updated SOLR-8319:
---
Attachment: SOLR-7642.patch

> NPE when creating pivot
> ---
>
> Key: SOLR-8319
> URL: https://issues.apache.org/jira/browse/SOLR-8319
> Project: Solr
>  Issue Type: Bug
>Reporter: Neil Ireson
>Priority: Major
> Attachments: SOLR-8319.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I get a NPE, the trace is shown at the end.
> The problem seems to be this line in the getSubset method:
>   Query query = ft.getFieldQuery(null, field, pivotValue);
> Which takes a value from the index and then analyses it to create a query. I 
> believe the problem is that when my analysis process is applied twice it 
> results in a null query. OK this might be seen as my issue because of dodgy 
> analysis, I thought it might be because I have the wrong order with 
> LengthFilterFactory before EnglishPossessiveFilterFactory and 
> KStemFilterFactory, i.e.:
> 
> 
>  
> So that "cat's" -> "cat" -> "", however any filter order I tried still 
> resulted in a NPE, and perhaps there is a viable case where parsing a term 
> twice results in a null query.
> The thing is I don't see why when the query term comes from the index it has 
> to undergo any analysis. If the term is from the index can it not simply be 
> created using a TermQuery, which I would imagine would also be faster. I 
> altered the "getFieldQuery" line above to the following and that has fixed my 
> NPE issue.
>   Query query = new TermQuery(new Term(field.getName(), pivotValue));
> So far this hasn't caused any other issues but perhaps that is due to my use 
> of Solr, rather than actually fixing an issue. 
> o.a.s.c.SolrCore java.lang.NullPointerException
> at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at 
> org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:91)
> at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:130)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:1296)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.getSubset(PivotFacetProcessor.java:375)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:305)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:228)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:170)
> at 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:262)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at 
> org.eclipse.jetty.server.HttpConnection.on

[GitHub] [lucene-solr] mikemccand merged pull request #2097: LUCENE-9537

[GitBox]

mikemccand merged pull request #2097:
URL: https://github.com/apache/lucene-solr/pull/2097


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-7642) Should launching Solr in cloud mode using a ZooKeeper chroot create the chroot znode if it doesn't exist?

[Isabelle Giguere (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275287#comment-17275287
 ] 

Isabelle Giguere commented on SOLR-7642:


New patch, adressing Jan's comments.

The patch needs refguide docs. 
- done

Should it also be documented in bin/solr -h? 
- done

Should "/solr" be an always-whitelisted path? 
- Not useful if -DcreateZkRoot=true, because any path will be ceated

Nipick: Consider terminology - is it zkRoot or zkChRoot? 
- Keeping 'createZkRoot' in the new patch.  It at least avoids changing scripts 
on our side ;)
- I'm not a fan of "chroot".  Doesn't 'ch' mean "change" ?  So "createZKChRoot" 
would mean "create the zookeeper change root" ?
- It could be createZkNode ?  createZkPath ?


I can't find the test class TestZkChroot.java in master branch, so, no unit 
test this time.

> Should launching Solr in cloud mode using a ZooKeeper chroot create the 
> chroot znode if it doesn't exist?
> -
>
> Key: SOLR-7642
> URL: https://issues.apache.org/jira/browse/SOLR-7642
> Project: Solr
>  Issue Type: Improvement
>Reporter: Timothy Potter
>Priority: Minor
> Attachments: SOLR-7642.patch, SOLR-7642.patch, SOLR-7642.patch, 
> SOLR-7642.patch, SOLR-7642_tag_7.5.0.patch, 
> SOLR-7642_tag_7.5.0_proposition.patch
>
>
> If you launch Solr for the first time in cloud mode using a ZooKeeper 
> connection string that includes a chroot leads to the following 
> initialization error:
> {code}
> ERROR - 2015-06-05 17:15:50.410; [   ] org.apache.solr.common.SolrException; 
> null:org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified 
> in ZkHost but the znode doesn't exist. localhost:2181/lan
> at 
> org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:339)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:140)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:110)
> at 
> org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
> at 
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at 
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> {code}
> The work-around for this is to use the scripts/cloud-scripts/zkcli.sh script 
> to create the chroot znode (bootstrap action does this).
> I'm wondering if we shouldn't just create the znode if it doesn't exist? Or 
> is that some violation of using a chroot?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-7642) Should launching Solr in cloud mode using a ZooKeeper chroot create the chroot znode if it doesn't exist?

[Isabelle Giguere (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275287#comment-17275287
 ] 

Isabelle Giguere edited comment on SOLR-7642 at 1/29/21, 7:45 PM:
--

New patch, addressing [~janhoy] comments.

The patch needs refguide docs. 
- done

Should it also be documented in bin/solr -h? 
- done

Should "/solr" be an always-whitelisted path? 
- Not useful if -DcreateZkRoot=true, because any path will be ceated

Nipick: Consider terminology - is it zkRoot or zkChRoot? 
- Keeping 'createZkRoot' in the new patch.  It at least avoids changing scripts 
on our side ;)
- I'm not a fan of "chroot".  Doesn't 'ch' mean "change" ?  So "createZKChRoot" 
would mean "create the zookeeper change root" ?
- It could be createZkNode ?  createZkPath ?


I can't find the test class TestZkChroot.java in master branch, so, no unit 
test this time.


was (Author: igiguere):
New patch, adressing Jan's comments.

The patch needs refguide docs. 
- done

Should it also be documented in bin/solr -h? 
- done

Should "/solr" be an always-whitelisted path? 
- Not useful if -DcreateZkRoot=true, because any path will be ceated

Nipick: Consider terminology - is it zkRoot or zkChRoot? 
- Keeping 'createZkRoot' in the new patch.  It at least avoids changing scripts 
on our side ;)
- I'm not a fan of "chroot".  Doesn't 'ch' mean "change" ?  So "createZKChRoot" 
would mean "create the zookeeper change root" ?
- It could be createZkNode ?  createZkPath ?


I can't find the test class TestZkChroot.java in master branch, so, no unit 
test this time.

> Should launching Solr in cloud mode using a ZooKeeper chroot create the 
> chroot znode if it doesn't exist?
> -
>
> Key: SOLR-7642
> URL: https://issues.apache.org/jira/browse/SOLR-7642
> Project: Solr
>  Issue Type: Improvement
>Reporter: Timothy Potter
>Priority: Minor
> Attachments: SOLR-7642.patch, SOLR-7642.patch, SOLR-7642.patch, 
> SOLR-7642.patch, SOLR-7642_tag_7.5.0.patch, 
> SOLR-7642_tag_7.5.0_proposition.patch
>
>
> If you launch Solr for the first time in cloud mode using a ZooKeeper 
> connection string that includes a chroot leads to the following 
> initialization error:
> {code}
> ERROR - 2015-06-05 17:15:50.410; [   ] org.apache.solr.common.SolrException; 
> null:org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified 
> in ZkHost but the znode doesn't exist. localhost:2181/lan
> at 
> org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:339)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:140)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:110)
> at 
> org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
> at 
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at 
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> {code}
> The work-around for this is to use the scripts/cloud-scripts/zkcli.sh script 
> to create the chroot znode (bootstrap action does this).
> I'm wondering if we shouldn't just create the znode if it doesn't exist? Or 
> is that some violation of using a chroot?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-7642) Should launching Solr in cloud mode using a ZooKeeper chroot create the chroot znode if it doesn't exist?

[Isabelle Giguere (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Isabelle Giguere updated SOLR-7642:
---
Attachment: SOLR-7642.patch

> Should launching Solr in cloud mode using a ZooKeeper chroot create the 
> chroot znode if it doesn't exist?
> -
>
> Key: SOLR-7642
> URL: https://issues.apache.org/jira/browse/SOLR-7642
> Project: Solr
>  Issue Type: Improvement
>Reporter: Timothy Potter
>Priority: Minor
> Attachments: SOLR-7642.patch, SOLR-7642.patch, SOLR-7642.patch, 
> SOLR-7642.patch, SOLR-7642_tag_7.5.0.patch, 
> SOLR-7642_tag_7.5.0_proposition.patch
>
>
> If you launch Solr for the first time in cloud mode using a ZooKeeper 
> connection string that includes a chroot leads to the following 
> initialization error:
> {code}
> ERROR - 2015-06-05 17:15:50.410; [   ] org.apache.solr.common.SolrException; 
> null:org.apache.solr.common.cloud.ZooKeeperException: A chroot was specified 
> in ZkHost but the znode doesn't exist. localhost:2181/lan
> at 
> org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:339)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:140)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:110)
> at 
> org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
> at 
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at 
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> {code}
> The work-around for this is to use the scripts/cloud-scripts/zkcli.sh script 
> to create the chroot znode (bootstrap action does this).
> I'm wondering if we shouldn't just create the znode if it doesn't exist? Or 
> is that some violation of using a chroot?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] mikemccand commented on pull request #2097: LUCENE-9537

[GitBox]

mikemccand commented on pull request #2097:
URL: https://github.com/apache/lucene-solr/pull/2097#issuecomment-770011220


   Thanks @cammiemw, that test now passes again!
   
   I ran top-level tests multiple times and each time, a solr test failed for 
what looked like (not certain) unrelated to this great change. On the 4th try, 
all solr tests finally passed.
   
   And `./gradlew precommit` passes as well.
   
   I will push soon!  Thank you for persisting @cammiemw and sorry for the slow 
iterations here.  I am excited that we are cross-fertilizing these two 
impactful open-source search engines!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr-operator] HoustonPutman opened a new pull request #206: Remove the SolrCollection and SolrCollectionAlias CRDs

[GitBox]

HoustonPutman opened a new pull request #206:
URL: https://github.com/apache/lucene-solr-operator/pull/206


   Resolves: #204 
   
   Still need to update the local tutorial to not use the Collection CRD.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr-operator] HoustonPutman merged pull request #203: Fix copy/paste typo addressed -> updated

[GitBox]

HoustonPutman merged pull request #203:
URL: https://github.com/apache/lucene-solr-operator/pull/203


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-8319) NPE when creating pivot

[Houston Putman (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275262#comment-17275262
 ] 

Houston Putman commented on SOLR-8319:
--

The reason for the NPE on different values of {{facet.limit}} is because of 
refinements. This logic is only called whenever refinements need to be made for 
pivot facet results. Refinements are made because the first-pass in getting 
facet results does not get exact results. It will ask for the top n terms from 
each shard. If limit=n and the top n terms all have results from all shards, 
then no refinement is needed, because the top n terms all have exact results. 
If some of the top n terms are not included in some shard results, then 
refinement requests have to be sent to those shards to get exact results.

When limit = -1, then no refinements are done because it asks for all results 
from every shard.
When limit <= 3, then the stopword values are not close-enough to the top 3 to 
need refinements.
When limit >= 4, then the stopwords are close enough to being selected for the 
results that they need to be refined.

> NPE when creating pivot
> ---
>
> Key: SOLR-8319
> URL: https://issues.apache.org/jira/browse/SOLR-8319
> Project: Solr
>  Issue Type: Bug
>Reporter: Neil Ireson
>Priority: Major
> Attachments: SOLR-8319.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I get a NPE, the trace is shown at the end.
> The problem seems to be this line in the getSubset method:
>   Query query = ft.getFieldQuery(null, field, pivotValue);
> Which takes a value from the index and then analyses it to create a query. I 
> believe the problem is that when my analysis process is applied twice it 
> results in a null query. OK this might be seen as my issue because of dodgy 
> analysis, I thought it might be because I have the wrong order with 
> LengthFilterFactory before EnglishPossessiveFilterFactory and 
> KStemFilterFactory, i.e.:
> 
> 
>  
> So that "cat's" -> "cat" -> "", however any filter order I tried still 
> resulted in a NPE, and perhaps there is a viable case where parsing a term 
> twice results in a null query.
> The thing is I don't see why when the query term comes from the index it has 
> to undergo any analysis. If the term is from the index can it not simply be 
> created using a TermQuery, which I would imagine would also be faster. I 
> altered the "getFieldQuery" line above to the following and that has fixed my 
> NPE issue.
>   Query query = new TermQuery(new Term(field.getName(), pivotValue));
> So far this hasn't caused any other issues but perhaps that is due to my use 
> of Solr, rather than actually fixing an issue. 
> o.a.s.c.SolrCore java.lang.NullPointerException
> at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at 
> org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:91)
> at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:130)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:1296)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.getSubset(PivotFacetProcessor.java:375)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:305)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:228)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:170)
> at 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:262)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at 
> org.eclipse.jetty.server.h

[GitHub] [lucene-solr] HoustonPutman opened a new pull request #2273: SOLR-8319: Add method for tokenized field search in FieldType

[GitBox]

HoustonPutman opened a new pull request #2273:
URL: https://github.com/apache/lucene-solr/pull/2273


   Addresses NPE found in https://issues.apache.org/jira/browse/SOLR-8319
   
   Will do some performance comparisons before merging as well.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-8319) NPE when creating pivot

[Houston Putman (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275256#comment-17275256
 ] 

Houston Putman commented on SOLR-8319:
--

I have done just a bit of digging, but the reason why a null query is returned 
is because the stop word itself contains no tokens, and [a query with no tokens 
gets returned as 
null|https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/util/QueryBuilder.java#L343].
 The default {{FieldType.getFieldQuery}} is not used, because 
{{TextField.getFieldQuery}} overrides it. This is the method that eventually 
(through other methods) calls the null-returning 
{{QueryBuilder.createFieldQuery}}.

I agree that doing analyzation of a single token is not necessary and can 
generally be skipped. I think it makes sense to add a method to {{FieldType}} 
that does a non-analyzed field search. From what I can tell, this would only be 
different from the {{FieldType.getFieldQuery}} implementation for 
{{TextField}}. But going through and doing the changes, I noticed that there 
was a TODO: in another place for this exact use case.

I think it would also be good to check for null, but I think this is a useful 
method to have anyways. And after this change there shouldn't be a code path to 
return null from that method. I'll put up a patch with my proposal. 

> NPE when creating pivot
> ---
>
> Key: SOLR-8319
> URL: https://issues.apache.org/jira/browse/SOLR-8319
> Project: Solr
>  Issue Type: Bug
>Reporter: Neil Ireson
>Priority: Major
> Attachments: SOLR-8319.patch
>
>
> I get a NPE, the trace is shown at the end.
> The problem seems to be this line in the getSubset method:
>   Query query = ft.getFieldQuery(null, field, pivotValue);
> Which takes a value from the index and then analyses it to create a query. I 
> believe the problem is that when my analysis process is applied twice it 
> results in a null query. OK this might be seen as my issue because of dodgy 
> analysis, I thought it might be because I have the wrong order with 
> LengthFilterFactory before EnglishPossessiveFilterFactory and 
> KStemFilterFactory, i.e.:
> 
> 
>  
> So that "cat's" -> "cat" -> "", however any filter order I tried still 
> resulted in a NPE, and perhaps there is a viable case where parsing a term 
> twice results in a null query.
> The thing is I don't see why when the query term comes from the index it has 
> to undergo any analysis. If the term is from the index can it not simply be 
> created using a TermQuery, which I would imagine would also be faster. I 
> altered the "getFieldQuery" line above to the following and that has fixed my 
> NPE issue.
>   Query query = new TermQuery(new Term(field.getName(), pivotValue));
> So far this hasn't caused any other issues but perhaps that is due to my use 
> of Solr, rather than actually fixing an issue. 
> o.a.s.c.SolrCore java.lang.NullPointerException
> at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at 
> org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:91)
> at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:130)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:1296)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.getSubset(PivotFacetProcessor.java:375)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:305)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:228)
> at 
> org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:170)
> at 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:262)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.ecli

[GitHub] [lucene-solr] mikemccand commented on pull request #2246: LUCENE-9694: New tool for creating a deterministic index

[GitBox]

mikemccand commented on pull request #2246:
URL: https://github.com/apache/lucene-solr/pull/2246#issuecomment-769974955


   Thank you @zhaih!  This will be a very useful tool for realistic 
benchmarking!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9694) New tool for creating a deterministic index

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/LUCENE-9694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275246#comment-17275246
 ] 

ASF subversion and git services commented on LUCENE-9694:
-

Commit e4cede0e8c5a640586c044f45e35448ef0f1862d in lucene-solr's branch 
refs/heads/master from Patrick Zhai
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e4cede0 ]

LUCENE-9694: New tool for creating a deterministic index (#2246)



> New tool for creating a deterministic index
> ---
>
> Key: LUCENE-9694
> URL: https://issues.apache.org/jira/browse/LUCENE-9694
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: general/tools
>Reporter: Haoyu Zhai
>Priority: Minor
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> Lucene's index is segmented, and sometimes number of segments and documents 
> arrangement greatly impact performance.
> Given a stable index sort, our team create a tool that records document 
> arrangement (called index map) of an index and rearrange another index 
> (consists of same documents) into the same structure (segment num, and 
> documents included in each segment).
> This tool could be also used in lucene benchmarks for a faster deterministic 
> index construction (if I understand correctly lucene benchmark is using a 
> single thread manner to achieve this).
>  
> We've already had some discussion in email
> [https://markmail.org/message/lbtdntclpnocmfuf]
> And I've implemented the first method, using {{IndexWriter.addIndexes}} and a 
> customized {{FilteredCodecReader}} to achieve the goal. The index 
> construction time is about 25min and time executing this tool is about 10min.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] mikemccand merged pull request #2246: LUCENE-9694: New tool for creating a deterministic index

[GitBox]

mikemccand merged pull request #2246:
URL: https://github.com/apache/lucene-solr/pull/2246


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Assigned] (SOLR-15121) Move XSLT (tr param) to scripting contrib

[David Eric Pugh (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Eric Pugh reassigned SOLR-15121:
--

Assignee: David Eric Pugh

> Move XSLT (tr param) to scripting contrib
> -
>
> Key: SOLR-15121
> URL: https://issues.apache.org/jira/browse/SOLR-15121
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Assignee: David Eric Pugh
>Priority: Blocker
> Fix For: master (9.0)
>
>
> The XSLT functionality, present in both XML /update loading, and also in the 
> response writer, ought to move to the "scripting" contrib module because XSLT 
> is a type of scripting.  XSLT is risky from a security standpoint, and so 
> should not be in solr-core.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] mikemccand commented on a change in pull request #2256: LUCENE-9507 Custom order for leaves in IndexReader and IndexWriter

[GitBox]

mikemccand commented on a change in pull request #2256:
URL: https://github.com/apache/lucene-solr/pull/2256#discussion_r566993804



##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##
@@ -941,6 +969,11 @@ public IndexWriter(Directory d, IndexWriterConfig conf) 
throws IOException {
 // obtain the write.lock. If the user configured a timeout,
 // we wrap with a sleeper and this might take some time.
 writeLock = d.obtainLock(WRITE_LOCK_NAME);
+if (config.getIndexSort() != null && leafSorter != null) {
+  throw new IllegalArgumentException(
+  "[IndexWriter] can't use index sort and leaf sorter at the same 
time!");

Review comment:
   Hmm, why is that?  I thought this change sorts whole segments, and index 
sorting sorts documents within one segment?  Why do they conflict?

##
File path: lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
##
@@ -933,6 +936,31 @@ protected final void ensureOpen() throws 
AlreadyClosedException {
* low-level IO error
*/
   public IndexWriter(Directory d, IndexWriterConfig conf) throws IOException {
+this(d, conf, null);
+  }
+
+  /**
+   * Constructs a new IndexWriter per the settings given in conf. 
If you want to make
+   * "live" changes to this writer instance, use {@link #getConfig()}.
+   *
+   * NOTE: after ths writer is created, the given configuration 
instance cannot be passed
+   * to another writer.
+   *
+   * @param d the index directory. The index is either created or appended 
according 
+   * conf.getOpenMode().
+   * @param conf the configuration settings according to which IndexWriter 
should be initialized.
+   * @param leafSorter a comparator for sorting leaf readers. Providing 
leafSorter is useful for
+   * indices on which it is expected to run many queries with particular 
sort criteria (e.g. for
+   * time-based indices this is usually a descending sort on timestamp). 
In this case {@code
+   * leafSorter} should sort leaves according to this sort criteria. 
Providing leafSorter allows
+   * to speed up this particular type of sort queries by early terminating 
while iterating
+   * though segments and segments' documents.
+   * @throws IOException if the directory cannot be read/written to, or if it 
does not exist and
+   * conf.getOpenMode() is OpenMode.APPEND or if 
there is any other
+   * low-level IO error
+   */
+  public IndexWriter(Directory d, IndexWriterConfig conf, 
Comparator leafSorter)

Review comment:
   Should `leafSorter` instead be added to `IndexWriterConfig`?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15121) Move XSLT (tr param) to scripting contrib

[David Smiley (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275220#comment-17275220
 ] 

David Smiley commented on SOLR-15121:
-

Additionally, the functionality should be guarded by the "trusted" configSet 
notion, just like the other scripting functionality is.

> Move XSLT (tr param) to scripting contrib
> -
>
> Key: SOLR-15121
> URL: https://issues.apache.org/jira/browse/SOLR-15121
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Priority: Blocker
> Fix For: master (9.0)
>
>
> The XSLT functionality, present in both XML /update loading, and also in the 
> response writer, ought to move to the "scripting" contrib module because XSLT 
> is a type of scripting.  XSLT is risky from a security standpoint, and so 
> should not be in solr-core.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gautamworah96 commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

gautamworah96 commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r566988968



##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -353,12 +349,65 @@ public FacetLabel getPath(int ordinal) throws IOException 
{
 }
 
 synchronized (categoryCache) {
-  categoryCache.put(catIDInteger, ret);
+  categoryCache.put(ordinal, ret);
 }
 
 return ret;
   }
 
+  private FacetLabel getPathFromCache(int ordinal) {
+ensureOpen();
+
+// TODO: can we use an int-based hash impl, such as IntToObjectMap,
+// wrapped as LRU?
+synchronized (categoryCache) {
+  FacetLabel res = categoryCache.get(ordinal);
+  if (res != null) {
+return res;
+  }
+}
+return null;
+  }
+
+  /* This API is only supported for indexes created with Lucene 8.7+ codec **/
+  public FacetLabel[] getBulkPath(int[] ordinal) throws IOException {
+FacetLabel[] bulkPath = new FacetLabel[ordinal.length];
+Map originalPosition = new HashMap<>();
+for (int i = 0; i < ordinal.length; i++) {
+  if (ordinal[i] < 0 || ordinal[i] >= indexReader.maxDoc()) {
+return null;
+  }
+  FacetLabel ordinalPath = getPathFromCache(ordinal[i]);
+  if (ordinalPath != null) {
+bulkPath[i] = ordinalPath;
+  }
+  originalPosition.put(ordinal[i], i);
+}
+
+Arrays.sort(ordinal);
+int readerIndex = 0;
+BinaryDocValues values = null;
+
+for (int ord : ordinal) {
+  if (bulkPath[originalPosition.get(ord)] == null) {
+if (values == null
+|| values.advanceExact(ord - 
indexReader.leaves().get(readerIndex).docBase) == false) {
+  readerIndex = ReaderUtil.subIndex(ord, indexReader.leaves());
+  LeafReader leafReader = 
indexReader.leaves().get(readerIndex).reader();
+  values = leafReader.getBinaryDocValues(Consts.FULL);
+  assert values.advanceExact(ord - 
indexReader.leaves().get(readerIndex).docBase);

Review comment:
   Thanks for catching this. Fixed it in the new commit





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-15121) Move XSLT (tr param) to scripting contrib

[David Smiley (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-15121:

Fix Version/s: master (9.0)

> Move XSLT (tr param) to scripting contrib
> -
>
> Key: SOLR-15121
> URL: https://issues.apache.org/jira/browse/SOLR-15121
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Priority: Blocker
> Fix For: master (9.0)
>
>
> The XSLT functionality, present in both XML /update loading, and also in the 
> response writer, ought to move to the "scripting" contrib module because XSLT 
> is a type of scripting.  XSLT is risky from a security standpoint, and so 
> should not be in solr-core.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-15121) Move XSLT (tr param) to scripting contrib

[David Smiley (Jira)]

David Smiley created SOLR-15121:
---

 Summary: Move XSLT (tr param) to scripting contrib
 Key: SOLR-15121
 URL: https://issues.apache.org/jira/browse/SOLR-15121
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: David Smiley


The XSLT functionality, present in both XML /update loading, and also in the 
response writer, ought to move to the "scripting" contrib module because XSLT 
is a type of scripting.  XSLT is risky from a security standpoint, and so 
should not be in solr-core.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gautamworah96 commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

gautamworah96 commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r566981233



##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -353,12 +349,65 @@ public FacetLabel getPath(int ordinal) throws IOException 
{
 }
 
 synchronized (categoryCache) {
-  categoryCache.put(catIDInteger, ret);
+  categoryCache.put(ordinal, ret);
 }
 
 return ret;
   }
 
+  private FacetLabel getPathFromCache(int ordinal) {
+ensureOpen();
+
+// TODO: can we use an int-based hash impl, such as IntToObjectMap,
+// wrapped as LRU?
+synchronized (categoryCache) {
+  FacetLabel res = categoryCache.get(ordinal);
+  if (res != null) {
+return res;
+  }
+}
+return null;
+  }
+
+  /* This API is only supported for indexes created with Lucene 8.7+ codec **/
+  public FacetLabel[] getBulkPath(int[] ordinal) throws IOException {
+FacetLabel[] bulkPath = new FacetLabel[ordinal.length];
+Map originalPosition = new HashMap<>();
+for (int i = 0; i < ordinal.length; i++) {
+  if (ordinal[i] < 0 || ordinal[i] >= indexReader.maxDoc()) {
+return null;
+  }
+  FacetLabel ordinalPath = getPathFromCache(ordinal[i]);
+  if (ordinalPath != null) {
+bulkPath[i] = ordinalPath;
+  }
+  originalPosition.put(ordinal[i], i);

Review comment:
   Hmmm. I think using a map to remember initial positions is still simpler 
to understand? or I could still change it in the next revision





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gautamworah96 commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

gautamworah96 commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r566980263



##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -353,12 +349,65 @@ public FacetLabel getPath(int ordinal) throws IOException 
{
 }
 
 synchronized (categoryCache) {
-  categoryCache.put(catIDInteger, ret);
+  categoryCache.put(ordinal, ret);
 }
 
 return ret;
   }
 
+  private FacetLabel getPathFromCache(int ordinal) {
+ensureOpen();
+
+// TODO: can we use an int-based hash impl, such as IntToObjectMap,
+// wrapped as LRU?
+synchronized (categoryCache) {
+  FacetLabel res = categoryCache.get(ordinal);
+  if (res != null) {
+return res;
+  }
+}
+return null;
+  }
+
+  /* This API is only supported for indexes created with Lucene 8.7+ codec **/

Review comment:
   Done in the recent commit





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14067) Move StatelessScriptUpdateProcessor to a contrib

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275213#comment-17275213
 ] 

ASF subversion and git services commented on SOLR-14067:


Commit 6d71a0aced96c3521796a9e16b055f041141a146 in lucene-solr's branch 
refs/heads/master from Eric Pugh
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6d71a0a ]

SOLR-14067: v4 Create /contrib/scripting module with ScriptingUpdateProcessor 
(#2257)

* Creating Scripting contrib module to centralize the less secure code related 
to scripts.

* tweak the changelog and update notice to explain why the name changed and the 
security posture thinking

* the test script happens to be a currency.xml, which made me think we were 
doing something specific to currency types, but instead any xml formatted file 
will suffice for the test.

* Update 
solr/contrib/scripting/src/java/org/apache/solr/scripting/update/ScriptUpdateProcessorFactory.java

* Update 
solr/contrib/scripting/src/java/org/apache/solr/scripting/update/package-info.java

* drop the ing, and be more specific on the name of the ref guide page

* comment out the script update chain.

The sample techproducts configSet is used by many of the solr unit tests, and 
by default doesn't have access to the jar file in the contrib module.   This is 
commented out, similar to how the lang contrib is.

* using a Mock for the script processor in order to keep the trusted configSets 
tests all together.

* tweak since we are using a mock script processor

Co-authored-by: David Smiley 

> Move StatelessScriptUpdateProcessor to a contrib
> 
>
> Key: SOLR-14067
> URL: https://issues.apache.org/jira/browse/SOLR-14067
> Project: Solr
>  Issue Type: Improvement
>Reporter: Ishan Chattopadhyaya
>Assignee: David Eric Pugh
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>
> Move server-side scripting out of core and into a new contrib.  This is 
> better for security.
> Former description:
> 
> We should eliminate all scripting capabilities within Solr. Let us start with 
> the StatelessScriptUpdateProcessor deprecation/removal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gautamworah96 commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

gautamworah96 commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r566978754



##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -314,7 +316,6 @@ public int getOrdinal(FacetLabel cp) throws IOException {
 
   @Override
   public FacetLabel getPath(int ordinal) throws IOException {
-ensureOpen();

Review comment:
   In the earlier iteration it was shifted to the helper method that looked 
up values in the cache. I have kept it in the outer method in the most recent 
commit





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] epugh merged pull request #2257: SOLR-14067: v4 Create /contrib/scripting module with ScriptingUpdateProcessor

[GitBox]

epugh merged pull request #2257:
URL: https://github.com/apache/lucene-solr/pull/2257


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gautamworah96 commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

gautamworah96 commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r566977877



##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -353,12 +349,65 @@ public FacetLabel getPath(int ordinal) throws IOException 
{
 }
 
 synchronized (categoryCache) {
-  categoryCache.put(catIDInteger, ret);
+  categoryCache.put(ordinal, ret);
 }
 
 return ret;
   }
 
+  private FacetLabel getPathFromCache(int ordinal) {
+ensureOpen();
+
+// TODO: can we use an int-based hash impl, such as IntToObjectMap,
+// wrapped as LRU?
+synchronized (categoryCache) {
+  FacetLabel res = categoryCache.get(ordinal);
+  if (res != null) {
+return res;
+  }
+}
+return null;
+  }
+
+  /* This API is only supported for indexes created with Lucene 8.7+ codec **/
+  public FacetLabel[] getBulkPath(int[] ordinal) throws IOException {
+FacetLabel[] bulkPath = new FacetLabel[ordinal.length];
+Map originalPosition = new HashMap<>();
+for (int i = 0; i < ordinal.length; i++) {
+  if (ordinal[i] < 0 || ordinal[i] >= indexReader.maxDoc()) {
+return null;
+  }
+  FacetLabel ordinalPath = getPathFromCache(ordinal[i]);
+  if (ordinalPath != null) {
+bulkPath[i] = ordinalPath;
+  }
+  originalPosition.put(ordinal[i], i);
+}
+
+Arrays.sort(ordinal);
+int readerIndex = 0;
+BinaryDocValues values = null;
+
+for (int ord : ordinal) {
+  if (bulkPath[originalPosition.get(ord)] == null) {

Review comment:
   It will not be `null` if we already find it in the cache in the initial 
step i.e the `FacetLabel` for this `ordinal` was found in the cache itself.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gautamworah96 commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

gautamworah96 commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r566976943



##
File path: 
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyReader.java
##
@@ -353,12 +349,65 @@ public FacetLabel getPath(int ordinal) throws IOException 
{
 }
 
 synchronized (categoryCache) {
-  categoryCache.put(catIDInteger, ret);
+  categoryCache.put(ordinal, ret);
 }
 
 return ret;
   }
 
+  private FacetLabel getPathFromCache(int ordinal) {
+ensureOpen();

Review comment:
   This seems much better. Updated. 
   My initial impression was that `ensureOpen()` needs to be called before 
looking up each ordinal.

##
File path: 
lucene/facet/src/test/org/apache/lucene/facet/taxonomy/directory/TestDirectoryTaxonomyReader.java
##
@@ -569,4 +569,31 @@ public void testAccountable() throws Exception {
 taxoReader.close();
 dir.close();
   }
+
+  public void testBulkPath() throws Exception {
+Directory src = newDirectory();
+DirectoryTaxonomyWriter w = new DirectoryTaxonomyWriter(src);
+String arr[] = new String[] {"a", "b", "c", "d", "e"};
+
+FacetLabel allpaths[] = new FacetLabel[arr.length];
+int allords[] = new int[arr.length];
+
+for (int i = 0; i < arr.length; i++) {
+  allpaths[i] = new FacetLabel(arr[i]);
+  w.addCategory(allpaths[i]);
+}
+w.commit();
+w.close();
+
+DirectoryTaxonomyReader r1 = new DirectoryTaxonomyReader(src);
+
+for (int i = 0; i < allpaths.length; i++) {
+  allords[i] = r1.getOrdinal(allpaths[i]);
+}
+
+FacetLabel allBulkpath[] = r1.getBulkPath(allords);
+assertArrayEquals(allpaths, allBulkpath);

Review comment:
   Added





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-15025) MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value

[Mike Drob (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved SOLR-15025.
--
Fix Version/s: 8.9
   master (9.0)
 Assignee: Mike Drob
   Resolution: Fixed

Change looks good as is, thanks for contributing Tim! I also back ported this 
to branch 8

> MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value
> -
>
> Key: SOLR-15025
> URL: https://issues.apache.org/jira/browse/SOLR-15025
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
>  Labels: beginner, newdev
> Fix For: master (9.0), 8.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> the api could also expand to take a time unit?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gautamworah96 commented on a change in pull request #2247: LUCENE-9476 Add getBulkPath API for the Taxonomy index

[GitBox]

gautamworah96 commented on a change in pull request #2247:
URL: https://github.com/apache/lucene-solr/pull/2247#discussion_r566976157



##
File path: 
lucene/facet/src/test/org/apache/lucene/facet/taxonomy/directory/TestDirectoryTaxonomyReader.java
##
@@ -569,4 +569,31 @@ public void testAccountable() throws Exception {
 taxoReader.close();
 dir.close();
   }
+
+  public void testBulkPath() throws Exception {

Review comment:
   Done. This is definitely better





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15025) MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274772#comment-17274772
 ] 

ASF subversion and git services commented on SOLR-15025:


Commit 5745703ccabdc849484b2e259aded8ea691457b6 in lucene-solr's branch 
refs/heads/branch_8x from Tim Dillon
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5745703 ]

SOLR-15025: MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value 
(#2193)

* Change timeout values to seconds

> MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value
> -
>
> Key: SOLR-15025
> URL: https://issues.apache.org/jira/browse/SOLR-15025
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Priority: Major
>  Labels: beginner, newdev
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> the api could also expand to take a time unit?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15025) MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274698#comment-17274698
 ] 

ASF subversion and git services commented on SOLR-15025:


Commit a7a434dbc45940553949214335810cef37b947bb in lucene-solr's branch 
refs/heads/master from Tim Dillon
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a7a434d ]

SOLR-15025: MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value 
(#2193)

* Change timeout values to seconds

> MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value
> -
>
> Key: SOLR-15025
> URL: https://issues.apache.org/jira/browse/SOLR-15025
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Priority: Major
>  Labels: beginner, newdev
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> the api could also expand to take a time unit?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] madrob merged pull request #2193: SOLR-15025: MiniSolrCloudCluster.waitForAllNodes ignores passed timeout value

[GitBox]

madrob merged pull request #2193:
URL: https://github.com/apache/lucene-solr/pull/2193


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-13209) NullPointerException from call in org.apache.solr.search.SolrIndexSearcher.getDocSet

[Isabelle Giguere (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-13209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274593#comment-17274593
 ] 

Isabelle Giguere commented on SOLR-13209:
-

Throwing an exception for a null query makes sense for the request in the 
description.  This request is invalid, missing a value for group.query.

But the exception should not be thrown from such a low-level method, because it 
could impact faceting too, where the user doesn't have control over the queries 
that are produced by the faceting process.  Refer to SOLR-8319.

Group queries parameters should be validated, and an exception thrown right 
away.

> NullPointerException from call in 
> org.apache.solr.search.SolrIndexSearcher.getDocSet
> 
>
> Key: SOLR-13209
> URL: https://issues.apache.org/jira/browse/SOLR-13209
> Project: Solr
>  Issue Type: Bug
>Affects Versions: master (9.0)
> Environment: h1. Steps to reproduce
> * Use a Linux machine.
> * Build commit {{ea2c8ba}} of Solr as described in the section below.
> * Build the films collection as described below.
> * Start the server using the command {{./bin/solr start -f -p 8983 -s 
> /tmp/home}}
> * Request the URL given in the bug description.
> h1. Compiling the server
> {noformat}
> git clone https://github.com/apache/lucene-solr
> cd lucene-solr
> git checkout ea2c8ba
> ant compile
> cd solr
> ant server
> {noformat}
> h1. Building the collection and reproducing the bug
> We followed [Exercise 
> 2|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html#exercise-2] from 
> the [Solr 
> Tutorial|http://lucene.apache.org/solr/guide/7_5/solr-tutorial.html].
> {noformat}
> mkdir -p /tmp/home
> echo '' > 
> /tmp/home/solr.xml
> {noformat}
> In one terminal start a Solr instance in foreground:
> {noformat}
> ./bin/solr start -f -p 8983 -s /tmp/home
> {noformat}
> In another terminal, create a collection of movies, with no shards and no 
> replication, and initialize it:
> {noformat}
> bin/solr create -c films
> curl -X POST -H 'Content-type:application/json' --data-binary '{"add-field": 
> {"name":"name", "type":"text_general", "multiValued":false, "stored":true}}' 
> http://localhost:8983/solr/films/schema
> curl -X POST -H 'Content-type:application/json' --data-binary 
> '{"add-copy-field" : {"source":"*","dest":"_text_"}}' 
> http://localhost:8983/solr/films/schema
> ./bin/post -c films example/films/films.json
> curl -v “URL_BUG”
> {noformat}
> Please check the issue description below to find the “URL_BUG” that will 
> allow you to reproduce the issue reported.
>Reporter: Cesar Rodriguez
>Priority: Minor
>  Labels: diffblue, newdev
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Requesting the following URL causes Solr to return an HTTP 500 error response:
> {noformat}
> http://localhost:8983/solr/films/select?group=true&group.query
> {noformat}
> The error response seems to be caused by the following uncaught exception:
> {noformat}
>  java.lang.NullPointerException
>   at 
> java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
>   at 
> org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:124)
>   at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:163)
>   at 
> org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:792)
>   at 
> org.apache.solr.search.Grouping$CommandQuery.createFirstPassCollector(Grouping.java:860)
>   at org.apache.solr.search.Grouping.execute(Grouping.java:327)
>   at 
> org.apache.solr.handler.component.QueryComponent.doProcessGroupedSearch(QueryComponent.java:1408)
>   at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:365)
>   at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2559)
> [...]
> {noformat}
> Method {{org.apache.solr.search.SolrIndexSearcher.getDocSet()}}, at line 792 
> calls {{filterCache.get(absQ)}} where {{absQ}} is a null pointer. I think 
> this null pointer comes in fact from the caller, but I don't fully follow the 
> logic of the code.
> To set up an environment to reproduce this bug, follow the description in the 
> ‘Environment’ field.
> We automatically found this issue and ~70 more like this using [Diffblue 
> Microservices Testing|https://www.diffblue.com/labs/?utm_source=solr-br]. 
> Find more information on this [fuzz testing 
> campaign|https://www.diffblue.com/blog/2018/12/19/diffblue-microservice-testing-a-sneak-peek-at-our-early-product-and-results?utm_source=solr-br].



--
This message was sent by Atlassia

[GitHub] [lucene-solr] megancarey commented on a change in pull request #2234: SOLR-15094: Replace all code references of coreNodeName to replicaName

[GitBox]

megancarey commented on a change in pull request #2234:
URL: https://github.com/apache/lucene-solr/pull/2234#discussion_r566948054



##
File path: solr/core/src/java/org/apache/solr/cloud/CloudDescriptor.java
##
@@ -64,7 +66,7 @@ public CloudDescriptor(CoreDescriptor cd, String coreName, 
Properties props) {
 // If no collection name is specified, we default to the core name
 this.collectionName = props.getProperty(CoreDescriptor.CORE_COLLECTION, 
coreName);
 this.roles = props.getProperty(CoreDescriptor.CORE_ROLES, null);
-this.nodeName = props.getProperty(CoreDescriptor.CORE_NODE_NAME);
+this.nodeName = props.getProperty(REPLICA_NAME);

Review comment:
   Can we rename `nodeName` internally here as well? I think it's a little 
misleading

##
File path: solr/core/src/java/org/apache/solr/cloud/ZkController.java
##
@@ -1438,14 +1438,14 @@ private void joinElection(CoreDescriptor cd, boolean 
afterExpiration, boolean jo
 // we only put a subset of props into the leader node
 props.put(ZkStateReader.CORE_NAME_PROP, cd.getName());
 props.put(ZkStateReader.NODE_NAME_PROP, getNodeName());
-props.put(ZkStateReader.CORE_NODE_NAME_PROP, coreNodeName);
+props.put(ZkStateReader.CORE_NODE_NAME_PROP, replicaName);

Review comment:
   Why is ZkStateReader.CORE_NODE_NAME_PROP staying?

##
File path: 
solr/solrj/src/test/org/apache/solr/client/solrj/impl/CloudSolrClientTest.java
##
@@ -880,7 +880,7 @@ public void testRetryUpdatesWhenClusterStateIsStale() 
throws Exception {
  .process(cluster.getSolrClient()).getStatus());
 cluster.waitForActiveCollection(COL, 1, 1);
 
-// determine the coreNodeName of only current replica
+// determine the replicaName of only current replica

Review comment:
   Nit: Rename variable on 888

##
File path: solr/core/src/java/org/apache/solr/cloud/ElectionContext.java
##
@@ -35,10 +35,10 @@
   volatile String leaderSeqPath;
   private SolrZkClient zkClient;
 
-  public ElectionContext(final String coreNodeName,
+  public ElectionContext(final String relicaName,

Review comment:
   Typo :) missing p

##
File path: solr/core/src/java/org/apache/solr/cloud/ZkController.java
##
@@ -1438,14 +1438,14 @@ private void joinElection(CoreDescriptor cd, boolean 
afterExpiration, boolean jo
 // we only put a subset of props into the leader node
 props.put(ZkStateReader.CORE_NAME_PROP, cd.getName());
 props.put(ZkStateReader.NODE_NAME_PROP, getNodeName());
-props.put(ZkStateReader.CORE_NODE_NAME_PROP, coreNodeName);
+props.put(ZkStateReader.CORE_NODE_NAME_PROP, replicaName);

Review comment:
   On line 1548 we use REPLICA_NAME instead - can we replace everywhere?

##
File path: solr/core/src/java/org/apache/solr/cloud/CloudUtil.java
##
@@ -56,7 +56,7 @@
   public static final int DEFAULT_TIMEOUT = 90;
 
   /**
-   * See if coreNodeName has been taken over by another baseUrl and unload core
+   * See if replicaName has been taken over by another baseUrl and unload core

Review comment:
   Update the log on line 80, and the variable name "cnn" as well

##
File path: 
solr/core/src/test/org/apache/solr/cloud/AssignBackwardCompatibilityTest.java
##
@@ -97,9 +97,9 @@ public void test() throws IOException, SolrServerException, 
KeeperException, Int
 Replica newReplica = 
getCollectionState(COLLECTION).getReplicas().stream()
 .filter(r -> r.getCoreName().equals(coreName))
 .findAny().get();
-String coreNodeName = newReplica.getName();
-assertFalse("Core node name is not unique", 
coreNodeNames.contains(coreName));
-coreNodeNames.add(coreNodeName);
+String replicaName = newReplica.getName();
+assertFalse("Core node name is not unique", 
replicaNames.contains(coreName));

Review comment:
   Update assert message to say "Replica name"

##
File path: solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java
##
@@ -847,7 +847,7 @@ final private void sendPrepRecoveryCmd(String 
leaderBaseUrl, String leaderCoreNa
 WaitForState prepCmd = new WaitForState();
 prepCmd.setCoreName(leaderCoreName);
 prepCmd.setNodeName(zkController.getNodeName());
-prepCmd.setCoreNodeName(coreZkNodeName);
+prepCmd.setCoreNodeName(replicaName);

Review comment:
   Use new method

##
File path: solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java
##
@@ -137,7 +137,7 @@ protected RecoveryStrategy(CoreContainer cc, CoreDescriptor 
cd, RecoveryListener
 zkController = cc.getZkController();
 zkStateReader = zkController.getZkStateReader();
 baseUrl = zkController.getBaseUrl();
-coreZkNodeName = cd.getCloudDescriptor().getCoreNodeName();
+replicaName = cd.getCloudDescriptor().getCoreNodeName();

Review comment:
   Update so we're not using deprecated metho

[jira] [Commented] (SOLR-15085) EmbeddedSolrServer calls shutdown on a provided CoreContainer

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274589#comment-17274589
 ] 

ASF subversion and git services commented on SOLR-15085:


Commit 2b8759383141ef96d881762c2e5e8bf6660c6047 in lucene-solr's branch 
refs/heads/branch_8x from Tim Owen
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2b87593 ]

SOLR-15085 Prevent EmbeddedSolrServer calling shutdown on a CoreContainer that 
was passed to it


> EmbeddedSolrServer calls shutdown on a provided CoreContainer
> -
>
> Key: SOLR-15085
> URL: https://issues.apache.org/jira/browse/SOLR-15085
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server, SolrJ
>Affects Versions: master (9.0)
>Reporter: Tim Owen
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> There are essentially 2 ways to create an EmbeddedSolrServer object, one by 
> passing in a CoreContainer object, and the other way creates one internally 
> on-the-fly. The current behaviour of the close method calls shutdown on the 
> CoreContainer, regardless of where it came from.
> I believe this is not good behaviour for a class that doesn't control the 
> lifecycle of the passed-in CoreContainer. In fact, there are 4 cases among 
> the codebase where a subclass of EmbeddedSolrServer is created just to 
> override this behaviour (with a comment saying it's unwanted).
> In my use-case I create EmbeddedSolrServer instances for cores as and when I 
> need to work with them, but the CoreContainer exists for the duration. I 
> don't want the whole container shut down when I'm done with just one of its 
> cores. You can workaround it by just not calling close on the 
> EmbeddedSolrServer object, but that's risky especially if you use a 
> try-with-resources as close is called automatically then.
> Fix is to keep track of whether the CoreContainer was created internally or 
> not, and only shut it down if internal. I will attach my patch PR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-15085) EmbeddedSolrServer calls shutdown on a provided CoreContainer

[Mike Drob (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-15085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved SOLR-15085.
--
Fix Version/s: 8.9
   master (9.0)
 Assignee: Mike Drob
   Resolution: Fixed

Thanks for the contribution, Tim! I added a line to CHANGES and upgrade notes 
about this as well.

> EmbeddedSolrServer calls shutdown on a provided CoreContainer
> -
>
> Key: SOLR-15085
> URL: https://issues.apache.org/jira/browse/SOLR-15085
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server, SolrJ
>Affects Versions: master (9.0)
>Reporter: Tim Owen
>Assignee: Mike Drob
>Priority: Minor
>  Labels: pull-request-available
> Fix For: master (9.0), 8.9
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> There are essentially 2 ways to create an EmbeddedSolrServer object, one by 
> passing in a CoreContainer object, and the other way creates one internally 
> on-the-fly. The current behaviour of the close method calls shutdown on the 
> CoreContainer, regardless of where it came from.
> I believe this is not good behaviour for a class that doesn't control the 
> lifecycle of the passed-in CoreContainer. In fact, there are 4 cases among 
> the codebase where a subclass of EmbeddedSolrServer is created just to 
> override this behaviour (with a comment saying it's unwanted).
> In my use-case I create EmbeddedSolrServer instances for cores as and when I 
> need to work with them, but the CoreContainer exists for the duration. I 
> don't want the whole container shut down when I'm done with just one of its 
> cores. You can workaround it by just not calling close on the 
> EmbeddedSolrServer object, but that's risky especially if you use a 
> try-with-resources as close is called automatically then.
> Fix is to keep track of whether the CoreContainer was created internally or 
> not, and only shut it down if internal. I will attach my patch PR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15085) EmbeddedSolrServer calls shutdown on a provided CoreContainer

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274588#comment-17274588
 ] 

ASF subversion and git services commented on SOLR-15085:


Commit 715caaae52ab3ab548d37e18a173b7d744054257 in lucene-solr's branch 
refs/heads/master from Tim Owen
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=715caaa ]

SOLR-15085 Prevent EmbeddedSolrServer calling shutdown on a CoreContainer that 
was passed to it


> EmbeddedSolrServer calls shutdown on a provided CoreContainer
> -
>
> Key: SOLR-15085
> URL: https://issues.apache.org/jira/browse/SOLR-15085
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server, SolrJ
>Affects Versions: master (9.0)
>Reporter: Tim Owen
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> There are essentially 2 ways to create an EmbeddedSolrServer object, one by 
> passing in a CoreContainer object, and the other way creates one internally 
> on-the-fly. The current behaviour of the close method calls shutdown on the 
> CoreContainer, regardless of where it came from.
> I believe this is not good behaviour for a class that doesn't control the 
> lifecycle of the passed-in CoreContainer. In fact, there are 4 cases among 
> the codebase where a subclass of EmbeddedSolrServer is created just to 
> override this behaviour (with a comment saying it's unwanted).
> In my use-case I create EmbeddedSolrServer instances for cores as and when I 
> need to work with them, but the CoreContainer exists for the duration. I 
> don't want the whole container shut down when I'm done with just one of its 
> cores. You can workaround it by just not calling close on the 
> EmbeddedSolrServer object, but that's risky especially if you use a 
> try-with-resources as close is called automatically then.
> Fix is to keep track of whether the CoreContainer was created internally or 
> not, and only shut it down if internal. I will attach my patch PR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] madrob closed pull request #2216: SOLR-15085 Prevent EmbeddedSolrServer calling shutdown on a CoreConta…

[GitBox]

madrob closed pull request #2216:
URL: https://github.com/apache/lucene-solr/pull/2216


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] madrob commented on pull request #2216: SOLR-15085 Prevent EmbeddedSolrServer calling shutdown on a CoreConta…

[GitBox]

madrob commented on pull request #2216:
URL: https://github.com/apache/lucene-solr/pull/2216#issuecomment-769935182


   Added upgrade notes, pushed to branch_8x and master! Thanks!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8319) NPE when creating pivot

[Isabelle Giguere (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274551#comment-17274551
 ] 

Isabelle Giguere edited comment on SOLR-8319 at 1/29/21, 5:00 PM:
--

The PR (937) in the linked issue SOLR-13209 suggest throwing a SolrException 
from SolrIndexSearcher.getDocSet(Query), which is certainly more elegant than 
an NPE.

I applied PR 937, on Solr 8.7.0, to see if I would get a SolrException for the 
facet request. I which case, I would have caught the exception, somewhere in 
the fact pivot processing (we shouldn't send an error message as a response for 
a valid facet pivot query).

But my facet pivot request still ends with NPE. The request goes through 
SolrIndexSearcher.numDocs(Query, DocSet) and 
SolrIndexSearcher.getPositiveDocSet(Query) :
{noformat}
Error from server at null: java.lang.NullPointerException at 
java.base/java.util.concurrent.ConcurrentHashMap.get(Unknown Source) at 
org.apache.solr.util.ConcurrentLFUCache.get(ConcurrentLFUCache.java:163) at 
org.apache.solr.search.LFUCache.get(LFUCache.java:198) at 
org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:846)
 at 
org.apache.solr.search.SolrIndexSearcher.numDocs(SolrIndexSearcher.java:2069) 
at 
org.apache.solr.handler.component.PivotFacetProcessor.getSubsetSize(PivotFacetProcessor.java:355)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:218)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:166)
 at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:280)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:360)
 at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214)
 at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415)
 at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
{noformat}
The fact is, in a facet pivot request, the user has no actual control on the 
Query that ends up in methods of SolrIndexSearcher. The queries are generated 
from the facet field content. Checking for null, as in the attached patch, may 
be the only thing we can do.

It is strange, however, that setting a value for facet.limit will return 
results... or an NPE, depending on the value. 
>From my experiments, a facet.limit value of -1 returns all results, other 
>values return partial results, but only up to a point, after that, it's NPE.
facet.limit value: 
-1 = all results
0 = no results
Other value = partial results OR NPE
1 = 1 result
2 = 2 results
3 = 3 results
4 = NPE

So... Whatever happens when facet.limit=-1 is handled properly, but for values 
greater than (what ?) are not.
Default value for facet.limit is 100.  So, it's not even handling it's own 
default correctly.



was (Author: igiguere):
The PR (937) in the linked issue SOLR-13209 suggest throwing a SolrException 
from SolrIndexSearcher.getDocSet(Query), which is certainly more elegant than 
an NPE.

I applied PR 937, on Solr 8.7.0, to see if I would get a SolrException for the 
facet request. I which case, I would have caught the exception, somewhere in 
the fact pivot processing (we shouldn't send an error message as a response for 
a valid facet pivot query).

But my facet pivot request still ends with NPE. The request goes through 
SolrIndexSearcher.numDocs(Query, DocSet) and 
SolrIndexSearcher.getPositiveDocSet(Query) :
{noformat}
Error from server at null: java.lang.NullPointerException at 
java.base/java.util.concurrent.ConcurrentHashMap.get(Unknown Source) at 
org.apache.solr.util.ConcurrentLFUCache.get(ConcurrentLFUCache.java:163) at 
org.apache.solr.search.LFUCache.get(LFUCache.java:198) at 
org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:846)
 at 
org.apache.solr.search.SolrIndexSearcher.numDocs(SolrIndexSearcher.java:2069) 
at 
org.apache.solr.handler.component.PivotFacetProcessor.getSubsetSize(PivotFacetProcessor.java:355)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:218)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:166)
 at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:280)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:360)
 at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214)
 at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) at 

[jira] [Resolved] (SOLR-15120) Reduce duplicative core creation work

[Mike Drob (Jira)]

 [ 
https://issues.apache.org/jira/browse/SOLR-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved SOLR-15120.
--
Fix Version/s: 8.9
   master (9.0)
 Assignee: Mike Drob
   Resolution: Fixed

> Reduce duplicative core creation work
> -
>
> Key: SOLR-15120
> URL: https://issues.apache.org/jira/browse/SOLR-15120
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mike Drob
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0), 8.9
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> While investigating collection creation performance, I've found some 
> duplicate work that we do for core creation, we can improve that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-15120) Reduce duplicative core creation work

[ASF subversion and git services (Jira)]

[ 
https://issues.apache.org/jira/browse/SOLR-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274552#comment-17274552
 ] 

ASF subversion and git services commented on SOLR-15120:


Commit 5a95d3f2942b65033b95f3e1806d48f3dc68f05f in lucene-solr's branch 
refs/heads/branch_8x from Mike Drob
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5a95d3f ]

SOLR-15120 Reduce duplicated core creation work

Use j.u.c collections instead of sync block
Rework how we load implicit handlers
Additional debug and trace logging for zookeeper comms


> Reduce duplicative core creation work
> -
>
> Key: SOLR-15120
> URL: https://issues.apache.org/jira/browse/SOLR-15120
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mike Drob
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> While investigating collection creation performance, I've found some 
> duplicate work that we do for core creation, we can improve that.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org