[jira] [Commented] (SOLR-13649) BasicAuth's 'blockUnknown' param should default to true

[Marcus Eagan (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905822#comment-16905822
 ] 

Marcus Eagan commented on SOLR-13649:
-

I've added an error in case the blockUnknown parameter is not set to make it 
easier for the community to adopt this change upon upgrading.

> BasicAuth's 'blockUnknown' param should default to true
> ---
>
> Key: SOLR-13649
> URL: https://issues.apache.org/jira/browse/SOLR-13649
> Project: Solr
>  Issue Type: Improvement
>  Components: Admin UI, Authentication, security
>Affects Versions: 7.7.2, 8.1.1
> Environment: All
>Reporter: Marcus Eagan
>Priority: Major
>  Labels: Authentication
> Fix For: master (9.0)
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> If someone seeks to enable basic authentication but they do not specify the 
> {{blockUnknown}} parameter, the default value is {{false}}. That default 
> behavior is a bit counterintuitive because if someone wishes to enable basic 
> authentication, you would expect that they would want all unknown users to 
> need to authenticate by default. I can imagine cases where you would not, but 
> those cases would be less frequent.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] MarcusSorealheis commented on issue #805: SOLR-13649 change the default behavior of the basic authentication plugin.

[GitBox]

MarcusSorealheis commented on issue #805: SOLR-13649 change the default 
behavior of the basic authentication plugin.
URL: https://github.com/apache/lucene-solr/pull/805#issuecomment-520690325
 
 
   
https://github.com/apache/lucene-solr/pull/805/files#diff-38fe7b489da085cdd2a483bd5a185aa4R82


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] MarcusSorealheis commented on issue #805: SOLR-13649 change the default behavior of the basic authentication plugin.

[GitBox]

MarcusSorealheis commented on issue #805: SOLR-13649 change the default 
behavior of the basic authentication plugin.
URL: https://github.com/apache/lucene-solr/pull/805#issuecomment-520690240
 
 
   @janhoy In the PRs most recent state, you can see that I am now throwing a 
`SolrException` in the case when the `blockUnknown` parameter is not set. This 
change will make upgrading less painful for the community.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-13457) Managing Timeout values in Solr

[Anindita Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905806#comment-16905806
 ] 

Anindita Gupta edited comment on SOLR-13457 at 8/13/19 4:35 AM:


In FacetStream.java, open() method Socket timeout 30 seconds & connection 
timeout 15 seconds has been added in Version #7.7 onward which is hard-coded 
and not configurable. 
 These numbers are very low when dealing with large number of documents for 
complex Streaming facet query and if we try to retrieve large number of records 
or starting Offset value is high, Timeout exception is occurring while waiting 
response from server.


was (Author: aninditagupta):
In FacetStream.java, open() method Socket timeout 30 seconds & connection 
timeout 15 seconds has been added in Version #7.7 onward which is hard-coded 
and not configurable. 
These numbers are very low when dealing with large number of documents for 
complex Streaming facet query and if we try to retrieve large number of records 
or starting Offset value is high, Timeout exception is occurring while waiting 
response from server.

How can we deal with this issue?

> Managing Timeout values in Solr
> ---
>
> Key: SOLR-13457
> URL: https://issues.apache.org/jira/browse/SOLR-13457
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Gus Heck
>Priority: Major
>
> Presently, Solr has a variety of timeouts for various connections or 
> operations. These timeouts have been added, tweaked and refined and in some 
> cases made configurable in an ad-hoc manner by the contributors of individual 
> features. Throughout the history of the project. This is all well and good 
> until one experiences a timeout during an otherwise valid use case and needs 
> to adjust it.
> This has also made managing timeouts in unit tests "interesting" as noted in 
> SOLR-13389.
> Probably nobody has the spare time to do a tour de force through the code and 
> coordinate every single timeout, so in this ticket I'd like to establish a 
> framework for categorizing time outs, a standard for how we make each 
> category configurable, and then add sub-tickets to address individual 
> timeouts.
> The intention is that eventually, there will be no "magic number" timeout 
> values in code, and one can predict where to find the configuration for a 
> timeout by determining it's category.
> Initial strawman categories (feel free to knock down or suggest alternatives):
>  # *Feature-Instance Timeout*: Timeouts that relate to a particular 
> instantiation of a feature, for example a database connection timeout for a 
> connection to a particular database by DIH. These should be set in the 
> configuration of that instance.
>  # *Optional Feature Timeout*: A timeout that only has meaning in the context 
> of a particular feature that is not required for solr to function... i.e. 
> something that can be turned on or off. Perhaps a timeout for communication 
> with an external ldap for authentication purposes. These should be configured 
> in the same configuration that enables this feature.
>  # *Global System Timeout*: A timeout that will always be an active part of 
> Solr these should be configured in a new  section of solr.xml. For 
> example the Jetty thread idle timeout, or the default timeout for http calls 
> between nodes.
>  # *Node Specific Timeout*: A timeout which may differ on different nodes. I 
> don't know of any of these, but I'll grant the possibility. These (and only 
> these) should be set by setting system properties. If we don't have any of 
> these, that's just fine :).
>  # *Client Timeout*: These are timeouts in solrj code that are active in code 
> running outside the server. They should be configurable via java api, and via 
> a config file of some sort from a single location defined in a sysprop or 
> sourced from classpath (in that order). When run on the server, the solrj 
> code should look for a *Global System Timeout* setting before consulting 
> sysprops or classpath.
> *Note that in no case is a hard-coded value the correct solution.*
> If we get a consensus on categories and their locations, then the next step 
> is to begin adding sub tickets to bring specific timeouts into compliance. 
> Every such ticket should include an update to the section of the ref guide 
> documenting the configuration to which the timeout has been added (e.g. docs 
> for solr.xml for Global System Timeouts) describing what exactly is affected 
> by the timeout, the maximum allowed value and how zero and negative numbers 
> are handled.
> It is of course true that some of these values will have the potential to 
> destroy system performance or integrity, and that should be mentioned in the 
> update to documentation.



--
This message was sent by 

[jira] [Commented] (SOLR-13457) Managing Timeout values in Solr

[Anindita Gupta (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905806#comment-16905806
 ] 

Anindita Gupta commented on SOLR-13457:
---

In FacetStream.java, open() method Socket timeout 30 seconds & connection 
timeout 15 seconds has been added in Version #7.7 onward which is hard-coded 
and not configurable. 
These numbers are very low when dealing with large number of documents for 
complex Streaming facet query and if we try to retrieve large number of records 
or starting Offset value is high, Timeout exception is occurring while waiting 
response from server.

How can we deal with this issue?

> Managing Timeout values in Solr
> ---
>
> Key: SOLR-13457
> URL: https://issues.apache.org/jira/browse/SOLR-13457
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: master (9.0)
>Reporter: Gus Heck
>Priority: Major
>
> Presently, Solr has a variety of timeouts for various connections or 
> operations. These timeouts have been added, tweaked and refined and in some 
> cases made configurable in an ad-hoc manner by the contributors of individual 
> features. Throughout the history of the project. This is all well and good 
> until one experiences a timeout during an otherwise valid use case and needs 
> to adjust it.
> This has also made managing timeouts in unit tests "interesting" as noted in 
> SOLR-13389.
> Probably nobody has the spare time to do a tour de force through the code and 
> coordinate every single timeout, so in this ticket I'd like to establish a 
> framework for categorizing time outs, a standard for how we make each 
> category configurable, and then add sub-tickets to address individual 
> timeouts.
> The intention is that eventually, there will be no "magic number" timeout 
> values in code, and one can predict where to find the configuration for a 
> timeout by determining it's category.
> Initial strawman categories (feel free to knock down or suggest alternatives):
>  # *Feature-Instance Timeout*: Timeouts that relate to a particular 
> instantiation of a feature, for example a database connection timeout for a 
> connection to a particular database by DIH. These should be set in the 
> configuration of that instance.
>  # *Optional Feature Timeout*: A timeout that only has meaning in the context 
> of a particular feature that is not required for solr to function... i.e. 
> something that can be turned on or off. Perhaps a timeout for communication 
> with an external ldap for authentication purposes. These should be configured 
> in the same configuration that enables this feature.
>  # *Global System Timeout*: A timeout that will always be an active part of 
> Solr these should be configured in a new  section of solr.xml. For 
> example the Jetty thread idle timeout, or the default timeout for http calls 
> between nodes.
>  # *Node Specific Timeout*: A timeout which may differ on different nodes. I 
> don't know of any of these, but I'll grant the possibility. These (and only 
> these) should be set by setting system properties. If we don't have any of 
> these, that's just fine :).
>  # *Client Timeout*: These are timeouts in solrj code that are active in code 
> running outside the server. They should be configurable via java api, and via 
> a config file of some sort from a single location defined in a sysprop or 
> sourced from classpath (in that order). When run on the server, the solrj 
> code should look for a *Global System Timeout* setting before consulting 
> sysprops or classpath.
> *Note that in no case is a hard-coded value the correct solution.*
> If we get a consensus on categories and their locations, then the next step 
> is to begin adding sub tickets to bring specific timeouts into compliance. 
> Every such ticket should include an update to the section of the ref guide 
> documenting the configuration to which the timeout has been added (e.g. docs 
> for solr.xml for Global System Timeouts) describing what exactly is affected 
> by the timeout, the maximum allowed value and how zero and negative numbers 
> are handled.
> It is of course true that some of these values will have the potential to 
> destroy system performance or integrity, and that should be mentioned in the 
> update to documentation.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk-13-rc) - Build # 1006 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/1006/
Java: 64bit/jdk-13-rc -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testClusterStateProviderAPI

Error Message:
[testClusterStateProviderAPI] expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: [testClusterStateProviderAPI] expected:<2> but was:<1>
at 
__randomizedtesting.SeedInfo.seed([E7C0B7EB161A5803:F8172BC76511A148]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at 
org.apache.solr.cloud.AliasIntegrationTest.testClusterStateProviderAPI(AliasIntegrationTest.java:299)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:830)




Build Log:
[...truncated 2089 lines...]
   [junit4] JVM J0: stderr was not empty, see: 

[JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk1.8.0) - Build # 269 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/269/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Timeout occurred while waiting response from server at: 
http://127.0.0.1:63693/_mb

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occurred while 
waiting response from server at: http://127.0.0.1:63693/_mb
at 
__randomizedtesting.SeedInfo.seed([3EB8B218F1E1165B:B6EC8DC25F1D7BA3]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:667)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:245)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1128)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:897)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:829)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:228)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:338)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1080)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)

[jira] [Commented] (SOLR-13464) no way for external clients to detect when changes to security config have taken effect

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905705#comment-16905705
 ] 

Shawn Heisey commented on SOLR-13464:
-

I'm thinking maybe if the security.json had a place for a serial number, 
similar to DNS, then the auth API could indicate the serial number of config 
that has been fully initialized.   When the config is changed via the API, 
advance the serial number, and recommend in the docs that users do the same 
when making changes themselves.

Or is that just ugly?


> no way for external clients to detect when changes to security config have 
> taken effect
> ---
>
> Key: SOLR-13464
> URL: https://issues.apache.org/jira/browse/SOLR-13464
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Priority: Major
>
> The basic functionality of the authorization/authentication REST APIs works 
> by persisting changes to a {{security.json}} file in ZooKeeper which is 
> monitored by every node via a Watcher.  When the watchers fire, the affected 
> plugin types are (re)-initialized ith the new settings.
> Since this information is "pulled" from ZK by the nodes, there is a (small) 
> inherent delay between when the REST API is hit by external clients, and when 
> each node learns of the changes.  An additional delay exists as the config is 
> "reloaded" to (re)initialize the plugins.
> Practically speaking these delays have very little impact on a "real" solr 
> cloud cluster, but they can be problematic in test cases -- while the 
> SecurityConfHandler on each node could be used to query the "current" 
> security.json file, it doesn't indicate if/when the plugins identified in the 
> "current" configuration are fully in use.
> For now, we have a "white box" work around available for MiniSolrCloudCluster 
> based tests by comparing the Plugins of each CoreContainer in use before and 
> after making known changes via the API (see commits identified below).
> This issue exists as a placeholder for future consideration of UX/API 
> improvements making it easier for external clients (w/o "white box" access to 
> solr internals) to know definitively if/when modified security settings take 
> effect.
> {panel:title=original jira description}
> I've been investigating some sporadic and hard to reproduce test failures 
> related to authentication in cloud mode, and i *think* (but have not directly 
> verified) that the common cause is that after uses one of the 
> {{/admin/auth...}} handlers to update some setting, there is an inherient and 
> unpredictible delay (due to ZK watches) until every node in the cluster has 
> had a chance to (re)load the new configuration and initialize the various 
> security plugins with the new settings.
> Which means, if a test client does a POST to some node to add/change/remove 
> some authn/authz settings, and then immediately hits the exact same node (or 
> any other node) to test that the effects of those settings exist, there is no 
> garuntee that they will have taken affect yet.
> {panel}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-13464) no way for external clients to detect when changes to security config have taken effect

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-13464:

Description: 
The basic functionality of the authorization/authentication REST APIs works by 
persisting changes to a {{security.json}} file in ZooKeeper which is monitored 
by every node via a Watcher.  When the watchers fire, the affected plugin types 
are (re)-initialized ith the new settings.
Since this information is "pulled" from ZK by the nodes, there is a (small) 
inherent delay between when the REST API is hit by external clients, and when 
each node learns of the changes.  An additional delay exists as the config is 
"reloaded" to (re)initialize the plugins.

Practically speaking these delays have very little impact on a "real" solr 
cloud cluster, but they can be problematic in test cases -- while the 
SecurityConfHandler on each node could be used to query the "current" 
security.json file, it doesn't indicate if/when the plugins identified in the 
"current" configuration are fully in use.

For now, we have a "white box" work around available for MiniSolrCloudCluster 
based tests by comparing the Plugins of each CoreContainer in use before and 
after making known changes via the API (see commits identified below).

This issue exists as a placeholder for future consideration of UX/API 
improvements making it easier for external clients (w/o "white box" access to 
solr internals) to know definitively if/when modified security settings take 
effect.


{panel:title=original jira description}
I've been investigating some sporadic and hard to reproduce test failures 
related to authentication in cloud mode, and i *think* (but have not directly 
verified) that the common cause is that after uses one of the 
{{/admin/auth...}} handlers to update some setting, there is an inherient and 
unpredictible delay (due to ZK watches) until every node in the cluster has had 
a chance to (re)load the new configuration and initialize the various security 
plugins with the new settings.

Which means, if a test client does a POST to some node to add/change/remove 
some authn/authz settings, and then immediately hits the exact same node (or 
any other node) to test that the effects of those settings exist, there is no 
garuntee that they will have taken affect yet.
{panel}

  was:
I've been investigating some sporadic and hard to reproduce test failures 
related to authentication in cloud mode, and i *think* (but have not directly 
verified) that the common cause is that after uses one of the 
{{/admin/auth...}} handlers to update some setting, there is an inherient and 
unpredictible delay (due to ZK watches) until every node in the cluster has had 
a chance to (re)load the new configuration and initialize the various security 
plugins with the new settings.

Which means, if a test client does a POST to some node to add/change/remove 
some authn/authz settings, and then immediately hits the exact same node (or 
any other node) to test that the effects of those settings exist, there is no 
garuntee that they will have taken affect yet.


 Issue Type: Improvement  (was: Bug)
Summary: no way for external clients to detect when changes to security 
config have taken effect  (was: Sporadic Auth + Cloud test failures, probably 
due to lag in nodes reloading security config)

since i was able to come up with a test workaround, i've shifted the type, 
summary, and description of this Jira to focus on future UX/API improvements 
for external clients

> no way for external clients to detect when changes to security config have 
> taken effect
> ---
>
> Key: SOLR-13464
> URL: https://issues.apache.org/jira/browse/SOLR-13464
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Priority: Major
>
> The basic functionality of the authorization/authentication REST APIs works 
> by persisting changes to a {{security.json}} file in ZooKeeper which is 
> monitored by every node via a Watcher.  When the watchers fire, the affected 
> plugin types are (re)-initialized ith the new settings.
> Since this information is "pulled" from ZK by the nodes, there is a (small) 
> inherent delay between when the REST API is hit by external clients, and when 
> each node learns of the changes.  An additional delay exists as the config is 
> "reloaded" to (re)initialize the plugins.
> Practically speaking these delays have very little impact on a "real" solr 
> cloud cluster, but they can be problematic in test cases -- while the 
> SecurityConfHandler on each node could be used to query the "current" 
> security.json file, it doesn't indicate if/when the plugins identified in the 
> "current" configuration are fully in use.
> For now, we have a "white box" work around available for 

[jira] [Commented] (SOLR-13464) Sporadic Auth + Cloud test failures, probably due to lag in nodes reloading security config

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905679#comment-16905679
 ] 

ASF subversion and git services commented on SOLR-13464:


Commit 06dd37e907df860125dbcaeb7a90ce5d7de684a1 in lucene-solr's branch 
refs/heads/branch_8x from Chris M. Hostetter
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=06dd37e ]

SOLR-13464: Test work arounds

 * Refactor existing work around in BasicAuthIntegrationTest up into 
SolrCloudAuthTestCase for re-use in JWTAuthPluginIntegrationTest

 * Simplify BasicAuthOnSingleNodeTest and PKIAuthenticationIntegrationTest to 
use their existing (static) security settings on creation of MiniSolrCloud.  
Since they no longer modify security.json once the nodes are alive, the issue 
no longer affects them

(cherry picked from commit c7822c393e6affa77c233f9e8e9bf9d8aeb12578)
(cherry picked from commit 0291db44bc8e092f7cb2f577f0ac8ab6fa6a5fd7)


> Sporadic Auth + Cloud test failures, probably due to lag in nodes reloading 
> security config
> ---
>
> Key: SOLR-13464
> URL: https://issues.apache.org/jira/browse/SOLR-13464
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Priority: Major
>
> I've been investigating some sporadic and hard to reproduce test failures 
> related to authentication in cloud mode, and i *think* (but have not directly 
> verified) that the common cause is that after uses one of the 
> {{/admin/auth...}} handlers to update some setting, there is an inherient and 
> unpredictible delay (due to ZK watches) until every node in the cluster has 
> had a chance to (re)load the new configuration and initialize the various 
> security plugins with the new settings.
> Which means, if a test client does a POST to some node to add/change/remove 
> some authn/authz settings, and then immediately hits the exact same node (or 
> any other node) to test that the effects of those settings exist, there is no 
> garuntee that they will have taken affect yet.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-8.x-Windows (64bit/jdk-13-rc) - Build # 395 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/395/
Java: 64bit/jdk-13-rc -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.client.solrj.io.stream.MathExpressionTest.testZipFDistribution

Error Message:
Zipf distribution not descending!!!

Stack Trace:
java.lang.Exception: Zipf distribution not descending!!!
at 
__randomizedtesting.SeedInfo.seed([A7971168B8765E1A:83227C4CAFDE5632]:0)
at 
org.apache.solr.client.solrj.io.stream.MathExpressionTest.testZipFDistribution(MathExpressionTest.java:3485)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:830)




Build Log:
[...truncated 17190 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.io.stream.MathExpressionTest
   [junit4]   2> 395077 INFO  
(SUITE-MathExpressionTest-seed#[A7971168B8765E1A]-worker) [ ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 395077 INFO  

[JENKINS] Lucene-Solr-repro-Java11 - Build # 314 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro-Java11/314/

[...truncated 28 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/443/consoleText

[repro] Revision: d468d71c030dba77fb7e6bed295e5766248a5944

[repro] Repro line:  ant test  -Dtestcase=TestCloudJSONFacetSKG 
-Dtests.method=testRandom -Dtests.seed=BFA12190BA347B6C -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=zh-Hans-MO 
-Dtests.timezone=Etc/GMT+4 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
768ca7c5a7fe5fcd89f047275319eafe783eb96c
[repro] git fetch

[...truncated 7 lines...]
[repro] git checkout d468d71c030dba77fb7e6bed295e5766248a5944

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   TestCloudJSONFacetSKG
[repro] ant compile-test

[...truncated 3315 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestCloudJSONFacetSKG" -Dtests.showOutput=onerror  
-Dtests.seed=BFA12190BA347B6C -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=zh-Hans-MO -Dtests.timezone=Etc/GMT+4 
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[...truncated 123543 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   2/5 failed: org.apache.solr.search.facet.TestCloudJSONFacetSKG
[repro] git checkout 768ca7c5a7fe5fcd89f047275319eafe783eb96c

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13464) Sporadic Auth + Cloud test failures, probably due to lag in nodes reloading security config

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905618#comment-16905618
 ] 

ASF subversion and git services commented on SOLR-13464:


Commit 0291db44bc8e092f7cb2f577f0ac8ab6fa6a5fd7 in lucene-solr's branch 
refs/heads/master from Chris M. Hostetter
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=0291db4 ]

SOLR-13464: fix javadoc typo that precommit somehow missed?


> Sporadic Auth + Cloud test failures, probably due to lag in nodes reloading 
> security config
> ---
>
> Key: SOLR-13464
> URL: https://issues.apache.org/jira/browse/SOLR-13464
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Priority: Major
>
> I've been investigating some sporadic and hard to reproduce test failures 
> related to authentication in cloud mode, and i *think* (but have not directly 
> verified) that the common cause is that after uses one of the 
> {{/admin/auth...}} handlers to update some setting, there is an inherient and 
> unpredictible delay (due to ZK watches) until every node in the cluster has 
> had a chance to (re)load the new configuration and initialize the various 
> security plugins with the new settings.
> Which means, if a test client does a POST to some node to add/change/remove 
> some authn/authz settings, and then immediately hits the exact same node (or 
> any other node) to test that the effects of those settings exist, there is no 
> garuntee that they will have taken affect yet.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 179 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/179/

No tests ran.

Build Log:
[...truncated 25 lines...]
ERROR: Failed to check out http://svn.apache.org/repos/asf/lucene/test-data
org.tmatesoft.svn.core.SVNException: svn: E175002: connection refused by the 
server
svn: E175002: OPTIONS request failed on '/repos/asf/lucene/test-data'
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:112)
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:96)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:765)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:352)
at 
org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:340)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVConnection.performHttpRequest(DAVConnection.java:910)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVConnection.exchangeCapabilities(DAVConnection.java:702)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVConnection.open(DAVConnection.java:113)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVRepository.openConnection(DAVRepository.java:1035)
at 
org.tmatesoft.svn.core.internal.io.dav.DAVRepository.getLatestRevision(DAVRepository.java:164)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.getRevisionNumber(SvnNgRepositoryAccess.java:119)
at 
org.tmatesoft.svn.core.internal.wc2.SvnRepositoryAccess.getLocations(SvnRepositoryAccess.java:178)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgRepositoryAccess.createRepositoryFor(SvnNgRepositoryAccess.java:43)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.checkout(SvnNgAbstractUpdate.java:831)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:26)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgCheckout.run(SvnNgCheckout.java:11)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at 
org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at 
org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at 
hudson.scm.subversion.CheckoutUpdater$SubversionUpdateTask.perform(CheckoutUpdater.java:133)
at 
hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at 
hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:176)
at 
hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:134)
at 
hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at 
hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at 
org.tmatesoft.svn.core.internal.util.SVNSocketConnection.run(SVNSocketConnection.java:57)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
... 4 more
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

[jira] [Commented] (SOLR-13464) Sporadic Auth + Cloud test failures, probably due to lag in nodes reloading security config

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905588#comment-16905588
 ] 

ASF subversion and git services commented on SOLR-13464:


Commit c7822c393e6affa77c233f9e8e9bf9d8aeb12578 in lucene-solr's branch 
refs/heads/master from Chris M. Hostetter
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c7822c3 ]

SOLR-13464: Test work arounds

 * Refactor existing work around in BasicAuthIntegrationTest up into 
SolrCloudAuthTestCase for re-use in JWTAuthPluginIntegrationTest

 * Simplify BasicAuthOnSingleNodeTest and PKIAuthenticationIntegrationTest to 
use their existing (static) security settings on creation of MiniSolrCloud.  
Since they no longer modify security.json once the nodes are alive, the issue 
no longer affects them


> Sporadic Auth + Cloud test failures, probably due to lag in nodes reloading 
> security config
> ---
>
> Key: SOLR-13464
> URL: https://issues.apache.org/jira/browse/SOLR-13464
> Project: Solr
>  Issue Type: Bug
>Reporter: Hoss Man
>Priority: Major
>
> I've been investigating some sporadic and hard to reproduce test failures 
> related to authentication in cloud mode, and i *think* (but have not directly 
> verified) that the common cause is that after uses one of the 
> {{/admin/auth...}} handlers to update some setting, there is an inherient and 
> unpredictible delay (due to ZK watches) until every node in the cluster has 
> had a chance to (re)load the new configuration and initialize the various 
> security plugins with the new settings.
> Which means, if a test client does a POST to some node to add/change/remove 
> some authn/authz settings, and then immediately hits the exact same node (or 
> any other node) to test that the effects of those settings exist, there is no 
> garuntee that they will have taken affect yet.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-12.0.1) - Build # 8082 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/8082/
Java: 64bit/jdk-12.0.1 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

12 tests failed.
FAILED:  
org.apache.solr.cloud.TestCloudSearcherWarming.testRepFactor1LeaderStartup

Error Message:
IOException occurred when talking to server at: https://127.0.0.1:54658/solr

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occurred when 
talking to server at: https://127.0.0.1:54658/solr
at 
__randomizedtesting.SeedInfo.seed([28055FB0A913DE7D:FF2D1268B8F39793]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:670)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:245)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1128)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:897)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:829)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:228)
at 
org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:547)
at 
org.apache.solr.cloud.TestCloudSearcherWarming.tearDown(TestCloudSearcherWarming.java:79)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
 

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 1419 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1419/

No tests ran.

Build Log:
[...truncated 24456 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: 
invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid 
part, must have at least one section (e.g., chapter, appendix, etc.)
 [java] Processed 2593 links (2119 relative) to 3411 anchors in 259 files
 [echo] Validated Links & Anchors via: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr-ref-guide/bare-bones-html/

-dist-changes:
 [copy] Copying 4 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/changes

package:

-unpack-solr-tgz:

-ensure-solr-tgz-exists:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked
[untar] Expanding: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/solr-9.0.0.tgz
 into 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked

generate-maven-artifacts:

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

[jira] [Commented] (LUCENE-8369) Remove the spatial module as it is obsolete

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905566#comment-16905566
 ] 

Michael McCandless commented on LUCENE-8369:


+1 for option 1 above.

> Remove the spatial module as it is obsolete
> ---
>
> Key: LUCENE-8369
> URL: https://issues.apache.org/jira/browse/LUCENE-8369
> Project: Lucene - Core
>  Issue Type: Task
>  Components: modules/spatial
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Attachments: LUCENE-8369.patch
>
>
> The "spatial" module is at this juncture nearly empty with only a couple 
> utilities that aren't used by anything in the entire codebase -- 
> GeoRelationUtils, and MortonEncoder.  Perhaps it should have been removed 
> earlier in LUCENE-7664 which was the removal of GeoPointField which was 
> essentially why the module existed.  Better late than never.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] yonik commented on issue #826: SOLR-13399: add splitByPrefix configuration to IndexSizeTrigger

[GitBox]

yonik commented on issue #826: SOLR-13399: add splitByPrefix configuration to 
IndexSizeTrigger
URL: https://github.com/apache/lucene-solr/pull/826#issuecomment-520581934
 
 
   OK, I implemented the change to use StringHelper.startsWith(), squashed the 
commits, added SOLR-13399 at the start of the commit message, and pushed to 
master & 8x.
   Thanks Megan!


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] yonik closed pull request #826: SOLR-13399: add splitByPrefix configuration to IndexSizeTrigger

[GitBox]

yonik closed pull request #826: SOLR-13399: add splitByPrefix configuration to 
IndexSizeTrigger
URL: https://github.com/apache/lucene-solr/pull/826
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13399) compositeId support for shard splitting

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905562#comment-16905562
 ] 

ASF subversion and git services commented on SOLR-13399:


Commit 7fc5c0e07d3b9d0651f9ad31f36eb021a9be8195 in lucene-solr's branch 
refs/heads/branch_8x from Megan Carey
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=7fc5c0e ]

SOLR-13399: Adding splitByPrefix param to IndexSizeTrigger; some splitByPrefix 
test and code cleanup


> compositeId support for shard splitting
> ---
>
> Key: SOLR-13399
> URL: https://issues.apache.org/jira/browse/SOLR-13399
> Project: Solr
>  Issue Type: New Feature
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-13399.patch, SOLR-13399.patch, 
> SOLR-13399_testfix.patch, SOLR-13399_useId.patch, 
> ShardSplitTest.master.seed_AE04B5C9BA6E9A4.log.txt
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Shard splitting does not currently have a way to automatically take into 
> account the actual distribution (number of documents) in each hash bucket 
> created by using compositeId hashing.
> We should probably add a parameter *splitByPrefix* to the *SPLITSHARD* 
> command that would look at the number of docs sharing each compositeId prefix 
> and use that to create roughly equal sized buckets by document count rather 
> than just assuming an equal distribution across the entire hash range.
> Like normal shard splitting, we should bias against splitting within hash 
> buckets unless necessary (since that leads to larger query fanout.) . Perhaps 
> this warrants a parameter that would control how much of a size mismatch is 
> tolerable before resorting to splitting within a bucket. 
> *allowedSizeDifference*?
> To more quickly calculate the number of docs in each bucket, we could index 
> the prefix in a different field.  Iterating over the terms for this field 
> would quickly give us the number of docs in each (i.e lucene keeps track of 
> the doc count for each term already.)  Perhaps the implementation could be a 
> flag on the *id* field... something like *indexPrefixes* and poly-fields that 
> would cause the indexing to be automatically done and alleviate having to 
> pass in an additional field during indexing and during the call to 
> *SPLITSHARD*.  This whole part is an optimization though and could be split 
> off into its own issue if desired.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9658) Caches should have an optional way to clean if idle for 'x' mins

[Andrzej Bialecki (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905553#comment-16905553
 ] 

Andrzej Bialecki  commented on SOLR-9658:
-

Here's an updated patch - I tried to address all your concerns:
 * removed the CacheListener API
 * refactored cache impls to allow inserting synthetic entries, and changed the 
unit tests to use these methods. It turned out that the management of 
{{oldestEntry}} needs to be improved in all caches when we allow the creation 
time in more recently added entries to go back...
 * currently unit tests test only the first part - "what happens when 
markAndSweep() is called on a cache with specific entries". I'll add the other 
part if this patch looks ok.

> Caches should have an optional way to clean if idle for 'x' mins
> 
>
> Key: SOLR-9658
> URL: https://issues.apache.org/jira/browse/SOLR-9658
> Project: Solr
>  Issue Type: New Feature
>Reporter: Noble Paul
>Assignee: Andrzej Bialecki 
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-9658.patch, SOLR-9658.patch, SOLR-9658.patch, 
> SOLR-9658.patch, SOLR-9658.patch
>
>
> If a cache is idle for long, it consumes precious memory. It should be 
> configurable to clear the cache if it was not accessed for 'x' secs. The 
> cache configuration can have an extra config {{maxIdleTime}} . if we wish it 
> to the cleaned after 10 mins of inactivity set it to {{maxIdleTime=600}}. 
> [~dragonsinth] would it be a solution for the memory leak you mentioned?



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9658) Caches should have an optional way to clean if idle for 'x' mins

[Andrzej Bialecki (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki  updated SOLR-9658:

Attachment: SOLR-9658.patch

> Caches should have an optional way to clean if idle for 'x' mins
> 
>
> Key: SOLR-9658
> URL: https://issues.apache.org/jira/browse/SOLR-9658
> Project: Solr
>  Issue Type: New Feature
>Reporter: Noble Paul
>Assignee: Andrzej Bialecki 
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-9658.patch, SOLR-9658.patch, SOLR-9658.patch, 
> SOLR-9658.patch, SOLR-9658.patch
>
>
> If a cache is idle for long, it consumes precious memory. It should be 
> configurable to clear the cache if it was not accessed for 'x' secs. The 
> cache configuration can have an extra config {{maxIdleTime}} . if we wish it 
> to the cleaned after 10 mins of inactivity set it to {{maxIdleTime=600}}. 
> [~dragonsinth] would it be a solution for the memory leak you mentioned?



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13399) compositeId support for shard splitting

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905544#comment-16905544
 ] 

ASF subversion and git services commented on SOLR-13399:


Commit 768ca7c5a7fe5fcd89f047275319eafe783eb96c in lucene-solr's branch 
refs/heads/master from Megan Carey
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=768ca7c ]

SOLR-13399: Adding splitByPrefix param to IndexSizeTrigger; some splitByPrefix 
test and code cleanup


> compositeId support for shard splitting
> ---
>
> Key: SOLR-13399
> URL: https://issues.apache.org/jira/browse/SOLR-13399
> Project: Solr
>  Issue Type: New Feature
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-13399.patch, SOLR-13399.patch, 
> SOLR-13399_testfix.patch, SOLR-13399_useId.patch, 
> ShardSplitTest.master.seed_AE04B5C9BA6E9A4.log.txt
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Shard splitting does not currently have a way to automatically take into 
> account the actual distribution (number of documents) in each hash bucket 
> created by using compositeId hashing.
> We should probably add a parameter *splitByPrefix* to the *SPLITSHARD* 
> command that would look at the number of docs sharing each compositeId prefix 
> and use that to create roughly equal sized buckets by document count rather 
> than just assuming an equal distribution across the entire hash range.
> Like normal shard splitting, we should bias against splitting within hash 
> buckets unless necessary (since that leads to larger query fanout.) . Perhaps 
> this warrants a parameter that would control how much of a size mismatch is 
> tolerable before resorting to splitting within a bucket. 
> *allowedSizeDifference*?
> To more quickly calculate the number of docs in each bucket, we could index 
> the prefix in a different field.  Iterating over the terms for this field 
> would quickly give us the number of docs in each (i.e lucene keeps track of 
> the doc count for each term already.)  Perhaps the implementation could be a 
> flag on the *id* field... something like *indexPrefixes* and poly-fields that 
> would cause the indexing to be automatically done and alleviate having to 
> pass in an additional field during indexing and during the call to 
> *SPLITSHARD*.  This whole part is an optimization though and could be split 
> off into its own issue if desired.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13240) UTILIZENODE action results in an exception

[Lucene/Solr QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905537#comment-16905537
 ] 

Lucene/Solr QA commented on SOLR-13240:
---

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  1m 35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  1m 35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  1m 35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  5m 
45s{color} | {color:green} solrj in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 26s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13240 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12977372/SOLR-13240.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene1-us-west 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / d468d71c03 |
| ant | version: Apache Ant(TM) version 1.10.5 compiled on March 28 2019 |
| Default Java | LTS |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/526/testReport/ |
| modules | C: solr/solrj U: solr/solrj |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/526/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> UTILIZENODE action results in an exception
> --
>
> Key: SOLR-13240
> URL: https://issues.apache.org/jira/browse/SOLR-13240
> Project: Solr
>  Issue Type: Bug
>  Components: AutoScaling
>Affects Versions: 7.6
>Reporter: Hendrik Haddorp
>Priority: Major
> Attachments: SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, 
> SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, solr-solrj-7.5.0.jar
>
>
> When I invoke the UTILIZENODE action the REST call fails like this after it 
> moved a few replicas:
> {
>   "responseHeader":{
> "status":500,
> "QTime":40220},
>   "Operation utilizenode caused 
> exception:":"java.lang.IllegalArgumentException:java.lang.IllegalArgumentException:
>  Comparison method violates its general contract!",
>   "exception":{
> "msg":"Comparison method violates its general contract!",
> "rspCode":-1},
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Comparison method violates its general contract!",
> "trace":"org.apache.solr.common.SolrException: Comparison method violates 
> its general contract!\n\tat 
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>  
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:274)\n\tat
>  
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:246)\n\tat
>  
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>  
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)\n\tat 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)\n\tat
>  org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)\n\tat 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
>  
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)\n\tat
>  
> 

[JENKINS] Lucene-Solr-8.x-Solaris (64bit/jdk1.8.0) - Build # 261 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Solaris/261/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  
org.apache.solr.update.processor.DimensionalRoutedAliasUpdateProcessorTest.testCatTime

Error Message:
took over 10 seconds after collection creation to update aliases

Stack Trace:
java.lang.AssertionError: took over 10 seconds after collection creation to 
update aliases
at 
__randomizedtesting.SeedInfo.seed([525209539DCB2D30:55594650A2387CD8]:0)
at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.solr.update.processor.RoutedAliasUpdateProcessorTest.waitColAndAlias(RoutedAliasUpdateProcessorTest.java:77)
at 
org.apache.solr.update.processor.DimensionalRoutedAliasUpdateProcessorTest.testCatTime(DimensionalRoutedAliasUpdateProcessorTest.java:480)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 443 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/443/

1 tests failed.
FAILED:  org.apache.solr.search.facet.TestCloudJSONFacetSKG.testRandom

Error Message:
Error from server at 
http://127.0.0.1:44709/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection:
 Error from server at null: Expected mime type application/octet-stream but got 
text/html.Error 500 Server Error 
 HTTP ERROR 500 Problem accessing 
/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection_shard1_replica_n1/select.
 Reason: Server ErrorCaused 
by:java.lang.AssertionError  at 
java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1896)  at 
java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)  at 
java.base/java.util.HashMap.putVal(HashMap.java:633)  at 
java.base/java.util.HashMap.put(HashMap.java:607)  at 
org.apache.solr.search.LRUCache.put(LRUCache.java:201)  at 
org.apache.solr.search.SolrCacheHolder.put(SolrCacheHolder.java:46)  at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1449)
  at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:568)  at 
org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1484)
  at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:398)
  at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:305)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:198)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:2581)  at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)  at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)  
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
 at org.eclipse.jetty.server.Server.handle(Server.java:505)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:311)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
  at java.base/java.lang.Thread.run(Thread.java:834)  http://eclipse.org/jetty;>Powered by Jetty:// 9.4.19.v20190610  
  

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at 
http://127.0.0.1:44709/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection:
 Error from server at null: Expected mime type application/octet-stream but got 
text/html. 


Error 500 Server Error

HTTP ERROR 500
Problem accessing 
/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection_shard1_replica_n1/select.
 Reason:
Server ErrorCaused by:java.lang.AssertionError
at 
java.base/java.util.HashMap$TreeNode.moveRootToFront(HashMap.java:1896)
at java.base/java.util.HashMap$TreeNode.putTreeVal(HashMap.java:2061)
at java.base/java.util.HashMap.putVal(HashMap.java:633)
at java.base/java.util.HashMap.put(HashMap.java:607)
at org.apache.solr.search.LRUCache.put(LRUCache.java:201)
at org.apache.solr.search.SolrCacheHolder.put(SolrCacheHolder.java:46)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1449)
at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:568)
at 

[GitHub] [lucene-solr] cpoerschke commented on a change in pull request #300: SOLR-11831: Skip second grouping step if group.limit is 1 (aka Las Vegas Patch)

[GitBox]

cpoerschke commented on a change in pull request #300: SOLR-11831: Skip second 
grouping step if group.limit is 1 (aka Las Vegas Patch)
URL: https://github.com/apache/lucene-solr/pull/300#discussion_r313013808
 
 

 ##
 File path: 
solr/core/src/java/org/apache/solr/search/grouping/distributed/responseprocessor/SkipSecondStepSearchGroupShardResponseProcessor.java
 ##
 @@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.solr.search.grouping.distributed.responseprocessor;
+
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.lucene.search.Sort;
+import org.apache.lucene.search.TotalHits;
+import org.apache.lucene.search.grouping.GroupDocs;
+import org.apache.lucene.search.grouping.SearchGroup;
+import org.apache.lucene.search.grouping.TopGroups;
+import org.apache.lucene.util.BytesRef;
+import org.apache.solr.handler.component.ResponseBuilder;
+import org.apache.solr.handler.component.ShardDoc;
+import org.apache.solr.handler.component.ShardRequest;
+import org.apache.solr.handler.component.ShardResponse;
+import org.apache.solr.search.SolrIndexSearcher;
+import org.apache.solr.search.grouping.GroupingSpecification;
+import 
org.apache.solr.search.grouping.distributed.shardresultserializer.SearchGroupsResultTransformer;
+
+public class SkipSecondStepSearchGroupShardResponseProcessor extends 
SearchGroupShardResponseProcessor {
+
+  @Override
+  protected SearchGroupsResultTransformer 
newSearchGroupsResultTransformer(SolrIndexSearcher solrIndexSearcher) {
+return new 
SearchGroupsResultTransformer.SkipSecondStepSearchResultResultTransformer(solrIndexSearcher);
+  }
+
+  @Override
+  protected SearchGroupsContainer newSearchGroupsContainer(ResponseBuilder rb) 
{
+return new 
SkipSecondStepSearchGroupsContainer(rb.getGroupingSpec().getFields());
+  }
+
+  @Override
+  public void process(ResponseBuilder rb, ShardRequest shardRequest) {
+super.process(rb, shardRequest);
+TopGroupsShardResponseProcessor.fillResultIds(rb);
+  }
+
+  protected static class SkipSecondStepSearchGroupsContainer extends 
SearchGroupsContainer {
+
+private final Map docIdToShard = new HashMap<>();
+
+public SkipSecondStepSearchGroupsContainer(String[] fields) {
+  super(fields);
+}
+
+@Override
+public void addSearchGroups(ShardResponse srsp, String field, 
Collection> searchGroups) {
+  super.addSearchGroups(srsp, field, searchGroups);
+  for (SearchGroup searchGroup : searchGroups) {
+assert(srsp.getShard() != null);
+docIdToShard.put(searchGroup.topDocSolrId, srsp.getShard());
+  }
+}
+
+@Override
+public void addMergedSearchGroups(ResponseBuilder rb, String groupField, 
Collection> mergedTopGroups ) {
+  // TODO: add comment or javadoc re: why this method is overridden as a 
no-op
 
 Review comment:
   The 
https://github.com/cpoerschke/lucene-solr/commit/d4af896d511b10b40263ee5383ed633609ff7ff1
 commit turns this 'TODO' into method-level javadocs. Adding similar 
method-level javadocs for the `addSearchGroupToShards` method below made me 
wonder if
   * (a) the `super` method call in it could be omitted, and
   * (b) if emptiness of the relevant ResponseBuilder structures might be used 
as a way of determining that the 'second step' stage can be skipped?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] cpoerschke commented on issue #300: SOLR-11831: Skip second grouping step if group.limit is 1 (aka Las Vegas Patch)

[GitBox]

cpoerschke commented on issue #300: SOLR-11831: Skip second grouping step if 
group.limit is 1 (aka Las Vegas Patch)
URL: https://github.com/apache/lucene-solr/pull/300#issuecomment-520496115
 
 
   > ... My 
https://github.com/cpoerschke/lucene-solr/commit/20129e7d3f7e12f442254e780e7da9a590a9036b
 commit has a couple of tweaks and suggestions, lumped together as one commit 
(sorry!) but do feel free to look and selectively pick as you like. ...
   
   Just pushed some more content to the 
https://github.com/cpoerschke/lucene-solr/commits/github-bloomberg-SOLR-11831-cpoerschke-11
 branch that the above-mentioned commit lives on.
   
   > ... the .`adoc` documentation changes are also on the known to-do list.
   
   
https://github.com/cpoerschke/lucene-solr/commit/50c5c6408af5d106abc4abc6a21bac8d8634de6a
 is a tiny start, also on that branch.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8943) Incorrect IDF in MultiPhraseQuery and SpanOrQuery

[Jim Ferenczi (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905332#comment-16905332
 ] 

Jim Ferenczi commented on LUCENE-8943:
--

{quote}

Your post made me think of the problem in another way. If we had something like 
MultiWordsSynonymQuery, we could have even more control. Similar to 
SynonymQuery we could use one IDF value for all synonyms. Synonym boost would 
work much more reliably.

{quote}

 

Yes, that's what I tried to explain in my post. It is a specific issue with 
multi-words synonyms so we should have a dedicated query. 

 

{quote}

Usually the values for pseudoStats would be computed bottom up (SpanWeight, 
PhraseWeight) from the subqueries. But we could implement a general 
MultiWordsSynonymQuery as subclass of BooleanQuery (only allowing disjunction) 
which would set (adapt) pseudoStats in all its subweights (docFreq as max 
docFreq of all synonyms just as SynonymQuery currently does).

{quote}

 

+1, that's how I'd start with this. We don't need to handle all type of queries 
though, only Term (e.g.: body:ny), conjunction of Term queries (e.g.: body:new 
AND body:york) and phrase queries (e.g.: "new york") should be accepted.

> Incorrect IDF in MultiPhraseQuery and SpanOrQuery
> -
>
> Key: LUCENE-8943
> URL: https://issues.apache.org/jira/browse/LUCENE-8943
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/query/scoring
>Affects Versions: 8.0
>Reporter: Christoph Goller
>Priority: Major
>
> I recently stumbled across a very old bug in the IDF computation for 
> MultiPhraseQuery and SpanOrQuery.
> BM25Similarity and TFIDFSimilarity / ClassicSimilarity have a method for 
> combining IDF values from more than on term / TermStatistics.
> I mean the method:
> Explanation idfExplain(CollectionStatistics collectionStats, TermStatistics 
> termStats[])
> It simply adds up the IDFs from all termStats[].
> This method is used e.g. in PhraseQuery where it makes sense. If we assume 
> that for the phrase "New York" the occurrences of both words are independent, 
> we can multiply their probabilitis and since IDFs are logarithmic we add them 
> up. Seems to be a reasonable approximation. However, this method is also used 
> to add up the IDFs of all terms in a MultiPhraseQuery as can be seen in:
> Similarity.SimScorer getStats(IndexSearcher searcher)
> A MultiPhraseQuery is actually a PhraseQuery with alternatives at individual 
> positions. IDFs of alternative terms for one position should not be added up. 
> Instead we should use the minimum value as an approcimation because this 
> corresponds to the docFreq of the most frequent term and we know that this is 
> a lower bound for the docFreq for this position.
> In SpanOrQuerry we have the same problem It uses buildSimWeight(...) from 
> SpanWeight and adds up all IDFs of all OR-clauses.
> If my arguments are not convincing, look at SynonymQuery / SynonymWeight in 
> the constructor:
> SynonymWeight(Query query, IndexSearcher searcher, ScoreMode scoreMode, float 
> boost) 
> A SynonymQuery is also a kind of OR-query and it uses the maximum of the 
> docFreq of all its alternative terms. I think this is how it should be.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] thomaswoeckinger edited a comment on issue #755: SOLR-13592: Introduce EmbeddedSolrTestBase for better integration tests

[GitBox]

thomaswoeckinger edited a comment on issue #755: SOLR-13592: Introduce 
EmbeddedSolrTestBase for better integration tests
URL: https://github.com/apache/lucene-solr/pull/755#issuecomment-520341697
 
 
   > My apologies Thomas, my gmail filters weren't letting through github 
notifications and I didn't see this until you pinged me in JIRA.
   > 
   
   :-), no problem, i am still interested to get this in!
   Do you know if there are any plans on the master branch migrating to junit 
5, this would allow to write tests using default interfaces, so it would be 
very easy to let them run under different scenarios like embedded, jetty, cloud 
and so on.
   
   > This all looks good to me, except for the whitespace formatting, which 
we've talked about on other PRs. Github makes it easy to hide 
introduction/removal of trailing whitespace, but other changes are harder to 
hide and make the diff noisier to read and harder to review with confidence: 
adding "final" modifiers to existing variables, breaking up long method 
signatures onto multiple lines, etc. Would you mind scrubbing some of those 
changes from this PR?
   
   Yes we talked about, my point: why using common format rules if even no one 
is using them.
   But anyway, i can take a look at the method signatures, if you like, but i 
think it is not that difficult to review at the current state.

   > 
   > In the meantime, I'm going to start "beasting" (run {{ant test-help}} for 
info) this PR to make sure that none of the affected tests have any flaky 
failures. I'll comment again in a day or two with some results. (If you want to 
help out, you can also run some beast-runs if your machine allows it.)
   > 
   > Thanks for your continued patience.
   
   i will have a look at it, the default test are all running on my site, i 
will comment on this PR when i have results.
   
   And you are very welcome with your comments and suggestions!
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Badapple report

[Erick Erickson]

Continued improvement I think. Or at least the improvements 3 weeks ago are 
working their way through the system. Note that the number of tests that _only_ 
failed three weeks ago is almost half the total. So I have some optimism that 
next week we’ll see a further large drop.


Here’s the synopsis, full report attached:
Number of AwaitsFix: 38 Number of BadApples: 11


Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0  had  28 failures
Week: 1  had  21 failures
Week: 2  had  47 failures
Week: 3  had  142 failures


**Annotated tests that didn't fail in the last 4 weeks.

  **Tests removed from the next two lists because they were specified in 
'doNotEnable' in the properties file
 MoveReplicaHDFSTest.testNormalFailedMove

  **Annotations will be removed from the following tests because they haven't 
failed in the last 4 rollups.

  **Methods: 3
   SolrZkClientTest.testSimpleUpdateACLs
   TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
   TestDistributedStatsComponentCardinality.test


Failures in Hoss' reports for the last 4 rollups.

There were 182 unannotated tests that failed in Hoss' rollups. Ordered by the 
date I downloaded the rollup file, newest->oldest. See above for the dates the 
files were collected 
These tests were NOT BadApple'd or AwaitsFix'd
All tests that failed 4 weeks running will be BadApple'd unless there are 
objections

Failures in the last 4 reports..
   Report   Pct runsfails   test
 0123   1.3  597 11  
AliasIntegrationTest.testClusterStateProviderAPI
 0123  26.7   65 17  
HdfsAutoAddReplicasIntegrationTest.testSimple
 0123   1.0  686  9  HttpPartitionWithTlogReplicasTest.test
 Will BadApple all tests above this line except ones listed at the 
top**


DO NOT ENABLE LIST:
MoveReplicaHDFSTest.testFailedMove
MoveReplicaHDFSTest.testNormalFailedMove
TestControlledRealTimeReopenThread.testCRTReopen
TestICUNormalizer2CharFilter.testRandomStrings
TestICUTokenizerCJK
TestImpersonationWithHadoopAuth.testForwarding
TestLTRReRankingPipeline.testDifferentTopN
TestRandomChains


DO NOT ANNOTATE LIST
CdcrBidirectionalTest.testBiDir
IndexSizeTriggerTest.testMergeIntegration
IndexSizeTriggerTest.testMixedBounds
IndexSizeTriggerTest.testSplitIntegration
IndexSizeTriggerTest.testTrigger
InfixSuggestersTest.testShutdownDuringBuild
ShardSplitTest.test
ShardSplitTest.testSplitMixedReplicaTypes
ShardSplitTest.testSplitWithChaosMonkey
TestLatLonShapeQueries.testRandomBig
TestRandomChains.testRandomChainsWithLargeStrings
TestTriggerIntegration.testSearchRate

Processing file (History bit 3): HOSS-2019-088-05.csv
Processing file (History bit 2): HOSS-2019-08-12.csv
Processing file (History bit 1): HOSS-2019-07-29.csv
Processing file (History bit 0): HOSS-2019-07-08.csv


Number of AwaitsFix: 38 Number of BadApples: 11


Raw fail count by week totals, most recent week first (corresponds to bits):
Week: 0  had  28 failures
Week: 1  had  21 failures
Week: 2  had  47 failures
Week: 3  had  142 failures


**Annotated tests that didn't fail in the last 4 weeks.

  **Tests removed from the next two lists because they were specified in 
'doNotEnable' in the properties file
 MoveReplicaHDFSTest.testNormalFailedMove

  **Annotations will be removed from the following tests because they haven't 
failed in the last 4 rollups.

  **Methods: 3
   SolrZkClientTest.testSimpleUpdateACLs
   TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
   TestDistributedStatsComponentCardinality.test


Failures in Hoss' reports for the last 4 rollups.

There were 182 unannotated tests that failed in Hoss' rollups. Ordered by the 
date I downloaded the rollup file, newest->oldest. See above for the dates the 
files were collected 
These tests were NOT BadApple'd or AwaitsFix'd
All tests that failed 4 weeks running will be BadApple'd unless there are 
objections

Failures in the last 4 reports..
   Report   Pct runsfails   test
 0123   1.3  597 11  
AliasIntegrationTest.testClusterStateProviderAPI
 0123  26.7   65 17  
HdfsAutoAddReplicasIntegrationTest.testSimple
 0123   1.0  686  9  HttpPartitionWithTlogReplicasTest.test
 Will BadApple all tests above this line except ones listed at the 
top**



 0121.4  276  3  TestCloudJSONFacetJoinDomain.testRandom
 0 23   6.2  685176  BasicAuthIntegrationTest.testBasicAuth
 0 23  16.7   49  5  
CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
 0 23   1.4  496  3  RollingRestartTest.test
 0 23   1.5  500 10  TestStressLiveNodes.testStress
 0 23   1.4  525  8  
TestUseDocValuesAsStored.testDuplicateMultiValued
 0 24.0  252  7  

[GitHub] [lucene-solr] yonik commented on a change in pull request #826: SOLR-13399: add splitByPrefix configuration to IndexSizeTrigger

[GitBox]

yonik commented on a change in pull request #826: SOLR-13399: add splitByPrefix 
configuration to IndexSizeTrigger
URL: https://github.com/apache/lucene-solr/pull/826#discussion_r312981845
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/handler/admin/SplitOp.java
 ##
 @@ -385,16 +385,8 @@ static String toSplitString(Collection 
splits) throws Exception
 
   // compare to current prefix bucket and see if this new term shares the 
same prefix
   if (term != null && term.length >= currPrefix.length && 
currPrefix.length > 0) {
-int i = 0;
-for (; i < currPrefix.length; i++) {
-  if (currPrefix.bytes[i] != term.bytes[term.offset + i]) {
-break;
-  }
-}
-
-if (i == currPrefix.length) {
-  // prefix was the same (common-case fast path)
-  // int count = termsEnum.docFreq();
+BytesRef termPrefix = new BytesRef(term.bytes, term.offset, 
currPrefix.length);
 
 Review comment:
   Found a better choice here that doesn't involve  object creation: 
StringHelper.startsWith()


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13240) UTILIZENODE action results in an exception

[Richard Goodman (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905296#comment-16905296
 ] 

Richard Goodman commented on SOLR-13240:


My bad [~cpoerschke] we use 7.4 in production here, so I just based it off 
that, however, I've re-done this against master, so I'm hoping it should now 
pass QA? _(fingers crossed)_ 

[^SOLR-13240.patch] 

> UTILIZENODE action results in an exception
> --
>
> Key: SOLR-13240
> URL: https://issues.apache.org/jira/browse/SOLR-13240
> Project: Solr
>  Issue Type: Bug
>  Components: AutoScaling
>Affects Versions: 7.6
>Reporter: Hendrik Haddorp
>Priority: Major
> Attachments: SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, 
> SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, solr-solrj-7.5.0.jar
>
>
> When I invoke the UTILIZENODE action the REST call fails like this after it 
> moved a few replicas:
> {
>   "responseHeader":{
> "status":500,
> "QTime":40220},
>   "Operation utilizenode caused 
> exception:":"java.lang.IllegalArgumentException:java.lang.IllegalArgumentException:
>  Comparison method violates its general contract!",
>   "exception":{
> "msg":"Comparison method violates its general contract!",
> "rspCode":-1},
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Comparison method violates its general contract!",
> "trace":"org.apache.solr.common.SolrException: Comparison method violates 
> its general contract!\n\tat 
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>  
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:274)\n\tat
>  
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:246)\n\tat
>  
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>  
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)\n\tat 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)\n\tat
>  org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)\n\tat 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
>  
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>  
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>  
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
>  
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  org.eclipse.jetty.server.Server.handle(Server.java:531)\n\tat 
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)\n\tat 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)\n\tat
>  
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)\n\tat
>  org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat 
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)\n\tat 
> 

[jira] [Updated] (SOLR-13240) UTILIZENODE action results in an exception

[Richard Goodman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-13240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Goodman updated SOLR-13240:
---
Attachment: SOLR-13240.patch

> UTILIZENODE action results in an exception
> --
>
> Key: SOLR-13240
> URL: https://issues.apache.org/jira/browse/SOLR-13240
> Project: Solr
>  Issue Type: Bug
>  Components: AutoScaling
>Affects Versions: 7.6
>Reporter: Hendrik Haddorp
>Priority: Major
> Attachments: SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, 
> SOLR-13240.patch, SOLR-13240.patch, SOLR-13240.patch, solr-solrj-7.5.0.jar
>
>
> When I invoke the UTILIZENODE action the REST call fails like this after it 
> moved a few replicas:
> {
>   "responseHeader":{
> "status":500,
> "QTime":40220},
>   "Operation utilizenode caused 
> exception:":"java.lang.IllegalArgumentException:java.lang.IllegalArgumentException:
>  Comparison method violates its general contract!",
>   "exception":{
> "msg":"Comparison method violates its general contract!",
> "rspCode":-1},
>   "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"Comparison method violates its general contract!",
> "trace":"org.apache.solr.common.SolrException: Comparison method violates 
> its general contract!\n\tat 
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>  
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:274)\n\tat
>  
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:246)\n\tat
>  
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>  
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:734)\n\tat 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:715)\n\tat
>  org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)\n\tat 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)\n\tat
>  
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat
>  
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat
>  
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
>  
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  org.eclipse.jetty.server.Server.handle(Server.java:531)\n\tat 
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)\n\tat 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)\n\tat
>  
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)\n\tat
>  org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat 
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)\n\tat 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)\n\tat
>  
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)\n\tat
>  
> 

[JENKINS] Lucene-Solr-Tests-8.x - Build # 395 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-8.x/395/

1 tests failed.
FAILED:  org.apache.solr.security.JWTAuthPluginIntegrationTest.testMetrics

Error Message:
Server returned HTTP response code: 401 for URL: 
http://127.0.0.1:39917/solr/jwtColl/query?q=*:*

Stack Trace:
java.io.IOException: Server returned HTTP response code: 401 for URL: 
http://127.0.0.1:39917/solr/jwtColl/query?q=*:*
at 
__randomizedtesting.SeedInfo.seed([129CCB3109B5EE02:EC8E199315BF99B1]:0)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at java.net.URLConnection.getContent(URLConnection.java:739)
at 
org.apache.solr.security.JWTAuthPluginIntegrationTest.get(JWTAuthPluginIntegrationTest.java:207)
at 
org.apache.solr.security.JWTAuthPluginIntegrationTest.testMetrics(JWTAuthPluginIntegrationTest.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-13-rc) - Build # 24530 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/24530/
Java: 64bit/jdk-13-rc -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testClusterStateProviderAPI

Error Message:
should be a routed alias

Stack Trace:
java.lang.AssertionError: should be a routed alias
at 
__randomizedtesting.SeedInfo.seed([784EFDAC52EBBAA8:6799618021E043E3]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.solr.cloud.AliasIntegrationTest.testClusterStateProviderAPI(AliasIntegrationTest.java:315)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:830)




Build Log:
[...truncated 1917 lines...]
   [junit4] JVM J1: stderr was not empty, see: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/temp/junit4-J1-20190812_123020_60016380312146198062237.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] 

[jira] [Commented] (SOLR-13242) RegexReplaceProcessorFactory not making accurate replacement

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905193#comment-16905193
 ] 

Gus Heck commented on SOLR-13242:
-

RegexReplaceUpdateProcessorFactory is a very simple class. As noted above it's 
just a wrapper around Matcher.replaceAll() 
{code:java}
return valueMutator(getSelector(), next, src -> {
  if (src instanceof CharSequence) {
CharSequence txt = (CharSequence) src;
return pattern.matcher(txt).replaceAll(replacement);
  }
  return src;
});
{code}
I notice this issue is dealing with patterns involving newlines and whitespace. 
Whitespace can be a hairy and complicated thing, since it's not rendered 
visibly unless your editor either fails to recognize it and prints ?'s... so 
the better, more complete your editor, the less likely you are to see the 
problem :). You often think you know what you're looking at but don't. It seems 
possible that you are running into unusual white space chars that break your 
pattern. One can see how fiddly this gets from things seen in Pattern.java:

 
||Predefined character classes||
|{{.}}|Any character (may or may not match [line 
terminators|https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html#lt])|
|{{\d}}|A digit: {{[0-9]}}|
|{{\D}}|A non-digit: {{[^0-9]}}|
|{{\h}}|A horizontal whitespace character: {{[ 
\t\xA0\u1680\u180e\u2000-\u200a\u202f\u205f\u3000]}}|
|{{\H}}|A non-horizontal whitespace character: {{[^\h]}}|
|{{\s}}|A whitespace character: {{[ \t\n\x0B\f\r]}}|
|{{\S}}|A non-whitespace character: {{[^\s]}}|
|{{\v}}|A vertical whitespace character: {{[\n\x0B\f\r\x85\u2028\u2029]}}|
|{{\V}}|A non-vertical whitespace character: {{[^\v]}}|
|{{\w}}|A word character: {{[a-zA-Z_0-9]}}|
|{{\W}}|A non-word character: {{[^\w]}}|

(\h|\v) appears to catch a lot more types of whitespace than \s for example. 
See also (with (?U)which turns on the unicode character class feature).
|{{\p\{Blank}}}|A space or a tab: 
{{[\p\{IsWhite_Space}&&[^\p\{gc=Zl}\p\{gc=Zp}\x0a\x0b\x0c\x0d\x85]]}}|

The comments above indicating that \W and \h instead of \s work better for the 
OP seem to support the hypothesis that the issue was caused by was oddball 
white space in the data, not a bug in Solr. [~edwinyeozl] can we resolve "this 
as not a problem"?

> RegexReplaceProcessorFactory not making accurate replacement
> 
>
> Key: SOLR-13242
> URL: https://issues.apache.org/jira/browse/SOLR-13242
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 7.6, 7.7, 7.7.1
>Reporter: Edwin Yeo Zheng Lin
>Priority: Major
>  Labels: regex, solr
>
> We are using the RegexReplaceProcessorFactory, and have tried with all of the 
> following configurations in solrconfig.xml:
>  
> 
>     content
>     (\s*\r?\n)\{2,}
>     
>     true
>   
> 
>     content
>     ([ \s]*\r?\n)\{2,}
>     
>     true
>   
>  
>     content
>     (\s*\n)\{2,}
>     
>     true
>   
>  
>     content
>     (\n\s*)\{2,}
>     
>     true
>   
>  
> The regex pattern of (\s*\r?\n)\{2,}, ([ \s]*\r?\n)\{2,}, (\s*\n)\{2,} and 
> (\n\s*)\{2,} are working perfectly in [regex101.com|http://regex101.com/], in 
> which all the \n will be replaced by only two 
> However, in Solr, there are cases (in Example 2 and 3 below) that has four 
>  in a row. This should not be the case, as we have already set it to 
> replace by two  regardless of how many \n are there in a row.
>  
>  
> *Example 1: The sentence that the above regex pattern is working correctly* 
> *Original content in EML [file:*|file://%2A/]  
> Dear Sir, 
>  
> I am terminating 
> *Original content:*    Dear Sir,  \n\n \n \n\n I am terminating
> *Index content:*     Dear Sir,  I am terminating 
>  
> *Example 2: The sentence that the above regex pattern is partially working 
> (as you can see, instead of 2 , there are 4 )*
> *Original content in EML [file:*|file://%2A/]
> _exalted_
> _Psalm 89:17_
>  
> 3 Choa Chu Kang Avenue 4    
> *Original content:* exalted  \n \n\n   Psalm 89:17   \n\n   \n\n  3 Choa Chu 
> Kang Avenue 4, Singapore
> *Index content:* exalted  Psalm 89:17     3 Choa Chu 
> Kang Avenue 4, Singapore
>  
> *Example 3: The sentence that the above regex pattern is partially working 
> (as you can see, instead of 2 , there are 4 )*
> *Original content in EML [file:*|file://%2A/]
> [http://www.concordpri.moe.edu.sg/]
>  
>  
>  
>  
> On Tue, Dec 18, 2018 at 10:07 AM    
> *Original content:* [http://www.concordpri.moe.edu.sg/]   \n\n   \n\n \n \n\n 
> \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n\n \n\n\n  On Tue, Dec 18, 2018 
> at 10:07 AM 
> *Index content:* [http://www.concordpri.moe.edu.sg/]     On 
> Tue, Dec 18, 2018 at 10:07 AM



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

[jira] [Commented] (LUCENE-8943) Incorrect IDF in MultiPhraseQuery and SpanOrQuery

[Christoph Goller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905178#comment-16905178
 ] 

Christoph Goller commented on LUCENE-8943:
--

I agree, we cannot realistically approximate the doc freq of phrases.
And yes, actually the scoring problem I brought up is a kind of synonym issue.

Usually, if we are using synonyms we want to score exact query matches higher 
than synonym matches. That's probably one of the reasons why SynonymQuery 
allows to specify boosts.

I am having lots of multiword synonyms. W2k16 e.g. is a synonym for "Windows 
Server 2016". Different boosts for multiword synonyms don't work reliably since 
matches for "Windows Server 2016" may score much higher than those of W2k16 due 
to huge IDFs.

I am not so much looking for an optimal BM25 scoring for Phrases / Multiphrases 
/ Spans. Instead I  am looking for a 
way to score them within BM25 so that boosts work as expected.

One step into this direction would be to limit IDF values in case of Phrases / 
Multiphrases / Spans. In BM25 it seems to be very important that IDF saturates 
and currently the behavior of Phrases / Multiphrases / Spans contradicts that. 
With the solution I proposed we can get rid of huge IDF values for Phrases / 
Multiphrases / Spans. Therefore I still think we should do it. Plus it would 
make scores more camparable and boosts would work more reliable.

Your post made me think of the problem in another way. If we had something like 
MultiWordsSynonymQuery, we could have even more control. Similar to 
SynonymQuery we could use one IDF value for all synonyms. Synonym boost would 
work much more reliably.

MultiWordsSynonymQuery could be very general. In my last post I suggested to 
approximate docFreq instead of IDFs in order to gurantee saturation. For 
implementing it, I thought about adding a member variable pseudoStats 
(TermStatistics) to Weight, which would be used for computing SimScorer. 
Usually the values for pseudoStats would be computed bottom up (SpanWeight, 
PhraseWeight) from the subqueries. But we could implement a general 
MultiWordsSynonymQuery as subclass of BooleanQuery (only allowing disjunction) 
which would set (adapt) pseudoStats in all its subweights (docFreq as max 
docFreq of all synonyms just as SynonymQuery currently does).

> Incorrect IDF in MultiPhraseQuery and SpanOrQuery
> -
>
> Key: LUCENE-8943
> URL: https://issues.apache.org/jira/browse/LUCENE-8943
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/query/scoring
>Affects Versions: 8.0
>Reporter: Christoph Goller
>Priority: Major
>
> I recently stumbled across a very old bug in the IDF computation for 
> MultiPhraseQuery and SpanOrQuery.
> BM25Similarity and TFIDFSimilarity / ClassicSimilarity have a method for 
> combining IDF values from more than on term / TermStatistics.
> I mean the method:
> Explanation idfExplain(CollectionStatistics collectionStats, TermStatistics 
> termStats[])
> It simply adds up the IDFs from all termStats[].
> This method is used e.g. in PhraseQuery where it makes sense. If we assume 
> that for the phrase "New York" the occurrences of both words are independent, 
> we can multiply their probabilitis and since IDFs are logarithmic we add them 
> up. Seems to be a reasonable approximation. However, this method is also used 
> to add up the IDFs of all terms in a MultiPhraseQuery as can be seen in:
> Similarity.SimScorer getStats(IndexSearcher searcher)
> A MultiPhraseQuery is actually a PhraseQuery with alternatives at individual 
> positions. IDFs of alternative terms for one position should not be added up. 
> Instead we should use the minimum value as an approcimation because this 
> corresponds to the docFreq of the most frequent term and we know that this is 
> a lower bound for the docFreq for this position.
> In SpanOrQuerry we have the same problem It uses buildSimWeight(...) from 
> SpanWeight and adds up all IDFs of all OR-clauses.
> If my arguments are not convincing, look at SynonymQuery / SynonymWeight in 
> the constructor:
> SynonymWeight(Query query, IndexSearcher searcher, ScoreMode scoreMode, float 
> boost) 
> A SynonymQuery is also a kind of OR-query and it uses the maximum of the 
> docFreq of all its alternative terms. I think this is how it should be.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: JDK 13 is now in the Release Candidate Phase

[Rory O'Donnell]

Thanks Uwe!

On 12/08/2019 13:36, Uwe Schindler wrote:


Hi Rory,

sorry for the delay: I was on vacation and now back in office (but 
only this week). I updated Jenkins to use the JDK 13 RC build and the 
first run is already going on. I also installed JDK 14 ea+9, let’s see 
what it says.


Uwe

*From:*Rory O'Donnell 
*Sent:* Saturday, August 10, 2019 12:26 PM
*To:* uwe.h.schind...@gmail.com; dawid.we...@cs.put.poznan.pl
*Cc:* rory.odonn...@oracle.com; Dalibor Topic 
; Balchandra Vaidya 
; Muneer Kolarkunnu 
; Deepak Damodaran 
; dev@lucene.apache.org

*Subject:* JDK 13 is now in the Release Candidate Phase

Hi Uwe & Dawid,

Haven't heard from you in some time, any issues to report as we 
announce a Release Candidate ?


*JDK 13 is now in the Release Candidate Phase *

Per the JDK 13 schedule [1], we are now in the Release Candidate phase.
The stabilization repository, jdk/jdk13, is open for P1 bug fixes per 
the JDK Release Process (JEP 3) [2].

All changes require approval via the Fix-Request Process [3].

For more details, see Mark Reinhold's email to jdk-dev mailing list [4]

  * Milestone Schedule:

  o Initial RC Build 33 - Aug 9, 2019
  o GAC - Aug 22, 2019
  o GAR - Sept 5, 2019
  o GA - Sept 17, 2019

*OpenJDK 13 build 33 is available at http://jdk.java.net/13/*

  * These early access, open source builds are provided under the GNU
General Public License, version 2, with the Classpath Exception
.
  * Schedule, status & features

  o http://openjdk.java.net/projects/jdk/13/

  * Release Notes

  o http://jdk.java.net/13/release-notes

  * Bug fixes reported by Open Source Projects  :

  o JDK-8228764 - fixed in b32 -reported by Apache Tomcat

*OpenJDK 14 EA build 9 is now available at http://jdk.java.net/14 *

  * These early access, open source builds are provided under the GNU
General Public License, version 2, with the Classpath Exception
.
  * Release Notes

  o http://jdk.java.net/14/release-notes

  * JEPs targeted to JDK 14

  o JEP 352  - Non-Volatile
Mapped Byte Buffers

  * Changes in this build


  * Bug fixes reported by Open Source Projects  :

  o JDK-8227170 - fixed in b8 -reported by Apache Ant
  o JDK-8228485 - fixed in b8 -reported by JaCoCo
  o JDK-8222791 - fixed in b7 -reported by Apache Lucene

*Project Panama Early-Access Builds*

  * Build jdk-14-panama+1-15 (2019/8/8) is available at
http://jdk.java.net/panama/
  * These early-access, open-source builds are provided under the GNU
General Public License, version 2, with the Classpath Exception
.

Regards,
Rory

[1] https://openjdk.java.net/projects/jdk/13/#Schedule
[2] https://openjdk.java.net/jeps/3
[3] https://openjdk.java.net/jeps/3#Fix-Request-Process
[4] 
https://mail.openjdk.java.net/pipermail/jdk-dev/2019-August/003250.html


--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland


--
Rgds, Rory O'Donnell
Quality Engineering Manager
Oracle EMEA, Dublin, Ireland

RE: JDK 13 is now in the Release Candidate Phase

[Uwe Schindler]

Hi Rory,

 

sorry for the delay: I was on vacation and now back in office (but only this 
week). I updated Jenkins to use the JDK 13 RC build and the first run is 
already going on. I also installed JDK 14 ea+9, let’s see what it says.

 

Uwe

 

 

From: Rory O'Donnell  
Sent: Saturday, August 10, 2019 12:26 PM
To: uwe.h.schind...@gmail.com; dawid.we...@cs.put.poznan.pl
Cc: rory.odonn...@oracle.com; Dalibor Topic ; 
Balchandra Vaidya ; Muneer Kolarkunnu 
; Deepak Damodaran 
; dev@lucene.apache.org
Subject: JDK 13 is now in the Release Candidate Phase

 

Hi Uwe & Dawid, 

Haven't heard from you in some time, any issues to report as we announce a 
Release Candidate ?

JDK 13 is now in the Release Candidate Phase 

Per the JDK 13 schedule [1], we are now in the Release Candidate phase. 
The stabilization repository, jdk/jdk13, is open for P1 bug fixes per the JDK 
Release Process (JEP 3) [2]. 
All changes require approval via the Fix-Request Process [3]. 

For more details, see Mark Reinhold's email to jdk-dev mailing list [4] 

*   Milestone Schedule:

*   Initial RC Build 33 - Aug 9, 2019
*   GAC - Aug 22, 2019
*   GAR - Sept 5, 2019
*   GA - Sept 17, 2019

OpenJDK 13 build 33 is available at http://jdk.java.net/13/ 

*   These early access, open source builds are provided under the GNU 
General Public License, version   
2, with the Classpath Exception.
*   Schedule, status & features

*   http://openjdk.java.net/projects/jdk/13/

*   Release Notes

*   http://jdk.java.net/13/release-notes

*   Bug fixes reported by Open Source Projects  : 

*   JDK-8228764 - fixed in b32 -reported by Apache Tomcat

OpenJDK 14 EA build 9 is now available at http://jdk.java.net/14 

*   These early access, open source builds are provided under the GNU 
General Public License, version   
2, with the Classpath Exception.
*   Release Notes 

*   http://jdk.java.net/14/release-notes

*   JEPs targeted to JDK 14

*   JEP 352   - Non-Volatile Mapped Byte 
Buffers

*   Changes in this build  

 
*   Bug fixes reported by Open Source Projects  : 

*   JDK-8227170 - fixed in b8 -reported by Apache Ant
*   JDK-8228485 - fixed in b8 -reported by JaCoCo
*   JDK-8222791 - fixed in b7 -reported by Apache Lucene

Project Panama Early-Access Builds

*   Build jdk-14-panama+1-15 (2019/8/8) is available at 
http://jdk.java.net/panama/
*   These early-access, open-source builds are provided under the GNU 
General Public License, version   
2, with the Classpath Exception.

Regards,
Rory

[1] https://openjdk.java.net/projects/jdk/13/#Schedule 
[2] https://openjdk.java.net/jeps/3 
[3] https://openjdk.java.net/jeps/3#Fix-Request-Process
[4] https://mail.openjdk.java.net/pipermail/jdk-dev/2019-August/003250.html



-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland 

[jira] [Resolved] (SOLR-13622) Add FileStream Streaming Expression

[Jason Gerlowski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-13622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gerlowski resolved SOLR-13622.

Resolution: Fixed

fucit.org doesn't show any StreamExpressionTest failures in the last 4 days, so 
I'm closing this accordingly.  Thanks again for catching the test issue Hoss, 
that's my bad.

> Add FileStream Streaming Expression
> ---
>
> Key: SOLR-13622
> URL: https://issues.apache.org/jira/browse/SOLR-13622
> Project: Solr
>  Issue Type: New Feature
>  Components: streaming expressions
>Reporter: Joel Bernstein
>Assignee: Jason Gerlowski
>Priority: Major
> Fix For: 8.3
>
> Attachments: SOLR-13622.patch, SOLR-13622.patch
>
>
> The FileStream will read files from a local filesystem and Stream back each 
> line of the file as a tuple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-13647) default solr.in.sh contains uncommented lines

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-13647.

Resolution: Fixed

> default solr.in.sh contains uncommented lines
> -
>
> Key: SOLR-13647
> URL: https://issues.apache.org/jira/browse/SOLR-13647
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.1.1
>Reporter: John
>Assignee: Jan Høydahl
>Priority: Trivial
> Fix For: 8.3
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> default version of this file should be completely commented
> ENABLE_REMOTE_JMX_OPTS had defaults



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13647) default solr.in.sh contains uncommented lines

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905118#comment-16905118
 ] 

ASF subversion and git services commented on SOLR-13647:


Commit 5ecc6dfaf890a117f21873725eba81feb1586d95 in lucene-solr's branch 
refs/heads/branch_8x from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5ecc6df ]

SOLR-13647: Default solr.in.sh contains incorrect default value

(cherry picked from commit d468d71c030dba77fb7e6bed295e5766248a5944)


> default solr.in.sh contains uncommented lines
> -
>
> Key: SOLR-13647
> URL: https://issues.apache.org/jira/browse/SOLR-13647
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.1.1
>Reporter: John
>Assignee: Jan Høydahl
>Priority: Trivial
> Fix For: 8.3
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> default version of this file should be completely commented
> ENABLE_REMOTE_JMX_OPTS had defaults



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] janhoy closed pull request #804: SOLR-13647 default solr.in.sh contains uncommented lines

[GitBox]

janhoy closed pull request #804: SOLR-13647 default solr.in.sh contains 
uncommented lines
URL: https://github.com/apache/lucene-solr/pull/804
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] janhoy commented on issue #804: SOLR-13647 default solr.in.sh contains uncommented lines

[GitBox]

janhoy commented on issue #804: SOLR-13647 default solr.in.sh contains 
uncommented lines
URL: https://github.com/apache/lucene-solr/pull/804#issuecomment-520395268
 
 
   Merged


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13647) default solr.in.sh contains uncommented lines

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905115#comment-16905115
 ] 

ASF subversion and git services commented on SOLR-13647:


Commit d468d71c030dba77fb7e6bed295e5766248a5944 in lucene-solr's branch 
refs/heads/master from Jan Høydahl
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d468d71 ]

SOLR-13647: Default solr.in.sh contains incorrect default value


> default solr.in.sh contains uncommented lines
> -
>
> Key: SOLR-13647
> URL: https://issues.apache.org/jira/browse/SOLR-13647
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.1.1
>Reporter: John
>Assignee: Jan Høydahl
>Priority: Trivial
> Fix For: 8.3
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> default version of this file should be completely commented
> ENABLE_REMOTE_JMX_OPTS had defaults



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-13647) default solr.in.sh contains uncommented lines

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-13647:
---
Fix Version/s: (was: 8.2)
   8.3

> default solr.in.sh contains uncommented lines
> -
>
> Key: SOLR-13647
> URL: https://issues.apache.org/jira/browse/SOLR-13647
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 8.1.1
>Reporter: John
>Assignee: Jan Høydahl
>Priority: Trivial
> Fix For: 8.3
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> default version of this file should be completely commented
> ENABLE_REMOTE_JMX_OPTS had defaults



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312869842
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java
 ##
 @@ -145,8 +145,14 @@ public void init(NamedList args) {
 
   }
 
+  protected MetricsInfo metricsInfo;
+
+
   @Override
   public void initializeMetrics(SolrMetricManager manager, String 
registryName, String tag, final String scope) {
+metricsInfo = new MetricsInfo(manager, registryName, 
getUniqueMetricTag(tag)) ;
+tag = metricsInfo.getTag();
 
 Review comment:
   This pattern now repeats itself basically in every component, which puts the 
burden on the developer and opens room for error. I think it would be much 
better if the framework was responsible for this part behind the scenes.
   
   I'm not sure about whether we need `MetricsInfo` class at all ... or maybe 
it should be created elsewhere and passed as an argument instead of the other 
arguments? Then there's no room for error whether to use this or that tag value.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312869842
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java
 ##
 @@ -145,8 +145,14 @@ public void init(NamedList args) {
 
   }
 
+  protected MetricsInfo metricsInfo;
+
+
   @Override
   public void initializeMetrics(SolrMetricManager manager, String 
registryName, String tag, final String scope) {
+metricsInfo = new MetricsInfo(manager, registryName, 
getUniqueMetricTag(tag)) ;
+tag = metricsInfo.getTag();
 
 Review comment:
   This pattern now repeats itself basically in every component, which puts the 
burden on the developer and opens room for error. I think it would be much 
better if the framework was responsible for this part behind the scenes.
   
   I'm not sure about whether we need `MetricsInfo` class at all ... or maybe 
it should be created elsewhere and passed as an argument?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312872053
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java
 ##
 @@ -319,6 +325,11 @@ public PluginInfo getPluginInfo(){
 return  pluginInfo;
   }
 
+  @Override
+  public void close()  {
+if (metricsInfo != null) metricsInfo.unregister();
 
 Review comment:
   Again, this is also a repeating pattern that is easy to forget (in fact you 
forgot about this in one place even in this patch). If it's something that has 
to be done reliably then I think the framework should take care of that.
   
   See for example how `SolrInfoBean.registerMetricName()` is designed - it 
uses result of another method to determine whether it's a no-op. Similarly, we 
could add `SolrMetricProducer.getMetricsInfo()` in addition to `AutoCloseable` 
and then always call `unregister` in `default void close()`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312818632
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java
 ##
 @@ -145,8 +145,14 @@ public void init(NamedList args) {
 
   }
 
+  protected MetricsInfo metricsInfo;
+
+
   @Override
   public void initializeMetrics(SolrMetricManager manager, String 
registryName, String tag, final String scope) {
+metricsInfo = new MetricsInfo(manager, registryName, 
getUniqueMetricTag(tag)) ;
+tag = metricsInfo.getTag();
+getUniqueMetricTag(tag);
 
 Review comment:
   Method call with no effect?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312820738
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/metrics/SolrMetricManager.java
 ##
 @@ -698,19 +726,22 @@ public int unregisterGauges(String registryName, String 
tag) {
 MetricRegistry registry = registry(registryName);
 AtomicInteger removed = new AtomicInteger();
 registry.removeMatching((name, metric) -> {
-  if (metric instanceof GaugeWrapper &&
-  tag.equals(((GaugeWrapper)metric).getTag())) {
-removed.incrementAndGet();
-return true;
-  } else {
-return false;
+  if (metric instanceof GaugeWrapper) {
+String tagVal = ((GaugeWrapper) metric).getTag();
+if(tagVal.equals(tag) ||
+tagVal.contains(tag)){
 
 Review comment:
   Maybe we should at least rename the method parameter to indicate that the 
`tag` is really a `containsTag` match? Otherwise users could expect that this 
method only unregisters exact matches.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312872979
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/search/LFUCache.java
 ##
 @@ -254,8 +255,13 @@ private static String calcHitRatio(long lookups, long 
hits) {
 return "0." + hundredths;
   }
 
+
+  private MetricsInfo metricsInfo;
+
   @Override
   public void initializeMetrics(SolrMetricManager manager, String 
registryName, String tag, String scope) {
+metricsInfo = new MetricsInfo(manager, registryName, 
getUniqueMetricTag(tag));
+tag = metricsInfo.getTag();
 
 Review comment:
   Not needed, since the call below uses metricsInfo.getTag().


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-8.2-Windows (64bit/jdk-12.0.1) - Build # 172 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.2-Windows/172/
Java: 64bit/jdk-12.0.1 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestSkipOverseerOperations.testSkipLeaderOperations

Error Message:
IOException occurred when talking to server at: https://127.0.0.1:51726/solr

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occurred when 
talking to server at: https://127.0.0.1:51726/solr
at 
__randomizedtesting.SeedInfo.seed([6C0BCD69B40846F3:9CE11E956B2B3699]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:670)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:245)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1128)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:897)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:829)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:228)
at 
org.apache.solr.cloud.TestSkipOverseerOperations.testSkipLeaderOperations(TestSkipOverseerOperations.java:123)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

[jira] [Commented] (SOLR-13242) RegexReplaceProcessorFactory not making accurate replacement

[Chongchen Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-13242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905044#comment-16905044
 ] 

Chongchen Chen commented on SOLR-13242:
---

I wrote a unit test, But I cannot reproduce your problem. Actually, the 
RegexReplaceProcessorFactory is only a wrapper for java.util.regex.Pattern. So 
I don't think there will be something wrong in it.  My code is:


{code:java}
package org.apache.solr.update.processor;

import org.apache.solr.SolrTestCaseJ4;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.core.SolrCore;
import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.response.SolrQueryResponse;
import org.apache.solr.servlet.SolrRequestParsers;
import org.apache.solr.update.AddUpdateCommand;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;

public class RegexReplaceProcessorFactoryTest extends SolrTestCaseJ4 {
  protected static FieldValueMutatingUpdateProcessor[] reProcessors;
  protected static SolrRequestParsers _parser;
  protected static ModifiableSolrParams parameters;
  private static RegexReplaceProcessorFactory[] factorys;
  private SolrInputDocument document;

  @BeforeClass
  public static void setUpBeforeClass() throws Exception {
System.setProperty("enable.update.log", "false"); // schema12 doesn't 
support _version_
initCore("solrconfig.xml", "schema12.xml");
SolrCore core = h.getCore();
_parser = new SolrRequestParsers( null );
SolrQueryResponse resp = null;
parameters = new ModifiableSolrParams();
parameters.set("fieldName", "mail");
String[] patterns = {"(\\s*\\r?\\n){2,}", "([ \\s]*\\r?\\n){2,}", 
"(\\s*\\n){2,}", "(\\n\\s*){2,}"};
parameters.set("replacement", "");
parameters.set("literalReplacement", "true");
factorys = new RegexReplaceProcessorFactory[patterns.length];
reProcessors = new FieldValueMutatingUpdateProcessor[patterns.length];
SolrQueryRequest req = _parser.buildRequestFrom(core, new 
ModifiableSolrParams(), null);
for (int i = 0; i < patterns.length; i++){
  parameters.set("pattern",  patterns[i]);
  factorys[i] = new RegexReplaceProcessorFactory();
  factorys[i].init(parameters.toNamedList());
  factorys[i].inform(core);
  reProcessors[i] = (FieldValueMutatingUpdateProcessor) 
factorys[i].getInstance(req, resp, null);
}
  }

  @AfterClass
  public static void tearDownAfterClass() throws Exception {
// null static members for gc
reProcessors = null;
_parser = null;
parameters = null;
factorys = null;
  }

  @Before
  public void setUp() throws Exception {
document = new SolrInputDocument();
super.setUp();
  }

  @Test
  public void testSOLR13242() throws Exception {
document.addField("id", "doc1");
document.addField("mail", "exalted \n \n\n Psalm 89:17   \n\n   \n\n  3 
Choa Chu Kang Avenue 4, Singapore");
processAdd(document);
System.out.println(document.getFieldValue("mail"));
  }

  private void processAdd(SolrInputDocument doc) throws Exception {
AddUpdateCommand addCommand = new AddUpdateCommand(null);
addCommand.solrDoc = doc;
for (int i = 0; i < reProcessors.length; i++){
  reProcessors[i].processAdd(addCommand);
}
  }

}

{code}

> RegexReplaceProcessorFactory not making accurate replacement
> 
>
> Key: SOLR-13242
> URL: https://issues.apache.org/jira/browse/SOLR-13242
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 7.6, 7.7, 7.7.1
>Reporter: Edwin Yeo Zheng Lin
>Priority: Major
>  Labels: regex, solr
>
> We are using the RegexReplaceProcessorFactory, and have tried with all of the 
> following configurations in solrconfig.xml:
>  
> 
>     content
>     (\s*\r?\n)\{2,}
>     
>     true
>   
> 
>     content
>     ([ \s]*\r?\n)\{2,}
>     
>     true
>   
>  
>     content
>     (\s*\n)\{2,}
>     
>     true
>   
>  
>     content
>     (\n\s*)\{2,}
>     
>     true
>   
>  
> The regex pattern of (\s*\r?\n)\{2,}, ([ \s]*\r?\n)\{2,}, (\s*\n)\{2,} and 
> (\n\s*)\{2,} are working perfectly in [regex101.com|http://regex101.com/], in 
> which all the \n will be replaced by only two 
> However, in Solr, there are cases (in Example 2 and 3 below) that has four 
>  in a row. This should not be the case, as we have already set it to 
> replace by two  regardless of how many \n are there in a row.
>  
>  
> *Example 1: The sentence that the above regex pattern is working correctly* 
> *Original content in EML [file:*|file://%2A/]  
> Dear Sir, 
>  
> I am terminating 
> *Original content:*    Dear Sir,  \n\n \n \n\n I am terminating
> *Index content:*     Dear Sir,  I am terminating 
>  
> *Example 2: The sentence that the 

[JENKINS] Lucene-Solr-repro - Build # 3513 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/3513/

[...truncated 28 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/178/consoleText

[repro] Revision: 5df5df9ec37c9fbaca9c0629482a95fb90c8d33b

[repro] Ant options: -Dtests.multiplier=2 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt
[repro] Repro line:  ant test  -Dtestcase=HdfsAutoAddReplicasIntegrationTest 
-Dtests.method=testSimple -Dtests.seed=FD0FEC07F813032F -Dtests.multiplier=2 
-Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=he -Dtests.timezone=America/Goose_Bay -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
f6f1b4244c40e5665b20a2a8ef9852c6dd827cb2
[repro] git fetch
[repro] git checkout 5df5df9ec37c9fbaca9c0629482a95fb90c8d33b

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   HdfsAutoAddReplicasIntegrationTest
[repro] ant compile-test

[...truncated 3577 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.HdfsAutoAddReplicasIntegrationTest" -Dtests.showOutput=onerror 
-Dtests.multiplier=2 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt
 -Dtests.seed=FD0FEC07F813032F -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=he -Dtests.timezone=America/Goose_Bay -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[...truncated 2917 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   1/5 failed: 
org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest
[repro] git checkout f6f1b4244c40e5665b20a2a8ef9852c6dd827cb2

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-13539) Atomic Update Multivalue remove does not work for field types UUID, Enums, Bool and Binary

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-13539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904989#comment-16904989
 ] 

Thomas Wöckinger commented on SOLR-13539:
-

Yes, i agree.

> Atomic Update Multivalue remove does not work for field types UUID, Enums, 
> Bool  and Binary
> ---
>
> Key: SOLR-13539
> URL: https://issues.apache.org/jira/browse/SOLR-13539
> Project: Solr
>  Issue Type: Bug
>  Components: UpdateRequestProcessors
>Affects Versions: 7.7.2, 8.1, 8.1.1
>Reporter: Thomas Wöckinger
>Priority: Critical
> Attachments: SOLR-13539.patch
>
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> When using JavaBinCodec the values of collections are of type 
> ByteArrayUtf8CharSequence, existing field values are Strings so the remove 
> Operation does not have any effect.
>  This is related to following field types: UUID, Enums, Bool and Binary



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] thomaswoeckinger commented on issue #755: SOLR-13592: Introduce EmbeddedSolrTestBase for better integration tests

[GitBox]

thomaswoeckinger commented on issue #755: SOLR-13592: Introduce 
EmbeddedSolrTestBase for better integration tests
URL: https://github.com/apache/lucene-solr/pull/755#issuecomment-520341697
 
 
   > My apologies Thomas, my gmail filters weren't letting through github 
notifications and I didn't see this until you pinged me in JIRA.
   > 
   
   :-), no problem, i am still interest to get this in!
   Do you know if there are any plans on the master branch migrating to junit 
5, this would allow to write tests using default interfaces, so it would be 
very easy to let them run under different scenarios like embedded, jetty, cloud 
and so on.
   
   > This all looks good to me, except for the whitespace formatting, which 
we've talked about on other PRs. Github makes it easy to hide 
introduction/removal of trailing whitespace, but other changes are harder to 
hide and make the diff noisier to read and harder to review with confidence: 
adding "final" modifiers to existing variables, breaking up long method 
signatures onto multiple lines, etc. Would you mind scrubbing some of those 
changes from this PR?
   
   Yes we talked about, my point: why using common format rules if even no one 
is using them.
   But anyway, i can take a look at the method signatures, if you like, but i 
think it is not that difficult to review at the current state.

   > 
   > In the meantime, I'm going to start "beasting" (run {{ant test-help}} for 
info) this PR to make sure that none of the affected tests have any flaky 
failures. I'll comment again in a day or two with some results. (If you want to 
help out, you can also run some beast-runs if your machine allows it.)
   > 
   > Thanks for your continued patience.
   
   i will have a look at it, the default test are all running on my site, i 
will comment on this PR when i have results.
   
   And you are very welcome with your comments and suggestions!
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312811000
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/metrics/SolrMetricProducer.java
 ##
 @@ -19,17 +19,58 @@
 /**
  * Used by objects that expose metrics through {@link SolrCoreMetricManager}.
  */
-public interface SolrMetricProducer {
+public interface SolrMetricProducer extends AutoCloseable {
+
+  /**
+   * Unique metric name is in the format of A/B/C
 
 Review comment:
   This is purely aesthetic, but maybe instead of `/` we should use a `.` as in 
all other metric-related namespaces?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312813568
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/metrics/SolrMetricProducer.java
 ##
 @@ -19,17 +19,58 @@
 /**
  * Used by objects that expose metrics through {@link SolrCoreMetricManager}.
  */
-public interface SolrMetricProducer {
+public interface SolrMetricProducer extends AutoCloseable {
+
+  /**
+   * Unique metric name is in the format of A/B/C
+   * A is the parent of B is the parent of C and so on.
+   * If object "B" is unregistered , C also must get unregistered.
+   * If object "A" is unregistered ,  B , C also must get unregistered.
+   *
+   * @param parentName
+   */
+  default String getUniqueMetricTag(String parentName) {
+String name = getClass().getSimpleName() + "@" + 
Integer.toHexString(hashCode());
+return parentName == null ?
+name :
+parentName + "/" + name;
+  }
+
+  default void close() throws Exception{
+
+  }
+
 
   /**
* Initializes metrics specific to this producer
-   * @param manager an instance of {@link SolrMetricManager}
+   *
+   * @param manager  an instance of {@link SolrMetricManager}
* @param registry registry name where metrics are registered
-   * @param tag a symbolic tag that represents this instance of the producer,
-   * or a group of related instances that have the same life-cycle. This tag is
-   * used when managing life-cycle of some metrics and is set when
-   * {@link #initializeMetrics(SolrMetricManager, String, String, String)} is 
called.
-   * @param scope scope of the metrics (eg. handler name) to separate metrics 
of
+   * @param tag  a symbolic tag that represents this instance of the 
producer,
+   * or a group of related instances that have the same 
life-cycle. This tag is
+   * used when managing life-cycle of some metrics and is set 
when
+   * {@link #initializeMetrics(SolrMetricManager, String, 
String, String)} is called.
+   * @param scopescope of the metrics (eg. handler name) to separate 
metrics of
*/
   void initializeMetrics(SolrMetricManager manager, String registry, String 
tag, String scope);
+
+  class MetricsInfo {
 
 Review comment:
   I think the name `MetricsInfo` is too generic. This is more like a decorator 
for a tag, so maybe `MetricsTag` ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312812678
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/metrics/SolrMetricProducer.java
 ##
 @@ -19,17 +19,58 @@
 /**
  * Used by objects that expose metrics through {@link SolrCoreMetricManager}.
  */
-public interface SolrMetricProducer {
+public interface SolrMetricProducer extends AutoCloseable {
+
+  /**
+   * Unique metric name is in the format of A/B/C
+   * A is the parent of B is the parent of C and so on.
+   * If object "B" is unregistered , C also must get unregistered.
+   * If object "A" is unregistered ,  B , C also must get unregistered.
+   *
+   * @param parentName
+   */
+  default String getUniqueMetricTag(String parentName) {
+String name = getClass().getSimpleName() + "@" + 
Integer.toHexString(hashCode());
+return parentName == null ?
+name :
+parentName + "/" + name;
+  }
+
+  default void close() throws Exception{
+
+  }
+
 
   /**
* Initializes metrics specific to this producer
-   * @param manager an instance of {@link SolrMetricManager}
+   *
+   * @param manager  an instance of {@link SolrMetricManager}
* @param registry registry name where metrics are registered
-   * @param tag a symbolic tag that represents this instance of the producer,
-   * or a group of related instances that have the same life-cycle. This tag is
-   * used when managing life-cycle of some metrics and is set when
-   * {@link #initializeMetrics(SolrMetricManager, String, String, String)} is 
called.
-   * @param scope scope of the metrics (eg. handler name) to separate metrics 
of
+   * @param tag  a symbolic tag that represents this instance of the 
producer,
+   * or a group of related instances that have the same 
life-cycle. This tag is
+   * used when managing life-cycle of some metrics and is set 
when
+   * {@link #initializeMetrics(SolrMetricManager, String, 
String, String)} is called.
+   * @param scopescope of the metrics (eg. handler name) to separate 
metrics of
*/
   void initializeMetrics(SolrMetricManager manager, String registry, String 
tag, String scope);
+
+  class MetricsInfo {
+final String registry;
+final SolrMetricManager metricManager;
+final String myTag;
+
+public MetricsInfo(SolrMetricManager metricManager, String registry, 
String tag) {
+  this.registry = registry;
+  this.metricManager = metricManager;
+  this.myTag = tag;
+}
+
+public String getTag(){
 
 Review comment:
   Space before curly bracket.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312814903
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/search/LFUCache.java
 ##
 @@ -227,6 +227,7 @@ public void close() {
 statsList.get(0).add(cache.getStats());
 statsList.remove(cache.getStats());
 cache.destroy();
+
 
 Review comment:
   `metricsInfo.unregister()` ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #825: SOLR-13677 All Metrics Gauges should be unregistered by the objects that registered them

[GitBox]

sigram commented on a change in pull request #825: SOLR-13677 All Metrics 
Gauges should be unregistered by the objects that registered them
URL: https://github.com/apache/lucene-solr/pull/825#discussion_r312811895
 
 

 ##
 File path: solr/core/src/java/org/apache/solr/search/FastLRUCache.java
 ##
 @@ -227,6 +227,7 @@ public void warm(SolrIndexSearcher searcher, SolrCache 
old) {
 
   @Override
   public void close() {
+if(metricsInfo != null) metricsInfo.unregister();
 
 Review comment:
   IIRC Apache code style inserts a space between `if` and condition.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-8.x-Windows (64bit/jdk-11.0.3) - Build # 394 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/394/
Java: 64bit/jdk-11.0.3 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.search.facet.TestCloudJSONFacetSKG.testRandom

Error Message:
No live SolrServers available to handle this 
request:[https://127.0.0.1:52867/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[https://127.0.0.1:52867/solr/org.apache.solr.search.facet.TestCloudJSONFacetSKG_collection]
at 
__randomizedtesting.SeedInfo.seed([62262C56A362EE79:106A09591202580A]:0)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:345)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1128)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:897)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:829)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:987)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1002)
at 
org.apache.solr.search.facet.TestCloudJSONFacetSKG.getNumFound(TestCloudJSONFacetSKG.java:669)
at 
org.apache.solr.search.facet.TestCloudJSONFacetSKG.verifySKGResults(TestCloudJSONFacetSKG.java:446)
at 
org.apache.solr.search.facet.TestCloudJSONFacetSKG.assertFacetSKGsAreCorrect(TestCloudJSONFacetSKG.java:392)
at 
org.apache.solr.search.facet.TestCloudJSONFacetSKG.assertFacetSKGsAreCorrect(TestCloudJSONFacetSKG.java:402)
at 
org.apache.solr.search.facet.TestCloudJSONFacetSKG.assertFacetSKGsAreCorrect(TestCloudJSONFacetSKG.java:402)
at 
org.apache.solr.search.facet.TestCloudJSONFacetSKG.assertFacetSKGsAreCorrect(TestCloudJSONFacetSKG.java:349)
at 
org.apache.solr.search.facet.TestCloudJSONFacetSKG.testRandom(TestCloudJSONFacetSKG.java:274)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (LUCENE-8949) Allow LeafFieldComparators to publish feature values

[Atri Sharma (JIRA)]

Atri Sharma created LUCENE-8949:
---

 Summary: Allow LeafFieldComparators to publish feature values
 Key: LUCENE-8949
 URL: https://issues.apache.org/jira/browse/LUCENE-8949
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Atri Sharma


We allow LeafFieldComparators to only accept a docID, get the equivalent 
feature value(s) and compare against the bottom/top of the values set for the 
comparator. This mandates that the values being compared against the bottom/top 
should originate from the same comparator. This does not allow use cases such 
as cross comparator value comparisons i.e. if a user wanted to compute the 
"global" minimum across multiple comparators.

 

FieldComparators expose an API to get the feature value corresponding to a 
docID. We should let LeafFieldComparators do the same. A new comparison method 
is not required since the parent FieldComparator's compare method can be used 
once the values are retrieved.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org