[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-10) - Build # 7298 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7298/
Java: 64bit/jdk-10 -XX:-UseCompressedOops -XX:+UseParallelGC

40 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
at 
__randomizedtesting.SeedInfo.seed([D6FE1E4A2A572C6B:85475CFAC846B991]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:404)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.solr.handler.component.DistributedSpellCheckComponentTest.test

Error Message:
Error from server at http://127.0.0.1:54515/oy_rd/f/collection1: Directory 

[jira] [Commented] (SOLR-12202) failed to run solr-exporter.cmd on Windows platform

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460546#comment-16460546
 ] 

ASF subversion and git services commented on SOLR-12202:


Commit a12d34c4c3bbb72e9fcd794f8ddca5fd15f62504 in lucene-solr's branch 
refs/heads/branch_7x from koji
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a12d34c ]

SOLR-12202: Fix errors in solr-exporter.cmd


> failed to run solr-exporter.cmd on Windows platform
> ---
>
> Key: SOLR-12202
> URL: https://issues.apache.org/jira/browse/SOLR-12202
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 7.3
>Reporter: Minoru Osuka
>Assignee: Koji Sekiguchi
>Priority: Major
> Attachments: SOLR-12202.patch
>
>
> failed to run solr-exporter.cmd on Windows platform due to following:
> - incorrect main class name.
> - incorrect classpath specification.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-7.3 - Build # 55 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.3/55/

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ZkControllerTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [Overseer] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.cloud.Overseer  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.cloud.Overseer.start(Overseer.java:545)  at 
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:850)
  at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170) 
 at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)  
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)  at 
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)  at 
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2068)
  at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331)
  at java.lang.Thread.run(Thread.java:748)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [Overseer]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.cloud.Overseer
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.cloud.Overseer.start(Overseer.java:545)
at 
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:850)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
at 
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
at 
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2068)
at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331)
at java.lang.Thread.run(Thread.java:748)


at __randomizedtesting.SeedInfo.seed([72D1F10E6BDB34AF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:301)
at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ZkControllerTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ZkControllerTest: 
1) 

[jira] [Commented] (SOLR-12202) failed to run solr-exporter.cmd on Windows platform

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460517#comment-16460517
 ] 

ASF subversion and git services commented on SOLR-12202:


Commit ee2198d6bd12bed1b75ac7abbd0e99c80d5557af in lucene-solr's branch 
refs/heads/master from koji
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ee2198d ]

SOLR-12202: Fix errors in solr-exporter.cmd


> failed to run solr-exporter.cmd on Windows platform
> ---
>
> Key: SOLR-12202
> URL: https://issues.apache.org/jira/browse/SOLR-12202
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 7.3
>Reporter: Minoru Osuka
>Assignee: Koji Sekiguchi
>Priority: Major
> Attachments: SOLR-12202.patch
>
>
> failed to run solr-exporter.cmd on Windows platform due to following:
> - incorrect main class name.
> - incorrect classpath specification.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-BadApples-master-Linux (32bit/jdk1.8.0_162) - Build # 30 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/30/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaFromClusterState

Error Message:
Collection not found: deleteFromClusterState_false

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: 
deleteFromClusterState_false
at 
__randomizedtesting.SeedInfo.seed([FD10528A53B62648:1389F9E76C885DFF]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:853)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at 
org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaFromClusterState(DeleteReplicaTest.java:187)
at 
org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaFromClusterState(DeleteReplicaTest.java:178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_162) - Build # 1834 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1834/
Java: 64bit/jdk1.8.0_162 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

5 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
waitFor not elapsed but produced an event

Stack Trace:
java.lang.AssertionError: waitFor not elapsed but produced an event
at 
__randomizedtesting.SeedInfo.seed([DA0D2D8BC7D7BEBE:B9C61B095E18CD93]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish 

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1833 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1833/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testProperties

Error Message:
Error from server at http://127.0.0.1:42175/solr: Collection : collection1meta 
is part of alias meta1 remove or modify the alias before removing this 
collection.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:42175/solr: Collection : collection1meta is 
part of alias meta1 remove or modify the alias before removing this collection.
at 
__randomizedtesting.SeedInfo.seed([4ACC590718BB5E8B:3BD2A0D1E946635]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:451)
at 
org.apache.solr.cloud.AliasIntegrationTest.tearDown(AliasIntegrationTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Updated] (SOLR-11881) Connection Reset Causing LIR

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-11881:
-
Description: 
We can see that a connection reset is causing LIR.

If a leader -> replica update get's a connection like this the leader will 
initiate LIR
{code:java}
2018-01-08 17:39:16.980 ERROR (qtp1010030701-468988) [c:collection s:shardX 
r:core_node56 collection_shardX_replicaY] o.a.s.u.p.DistributedUpdateProcessor 
Setting up to try to start recovery on replica 
https://host08.domain:8985/solr/collection_shardX_replicaY/
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at 
org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
at 
org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
at 
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at 
org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:312)
at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
>From https://issues.apache.org/jira/browse/SOLR-6931 Mark says "On a heavy 
>working SolrCloud cluster, even a rare response like this from a replica can 
>cause a recovery and heavy cluster disruption" .

Looking at SOLR-6931 we added a http retry handler but we only retry on GET 
requests. Updates are POST requests 
{{ConcurrentUpdateSolrClient#sendUpdateStream}}

Update requests between the leader and replica should be retry-able since they 
have been versioned.

  was:
We can see that a connection reset is causing LIR.


If a leader -> replica update get's a connection like this the leader will 
initiate LIR

{code}
2018-01-08 17:39:16.980 ERROR (qtp1010030701-468988) [c:person s:shard7_1 
r:core_node56 x:person_shard7_1_replica1] o.a.s.u.p.DistributedUpdateProcessor 
Setting up to try to start recovery on replica 
https://host08.domain:8985/solr/person_shard7_1_replica2/
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at 
org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
at 
org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
at 
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at 
org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at 

[jira] [Commented] (LUCENE-7960) NGram filters -- add option to keep short terms

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460312#comment-16460312
 ] 

Shawn Heisey commented on LUCENE-7960:
--

Updated patch added.  Deprecates the existing 3-arg constructors, and removes 
all usage of the deprecated constructors from the codebase.  Tests in 
lucene/analysis and precommit at the root are passing.


> NGram filters -- add option to keep short terms
> ---
>
> Key: LUCENE-7960
> URL: https://issues.apache.org/jira/browse/LUCENE-7960
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Shawn Heisey
>Priority: Major
> Attachments: LUCENE-7960.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When ngram or edgengram filters are used, any terms that are shorter than the 
> minGramSize are completely removed from the token stream.
> This is probably 100% what was intended, but I've seen it cause a lot of 
> problems for users.  I am not suggesting that the default behavior be 
> changed.  That would be far too disruptive to the existing user base.
> I do think there should be a new boolean option, with a name like 
> keepShortTerms, that defaults to false, to allow the short terms to be 
> preserved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7960) NGram filters -- add option to keep short terms

[Shawn Heisey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shawn Heisey updated LUCENE-7960:
-
Attachment: LUCENE-7960.patch

> NGram filters -- add option to keep short terms
> ---
>
> Key: LUCENE-7960
> URL: https://issues.apache.org/jira/browse/LUCENE-7960
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Shawn Heisey
>Priority: Major
> Attachments: LUCENE-7960.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When ngram or edgengram filters are used, any terms that are shorter than the 
> minGramSize are completely removed from the token stream.
> This is probably 100% what was intended, but I've seen it cause a lot of 
> problems for users.  I am not suggesting that the default behavior be 
> changed.  That would be far too disruptive to the existing user base.
> I do think there should be a new boolean option, with a name like 
> keepShortTerms, that defaults to false, to allow the short terms to be 
> preserved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7960) NGram filters -- add option to keep short terms

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460304#comment-16460304
 ] 

Shawn Heisey commented on LUCENE-7960:
--

Applying the PR as-is does seem to work.  All the tests are passing.  I'm 
working on some minor alterations.  I've got precommit running, so far it looks 
good.

> NGram filters -- add option to keep short terms
> ---
>
> Key: LUCENE-7960
> URL: https://issues.apache.org/jira/browse/LUCENE-7960
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Shawn Heisey
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When ngram or edgengram filters are used, any terms that are shorter than the 
> minGramSize are completely removed from the token stream.
> This is probably 100% what was intended, but I've seen it cause a lot of 
> problems for users.  I am not suggesting that the default behavior be 
> changed.  That would be far too disruptive to the existing user base.
> I do think there should be a new boolean option, with a name like 
> keepShortTerms, that defaults to false, to allow the short terms to be 
> preserved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11881) Connection Reset Causing LIR

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-11881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460289#comment-16460289
 ] 

Tomás Fernández Löbbe commented on SOLR-11881:
--

Attached a rough patch that handles the retry in SolrCmdDistributor:
* I only added retry to requests from leader to it's replicas.
* Didn't add any tests yet, I've been running the ChaosMonkey to see how the 
retries behave
* I change the retry exception from only {{ConnectException}} to 
{{SocketException}} or {{NoHttpResponseException}}
* I plan to reduce the number of retries for this case (25 sounds like a lot, I 
was thinking of 5 or 10 max, but I'm open to suggestions)

[~varunthacker], [~markrmil...@gmail.com] let me know what you think

> Connection Reset Causing LIR
> 
>
> Key: SOLR-11881
> URL: https://issues.apache.org/jira/browse/SOLR-11881
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-11881-SolrCmdDistributor.patch, SOLR-11881.patch, 
> SOLR-11881.patch
>
>
> We can see that a connection reset is causing LIR.
> If a leader -> replica update get's a connection like this the leader will 
> initiate LIR
> {code}
> 2018-01-08 17:39:16.980 ERROR (qtp1010030701-468988) [c:person s:shard7_1 
> r:core_node56 x:person_shard7_1_replica1] 
> o.a.s.u.p.DistributedUpdateProcessor Setting up to try to start recovery on 
> replica https://host08.domain:8985/solr/person_shard7_1_replica2/
> java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
> at sun.security.ssl.InputRecord.read(InputRecord.java:503)
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
> at 
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
> at 
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
> at 
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
> at 
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
> at 
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
> at 
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
> at 
> org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:312)
> at 
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185)
> at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> From https://issues.apache.org/jira/browse/SOLR-6931 Mark says "On a heavy 
> working SolrCloud cluster, even a rare response like this from a replica can 
> cause a recovery and heavy cluster disruption" . 
> Looking at SOLR-6931 we added a http retry handler but we only retry on GET 
> requests. Updates are POST requests 
> {{ConcurrentUpdateSolrClient#sendUpdateStream}}
> Update requests between the leader and replica should be retry-able since 
> they have been versioned. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11881) Connection Reset Causing LIR

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-11881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe updated SOLR-11881:
-
Attachment: SOLR-11881-SolrCmdDistributor.patch

> Connection Reset Causing LIR
> 
>
> Key: SOLR-11881
> URL: https://issues.apache.org/jira/browse/SOLR-11881
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-11881-SolrCmdDistributor.patch, SOLR-11881.patch, 
> SOLR-11881.patch
>
>
> We can see that a connection reset is causing LIR.
> If a leader -> replica update get's a connection like this the leader will 
> initiate LIR
> {code}
> 2018-01-08 17:39:16.980 ERROR (qtp1010030701-468988) [c:person s:shard7_1 
> r:core_node56 x:person_shard7_1_replica1] 
> o.a.s.u.p.DistributedUpdateProcessor Setting up to try to start recovery on 
> replica https://host08.domain:8985/solr/person_shard7_1_replica2/
> java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
> at sun.security.ssl.InputRecord.read(InputRecord.java:503)
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
> at 
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
> at 
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
> at 
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
> at 
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
> at 
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
> at 
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
> at 
> org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:312)
> at 
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185)
> at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> From https://issues.apache.org/jira/browse/SOLR-6931 Mark says "On a heavy 
> working SolrCloud cluster, even a rare response like this from a replica can 
> cause a recovery and heavy cluster disruption" . 
> Looking at SOLR-6931 we added a http retry handler but we only retry on GET 
> requests. Updates are POST requests 
> {{ConcurrentUpdateSolrClient#sendUpdateStream}}
> Update requests between the leader and replica should be retry-able since 
> they have been versioned. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7960) NGram filters -- add option to keep short terms

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460226#comment-16460226
 ] 

Shawn Heisey commented on LUCENE-7960:
--

I have basically come to the conclusion that I have absolutely no idea how this 
stuff works and cannot make any sense out of what the patch does, or even what 
the classes are doing *before* the modifications.

> NGram filters -- add option to keep short terms
> ---
>
> Key: LUCENE-7960
> URL: https://issues.apache.org/jira/browse/LUCENE-7960
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Shawn Heisey
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When ngram or edgengram filters are used, any terms that are shorter than the 
> minGramSize are completely removed from the token stream.
> This is probably 100% what was intended, but I've seen it cause a lot of 
> problems for users.  I am not suggesting that the default behavior be 
> changed.  That would be far too disruptive to the existing user base.
> I do think there should be a new boolean option, with a name like 
> keepShortTerms, that defaults to false, to allow the short terms to be 
> preserved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_162) - Build # 1833 - Unstable!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8288) ContextQuery "." for RegexCompletionQuery produces an assertion failure

[Julie Tibshirani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julie Tibshirani updated LUCENE-8288:
-
Attachment: LUCENE-8288-repro.patch

> ContextQuery "." for RegexCompletionQuery produces an assertion failure
> ---
>
> Key: LUCENE-8288
> URL: https://issues.apache.org/jira/browse/LUCENE-8288
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Julie Tibshirani
>Priority: Major
> Attachments: LUCENE-8288-repro.patch
>
>
> When a RegexCompletionQuery of "." is provided to ContextQuery, the following 
> assertion failure occurs:
> {code:java}
> java.lang.AssertionError: input should not end with a context separator 
> followed by SEP_LABEL
> at 
> org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setInnerWeight(ContextQuery.java:299)
> at 
> org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setNextMatch(ContextQuery.java:275)
> at 
> org.apache.lucene.search.suggest.document.NRTSuggester.lookup(NRTSuggester.java:221)
> at 
> org.apache.lucene.search.suggest.document.CompletionScorer.score(CompletionScorer.java:70)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at 
> org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:78)
> at 
> org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:58)
> at 
> org.apache.lucene.search.suggest.document.TestContextQuery.testDotRegexQuery(TestContextQuery.java:188)
> {code}
> Note that this is a related, but distinct issue from 
> https://issues.apache.org/jira/browse/LUCENE-8287, where the 
> RegexCompletionQuery is empty.
> The attached patch provides a reproduction of the issue, as the test case 
> TestContextQuery#testRegexDotQuery. To reproduce, Java assertions must be 
> enabled (as in the default configuration for tests).
> The patch also provides a test case for the normal behavior of an empty 
> RegexCompletionQuery, when it is not wrapped in ContextQuery 
> (TestRegexCompletionQuery#testRegexDotQuery). In this case, there is no 
> error, and all suggestions are returned.
> From a quick look, it seems as though "." doesn't capture any characters past 
>  CompletionAnalyzer.SEP_LABEL, so the matching prefix in 
> ContextCompletionWeight#setInnerWeight is unexpectedly empty.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8288) ContextQuery "." for RegexCompletionQuery produces an assertion failure

[Julie Tibshirani (JIRA)]

Julie Tibshirani created LUCENE-8288:


 Summary: ContextQuery "." for RegexCompletionQuery produces an 
assertion failure
 Key: LUCENE-8288
 URL: https://issues.apache.org/jira/browse/LUCENE-8288
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Julie Tibshirani


When a RegexCompletionQuery of "." is provided to ContextQuery, the following 
assertion failure occurs:
{code:java}
java.lang.AssertionError: input should not end with a context separator 
followed by SEP_LABEL

at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setInnerWeight(ContextQuery.java:299)
at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setNextMatch(ContextQuery.java:275)
at 
org.apache.lucene.search.suggest.document.NRTSuggester.lookup(NRTSuggester.java:221)
at 
org.apache.lucene.search.suggest.document.CompletionScorer.score(CompletionScorer.java:70)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:78)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:58)
at 
org.apache.lucene.search.suggest.document.TestContextQuery.testDotRegexQuery(TestContextQuery.java:188)

{code}
Note that this is a related, but distinct issue from 
https://issues.apache.org/jira/browse/LUCENE-8287, where the 
RegexCompletionQuery is empty.

The attached patch provides a reproduction of the issue, as the test case 
TestContextQuery#testRegexDotQuery. To reproduce, Java assertions must be 
enabled (as in the default configuration for tests).

The patch also provides a test case for the normal behavior of an empty 
RegexCompletionQuery, when it is not wrapped in ContextQuery 
(TestRegexCompletionQuery#testRegexDotQuery). In this case, there is no error, 
and all suggestions are returned.

>From a quick look, it seems as though "." doesn't capture any characters past  
>CompletionAnalyzer.SEP_LABEL, so the matching prefix in 
>ContextCompletionWeight#setInnerWeight is unexpectedly empty.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4599 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4599/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

4 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at 
__randomizedtesting.SeedInfo.seed([8629FB0453B12E32:E5E2CD86CA7E5D1F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:187)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
last state: 

[jira] [Updated] (LUCENE-8287) ContextQuery with empty RegexCompletionQuery produces an assertion failure

[Julie Tibshirani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julie Tibshirani updated LUCENE-8287:
-
Description: 
When an empty RegexCompletionQuery is provided to ContextQuery, the following 
assertion failure occurs:
{code:java}
java.lang.AssertionError: input should not end with the context separator
at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setInnerWeight(ContextQuery.java:296)
at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setNextMatch(ContextQuery.java:275)
at 
org.apache.lucene.search.suggest.document.NRTSuggester.lookup(NRTSuggester.java:221)
at 
org.apache.lucene.search.suggest.document.CompletionScorer.score(CompletionScorer.java:70)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:78)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:58)
at 
org.apache.lucene.search.suggest.document.TestContextQuery.testEmptyRegexQuery(TestContextQuery.java:193)
{code}
This is a bit of an edge case, but may be concerning since without assertions 
enabled, you can go on to access IntsRef indices that are out of bounds.

The attached patch provides a reproduction of the issue, as the test case 
TestContextQuery#testEmptyRegexQuery. Note that to reproduce, Java assertions 
must be enabled (as in the default configuration for tests).

The patch also provides a test case for the normal behavior of an empty 
RegexCompletionQuery, when it is not wrapped in ContextQuery 
(TestRegexCompletionQuery#testEmptyRegexQuery). In this case, there is no 
error, and all suggestions are returned. 

  was:
When an empty RegexCompletionQuery is provided to ContextQuery, the following 
assertion failure occurs:
{code:java}
java.lang.AssertionError: input should not end with the context separator
at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setInnerWeight(ContextQuery.java:296)
at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setNextMatch(ContextQuery.java:275)
at 
org.apache.lucene.search.suggest.document.NRTSuggester.lookup(NRTSuggester.java:221)
at 
org.apache.lucene.search.suggest.document.CompletionScorer.score(CompletionScorer.java:70)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:78)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:58)
at 
org.apache.lucene.search.suggest.document.TestContextQuery.testEmptyRegexQuery(TestContextQuery.java:193)
{code}
This is a bit of an edge case, but may be concerning as without assertions 
enabled, you can go on to access IntsRef indices that are out of bounds.

The attached patch provides a reproduction of the issue, as the test case 
TestContextQuery#testEmptyRegexQuery. Note that to reproduce, Java assertions 
must be enabled (as in the default configuration for tests).

The patch also provides a test case for the normal behavior of an empty 
RegexCompletionQuery, when it is not wrapped in ContextQuery 
(TestRegexCompletionQuery#testEmptyRegexQuery). In this case, there is no 
error, and all suggestions are returned. 


> ContextQuery with empty RegexCompletionQuery produces an assertion failure
> --
>
> Key: LUCENE-8287
> URL: https://issues.apache.org/jira/browse/LUCENE-8287
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Julie Tibshirani
>Priority: Major
> Attachments: LUCENE-8287-repro.patch
>
>
> When an empty RegexCompletionQuery is provided to ContextQuery, the following 
> assertion failure occurs:
> {code:java}
> java.lang.AssertionError: input should not end with the context separator
> at 
> org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setInnerWeight(ContextQuery.java:296)
> at 
> org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setNextMatch(ContextQuery.java:275)
> at 
> org.apache.lucene.search.suggest.document.NRTSuggester.lookup(NRTSuggester.java:221)
> at 
> org.apache.lucene.search.suggest.document.CompletionScorer.score(CompletionScorer.java:70)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at 
> org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:78)
> at 
> org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:58)
> at 
> org.apache.lucene.search.suggest.document.TestContextQuery.testEmptyRegexQuery(TestContextQuery.java:193)
> {code}
> This is a bit 

Need eyes on LUCENE-7960

[Shawn Heisey]

Can someone who groks analysis code look at the PR for
https://issues.apache.org/jira/browse/LUCENE-7960 ?  I suspect that the
patch is making changes beyond what's required to implement the issue,
but my understanding of the code is too limited to know.

I already know that it should be adding a constructor, not changing it. 
Secondary question is whether the current three-arg constructor on these
classes should be deprecated or kept.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8287) ContextQuery with empty RegexCompletionQuery produces an assertion failure

[Julie Tibshirani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julie Tibshirani updated LUCENE-8287:
-
Attachment: LUCENE-8287-repro.patch

> ContextQuery with empty RegexCompletionQuery produces an assertion failure
> --
>
> Key: LUCENE-8287
> URL: https://issues.apache.org/jira/browse/LUCENE-8287
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Julie Tibshirani
>Priority: Major
> Attachments: LUCENE-8287-repro.patch
>
>
> When an empty RegexCompletionQuery is provided to ContextQuery, the following 
> assertion failure occurs:
> {code:java}
> java.lang.AssertionError: input should not end with the context separator
> at 
> org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setInnerWeight(ContextQuery.java:296)
> at 
> org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setNextMatch(ContextQuery.java:275)
> at 
> org.apache.lucene.search.suggest.document.NRTSuggester.lookup(NRTSuggester.java:221)
> at 
> org.apache.lucene.search.suggest.document.CompletionScorer.score(CompletionScorer.java:70)
> at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
> at 
> org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:78)
> at 
> org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:58)
> at 
> org.apache.lucene.search.suggest.document.TestContextQuery.testEmptyRegexQuery(TestContextQuery.java:193)
> {code}
> This is a bit of an edge case, but may be concerning as without assertions 
> enabled, you can go on to access IntsRef indices that are out of bounds.
> The attached patch provides a reproduction of the issue, as the test case 
> TestContextQuery#testEmptyRegexQuery. Note that to reproduce, Java assertions 
> must be enabled (as in the default configuration for tests).
> The patch also provides a test case for the normal behavior of an empty 
> RegexCompletionQuery, when it is not wrapped in ContextQuery 
> (TestRegexCompletionQuery#testEmptyRegexQuery). In this case, there is no 
> error, and all suggestions are returned. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8287) ContextQuery with empty RegexCompletionQuery produces an assertion failure

[Julie Tibshirani (JIRA)]

Julie Tibshirani created LUCENE-8287:


 Summary: ContextQuery with empty RegexCompletionQuery produces an 
assertion failure
 Key: LUCENE-8287
 URL: https://issues.apache.org/jira/browse/LUCENE-8287
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Julie Tibshirani


When an empty RegexCompletionQuery is provided to ContextQuery, the following 
assertion failure occurs:
{code:java}
java.lang.AssertionError: input should not end with the context separator
at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setInnerWeight(ContextQuery.java:296)
at 
org.apache.lucene.search.suggest.document.ContextQuery$ContextCompletionWeight.setNextMatch(ContextQuery.java:275)
at 
org.apache.lucene.search.suggest.document.NRTSuggester.lookup(NRTSuggester.java:221)
at 
org.apache.lucene.search.suggest.document.CompletionScorer.score(CompletionScorer.java:70)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:78)
at 
org.apache.lucene.search.suggest.document.SuggestIndexSearcher.suggest(SuggestIndexSearcher.java:58)
at 
org.apache.lucene.search.suggest.document.TestContextQuery.testEmptyRegexQuery(TestContextQuery.java:193)
{code}
This is a bit of an edge case, but may be concerning as without assertions 
enabled, you can go on to access IntsRef indices that are out of bounds.

The attached patch provides a reproduction of the issue, as the test case 
TestContextQuery#testEmptyRegexQuery. Note that to reproduce, Java assertions 
must be enabled (as in the default configuration for tests).

The patch also provides a test case for the normal behavior of an empty 
RegexCompletionQuery, when it is not wrapped in ContextQuery 
(TestRegexCompletionQuery#testEmptyRegexQuery). In this case, there is no 
error, and all suggestions are returned. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12297) Create a good SolrClient for SolrCloud.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460140#comment-16460140
 ] 

Mark Miller edited comment on SOLR-12297 at 5/1/18 9:07 PM:


My plan for this would be done in stages:

*Stage1*: Add this new Http2SolrClient. It would initially be used with Http1.1 
and serve as a better alternative to both ConcurrentUpdateSolrClient (other 
than for very special use cases) and HttpSolrClient. It would offer non 
blocking IO and async request options. It may not offer every feature of 
HttpSolrClient (mostly non basic auth security).

*Stage2*: Over time, Http2SolrClient offers every feature of HttpSolrClient.

*Stage3*: We replace internal usage of HttpSolrClient with Http2SolrClient. Now 
we can freely explore async options or changes over time.

*Stage4*: We wait for a major version to switch to Http2 from Http1.1, or we 
offer Http2 as an option, or use two connectors and offer Http2 on another port.

*Stage5*: We can explore taking advantage of other Http2 options that we don't 
get for free as we do multiplexing.

*Stage6*: We consider removing HttpSolrClient and ConcurrentUpdateSolrClient in 
future versions of Solr.


was (Author: markrmil...@gmail.com):
My plan for this would be done in stages:

Stage1: Add this new Http2SolrClient. It would initially be used with Http1.1 
and serve as a better alternative to ConcurrentUpdateSolrClient (other than for 
very special use cases) and HttpSolrClient. It would offer non blocking IO and 
async request options. It may not offer every feature of HttpSolrClient (mostly 
non basic auth security).

Stage2: Over time, Http2SolrClient offers every feature of HttpSolrClient.

Stage3: We replace internal usage of HttpSolrClient with Http2SolrClient. Now 
we can freely explore async options or changes over time.

Stage4: We wait for a major version to switch to Http2 from Http1.1, or we 
offer Http2 as an option, or use two connectors and offer Http2 on another port.

Stage5: We can explore taking advantage of other Http2 options that we don't 
get for free, like multiplexing.

> Create a good SolrClient for SolrCloud.
> ---
>
> Key: SOLR-12297
> URL: https://issues.apache.org/jira/browse/SOLR-12297
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> Blocking or async support as well as HTTP2 compatible with multiplexing.
> Once it supports enough and is stable, replace internal usage, allowing 
> async, and eventually move to HTTP2 connector and allow multiplexing. Could 
> support HTTP1.1 and HTTP2 on different ports depending on state of the world 
> then.
> The goal of the client itself is to work against HTTP1.1 or HTTP2 with 
> minimal or no code path differences and the same for async requests (should 
> initially work for both 1.1 and 2 and share majority of code).
> The client should also be able to replace HttpSolrClient and plug into the 
> other clients the same way.
> I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually 
> though.
> I evaluated some clients and while there are a few options, I went with 
> Jetty's HttpClient. It's more mature than Apache HttpClient's support (in 5 
> beta) and we would have to update to a new API for Apache HttpClient anyway.
> Meanwhile, the Jetty guys have been very supportive of helping Solr with any 
> issues and I like having the client and server from the same project.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8274) Add per-request MDC logging based on user-provided value.

[Otis Gospodnetic (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460158#comment-16460158
 ] 

Otis Gospodnetic commented on SOLR-8274:


Perhaps a more modern way to approach this is to instrument Solr.  OpenTracing 
comes to mind.  See 
[https://sematext.com/blog/opentracing-distributed-tracing-emerging-industry-standard/]
 for a quick overview.  See also [https://github.com/opentracing-contrib] 

> Add per-request MDC logging based on user-provided value.
> -
>
> Key: SOLR-8274
> URL: https://issues.apache.org/jira/browse/SOLR-8274
> Project: Solr
>  Issue Type: Improvement
>  Components: logging
>Reporter: Jason Gerlowski
>Priority: Minor
> Attachments: SOLR-8274.patch
>
>
> *Problem 1* Currently, there's no way (AFAIK) to find all log messages 
> associated with a particular request.
> *Problem 2* There's also no easy way for multi-tenant Solr setups to find all 
> log messages associated with a particular customer/tenant.
> Both of these problems would be more manageable if Solr could be configured 
> to record an MDC tag based on a header, or some other user provided value.
> This would allow admins to group together logs about a single request.  If 
> the same header value is repeated multiple times this functionality could 
> also be used to group together arbitrary requests, such as those that come 
> from a particular user, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12290) Do not close any servlet streams and improve our servlet stream closing prevention code for users and devs.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460143#comment-16460143
 ] 

Mark Miller commented on SOLR-12290:


For a long term strategy to improve all of this yet again (of course all the 
previous HttpClient Http1.1 connection failure work was always a stop gap band 
aid and as we found out extremely hard to fully nail down), I filed SOLR-12297.

> Do not close any servlet streams and improve our servlet stream closing 
> prevention code for users and devs.
> ---
>
> Key: SOLR-12290
> URL: https://issues.apache.org/jira/browse/SOLR-12290
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12290.patch, SOLR-12290.patch, SOLR-12290.patch, 
> SOLR-12290.patch
>
>
> Original Summary:
> When you fetch a file for replication we close the request output stream 
> after writing the file which ruins the connection for reuse.
> We can't close response output streams, we need to reuse these connections. 
> If we do close them, clients are hit with connection problems when they try 
> and reuse the connection from their pool.
> New Summary:
> At some point the above was addressed during refactoring. We should remove 
> these neutered closes and review our close shield code.
> If you are here to track down why this is done:
> Connection reuse requires that we read all streams and do not close them - 
> instead the container itself must manage request and response streams. If we 
> allow them to be closed, not only do we lose some connection reuse, but we 
> can cause spurious client errors that can cause expensive recoveries for no 
> reason. The spec allows us to count on the container to manage streams. It's 
> our job simply to not close them and to always read them fully, from client 
> and server. 
> Java itself can help with always reading the streams fully up to some small 
> default amount of unread stream slack, but that is very dangerous to count 
> on, so we always manually eat up anything on the streams our normal logic 
> ends up not reading for whatever reason.
> We also cannot call abort without ruining the connection or sendError. These 
> should be options of very last resort (requiring a blood sacrifice) or when 
> shutting down.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12297) Create a good SolrClient for SolrCloud.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460140#comment-16460140
 ] 

Mark Miller commented on SOLR-12297:


My plan for this would be done in stages:

Stage1: Add this new Http2SolrClient. It would initially be used with Http1.1 
and serve as a better alternative to ConcurrentUpdateSolrClient (other than for 
very special use cases) and HttpSolrClient. It would offer non blocking IO and 
async request options. It may not offer every feature of HttpSolrClient (mostly 
non basic auth security).

Stage2: Over time, Http2SolrClient offers every feature of HttpSolrClient.

Stage3: We replace internal usage of HttpSolrClient with Http2SolrClient. Now 
we can freely explore async options or changes over time.

Stage4: We wait for a major version to switch to Http2 from Http1.1, or we 
offer Http2 as an option, or use two connectors and offer Http2 on another port.

Stage5: We can explore taking advantage of other Http2 options that we don't 
get for free, like multiplexing.

> Create a good SolrClient for SolrCloud.
> ---
>
> Key: SOLR-12297
> URL: https://issues.apache.org/jira/browse/SOLR-12297
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> Blocking or async support as well as HTTP2 compatible with multiplexing.
> Once it supports enough and is stable, replace internal usage, allowing 
> async, and eventually move to HTTP2 connector and allow multiplexing. Could 
> support HTTP1.1 and HTTP2 on different ports depending on state of the world 
> then.
> The goal of the client itself is to work against HTTP1.1 or HTTP2 with 
> minimal or no code path differences and the same for async requests (should 
> initially work for both 1.1 and 2 and share majority of code).
> The client should also be able to replace HttpSolrClient and plug into the 
> other clients the same way.
> I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually 
> though.
> I evaluated some clients and while there are a few options, I went with 
> Jetty's HttpClient. It's more mature than Apache HttpClient's support (in 5 
> beta) and we would have to update to a new API for Apache HttpClient anyway.
> Meanwhile, the Jetty guys have been very supportive of helping Solr with any 
> issues and I like having the client and server from the same project.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11990) Make it possible to co-locate replicas of multiple collections together in a node using policy

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460096#comment-16460096
 ] 

Shalin Shekhar Mangar commented on SOLR-11990:
--

I've attached another approach which does not modify the policy framework 
directly but instead modifies Suggesters and Collection APIs to maintain the 
co-location of two collections. This is still a work in progress and has many 
nocommits.

> Make it possible to co-locate replicas of multiple collections together in a 
> node using policy
> --
>
> Key: SOLR-11990
> URL: https://issues.apache.org/jira/browse/SOLR-11990
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-11990.patch, SOLR-11990.patch, SOLR-11990.patch
>
>
> It is necessary to co-locate replicas of different collection together in a 
> node when cross-collection joins are performed. The policy rules framework 
> should support this use-case.
> Example: Co-locate exactly 1 replica of collection A in each node where a 
> replica of collection B is present.
> {code}
> {"replica":">0", "collection":"A", "shard":"#EACH", "withCollection":"B"}
> {code}
> This requires changing create collection, create shard and add replica APIs 
> as well because we want a replica of collection A to be created first before 
> a replica of collection B is created so that join queries etc are always 
> possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11990) Make it possible to co-locate replicas of multiple collections together in a node using policy

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-11990:
-
Attachment: SOLR-11990.patch

> Make it possible to co-locate replicas of multiple collections together in a 
> node using policy
> --
>
> Key: SOLR-11990
> URL: https://issues.apache.org/jira/browse/SOLR-11990
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-11990.patch, SOLR-11990.patch, SOLR-11990.patch
>
>
> It is necessary to co-locate replicas of different collection together in a 
> node when cross-collection joins are performed. The policy rules framework 
> should support this use-case.
> Example: Co-locate exactly 1 replica of collection A in each node where a 
> replica of collection B is present.
> {code}
> {"replica":">0", "collection":"A", "shard":"#EACH", "withCollection":"B"}
> {code}
> This requires changing create collection, create shard and add replica APIs 
> as well because we want a replica of collection A to be created first before 
> a replica of collection B is created so that join queries etc are always 
> possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Windows (64bit/jdk-11-ea+5) - Build # 573 - Still Unstable!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Weekly BadApple report:

[Erick Erickson]

OK, won't BadApple any of the CreateRoutedAliasTest then.

On Tue, May 1, 2018 at 9:11 AM, David Smiley  wrote:
> I'm actively working on the Alias ones.  Some have identified root causes
> relating to eventual-consistency timings.  Since I don't have a proper fix
> yet, in the interim I'm tempted to add a temporary Thread.sleep(200) at
> certain places in pertinent tests.
>
> On Tue, May 1, 2018 at 12:59 AM Erick Erickson 
> wrote:
>>
>> In order to reduce some of the make-work, I collect failed tests Fri->Mon
>> so the BadApple'd tests on Thursday don't clutter things up.
>>
>> BadApple candidates: I'll BadApple these on Thursday unless there are
>> objections
>>
>>
>> org.apache.solr.client.solrj.io.stream.MathExpressionTest.testGammaDistribution
>>org.apache.solr.cloud.AddReplicaTest.test
>>
>> org.apache.solr.cloud.CreateRoutedAliasTest.testCollectionNamesMustBeAbsent
>>org.apache.solr.cloud.CreateRoutedAliasTest.testTimezoneAbsoluteDate
>>org.apache.solr.cloud.CreateRoutedAliasTest.testV1
>>org.apache.solr.cloud.CreateRoutedAliasTest.testV2
>>org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly
>>
>> org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderAndMixedReplicas
>>org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderOldReplica
>>
>> org.apache.solr.cloud.LIRRollingUpdatesTest.testOldLeaderAndMixedReplicas
>>
>> org.apache.solr.cloud.LeaderElectionIntegrationTest.testSimpleSliceLeaderElection
>>org.apache.solr.cloud.OverseerRolesTest.testOverseerRole
>>
>> org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection
>>org.apache.solr.cloud.TestPullReplica.testKillLeader
>>org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test
>>org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLost
>>org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.testRestoreState
>>
>> org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication
>>org.apache.solr.schema.TestCloudSchemaless.test
>>org.apache.solr.search.TestStressRecovery.testStressRecovery
>>org.apache.solr.update.TestInPlaceUpdatesDistrib.test
>>
>>
>> I believe these are being worked on, so I will _not_ BadApple them
>> even though they fail.
>> What about the other autoscaling tests?
>>
>>
>> org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration
>>
>> org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
>>org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
>>
>>
>> Number of AwaitsFix: 21 Number of BadApples: 90
>>
>> *AwaitsFix Annotations:
>>
>>
>> Lucene AwaitsFix
>> GeoPolygonTest.java
>>testLUCENE8276_case3()
>>
>> //@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8276;)
>>
>> GeoPolygonTest.java
>>testLUCENE8280()
>>
>> //@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8280;)
>>
>> GeoPolygonTest.java
>>testLUCENE8281()
>>
>> //@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8281;)
>>
>> RandomGeoPolygonTest.java
>>testCompareBigPolygons()
>>
>> //@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8281;)
>>
>> RandomGeoPolygonTest.java
>>testCompareSmallPolygons()
>>
>> //@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8281;)
>>
>> TestControlledRealTimeReopenThread.java
>>testCRTReopen()
>>@AwaitsFix(bugUrl =
>> "https://issues.apache.org/jira/browse/LUCENE-5737;)
>>
>> TestICUNormalizer2CharFilter.java
>>testRandomStrings()
>>@AwaitsFix(bugUrl =
>> "https://issues.apache.org/jira/browse/LUCENE-5595;)
>>
>> TestICUTokenizerCJK.java
>>TestICUTokenizerCJK suite
>>@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8222;)
>>
>> TestMoreLikeThis.java
>>testMultiFieldShouldReturnPerFieldBooleanQuery()
>>@AwaitsFix(bugUrl =
>> "https://issues.apache.org/jira/browse/LUCENE-7161;)
>>
>> UIMABaseAnalyzerTest.java
>>testRandomStrings()
>>@Test @AwaitsFix(bugUrl =
>> "https://issues.apache.org/jira/browse/LUCENE-3869;)
>>
>> UIMABaseAnalyzerTest.java
>>testRandomStringsWithConfigurationParameters()
>>@Test @AwaitsFix(bugUrl =
>> "https://issues.apache.org/jira/browse/LUCENE-3869;)
>>
>> UIMATypeAwareAnalyzerTest.java
>>testRandomStrings()
>>@Test @AwaitsFix(bugUrl =
>> "https://issues.apache.org/jira/browse/LUCENE-3869;)
>>
>>
>> Solr AwaitsFix
>> ReplaceNodeNoTargetTest.java
>>ReplaceNodeNoTargetTest suite
>>@LuceneTestCase.AwaitsFix(bugUrl =
>> "https://issues.apache.org/jira/browse/SOLR-11067;)
>>
>> TestCollapseQParserPlugin.java
>>testStringCollapse()
>>@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974;)
>>
>> TestImpersonationWithHadoopAuth.java
>>testForwarding()
>>@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893;)
>>

Re: SolrCloud test fails RE "Can't find resource"

[Erick Erickson]

NP, I've been swamped lately anyway. If you do see anything you want
me to beast, just give me their names. It's one of those things that
takes 5 minutes to set up then a long time doing nothing.

Let me know if/when you want me to give it a whirl

Erick

On Tue, May 1, 2018 at 11:02 AM, David Smiley  wrote:
> (sorry for my belated response)
> Beasting a zillion times sounds like a decent option.  First I want to
> compile a list to see if there are any apparent patterns though.
>
> On Fri, Apr 27, 2018 at 11:16 AM Erick Erickson 
> wrote:
>>
>> Do you think making a test that just started up a
>> MiniSolrCloudCluster, created a collection then quit and then beasting
>> it a zillion times might reproduce? I can help beast such a thing if
>> it was available.
>>
>> I think there are a few root causes that are sporadically showing up
>> here and there, any we can track down will help enormously in reducing
>> noise.
>>
>> Erick
>>
>> On Fri, Apr 27, 2018 at 6:34 AM, David Smiley 
>> wrote:
>> > Just thinking out loud here about something spooky...
>> >
>> > I've noticed some relatively rare and seemingly random SolrCloud tests
>> > that
>> > fail to create a collection because an expected configSet resource isn't
>> > found (isn't in Zookeeper where it ought to be).  The initial exception
>> > will
>> > have "Can't find resource" then the name of the resource -- sometimes
>> > it's
>> > solrconfig.xml or other times some language specific stop-word file or
>> > something.  This happens when a collection is being created, but fails
>> > because a core/replica won't load because the configSet is faulty
>> > because a
>> > required file isn't there.  I do *not* believe this is some sort of
>> > visibility race of the configSet wherein a sleep/wait before creating
>> > the
>> > collection would help, because I've seen an entire test suite (test
>> > class
>> > file) of many tests (test methods) all fail for the same reason. In that
>> > case the MiniSolrCloudCluster was simply initialized in beforeClass and
>> > thus
>> > ought to have "_default" ready to be used before the test methods, yet
>> > every
>> > test method failed for the same reason.   This is very strange; the code
>> > that uploads the _default configSet seems sound.  Yet there's some
>> > sporadic
>> > bug so I guess somewhere there's either some race or there's an
>> > underlying
>> > upload failure that is being ignored.
>> >
>> > I think I ought to compile a list of tests that have shown this error
>> > and
>> > the dates(s) / build info in which it occurred.  Maybe there's a pattern
>> > /
>> > something they have in common.
>> > --
>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>> > http://www.solrenterprisesearchserver.com
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10243) Fix TestExtractionDateUtil.testParseDate sporadic failures

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460037#comment-16460037
 ] 

Steve Rowe commented on SOLR-10243:
---

Another recent reproducing failure, from 
[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1822/]:

{noformat}
Checking out Revision e3a98171744e25f446c3c6d5df41a3f65a54737c 
(refs/remotes/origin/master)
[...]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestExtractionDateUtil -Dtests.method=testParseDate 
-Dtests.seed=79065E512B0EB1FA -Dtests.slow=true -Dtests.locale=sk 
-Dtests.timezone=America/Metlakatla -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.09s J1 | TestExtractionDateUtil.testParseDate <<<
   [junit4]> Throwable #1: java.lang.AssertionError: Incorrect parsed 
timestamp: 1226583351000 != 1226579751000 (Thu Nov 13 04:35:51 AKST 2008)
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([79065E512B0EB1FA:331F266450A7C64F]:0)
   [junit4]>at 
org.apache.solr.handler.extraction.TestExtractionDateUtil.assertParsedDate(TestExtractionDateUtil.java:59)
   [junit4]>at 
org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate(TestExtractionDateUtil.java:54)
   [junit4]>at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, 
docValues:{}, maxPointsInLeafNode=1611, maxMBSortInHeap=6.381790188035722, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@10525726),
 locale=sk, timezone=America/Metlakatla
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 
(64-bit)/cpus=3,threads=1,free=63548536,total=97320960
{noformat}


> Fix TestExtractionDateUtil.testParseDate sporadic failures
> --
>
> Key: SOLR-10243
> URL: https://issues.apache.org/jira/browse/SOLR-10243
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Priority: Major
>
> Jenkins test failure:
> {{ant test  -Dtestcase=TestExtractionDateUtil -Dtests.method=testParseDate 
> -Dtests.seed=B72AC4792F31F74B -Dtests.slow=true -Dtests.locale=lv 
> -Dtests.timezone=America/Metlakatla -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8}}   It reproduces on 6x for me but not master.
> I reviewed this briefly and there seems to be a locale assumption in the test.
> 1 tests failed.
> FAILED:  
> org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate
> Error Message:
> Incorrect parsed timestamp: 1226583351000 != 1226579751000 (Thu Nov 13 
> 04:35:51 AKST 2008)
> Stack Trace:
> java.lang.AssertionError: Incorrect parsed timestamp: 1226583351000 != 
> 1226579751000 (Thu Nov 13 04:35:51 AKST 2008)
> at 
> __randomizedtesting.SeedInfo.seed([B72AC4792F31F74B:FD33BC4C549880FE]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at 
> org.apache.solr.handler.extraction.TestExtractionDateUtil.assertParsedDate(TestExtractionDateUtil.java:59)
> at 
> org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate(TestExtractionDateUtil.java:54)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12302) Reproducing BasicDistributedZkTest failure

[Steve Rowe (JIRA)]

Steve Rowe created SOLR-12302:
-

 Summary: Reproducing BasicDistributedZkTest failure
 Key: SOLR-12302
 URL: https://issues.apache.org/jira/browse/SOLR-12302
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrCloud, Tests
Reporter: Steve Rowe


>From [https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/27], 
>reproduced 5/5 iterations:

{noformat}
Checking out Revision 1409ab8f84ab0949b1da095f03dc94d3b74db5cf 
(refs/remotes/origin/master)
[...]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=BasicDistributedZkTest -Dtests.method=test 
-Dtests.seed=AC56303341F3D845 -Dtests.multiplier=3 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=es-CO -Dtests.timezone=America/Cayman 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   45.6s J0 | BasicDistributedZkTest.test <<<
   [junit4]> Throwable #1: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:37571/_ya/c/collection1: Async exception during 
distributed update: Error from server at 
http://127.0.0.1:34927/_ya/c/collection1_shard1_replica_n43: Bad Request
   [junit4]> request: 
http://127.0.0.1:34927/_ya/c/collection1_shard1_replica_n43/update?update.chain=distrib-dup-test-chain-explicit=TOLEADER=http%3A%2F%2F127.0.0.1%3A37571%2F_ya%2Fc%2Fcollection1_shard2_replica_n41%2F=javabin=2
   [junit4]> Remote error message: Exception writing document id 46 to the 
index; possible analysis error.
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([AC56303341F3D845:24020FE9EF0FB5BD]:0)
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]>at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]>at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
   [junit4]>at 
org.apache.solr.BaseDistributedSearchTestCase.add(BaseDistributedSearchTestCase.java:507)
   [junit4]>at 
org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:684)
   [junit4]>at 
org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:378)
   [junit4]>at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
   [junit4]>at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
   [junit4]>at java.lang.Thread.run(Thread.java:748)
[...]
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{regex_dup_A_s=Lucene50(blocksize=128), 
regex_dup_B_s=PostingsFormat(name=Asserting), 
SubjectTerms_mfacet=PostingsFormat(name=Asserting), 
multiDefault=Lucene50(blocksize=128), genre_s=Lucene50(blocksize=128), 
author_t=Lucene50(blocksize=128), series_t=Lucene50(blocksize=128), 
rnd_b=PostingsFormat(name=LuceneVarGapFixedInterval), 
oddField_s=Lucene50(blocksize=128), a_t=PostingsFormat(name=Asserting), 
cat=PostingsFormat(name=Asserting), foo_b=Lucene50(blocksize=128), 
name=PostingsFormat(name=LuceneVarGapFixedInterval), 
inStock=Lucene50(blocksize=128), 
id=PostingsFormat(name=LuceneVarGapFixedInterval), 
text=Lucene50(blocksize=128)}, 
docValues:{other_tl1=DocValuesFormat(name=Lucene70), 
range_facet_l_dv=DocValuesFormat(name=Memory), 
n_l1=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Lucene70), 
n_td1=DocValuesFormat(name=Asserting), n_d1=DocValuesFormat(name=Lucene70), 
range_facet_l=DocValuesFormat(name=Lucene70), 
n_f1=DocValuesFormat(name=Asserting), n_tl1=DocValuesFormat(name=Asserting), 
n_tf1=DocValuesFormat(name=Lucene70), price=DocValuesFormat(name=Direct), 
sequence_i=DocValuesFormat(name=Direct), 
intDvoDefault=DocValuesFormat(name=Direct), 
timestamp=DocValuesFormat(name=Lucene70), 
foo_i=DocValuesFormat(name=Asserting), val_i=DocValuesFormat(name=Memory), 
n_dt1=DocValuesFormat(name=Asserting), a_i1=DocValuesFormat(name=Lucene70), 
n_ti1=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Lucene70), 
n_tdt1=DocValuesFormat(name=Lucene70), id_i1=DocValuesFormat(name=Asserting), 
foo_d=DocValuesFormat(name=Memory), 
range_facet_i_dv=DocValuesFormat(name=Lucene70), 
foo_f=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=752, 
maxMBSortInHeap=5.295098720355578, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@53d7cd2),
 locale=es-CO, timezone=America/Cayman
   

[jira] [Created] (SOLR-12301) Umbrella issue: paramaterize logging calls in Solr, use consistent naming conventions for the logger

[Erick Erickson (JIRA)]

Erick Erickson created SOLR-12301:
-

 Summary: Umbrella issue: paramaterize logging calls in Solr, use 
consistent naming conventions for the logger
 Key: SOLR-12301
 URL: https://issues.apache.org/jira/browse/SOLR-12301
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Erick Erickson
Assignee: Erick Erickson


See the discussion at SOLR-12286 for a lot of background, but the short form is 
that logging calls of the form

log.info("somehting" + "something");
and
log.info("soemthing {}", object.toString());

generate useless garbage even when logging at a more restrictive level (say 
WARN), whereas

log.info("somehting {}", "something");
and
log.infl("soemthing {}", object);

do not. The first form is something of a relic, and there are even some uses of 
the second.

So, let's tackle subsets of the source code as subordinate JIRAs





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7788) fail precommit on unparameterised log.trace messages

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460006#comment-16460006
 ] 

Erick Erickson commented on LUCENE-7788:


[~cpoerschke] As you've probably seen, I've been agitating for a thorough 
review of our usage of logging. I think some of us will be making some active 
progress along these lines, then be able to fail the precommit.

So I have several questions:

1> WDYT about failing all logging messages that aren't parameterised? Is there 
any reason _any_ logging message should not be parameterised?

2> Let's say we fix up one directory (solr/core for example). Can we turn on 
the precommit check on a per-directory basis?

3> Since we're going through the review in the first place we can regularize 
the names of the loggers to whatever we want. It looks like "log" is the least 
number of changes so it wins by default. WDYT about adding a precommit check 
for that too?

> fail precommit on unparameterised log.trace messages
> 
>
> Key: LUCENE-7788
> URL: https://issues.apache.org/jira/browse/LUCENE-7788
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: LUCENE-7788.patch, LUCENE-7788.patch
>
>
> SOLR-10415 would be removing existing unparameterised log.trace messages use 
> and once that is in place then this ticket's one-line change would be for 
> 'ant precommit' to reject any future unparameterised log.trace message use.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12297) Create a good SolrClient for SolrCloud.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-12297:
---
Description: 
Blocking or async support as well as HTTP2 compatible with multiplexing.

Once it supports enough and is stable, replace internal usage, allowing async, 
and eventually move to HTTP2 connector and allow multiplexing. Could support 
HTTP1.1 and HTTP2 on different ports depending on state of the world then.

The goal of the client itself is to work against HTTP1.1 or HTTP2 with minimal 
or no code path differences and the same for async requests (should initially 
work for both 1.1 and 2 and share majority of code).

The client should also be able to replace HttpSolrClient and plug into the 
other clients the same way.

I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually 
though.

I evaluated some clients and while there are a few options, I went with Jetty's 
HttpClient. It's more mature than Apache HttpClient's support (in 5 beta) and 
we would have to update to a new API for Apache HttpClient anyway.

Meanwhile, the Jetty guys have been very supportive of helping Solr with any 
issues and I like having the client and server from the same project.

  was:
Blocking or async support as well as HTTP2 compatible with multiplexing.

Once it supports enough and is stable, replace internal usage, allowing async, 
and eventually move to HTTP2 connector and allow multiplexing. Could support 
HTTP1.1 and HTTP2 on different ports depending on state of the world then.

The goal of the client itself is to work against HTTP1.1 or HTTP2 with minimal 
or no code path differences and the same for async requests (should initially 
work for both 1.1 and 2 and share majority of code).

The client should also be able to replace HttpSolrClient and plug into the 
other clients the same way.

I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually 
though.


> Create a good SolrClient for SolrCloud.
> ---
>
> Key: SOLR-12297
> URL: https://issues.apache.org/jira/browse/SOLR-12297
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> Blocking or async support as well as HTTP2 compatible with multiplexing.
> Once it supports enough and is stable, replace internal usage, allowing 
> async, and eventually move to HTTP2 connector and allow multiplexing. Could 
> support HTTP1.1 and HTTP2 on different ports depending on state of the world 
> then.
> The goal of the client itself is to work against HTTP1.1 or HTTP2 with 
> minimal or no code path differences and the same for async requests (should 
> initially work for both 1.1 and 2 and share majority of code).
> The client should also be able to replace HttpSolrClient and plug into the 
> other clients the same way.
> I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually 
> though.
> I evaluated some clients and while there are a few options, I went with 
> Jetty's HttpClient. It's more mature than Apache HttpClient's support (in 5 
> beta) and we would have to update to a new API for Apache HttpClient anyway.
> Meanwhile, the Jetty guys have been very supportive of helping Solr with any 
> issues and I like having the client and server from the same project.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12297) Create a good SolrClient for SolrCloud.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-12297:
---
Description: 
Blocking or async support as well as HTTP2 compatible with multiplexing.

Once it supports enough and is stable, replace internal usage, allowing async, 
and eventually move to HTTP2 connector and allow multiplexing. Could support 
HTTP1.1 and HTTP2 on different ports depending on state of the world then.

The goal of the client itself is to work against HTTP1.1 or HTTP2 with minimal 
or no code path differences and the same for async requests (should initially 
work for both 1.1 and 2 and share majority of code).

The client should also be able to replace HttpSolrClient and plug into the 
other clients the same way.

I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually 
though.

  was:
Blocking or async support as well as HTTP2 compatible with multiplexing.

Once it supports enough and is stable, replace internal usage, allowing async, 
and eventually move to HTTP2 connector and allow multiplexing. Could support 
HTTP1.1 and HTTP2 on different ports depending on state of the world then.


> Create a good SolrClient for SolrCloud.
> ---
>
> Key: SOLR-12297
> URL: https://issues.apache.org/jira/browse/SOLR-12297
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> Blocking or async support as well as HTTP2 compatible with multiplexing.
> Once it supports enough and is stable, replace internal usage, allowing 
> async, and eventually move to HTTP2 connector and allow multiplexing. Could 
> support HTTP1.1 and HTTP2 on different ports depending on state of the world 
> then.
> The goal of the client itself is to work against HTTP1.1 or HTTP2 with 
> minimal or no code path differences and the same for async requests (should 
> initially work for both 1.1 and 2 and share majority of code).
> The client should also be able to replace HttpSolrClient and plug into the 
> other clients the same way.
> I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually 
> though.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12286) Wrap log instances in "if (LOG.isLevelEnabled) { log... }

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-12286.
---
Resolution: Won't Fix

Using parameterized logging will take care of almost all the cases.

> Wrap log instances in "if (LOG.isLevelEnabled) { log... }
> -
>
> Key: SOLR-12286
> URL: https://issues.apache.org/jira/browse/SOLR-12286
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> I've been playing around with the question "are objects generated when 
> logging if I use pattern." and "it depends" (tm). I'll attach a test 
> program for anyone to use. Attach VisualVM to it and sample memory when 
> various patterns are enabled.
> The nub of it is this: with the log level set at ERROR, no messages from any 
> of these are printed, yet the number of objects created is vastly different:
> {code}
>   while (true) {
>   // "test" is an instance of a class with an overridden toString() 
> method.
>   // These generate a zillion spurious objects.
>   logger.info("This is a test {} {} {}", test, rand.nextInt(), 
> "whatever");
>   logger.info("This is a test {}", rand.nextInt());
>   logger.info("This is really bad" + test);
>   
>   // These don't generate spurious objects
>   logger.info("This is a test {} {}", test, "whatever");
>   logger.info("This is really bad" + "whatever");
>   }
> {code}
> Simply generating a new random number doesn't create zillions of objects.
> I don't particularly care _why_ the differences exist, the take-away is that 
> if we simply establish the rule "wrap log.level() messages with if..." then 
> we'll avoid the problems altogether. That's _much_ easier to both remember 
> and enforce than trying to understand and remember when pattern A is 
> acceptable and pattern B is not.
> Maybe we could even make this a precommit failure?
> Solr has enough "interesting" things happen re: GC that we don't need to 
> needlessly add to GC pressure.
> Parameterizing is still a good idea as in SOLR-10415 (I'll link)
> Here's the full program, there's not a lot of sophistication here:
> {code}
> import org.apache.logging.log4j.Level;
> import org.apache.logging.log4j.Logger;
> import org.apache.logging.log4j.LogManager;
> import org.apache.logging.log4j.core.LoggerContext;
> import org.apache.logging.log4j.core.config.Configuration;
> import org.apache.logging.log4j.core.config.LoggerConfig;
> import java.util.Random;
> public class Log4J2Test {
>   // Define a static logger variable so that it references the
>   // Logger instance named "MyApp".
>   private static final Logger logger = LogManager.getLogger(Log4J2Test.class);
>   static Random rand = new Random();
>   public static void main(final String... args) {
> // Set up a simple configuration that logs on the console.
> logger.trace("Entering application.");
> LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
> Configuration config = ctx.getConfiguration();
> LoggerConfig loggerConfig = 
> config.getLoggerConfig(LogManager.ROOT_LOGGER_NAME);
> loggerConfig.setLevel(Level.ERROR);
> ctx.updateLoggers();
> Test test = new Test();
> logger.error("Ensure something comes out.");
> while (true) {
>   if (logger.isInfoEnabled()) {
> // These generate a zillion objects.
> logger.info("This is a test {} {} {}", test, rand.nextInt(), 
> "whatever");
> logger.info("This is a test {}", rand.nextInt());
> logger.info("This is really bad" + test);
> // These generates no spurious objects
> logger.info("This is a test {} {}", test, "whatever");
> logger.info("This is really bad" + "whatever");
>   }
> }
>   }
> }
> class Test {
>   static Random rand = new Random();
>   public String toString() {
> return "This is some random string" + rand.nextInt();
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12286) Wrap log instances in "if (LOG.isLevelEnabled) { log... }

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459993#comment-16459993
 ] 

Erick Erickson commented on SOLR-12286:
---

Uwe: I can always count on you to further my education!

That comment was confused, I wanted to make sure people knew there was a 
"toString()" method on the class in case it mattered, I hadn't really figured 
out that the rand.nextInt() was where the problem came from.

I still have some evidence though that this construct is _not_ eliminated by 
HotSpot:
logger.info("This is a test {}",rand.nextInt());
.vs.
logger.info("stuff {}", eoe++);

For both cases I put a sleep in before my tight loop that let me get attached 
with visualVM. In the first case, as soon as the sleep is done the number of 
generated Integer objects shoots up. In the second there's a noticeable delay 
(seconds long). Michael's comment about the Integer cache?

But that's a nit, I don't think it's worth any more time. I don't see much 
value in going any further with it. rand.nextInt() is totally artificial..

Michael: You're probably right there. Interestingly if I put a sleep for long 
enough to get VisualVM attached to the program, there are a few seconds after 
the sleep that don't show object creation (compiled and run outside the IDE) 
with the below, then it shoots up, but it looks like after we generate a 
billion distinct integers or something.

int eoe = 0;
logger.info("stuff {}", eoe++)

So integers aren't all that interesting, Bad Test!

This construct is still evil, no surprise there. Uwe's lambda trick might do 
it, but that's not available:
logger.info("stuff {}", test.toString()));

So all in all, I think this is  a tempest in a teapot. I didn't particularly 
_like_ the suggestion that we wrap all these calls, but OTOH I also didn't like 
the idea of creating lots of spurious objects.

I'll close this "won't fix" and we'll deal with this by doing the parameterized 
logging calls. I think that'll be sufficient.

> Wrap log instances in "if (LOG.isLevelEnabled) { log... }
> -
>
> Key: SOLR-12286
> URL: https://issues.apache.org/jira/browse/SOLR-12286
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> I've been playing around with the question "are objects generated when 
> logging if I use pattern." and "it depends" (tm). I'll attach a test 
> program for anyone to use. Attach VisualVM to it and sample memory when 
> various patterns are enabled.
> The nub of it is this: with the log level set at ERROR, no messages from any 
> of these are printed, yet the number of objects created is vastly different:
> {code}
>   while (true) {
>   // "test" is an instance of a class with an overridden toString() 
> method.
>   // These generate a zillion spurious objects.
>   logger.info("This is a test {} {} {}", test, rand.nextInt(), 
> "whatever");
>   logger.info("This is a test {}", rand.nextInt());
>   logger.info("This is really bad" + test);
>   
>   // These don't generate spurious objects
>   logger.info("This is a test {} {}", test, "whatever");
>   logger.info("This is really bad" + "whatever");
>   }
> {code}
> Simply generating a new random number doesn't create zillions of objects.
> I don't particularly care _why_ the differences exist, the take-away is that 
> if we simply establish the rule "wrap log.level() messages with if..." then 
> we'll avoid the problems altogether. That's _much_ easier to both remember 
> and enforce than trying to understand and remember when pattern A is 
> acceptable and pattern B is not.
> Maybe we could even make this a precommit failure?
> Solr has enough "interesting" things happen re: GC that we don't need to 
> needlessly add to GC pressure.
> Parameterizing is still a good idea as in SOLR-10415 (I'll link)
> Here's the full program, there's not a lot of sophistication here:
> {code}
> import org.apache.logging.log4j.Level;
> import org.apache.logging.log4j.Logger;
> import org.apache.logging.log4j.LogManager;
> import org.apache.logging.log4j.core.LoggerContext;
> import org.apache.logging.log4j.core.config.Configuration;
> import org.apache.logging.log4j.core.config.LoggerConfig;
> import java.util.Random;
> public class Log4J2Test {
>   // Define a static logger variable so that it references the
>   // Logger instance named "MyApp".
>   private static final Logger logger = LogManager.getLogger(Log4J2Test.class);
>   static Random rand = new Random();
>   public static void main(final String... args) {
> // Set up a simple configuration that logs on the console.
> logger.trace("Entering application.");
> 

[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 51 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/51/

10 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ZkControllerTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [Overseer] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.cloud.Overseer  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.cloud.Overseer.start(Overseer.java:545)  at 
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:851)
  at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170) 
 at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)  
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)  at 
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)  at 
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2081)
  at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331)
  at java.lang.Thread.run(Thread.java:748)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [Overseer]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.cloud.Overseer
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.cloud.Overseer.start(Overseer.java:545)
at 
org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:851)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307)
at 
org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:393)
at 
org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2081)
at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:331)
at java.lang.Thread.run(Thread.java:748)


at __randomizedtesting.SeedInfo.seed([8816A03D966D33E5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:303)
at sun.reflect.GeneratedMethodAccessor53.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ZkControllerTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ZkControllerTest: 
1) 

Re: SolrCloud test fails RE "Can't find resource"

[David Smiley]

(sorry for my belated response)
Beasting a zillion times sounds like a decent option.  First I want to
compile a list to see if there are any apparent patterns though.

On Fri, Apr 27, 2018 at 11:16 AM Erick Erickson 
wrote:

> Do you think making a test that just started up a
> MiniSolrCloudCluster, created a collection then quit and then beasting
> it a zillion times might reproduce? I can help beast such a thing if
> it was available.
>
> I think there are a few root causes that are sporadically showing up
> here and there, any we can track down will help enormously in reducing
> noise.
>
> Erick
>
> On Fri, Apr 27, 2018 at 6:34 AM, David Smiley 
> wrote:
> > Just thinking out loud here about something spooky...
> >
> > I've noticed some relatively rare and seemingly random SolrCloud tests
> that
> > fail to create a collection because an expected configSet resource isn't
> > found (isn't in Zookeeper where it ought to be).  The initial exception
> will
> > have "Can't find resource" then the name of the resource -- sometimes
> it's
> > solrconfig.xml or other times some language specific stop-word file or
> > something.  This happens when a collection is being created, but fails
> > because a core/replica won't load because the configSet is faulty
> because a
> > required file isn't there.  I do *not* believe this is some sort of
> > visibility race of the configSet wherein a sleep/wait before creating the
> > collection would help, because I've seen an entire test suite (test class
> > file) of many tests (test methods) all fail for the same reason. In that
> > case the MiniSolrCloudCluster was simply initialized in beforeClass and
> thus
> > ought to have "_default" ready to be used before the test methods, yet
> every
> > test method failed for the same reason.   This is very strange; the code
> > that uploads the _default configSet seems sound.  Yet there's some
> sporadic
> > bug so I guess somewhere there's either some race or there's an
> underlying
> > upload failure that is being ignored.
> >
> > I think I ought to compile a list of tests that have shown this error and
> > the dates(s) / build info in which it occurred.  Maybe there's a pattern
> /
> > something they have in common.
> > --
> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> > http://www.solrenterprisesearchserver.com
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
> --
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

[jira] [Commented] (LUCENE-8286) UnifiedHighlighter should support the new Weight.matches API for better match accuracy

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459899#comment-16459899
 ] 

David Smiley commented on LUCENE-8286:
--

Per chance do you have any WIP code for this or do any concerns come to your 
mind [~romseygeek]?
Perhaps I'll get around to this issue in a couple weeks.

> UnifiedHighlighter should support the new Weight.matches API for better match 
> accuracy
> --
>
> Key: LUCENE-8286
> URL: https://issues.apache.org/jira/browse/LUCENE-8286
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Reporter: David Smiley
>Priority: Major
>
> The new Weight.matches() API should allow the UnifiedHighlighter to more 
> accurately highlight some BooleanQuery patterns correctly -- see LUCENE-7903.
> In addition, this API should make the job of highlighting easier, reducing 
> the LOC and related complexities, especially the UH's PhraseHelper.  Note: 
> reducing/removing PhraseHelper is not a near-term goal since Weight.matches 
> is experimental and incomplete, and perhaps we'll discover some gaps in 
> flexibility/functionality.
> This issue should introduce a new UnifiedHighlighter.HighlightFlag enum 
> option for this method of highlighting.   Perhaps call it {{WEIGHT_MATCHES}}? 
>  Longer term it could go away and it'll be implied if you specify enum values 
> for PHRASES & MULTI_TERM_QUERY?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12300) Unpopulated SolrDocument using Custom DocTransformer

[Landon Petzoldt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Landon Petzoldt updated SOLR-12300:
---
Description: 
When attempting to tag more than 2 fields with transformers, the documents' 
fields are not populated except for the id field. This seems to be specific to 
Solr 7+ as this was not an issue in Solr 6.4.2. The issue only seems to be 
present for custom transformers, and the default transformers seem to populate 
correctly.

Steps for Reproduction in Solr 7.2 or 7.3
 # Create Java project and import {{*solr-core*}} and {{*solr-solrj*}} library 
jars.
 # Create classes {{*BlankTransformerFactory*}} and {{*BlankTransformer*}} [see 
gist|https://gist.github.com/clurdish/8683e56ea1b93978f7027844537a0232]
 # Build project and put resulting jar into {{*solr\contrib\plugins*}}
 # Create sample solr core {{*testcore*}}
 # Add configuration to the core's {{*solrconfig.xml*}} (see below)
 # Load sample documents into core (see below)
 # (2 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer}}
 *_all documents are returned correctly_*
# (3 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer:[blanktrans]}}
 *_only id field is returned_*


*{{solrconfig.xml}}*
...
{{}}
...
{{}}
...

*{{sample_data.json}}*
{
  "id": "1",
  "Title": ["The Big Tree"],
  "layer": ["fims"]
},
{
  "id": "2",
  "Title": ["Far Far Away"],
  "layer": ["buildings"]
},
{
  "id": "3",
  "Title": ["Way Too Long"],
  "Author": ["Slim Jim McGee"],
  "layer": ["fims"]
},
{
  "id": "4",
  "Author": ["Rumplestiltskin"],
  "layer": ["tasks"]
}

  was:
When attempting to tag more than 2 fields with transformers, the documents' 
fields are not populated except for the id field. This seems to be specific to 
Solr 7+ as this was not an issue in Solr 6.4.2. The issue only seems to be 
present for custom transformers, and the default transformers seem to populate 
correctly.

Steps for Reproduction in Solr 7.2 or 7.3
 # Create Java project and import {{*solr-core*}} and {{*solr-solrj*}} library 
jars.
 # Create classes {{*BlankTransformerFactory*}} and {{*BlankTransformer*}} [see 
gist|https://gist.github.com/clurdish/8683e56ea1b93978f7027844537a0232]
 # Build project and put resulting jar into {{*solr\contrib\plugins*}}
 # Create sample solr core {{*testcore*}}
 # Add configuration to the core's {{*solrconfig.xml*}} (see below)
 # Load sample documents into core (see below)
 # (2 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer}}
 *_all documents are returned correctly_*
# (3 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer:[blanktrans]}}
 *_only id field is returned_*


*{{solrconfig.xml}}*
{quote}...
{{}}
...
{{}}
...{quote}

*{{sample_data.json}}*
{
  "id": "1",
  "Title": ["The Big Tree"],
  "layer": ["fims"]
},
{
  "id": "2",
  "Title": ["Far Far Away"],
  "layer": ["buildings"]
},
{
  "id": "3",
  "Title": ["Way Too Long"],
  "Author": ["Slim Jim McGee"],
  "layer": ["fims"]
},
{
  "id": "4",
  "Author": ["Rumplestiltskin"],
  "layer": ["tasks"]
}


> Unpopulated SolrDocument using Custom DocTransformer
> 
>
> Key: SOLR-12300
> URL: https://issues.apache.org/jira/browse/SOLR-12300
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.2, 7.3
> Environment: Microsoft Windows 10 Enterprise
> Version 10.0.14393 Build 14393
>Reporter: Landon Petzoldt
>Priority: Major
>
> When attempting to tag more than 2 fields with transformers, the documents' 
> fields are not populated except for the id field. This seems to be specific 
> to Solr 7+ as this was not an issue in Solr 6.4.2. The issue only seems to be 
> present for custom transformers, and the default transformers seem to 
> populate correctly.
> Steps for Reproduction in Solr 7.2 or 7.3
>  # Create Java project and import {{*solr-core*}} and {{*solr-solrj*}} 
> library jars.
>  # Create classes {{*BlankTransformerFactory*}} and {{*BlankTransformer*}} 
> [see gist|https://gist.github.com/clurdish/8683e56ea1b93978f7027844537a0232]
>  # Build project and put resulting jar into {{*solr\contrib\plugins*}}
>  # Create sample solr core {{*testcore*}}
>  # Add configuration to the core's {{*solrconfig.xml*}} (see below)
>  # Load sample documents into core (see below)
>  # (2 fields) Navigate to 
> {{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer}}
>  *_all documents are returned correctly_*
> # (3 fields) Navigate to 
> 

[jira] [Updated] (SOLR-12300) Unpopulated SolrDocument using Custom DocTransformer

[Landon Petzoldt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Landon Petzoldt updated SOLR-12300:
---
Description: 
When attempting to tag more than 2 fields with transformers, the documents' 
fields are not populated except for the id field. This seems to be specific to 
Solr 7+ as this was not an issue in Solr 6.4.2. The issue only seems to be 
present for custom transformers, and the default transformers seem to populate 
correctly.

Steps for Reproduction in Solr 7.2 or 7.3
 # Create Java project and import {{*solr-core*}} and {{*solr-solrj*}} library 
jars.
 # Create classes {{*BlankTransformerFactory*}} and {{*BlankTransformer*}} [see 
gist|https://gist.github.com/clurdish/8683e56ea1b93978f7027844537a0232]
 # Build project and put resulting jar into {{*solr\contrib\plugins*}}
 # Create sample solr core {{*testcore*}}
 # Add configuration to the core's {{*solrconfig.xml*}} (see below)
 # Load sample documents into core (see below)
 # (2 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer}}
 *_all documents are returned correctly_*
# (3 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer:[blanktrans]}}
 *_only id field is returned_*


*{{solrconfig.xml}}*
{quote}...
{{}}
...
{{}}
...{quote}

*{{sample_data.json}}*
{
  "id": "1",
  "Title": ["The Big Tree"],
  "layer": ["fims"]
},
{
  "id": "2",
  "Title": ["Far Far Away"],
  "layer": ["buildings"]
},
{
  "id": "3",
  "Title": ["Way Too Long"],
  "Author": ["Slim Jim McGee"],
  "layer": ["fims"]
},
{
  "id": "4",
  "Author": ["Rumplestiltskin"],
  "layer": ["tasks"]
}

  was:
When attempting to tag more than 2 fields with transformers, the documents' 
fields are not populated except for the id field. This seems to be specific to 
Solr 7+ as this was not an issue in Solr 6.4.2. The issue only seems to be 
present for custom transformers, and the default transformers seem to populate 
correctly.

Steps for Reproduction in Solr 7.2 or 7.3
 # Create Java project and import {{*solr-core*}} and {{*solr-solrj*}} library 
jars.
 # Create classes {{*BlankTransformerFactory*}} and {{*BlankTransformer*}} [see 
gist|https://gist.github.com/clurdish/8683e56ea1b93978f7027844537a0232]
 # Build project and put resulting jar into {{*solr\contrib\plugins*}}
 # Create sample solr core {{*testcore*}}
 # Add configuration to the core's {{*solrconfig.xml*}} (see below)
 # Load sample documents into core (see below)
 # (2 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer}}
 *_all documents are returned correctly_*
# (3 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer:[blanktrans]}}
 *_only id field is returned_*


*{{solrconfig.xml}}*
{quote}...
{{}}
...
{{}}
...{quote}

*{{sample_data.json}}*
{quote}{
  "id": "1",
  "Title": ["The Big Tree"],
  "layer": ["fims"]
},
{
  "id": "2",
  "Title": ["Far Far Away"],
  "layer": ["buildings"]
},
{
  "id": "3",
  "Title": ["Way Too Long"],
  "Author": ["Slim Jim McGee"],
  "layer": ["fims"]
},
{
  "id": "4",
  "Author": ["Rumplestiltskin"],
  "layer": ["tasks"]
}{quote}


> Unpopulated SolrDocument using Custom DocTransformer
> 
>
> Key: SOLR-12300
> URL: https://issues.apache.org/jira/browse/SOLR-12300
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.2, 7.3
> Environment: Microsoft Windows 10 Enterprise
> Version 10.0.14393 Build 14393
>Reporter: Landon Petzoldt
>Priority: Major
>
> When attempting to tag more than 2 fields with transformers, the documents' 
> fields are not populated except for the id field. This seems to be specific 
> to Solr 7+ as this was not an issue in Solr 6.4.2. The issue only seems to be 
> present for custom transformers, and the default transformers seem to 
> populate correctly.
> Steps for Reproduction in Solr 7.2 or 7.3
>  # Create Java project and import {{*solr-core*}} and {{*solr-solrj*}} 
> library jars.
>  # Create classes {{*BlankTransformerFactory*}} and {{*BlankTransformer*}} 
> [see gist|https://gist.github.com/clurdish/8683e56ea1b93978f7027844537a0232]
>  # Build project and put resulting jar into {{*solr\contrib\plugins*}}
>  # Create sample solr core {{*testcore*}}
>  # Add configuration to the core's {{*solrconfig.xml*}} (see below)
>  # Load sample documents into core (see below)
>  # (2 fields) Navigate to 
> {{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer}}
>  *_all documents are returned correctly_*
> # (3 fields) Navigate 

[jira] [Created] (SOLR-12300) Unpopulated SolrDocument using Custom DocTransformer

[Landon Petzoldt (JIRA)]

Landon Petzoldt created SOLR-12300:
--

 Summary: Unpopulated SolrDocument using Custom DocTransformer
 Key: SOLR-12300
 URL: https://issues.apache.org/jira/browse/SOLR-12300
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.2, 7.3
 Environment: Microsoft Windows 10 Enterprise

Version 10.0.14393 Build 14393
Reporter: Landon Petzoldt


When attempting to tag more than 2 fields with transformers, the documents' 
fields are not populated except for the id field. This seems to be specific to 
Solr 7+ as this was not an issue in Solr 6.4.2. The issue only seems to be 
present for custom transformers, and the default transformers seem to populate 
correctly.

Steps for Reproduction in Solr 7.2 or 7.3
 # Create Java project and import {{*solr-core*}} and {{*solr-solrj*}} library 
jars.
 # Create classes {{*BlankTransformerFactory*}} and {{*BlankTransformer*}} [see 
gist|https://gist.github.com/clurdish/8683e56ea1b93978f7027844537a0232]
 # Build project and put resulting jar into {{*solr\contrib\plugins*}}
 # Create sample solr core {{*testcore*}}
 # Add configuration to the core's {{*solrconfig.xml*}} (see below)
 # Load sample documents into core (see below)
 # (2 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer}}
 *_all documents are returned correctly_*
# (3 fields) Navigate to 
{{http://localhost:8983/solr/testcore/select?q=*:*=true=Author:[blanktrans],Title:[blanktrans],id,layer:[blanktrans]}}
 *_only id field is returned_*


*{{solrconfig.xml}}*
{quote}...
{{}}
...
{{}}
...{quote}

*{{sample_data.json}}*
{quote}{
  "id": "1",
  "Title": ["The Big Tree"],
  "layer": ["fims"]
},
{
  "id": "2",
  "Title": ["Far Far Away"],
  "layer": ["buildings"]
},
{
  "id": "3",
  "Title": ["Way Too Long"],
  "Author": ["Slim Jim McGee"],
  "layer": ["fims"]
},
{
  "id": "4",
  "Author": ["Rumplestiltskin"],
  "layer": ["tasks"]
}{quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8286) UnifiedHighlighter should support the new Weight.matches API for better match accuracy

[David Smiley (JIRA)]

David Smiley created LUCENE-8286:


 Summary: UnifiedHighlighter should support the new Weight.matches 
API for better match accuracy
 Key: LUCENE-8286
 URL: https://issues.apache.org/jira/browse/LUCENE-8286
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/highlighter
Reporter: David Smiley


The new Weight.matches() API should allow the UnifiedHighlighter to more 
accurately highlight some BooleanQuery patterns correctly -- see LUCENE-7903.

In addition, this API should make the job of highlighting easier, reducing the 
LOC and related complexities, especially the UH's PhraseHelper.  Note: 
reducing/removing PhraseHelper is not a near-term goal since Weight.matches is 
experimental and incomplete, and perhaps we'll discover some gaps in 
flexibility/functionality.

This issue should introduce a new UnifiedHighlighter.HighlightFlag enum option 
for this method of highlighting.   Perhaps call it {{WEIGHT_MATCHES}}?  Longer 
term it could go away and it'll be implied if you specify enum values for 
PHRASES & MULTI_TERM_QUERY?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459884#comment-16459884
 ] 

ASF subversion and git services commented on SOLR-8998:
---

Commit b347f031dbec8fe76ee15f85cdcc62efbf63a80d in lucene-solr's branch 
refs/heads/branch_7x from [~mkhludnev]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b347f03 ]

SOLR-8998: uniqueBlock() aggreagation for singlevalue string fields in 
json.facet


> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
>  Issue Type: New Feature
>  Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch, 
> SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and 
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on 
> derived values from their children.  The most important part (and the most 
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which 
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked 
> index only. For now it it supports singlevalue string fields, docValues 
> usually make sense.   
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459877#comment-16459877
 ] 

ASF subversion and git services commented on SOLR-8998:
---

Commit ee7b52f4c6fe55f0d07ce8228c246b61d1f2b5fb in lucene-solr's branch 
refs/heads/master from [~mkhludnev]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ee7b52f ]

SOLR-8998: uniqueBlock() aggreagation for singlevalue string fields in 
json.facet


> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
>  Issue Type: New Feature
>  Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch, 
> SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and 
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on 
> derived values from their children.  The most important part (and the most 
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which 
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked 
> index only. For now it it supports singlevalue string fields, docValues 
> usually make sense.   
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7960) NGram filters -- add option to keep short terms

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459873#comment-16459873
 ] 

Shawn Heisey commented on LUCENE-7960:
--

I've gotten a look at the PR.

Changing the signature on an existing constructor isn't a good idea.  Lucene is 
a public API and there will be user code using that constructor that must 
continue to work if Lucene is upgraded.  We should add a new constructor and 
have the existing constructor(s) call that one with default values.

The only question about that is whether the existing constructor should be 
deprecated in stable and removed in master.  I'm not sure who to ask.

There are some variable renames.  They don't look like problems, especially 
because the visibility is private, but I'd like to get the opinion of someone 
who has deeper Lucene knowledge.

I'm having a difficult time following the modifications to the filter logic.  
Some of the modifications look like they're not directly related to 
implementing this issue, but I can't tell for sure.


> NGram filters -- add option to keep short terms
> ---
>
> Key: LUCENE-7960
> URL: https://issues.apache.org/jira/browse/LUCENE-7960
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: Shawn Heisey
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When ngram or edgengram filters are used, any terms that are shorter than the 
> minGramSize are completely removed from the token stream.
> This is probably 100% what was intended, but I've seen it cause a lot of 
> problems for users.  I am not suggesting that the default behavior be 
> changed.  That would be far too disruptive to the existing user base.
> I do think there should be a new boolean option, with a name like 
> keepShortTerms, that defaults to false, to allow the short terms to be 
> preserved.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12297) Create a good SolrClient for SolrCloud.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459859#comment-16459859
 ] 

Mark Miller commented on SOLR-12297:


It's very easy to add an HTTP2 connector to Jetty, just swap in the right 
config, it's a lot more work to have HTTP2 user clients and internal clients 
that can actually do anything with it. Our clients don't support talking to 
HTTP2 so I'm not sure how that would actually work unless it was single server 
Solr with no internal communication. The clients also need new async API 
options. This is about that work.

One option is Apache HttpClient 5 I guess, but I've been playing around with 
Jetty HttpClient instead, given there past interest and effort with helping the 
project and the fact that we use Jetty for the container.

> Create a good SolrClient for SolrCloud.
> ---
>
> Key: SOLR-12297
> URL: https://issues.apache.org/jira/browse/SOLR-12297
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> Blocking or async support as well as HTTP2 compatible with multiplexing.
> Once it supports enough and is stable, replace internal usage, allowing 
> async, and eventually move to HTTP2 connector and allow multiplexing. Could 
> support HTTP1.1 and HTTP2 on different ports depending on state of the world 
> then.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.3-Linux (64bit/jdk-11-ea+5) - Build # 157 - Unstable!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12292) Make it easier to configure Solr with CORS

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459830#comment-16459830
 ] 

David Smiley commented on SOLR-12292:
-

Even though this issue is explicitly about admin calls, I want to say that IMO 
SearchHandler requests ought to have CORS headers that by default allow any 
origin.  After all we already support jsonp and thus search requests are 
effectively exposed to any host already.

> Make it easier to configure Solr with CORS
> --
>
> Key: SOLR-12292
> URL: https://issues.apache.org/jira/browse/SOLR-12292
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Reporter: Jan Høydahl
>Priority: Major
>
> While working on SOLR-8207 I wanted to collect info from other SolrCloud 
> nodes from the AdminUI. However this is blocked by 
> [CORS|https://en.wikipedia.org/wiki/Cross-origin_resource_sharing] policy. In 
> that Jira I instead did the fan-out on the Solr server side for the two 
> handler I needed.
> It would be nice if all nodes in a SolrCloud cluster could automatically 
> accept any other node as a legal origin, and make it easy for users to add 
> other origins by config.
> If we use the [Jetty CORS 
> filter|http://www.eclipse.org/jetty/documentation/9.4.9.v20180320/cross-origin-filter.html]
>  in web.xml, perhaps we could parse a env.var from solr.in.xx and inject into 
> the {{allowedOrigins}} property of that filter? There is also SOLR-6059 which 
> tries to implement CORS inside of Solr handlers and not in Jetty. Don't know 
> pros/cons of those.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12297) Create a good SolrClient for SolrCloud.

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459809#comment-16459809
 ] 

David Smiley commented on SOLR-12297:
-

FWIW I once reconfigured Solr's Jetty to support HTTP2.  It took a little 
futzing around but it worked.  No programming changes needed.  I could try and 
dig up my config on that; I'd need to do a diff.  It was a for a POC.

> Create a good SolrClient for SolrCloud.
> ---
>
> Key: SOLR-12297
> URL: https://issues.apache.org/jira/browse/SOLR-12297
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> Blocking or async support as well as HTTP2 compatible with multiplexing.
> Once it supports enough and is stable, replace internal usage, allowing 
> async, and eventually move to HTTP2 connector and allow multiplexing. Could 
> support HTTP1.1 and HTTP2 on different ports depending on state of the world 
> then.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Weekly BadApple report:

[David Smiley]

I'm actively working on the Alias ones.  Some have identified root causes
relating to eventual-consistency timings.  Since I don't have a proper fix
yet, in the interim I'm tempted to add a temporary Thread.sleep(200) at
certain places in pertinent tests.

On Tue, May 1, 2018 at 12:59 AM Erick Erickson 
wrote:

> In order to reduce some of the make-work, I collect failed tests Fri->Mon
> so the BadApple'd tests on Thursday don't clutter things up.
>
> BadApple candidates: I'll BadApple these on Thursday unless there are
> objections
>
>
>  
> org.apache.solr.client.solrj.io.stream.MathExpressionTest.testGammaDistribution
>org.apache.solr.cloud.AddReplicaTest.test
>
>  org.apache.solr.cloud.CreateRoutedAliasTest.testCollectionNamesMustBeAbsent
>org.apache.solr.cloud.CreateRoutedAliasTest.testTimezoneAbsoluteDate
>org.apache.solr.cloud.CreateRoutedAliasTest.testV1
>org.apache.solr.cloud.CreateRoutedAliasTest.testV2
>org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly
>
>  org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderAndMixedReplicas
>org.apache.solr.cloud.LIRRollingUpdatesTest.testNewLeaderOldReplica
>
>  org.apache.solr.cloud.LIRRollingUpdatesTest.testOldLeaderAndMixedReplicas
>
>  
> org.apache.solr.cloud.LeaderElectionIntegrationTest.testSimpleSliceLeaderElection
>org.apache.solr.cloud.OverseerRolesTest.testOverseerRole
>
>  
> org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection
>org.apache.solr.cloud.TestPullReplica.testKillLeader
>org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test
>org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLost
>org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.testRestoreState
>
>  
> org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication
>org.apache.solr.schema.TestCloudSchemaless.test
>org.apache.solr.search.TestStressRecovery.testStressRecovery
>org.apache.solr.update.TestInPlaceUpdatesDistrib.test
>
>
> I believe these are being worked on, so I will _not_ BadApple them
> even though they fail.
> What about the other autoscaling tests?
>
>
>  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration
>
>  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
>org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger
>
>
> Number of AwaitsFix: 21 Number of BadApples: 90
>
> *AwaitsFix Annotations:
>
>
> Lucene AwaitsFix
> GeoPolygonTest.java
>testLUCENE8276_case3()
>//@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8276
> ")
>
> GeoPolygonTest.java
>testLUCENE8280()
>//@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8280
> ")
>
> GeoPolygonTest.java
>testLUCENE8281()
>//@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8281
> ")
>
> RandomGeoPolygonTest.java
>testCompareBigPolygons()
>//@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8281
> ")
>
> RandomGeoPolygonTest.java
>testCompareSmallPolygons()
>//@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8281
> ")
>
> TestControlledRealTimeReopenThread.java
>testCRTReopen()
>@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5737
> ")
>
> TestICUNormalizer2CharFilter.java
>testRandomStrings()
>@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5595
> ")
>
> TestICUTokenizerCJK.java
>TestICUTokenizerCJK suite
>@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/LUCENE-8222;)
>
> TestMoreLikeThis.java
>testMultiFieldShouldReturnPerFieldBooleanQuery()
>@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-7161
> ")
>
> UIMABaseAnalyzerTest.java
>testRandomStrings()
>@Test @AwaitsFix(bugUrl =
> "https://issues.apache.org/jira/browse/LUCENE-3869;)
>
> UIMABaseAnalyzerTest.java
>testRandomStringsWithConfigurationParameters()
>@Test @AwaitsFix(bugUrl =
> "https://issues.apache.org/jira/browse/LUCENE-3869;)
>
> UIMATypeAwareAnalyzerTest.java
>testRandomStrings()
>@Test @AwaitsFix(bugUrl =
> "https://issues.apache.org/jira/browse/LUCENE-3869;)
>
>
> Solr AwaitsFix
> ReplaceNodeNoTargetTest.java
>ReplaceNodeNoTargetTest suite
>@LuceneTestCase.AwaitsFix(bugUrl =
> "https://issues.apache.org/jira/browse/SOLR-11067;)
>
> TestCollapseQParserPlugin.java
>testStringCollapse()
>@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974;)
>
> TestImpersonationWithHadoopAuth.java
>testForwarding()
>@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893;)
>
> TestLTRReRankingPipeline.java
>testDifferentTopN()
>@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11134;)
>
> TestMinMaxOnMultiValuedField.java
>testDoubleFieldCache()
>@AwaitsFix(bugUrl = 

[jira] [Created] (LUCENE-8285) FSTOrdPostingsFormat ironically does not support ord()

[David Smiley (JIRA)]

David Smiley created LUCENE-8285:


 Summary: FSTOrdPostingsFormat ironically does not support ord()
 Key: LUCENE-8285
 URL: https://issues.apache.org/jira/browse/LUCENE-8285
 Project: Lucene - Core
  Issue Type: Improvement
  Components: core/codecs
Reporter: David Smiley


Ironically, FSTOrdPostingsFormat has a TermsEnum.ord() that throws 
UnsupportedOperationException.  There are TODOs here and some discussion in 
LUCENE-3069 (CC [~billy]).  Isn't supporting ord() entirely the point of this 
postings format?

Additionally, if ord() was supported,couldn't TermState be effectively the 
ordinal, thus making it lighter-weight/cheaper?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9685) tag a query in JSON syntax

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429838#comment-16429838
 ] 

Mikhail Khludnev edited comment on SOLR-9685 at 5/1/18 2:52 PM:


[~squallsama], I'm afraid the proposed syntax is a way verbose. I propose to 
think about particular usage. 
[Here|https://lucene.apache.org/solr/guide/7_1/json-query-dsl.html#use-json-query-dsl-in-json-request-api]
 we have 
{code}
{
"query": {
"bool": {
"must_not": "{!frange u:3.0}ranking"
}
},
"filter: [
"title:solr",
{ "lucene" : {"df: "content", query : "lucene solr" }}
]
}
{code}
I'd like to tag it like this
{code}
{
"query": {
"bool": {
"must_not": "{!frange u:3.0}ranking"
},
   "tag":"top"
},
"filter: [
{"query":"title:solr",   // the most tricky one  
  "tag":"title"
},
{ "lucene" : {"df: "content", query : "lucene solr" },
  "tag":"content" 
}
]
}
{code}

Opinions? 
*UPD*
repeating {{tag}} is boring, let's shorten it to the single hash like this
{code}
{
"query": {
  "#top":{
  "bool": {
"must_not": "{!frange u:3.0}ranking"
   }  }
},
"filter: [
{"#title":"title:solr"},
{ "#content": {"lucene" : {"df: "content", query : "lucene solr" }}}
]
}
{code} 


was (Author: mkhludnev):
[~squallsama], I'm afraid the proposed syntax is a way verbose. I propose to 
think about particular usage. 
[Here|https://lucene.apache.org/solr/guide/7_1/json-query-dsl.html#use-json-query-dsl-in-json-request-api]
 we have 
{code}
{
"query": {
"bool": {
"must_not": "{!frange u:3.0}ranking"
}
},
"filter: [
"title:solr",
{ "lucene" : {"df: "content", query : "lucene solr" }}
]
}
{code}
I'd like to tag it like this
{code}
{
"query": {
"bool": {
"must_not": "{!frange u:3.0}ranking"
},
   "tag":"top"
},
"filter: [
{"query":"title:solr",   // the most tricky one  
  "tag":"title"
},
{ "lucene" : {"df: "content", query : "lucene solr" },
  "tag":"content" 
}
]
}
{code}

Opinions? 

> tag a query in JSON syntax
> --
>
> Key: SOLR-9685
> URL: https://issues.apache.org/jira/browse/SOLR-9685
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module, JSON Request API
>Reporter: Yonik Seeley
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There should be a way to tag a query/filter in JSON syntax.
> Perhaps these two forms could be equivalent:
> {code}
> "{!tag=COLOR}color:blue"
> { tagged : { COLOR : "color:blue" }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8998) JSON Facet API child roll-ups

[Dr Oleg Savrasov (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459719#comment-16459719
 ] 

Dr Oleg Savrasov edited comment on SOLR-8998 at 5/1/18 2:33 PM:


I like the idea. Usage of proposed {{uniqueBlock(_root_)}} aggregation looks 
very similar to {{unique(_root_)}} workaround. But the implementation avoids 
accumulating {{FixedBitSet}} for each slot. The only thing I worry about is 
found hack with {{limit:-1}}. I believe {{uniqueBlock}} should perform much 
better with it, but I don't see any way to use it by default without modifying 
main facet machinery. Shall we somehow document that it's recommended to use 
{{uniqueBlock}} aggregation with {{limit:-1}} facet parameter?


was (Author: osavrasov):
I like the idea. Usage of proposed uniqueBlock(_root_) aggregation looks very 
similar to unique(_root_) workaround. But the implementation avoids 
accumulating FixedBitSet for each slot. The only thing I worry about is found 
hack with limit:-1. I believe uniqueBlock should perform much better with it, 
but I don't see any way to use it by default without modifying main facet 
machinery. Shall we somehow document that it's recommended to use uniqueBlock 
aggregation with limit:-1 facet parameter?

> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
>  Issue Type: New Feature
>  Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch, 
> SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and 
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on 
> derived values from their children.  The most important part (and the most 
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which 
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked 
> index only. For now it it supports singlevalue string fields, docValues 
> usually make sense.   
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups

[Dr Oleg Savrasov (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459719#comment-16459719
 ] 

Dr Oleg Savrasov commented on SOLR-8998:


I like the idea. Usage of proposed uniqueBlock(_root_) aggregation looks very 
similar to unique(_root_) workaround. But the implementation avoids 
accumulating FixedBitSet for each slot. The only thing I worry about is found 
hack with limit:-1. I believe uniqueBlock should perform much better with it, 
but I don't see any way to use it by default without modifying main facet 
machinery. Shall we somehow document that it's recommended to use uniqueBlock 
aggregation with limit:-1 facet parameter?

> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
>  Issue Type: New Feature
>  Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch, 
> SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and 
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on 
> derived values from their children.  The most important part (and the most 
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which 
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked 
> index only. For now it it supports singlevalue string fields, docValues 
> usually make sense.   
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12298) Index Full nested document Hierarchy For Queries (umbrella issue)

[mosh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459631#comment-16459631
 ] 

mosh edited comment on SOLR-12298 at 5/1/18 2:21 PM:
-

Approach: I see [~janhoy]'s 
[proposal|http://lucene.472066.n3.nabble.com/nesting-Any-way-to-return-the-whole-hierarchical-structure-when-doing-Block-Join-queries-td4265933.html#a4380320]
 as a starting point for this issue, as it addresses most of the problems, as 
well as [this|https://www.youtube.com/watch?v=qV0fIg-LGBE] talk on Solr 
Revolution 2016: "Working with Deeply Nested Documents in Apache Solr", as the 
starting points to this issue.

Firstly, the way a nested document is indexed has to be changed.
 I propose we add the following fields:
 # __parent__
 # __level__
 # __path__

__parent__: This field wild will store the document's parent docId, to be used 
for building the whole hierarchy, using a new document transformer, as 
suggested by Jan on the mailing list.

__level__: This field will store the level of the specified field in the 
document, using an int value. This field can be used for the parentFilter, 
eliminating the need to provide a parentFilter, which will be set by default as 
"__level__:queriedFieldLevel".

__path__: This field will contain the full path, separated by a specific 
reserved char e.g., '.'
 for example: "first.second.third".
 This will enable users to search for a specific path, or provide a regular 
expression to search for fields sharing the same name in different levels of 
the document, filtering using the _level_ key if needed.

To make this happen at index time, changes have to be made to the JSON loader, 
which will add the above fields, as well as the __root__ field, which holds the 
documents top most level docId. This will only happen when a specified 
parameter is added to the update request, e.g. "nested=true".

The new child doc transformer will be able to either reassemble the whole 
document structure, or do so from a specific level, if specified.
 Full hierarchy reconstruction can be done relatively cheaply, using the 
__root__ _field to get to the highest level document, and querying the block 
for its children, ordering the query by the _level__ field.


was (Author: moshebla):
Approach: I see [~janhoy]'s 
[proposal|http://lucene.472066.n3.nabble.com/nesting-Any-way-to-return-the-whole-hierarchical-structure-when-doing-Block-Join-queries-td4265933.html#a4380320]
 as a starting point for this issue, as it addresses most of the problems, as 
well as [this|https://www.youtube.com/watch?v=qV0fIg-LGBE] talk on Solr 
Revolution 2016: "Working with Deeply Nested Documents in Apache Solr", as the 
starting points to this issue.

Firstly, the way a nested document is indexed has to be changed.
 I propose we add the following fields:
 # __parent__
 # __level__
 # __path__

__parent__: This field wild will store the document's parent docId, to be used 
for building the whole hierarchy, using a new document transformer, as 
suggested by Jan on the mailing list.

__level__: This field will store the level of the specified field in the 
document, using an int value. This field can be used for the parentFilter, 
eliminating the need to provide a parentFilter, which will be set by default as 
"__level__:queriedFieldLevel".

__path__: This field will contain the full path, separated by a specific 
reserved char e.g., '.'
 for example: "first.second.third".
 This will enable users to search for a specific path, or provide a regular 
expression to search for fields sharing the same name in different levels of 
the document, filtering using the _level_ key if needed.

To make this happen at index time, changes have to be made to the JSON loader, 
which will add the above fields, as well as the __root__ field, which holds the 
documents top most level docId. This will only happen when a specified 
parameter is added to the update request, e.g. "nested=true".

The new child doc transformer will be able to either reassemble the whole 
document structure, or do so from a specific level, if specified.
 Full hierarchy reconstruction can be done relatively cheaply, using the 
__root__ field to get to the highest level document, and querying the block for 
its children, ordering the query by the __level__ field.

> Index Full nested document Hierarchy For Queries (umbrella issue)
> -
>
> Key: SOLR-12298
> URL: https://issues.apache.org/jira/browse/SOLR-12298
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Major
>
> Solr ought to have the ability to index deeply nested objects, while storing 
> the original document hierarchy.
> Currently the client has to index the child document's 

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_162) - Build # 1831 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1831/
Java: 64bit/jdk1.8.0_162 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.client.solrj.io.stream.StreamDecoratorTest.testParallelTerminatingDaemonUpdateStream

Error Message:
--> 
http://127.0.0.1:43031/solr/collection1:java.util.ConcurrentModificationException

Stack Trace:
java.io.IOException: --> 
http://127.0.0.1:43031/solr/collection1:java.util.ConcurrentModificationException
at 
__randomizedtesting.SeedInfo.seed([4516FF5CC31B2F17:B47227349771E009]:0)
at 
org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:222)
at 
org.apache.solr.client.solrj.io.stream.StreamDecoratorTest.testParallelTerminatingDaemonUpdateStream(StreamDecoratorTest.java:2735)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 16003 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.io.stream.StreamDecoratorTest
   

[jira] [Commented] (SOLR-12276) Admin UI - Convert from "AngularJS" to "Angular"

[Upayavira (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459700#comment-16459700
 ] 

Upayavira commented on SOLR-12276:
--

[~jdyer] I'm not following Solr much at the moment, but it is great to see this 
ticket! If there is anything you need explaining regarding the original Angular 
UI, feel free to ask.

> Admin UI - Convert from "AngularJS" to "Angular"
> 
>
> Key: SOLR-12276
> URL: https://issues.apache.org/jira/browse/SOLR-12276
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Admin UI
>Reporter: James Dyer
>Priority: Minor
>  Labels: Angular, AngularJS, angular-migration
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> With SOLR-12196 it was noted the current Solr Admin UI runs AngularJS (1.x), 
> which is to be End-of-Life later this year.  Various options were proposed 
> for what to do next.  One option is to keep the existing functionality but 
> migrate to a newer UI framework.  This ticket is to migrate the existing UI 
> to Angular (2+).
> See [this readme 
> file|https://github.com/jdyer1/lucene-solr/tree/feature/angular-conversion-solr-admin/solr/webapp].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11384) add support for distributed graph query

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459678#comment-16459678
 ] 

Gus Heck commented on SOLR-11384:
-

This would make the existing 
[https://lucene.apache.org/solr/guide/7_3/other-parsers.html#graph-query-parser]
 work across multiple cores. That feature is useful for things like complex 
hierarchy based security expressed as (cacheable) filter queries. Last I looked 
streaming expressions can't be used as a filter on regular queries (though it's 
been some time since I looked) and would need to be calculated every time. 

> add support for distributed graph query
> ---
>
> Key: SOLR-11384
> URL: https://issues.apache.org/jira/browse/SOLR-11384
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Kevin Watters
>Priority: Minor
>
> Creating this ticket to track the work that I've done on the distributed 
> graph traversal support in solr.
> Current GraphQuery will only work on a single core, which introduces some 
> limits on where it can be used and also complexities if you want to scale it. 
>  I believe there's a strong desire to support a fully distributed method of 
> doing the Graph Query.  I'm working on a patch, it's not complete yet, but if 
> anyone would like to have a look at the approach and implementation,  I 
> welcome much feedback.  
> The flow for the distributed graph query is almost exactly the same as the 
> normal graph query.  The only difference is how it discovers the "frontier 
> query" at each level of the traversal.  
> When a distribute graph query request comes in, each shard begins by running 
> the root query, to know where to start on it's shard.  Each participating 
> shard then discovers it's edges for the next hop.  Those edges are then 
> broadcast to all other participating shards.  The shard then receives all the 
> parts of the frontier query , assembles it, and executes it.
> This process continues on each shard until there are no new edges left, or 
> the maxDepth of the traversal has finished.
> The approach is to introduce a FrontierBroker that resides as a singleton on 
> each one of the solr nodes in the cluster.  When a graph query is created, it 
> can do a getInstance() on it so it can listen on the frontier parts coming in.
> Initially, I was using an external Kafka broker to handle this, and it did 
> work pretty well.  The new approach is migrating the FrontierBroker to be a 
> request handler in Solr, and potentially to use the SolrJ client to publish 
> the edges to each node in the cluster.
> There are a few outstanding design questions, first being, how do we know 
> what the list of shards are that are participating in the current query 
> request?  Is that easy info to get at?
> Second,  currently, we are serializing a query object between the shards, 
> perhaps we should consider a slightly different abstraction, and serialize 
> lists of "edge" objects between the nodes.   The point of this would be to 
> batch the exploration/traversal of current frontier to help avoid large 
> bursts of memory being required.
> Thrid, what sort of caching strategy should be introduced for the frontier 
> queries, if any?  And if we do some caching there, how/when should the 
> entries be expired and auto-warmed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12299) More Like This Params Refactor

[Alessandro Benedetti (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459662#comment-16459662
 ] 

Alessandro Benedetti commented on SOLR-12299:
-

It is better to start small, from the params.
other refactors will follow and be part of the bigger Jira issue.

> More Like This Params Refactor
> --
>
> Key: SOLR-12299
> URL: https://issues.apache.org/jira/browse/SOLR-12299
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Reporter: Alessandro Benedetti
>Priority: Major
>
> More Like This ca be refactored to improve the code readability, test 
> coverage and maintenance.
> Scope of this Jira issue is to start the More Like This refactor from the 
> More Like This Params.
> This Jira will not improve the current More Like This but just keep the same 
> functionality with a refactored code.
> Other Jira issues will follow improving the overall code readability, test 
> coverage and maintenance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12299) More Like This Params Refactor

[Alessandro Benedetti (JIRA)]

Alessandro Benedetti created SOLR-12299:
---

 Summary: More Like This Params Refactor
 Key: SOLR-12299
 URL: https://issues.apache.org/jira/browse/SOLR-12299
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: MoreLikeThis
Reporter: Alessandro Benedetti


More Like This ca be refactored to improve the code readability, test coverage 
and maintenance.
Scope of this Jira issue is to start the More Like This refactor from the More 
Like This Params.
This Jira will not improve the current More Like This but just keep the same 
functionality with a refactored code.

Other Jira issues will follow improving the overall code readability, test 
coverage and maintenance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

ApacheCon North America 2018 schedule is now live.

[Rich Bowen]

Dear Apache Enthusiast,

We are pleased to announce our schedule for ApacheCon North America 
2018. ApacheCon will be held September 23-27 at the Montreal Marriott 
Chateau Champlain in Montreal, Canada.


Registration is open! The early bird rate of $575 lasts until July 21, 
at which time it goes up to $800. And the room block at the Marriott 
($225 CAD per night, including wifi) closes on August 24th.


We will be featuring more than 100 sessions on Apache projects. The 
schedule is now online at https://apachecon.com/acna18/


The schedule includes full tracks of content from Cloudstack[1], 
Tomcat[2], and our GeoSpatial community[3].


We will have 4 keynote speakers, two of whom are Apache members, and two 
from the wider community.


On Tuesday, Apache member and former board member Cliff Schmidt will be 
speaking about how Amplio uses technology to educate and improve the 
quality of life of people living in very difficult parts of the 
world[4]. And Apache Fineract VP Myrle Krantz will speak about how Open 
Source banking is helping the global fight against poverty[5].


Then, on Wednesday, we’ll hear from Bridget Kromhout, Principal Cloud 
Developer Advocate from Microsoft, about the really hard problem in 
software - the people[6]. And Euan McLeod, ‎VP VIPER at ‎Comcast will 
show us the many ways that Apache software delivers your favorite shows 
to your living room[7].


ApacheCon will also feature old favorites like the Lightning Talks, the 
Hackathon (running the duration of the event), PGP key signing, and lots 
of hallway-track time to get to know your project community better.


Follow us on Twitter, @ApacheCon, and join the disc...@apachecon.com 
mailing list (send email to discuss-subscr...@apachecon.com) to stay up 
to date with developments. And if your company wants to sponsor this 
event, get in touch at h...@apachecon.com for opportunities that are 
still available.


See you in Montreal!

Rich Bowen
VP Conferences, The Apache Software Foundation
h...@apachecon.com
@ApacheCon

[1] http://cloudstackcollab.org/
[2] http://tomcat.apache.org/conference.html
[3] http://apachecon.dukecon.org/acna/2018/#/schedule?search=geospatial
[4] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/df977fd305a31b903
[5] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/22c6c30412a3828d6
[6] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/fbbb2384fa91ebc6b
[7] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/88d50c3613852c2de

ApacheCon North America 2018 schedule is now live.

[Rich Bowen]

Dear Apache Enthusiast,

We are pleased to announce our schedule for ApacheCon North America 
2018. ApacheCon will be held September 23-27 at the Montreal Marriott 
Chateau Champlain in Montreal, Canada.


Registration is open! The early bird rate of $575 lasts until July 21, 
at which time it goes up to $800. And the room block at the Marriott 
($225 CAD per night, including wifi) closes on August 24th.


We will be featuring more than 100 sessions on Apache projects. The 
schedule is now online at https://apachecon.com/acna18/


The schedule includes full tracks of content from Cloudstack[1], 
Tomcat[2], and our GeoSpatial community[3].


We will have 4 keynote speakers, two of whom are Apache members, and two 
from the wider community.


On Tuesday, Apache member and former board member Cliff Schmidt will be 
speaking about how Amplio uses technology to educate and improve the 
quality of life of people living in very difficult parts of the 
world[4]. And Apache Fineract VP Myrle Krantz will speak about how Open 
Source banking is helping the global fight against poverty[5].


Then, on Wednesday, we’ll hear from Bridget Kromhout, Principal Cloud 
Developer Advocate from Microsoft, about the really hard problem in 
software - the people[6]. And Euan McLeod, ‎VP VIPER at ‎Comcast will 
show us the many ways that Apache software delivers your favorite shows 
to your living room[7].


ApacheCon will also feature old favorites like the Lightning Talks, the 
Hackathon (running the duration of the event), PGP key signing, and lots 
of hallway-track time to get to know your project community better.


Follow us on Twitter, @ApacheCon, and join the disc...@apachecon.com 
mailing list (send email to discuss-subscr...@apachecon.com) to stay up 
to date with developments. And if your company wants to sponsor this 
event, get in touch at h...@apachecon.com for opportunities that are 
still available.


See you in Montreal!

Rich Bowen
VP Conferences, The Apache Software Foundation
h...@apachecon.com
@ApacheCon

[1] http://cloudstackcollab.org/
[2] http://tomcat.apache.org/conference.html
[3] http://apachecon.dukecon.org/acna/2018/#/schedule?search=geospatial
[4] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/df977fd305a31b903
[5] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/22c6c30412a3828d6
[6] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/fbbb2384fa91ebc6b
[7] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/88d50c3613852c2de

ApacheCon North America 2018 schedule is now live.

[Rich Bowen]

Dear Apache Enthusiast,

We are pleased to announce our schedule for ApacheCon North America 
2018. ApacheCon will be held September 23-27 at the Montreal Marriott 
Chateau Champlain in Montreal, Canada.


Registration is open! The early bird rate of $575 lasts until July 21, 
at which time it goes up to $800. And the room block at the Marriott 
($225 CAD per night, including wifi) closes on August 24th.


We will be featuring more than 100 sessions on Apache projects. The 
schedule is now online at https://apachecon.com/acna18/


The schedule includes full tracks of content from Cloudstack[1], 
Tomcat[2], and our GeoSpatial community[3].


We will have 4 keynote speakers, two of whom are Apache members, and two 
from the wider community.


On Tuesday, Apache member and former board member Cliff Schmidt will be 
speaking about how Amplio uses technology to educate and improve the 
quality of life of people living in very difficult parts of the 
world[4]. And Apache Fineract VP Myrle Krantz will speak about how Open 
Source banking is helping the global fight against poverty[5].


Then, on Wednesday, we’ll hear from Bridget Kromhout, Principal Cloud 
Developer Advocate from Microsoft, about the really hard problem in 
software - the people[6]. And Euan McLeod, ‎VP VIPER at ‎Comcast will 
show us the many ways that Apache software delivers your favorite shows 
to your living room[7].


ApacheCon will also feature old favorites like the Lightning Talks, the 
Hackathon (running the duration of the event), PGP key signing, and lots 
of hallway-track time to get to know your project community better.


Follow us on Twitter, @ApacheCon, and join the disc...@apachecon.com 
mailing list (send email to discuss-subscr...@apachecon.com) to stay up 
to date with developments. And if your company wants to sponsor this 
event, get in touch at h...@apachecon.com for opportunities that are 
still available.


See you in Montreal!

Rich Bowen
VP Conferences, The Apache Software Foundation
h...@apachecon.com
@ApacheCon

[1] http://cloudstackcollab.org/
[2] http://tomcat.apache.org/conference.html
[3] http://apachecon.dukecon.org/acna/2018/#/schedule?search=geospatial
[4] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/df977fd305a31b903
[5] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/22c6c30412a3828d6
[6] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/fbbb2384fa91ebc6b
[7] 
http://apachecon.dukecon.org/acna/2018/#/scheduledEvent/88d50c3613852c2de


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11384) add support for distributed graph query

[Jeroen Steggink (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459653#comment-16459653
 ] 

Jeroen Steggink commented on SOLR-11384:


Is this still relevant? We have a streaming expression for graph traversal.

> add support for distributed graph query
> ---
>
> Key: SOLR-11384
> URL: https://issues.apache.org/jira/browse/SOLR-11384
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Kevin Watters
>Priority: Minor
>
> Creating this ticket to track the work that I've done on the distributed 
> graph traversal support in solr.
> Current GraphQuery will only work on a single core, which introduces some 
> limits on where it can be used and also complexities if you want to scale it. 
>  I believe there's a strong desire to support a fully distributed method of 
> doing the Graph Query.  I'm working on a patch, it's not complete yet, but if 
> anyone would like to have a look at the approach and implementation,  I 
> welcome much feedback.  
> The flow for the distributed graph query is almost exactly the same as the 
> normal graph query.  The only difference is how it discovers the "frontier 
> query" at each level of the traversal.  
> When a distribute graph query request comes in, each shard begins by running 
> the root query, to know where to start on it's shard.  Each participating 
> shard then discovers it's edges for the next hop.  Those edges are then 
> broadcast to all other participating shards.  The shard then receives all the 
> parts of the frontier query , assembles it, and executes it.
> This process continues on each shard until there are no new edges left, or 
> the maxDepth of the traversal has finished.
> The approach is to introduce a FrontierBroker that resides as a singleton on 
> each one of the solr nodes in the cluster.  When a graph query is created, it 
> can do a getInstance() on it so it can listen on the frontier parts coming in.
> Initially, I was using an external Kafka broker to handle this, and it did 
> work pretty well.  The new approach is migrating the FrontierBroker to be a 
> request handler in Solr, and potentially to use the SolrJ client to publish 
> the edges to each node in the cluster.
> There are a few outstanding design questions, first being, how do we know 
> what the list of shards are that are participating in the current query 
> request?  Is that easy info to get at?
> Second,  currently, we are serializing a query object between the shards, 
> perhaps we should consider a slightly different abstraction, and serialize 
> lists of "edge" objects between the nodes.   The point of this would be to 
> batch the exploration/traversal of current frontier to help avoid large 
> bursts of memory being required.
> Thrid, what sort of caching strategy should be introduced for the frontier 
> queries, if any?  And if we do some caching there, how/when should the 
> entries be expired and auto-warmed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12298) Index Full nested document Hierarchy For Queries (umbrella issue)

[mosh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459631#comment-16459631
 ] 

mosh edited comment on SOLR-12298 at 5/1/18 12:05 PM:
--

Approach: I see [~janhoy]'s 
[proposal|http://lucene.472066.n3.nabble.com/nesting-Any-way-to-return-the-whole-hierarchical-structure-when-doing-Block-Join-queries-td4265933.html#a4380320]
 as a starting point for this issue, as it addresses most of the problems, as 
well as [this|https://www.youtube.com/watch?v=qV0fIg-LGBE] talk on Solr 
Revolution 2016: "Working with Deeply Nested Documents in Apache Solr", as the 
starting points to this issue.

Firstly, the way a nested document is indexed has to be changed.
 I propose we add the following fields:
 # __parent__
 # __level__
 # __path__

__parent__: This field wild will store the document's parent docId, to be used 
for building the whole hierarchy, using a new document transformer, as 
suggested by Jan on the mailing list.

__level__: This field will store the level of the specified field in the 
document, using an int value. This field can be used for the parentFilter, 
eliminating the need to provide a parentFilter, which will be set by default as 
"__level__:queriedFieldLevel".

__path__: This field will contain the full path, separated by a specific 
reserved char e.g., '.'
 for example: "first.second.third".
 This will enable users to search for a specific path, or provide a regular 
expression to search for fields sharing the same name in different levels of 
the document, filtering using the _level_ key if needed.

To make this happen at index time, changes have to be made to the JSON loader, 
which will add the above fields, as well as the __root__ field, which holds the 
documents top most level docId. This will only happen when a specified 
parameter is added to the update request, e.g. "nested=true".

The new child doc transformer will be able to either reassemble the whole 
document structure, or do so from a specific level, if specified.
 Full hierarchy reconstruction can be done relatively cheaply, using the 
__root__ field to get to the highest level document, and querying the block for 
its children, ordering the query by the __level__ field.


was (Author: moshebla):
Approach: I see [~janhoy]'s 
[proposal|http://lucene.472066.n3.nabble.com/nesting-Any-way-to-return-the-whole-hierarchical-structure-when-doing-Block-Join-queries-td4265933.html#a4380320]
 as a starting point for this issue, as it addresses most of the problems, as 
well as [this|https://www.youtube.com/watch?v=qV0fIg-LGBE] talk on Solr 
Revolution 2016: "Working with Deeply Nested Documents in Apache Solr", as the 
starting points to this issue.

Firstly, the way a nested document is indexed has to be changed.
 I propose we add the following fields:
 # __parent__
 # __level__
 # __path__

_parent_: This field wild will store the document's parent docId, to be used 
for building the whole hierarchy, using a new document transformer, as 
suggested by Jan on the mailing list.

_level_: This field will store the level of the specified field in the 
document, using an int value. This field can be used for the parentFilter, 
eliminating the need to provide a parentFilter, which will be set by default as 
"_level_:queriedFieldLevel".

_path_: This field will contain the full path, separated by a specific reserved 
char e.g., '.'
 for example: "first.second.third".
 This will enable users to search for a specific path, or provide a regular 
expression to search for fields sharing the same name in different levels of 
the document, filtering using the _level_ key if needed.

To make this happen at index time, changes have to be made to the JSON loader, 
which will add the above fields, as well as the _root_ field, which holds the 
documents top most level docId. This will only happen when a specified 
parameter is added to the update request, e.g. "nested=true".

The new child doc transformer will be able to either reassemble the whole 
document structure, or do so from a specific level, if specified.
 Full hierarchy reconstruction can be done relatively cheaply, using the _root_ 
field to get to the highest level document, and querying the block for its 
children, ordering the query by the _level_ field.

> Index Full nested document Hierarchy For Queries (umbrella issue)
> -
>
> Key: SOLR-12298
> URL: https://issues.apache.org/jira/browse/SOLR-12298
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Major
>
> Solr ought to have the ability to index deeply nested objects, while storing 
> the original document hierarchy.
> Currently the client has to index the child document's full path 

[jira] [Comment Edited] (SOLR-12298) Index Full nested document Hierarchy For Queries (umbrella issue)

[mosh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459631#comment-16459631
 ] 

mosh edited comment on SOLR-12298 at 5/1/18 12:04 PM:
--

Approach: I see [~janhoy]'s 
[proposal|http://lucene.472066.n3.nabble.com/nesting-Any-way-to-return-the-whole-hierarchical-structure-when-doing-Block-Join-queries-td4265933.html#a4380320]
 as a starting point for this issue, as it addresses most of the problems, as 
well as [this|https://www.youtube.com/watch?v=qV0fIg-LGBE] talk on Solr 
Revolution 2016: "Working with Deeply Nested Documents in Apache Solr", as the 
starting points to this issue.

Firstly, the way a nested document is indexed has to be changed.
 I propose we add the following fields:
 # __parent__
 # __level__
 # __path__

_parent_: This field wild will store the document's parent docId, to be used 
for building the whole hierarchy, using a new document transformer, as 
suggested by Jan on the mailing list.

_level_: This field will store the level of the specified field in the 
document, using an int value. This field can be used for the parentFilter, 
eliminating the need to provide a parentFilter, which will be set by default as 
"_level_:queriedFieldLevel".

_path_: This field will contain the full path, separated by a specific reserved 
char e.g., '.'
 for example: "first.second.third".
 This will enable users to search for a specific path, or provide a regular 
expression to search for fields sharing the same name in different levels of 
the document, filtering using the _level_ key if needed.

To make this happen at index time, changes have to be made to the JSON loader, 
which will add the above fields, as well as the _root_ field, which holds the 
documents top most level docId. This will only happen when a specified 
parameter is added to the update request, e.g. "nested=true".

The new child doc transformer will be able to either reassemble the whole 
document structure, or do so from a specific level, if specified.
 Full hierarchy reconstruction can be done relatively cheaply, using the _root_ 
field to get to the highest level document, and querying the block for its 
children, ordering the query by the _level_ field.


was (Author: moshebla):
Approach: I see [~janhoy]'s 
[proposal|http://lucene.472066.n3.nabble.com/nesting-Any-way-to-return-the-whole-hierarchical-structure-when-doing-Block-Join-queries-td4265933.html#a4380320]
 as a starting point for this issue, as it addresses most of the problems, as 
well as [this|https://www.youtube.com/watch?v=qV0fIg-LGBE] talk on Solr 
Revolution 2016: "Working with Deeply Nested Documents in Apache Solr", as the 
starting points to this issue.

Firstly, the way a nested document is indexed has to be changed.
I propose we add the following fields:
 # _parent_
 # _level_
 # _path_

_parent_: This field wild will store the document's parent docId, to be used 
for building the whole hierarchy, using a new document transformer, as 
suggested by Jan on the mailing list.

_level_: This field will store the level of the specified field in the 
document, using an int value. This field can be used for the parentFilter, 
eliminating the need to provide a parentFilter, which will be set by default as 
"_level_:queriedFieldLevel".

_path_: This field will contain the full path, separated by a specific reserved 
char e.g., '.'
for example: "first.second.third".
This will enable users to search for a specific path, or provide a regular 
expression to search for fields sharing the same name in different levels of 
the document, filtering using the _level_ key if needed.

To make this happen at index time, changes have to be made to the JSON loader, 
which will add the above fields, as well as the _root_ field, which holds the 
documents top most level docId. This will only happen when a specified 
parameter is added to the update request, e.g. "nested=true".

The new child doc transformer will be able to either reassemble the whole 
document structure, or do so from a specific level, if specified.
Full hierarchy reconstruction can be done relatively cheaply, using the _root_ 
field to get to the highest level document, and querying the block for its 
children, ordering the query by the _level_ field.

> Index Full nested document Hierarchy For Queries (umbrella issue)
> -
>
> Key: SOLR-12298
> URL: https://issues.apache.org/jira/browse/SOLR-12298
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Major
>
> Solr ought to have the ability to index deeply nested objects, while storing 
> the original document hierarchy.
> Currently the client has to index the child document's full path and level to 
> manually 

[jira] [Comment Edited] (SOLR-12298) Index Full nested document Hierarchy For Queries (umbrella issue)

[mosh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459631#comment-16459631
 ] 

mosh edited comment on SOLR-12298 at 5/1/18 12:03 PM:
--

Approach: I see [~janhoy]'s 
[proposal|http://lucene.472066.n3.nabble.com/nesting-Any-way-to-return-the-whole-hierarchical-structure-when-doing-Block-Join-queries-td4265933.html#a4380320]
 as a starting point for this issue, as it addresses most of the problems, as 
well as [this|https://www.youtube.com/watch?v=qV0fIg-LGBE] talk on Solr 
Revolution 2016: "Working with Deeply Nested Documents in Apache Solr", as the 
starting points to this issue.

Firstly, the way a nested document is indexed has to be changed.
I propose we add the following fields:
 # _parent_
 # _level_
 # _path_

_parent_: This field wild will store the document's parent docId, to be used 
for building the whole hierarchy, using a new document transformer, as 
suggested by Jan on the mailing list.

_level_: This field will store the level of the specified field in the 
document, using an int value. This field can be used for the parentFilter, 
eliminating the need to provide a parentFilter, which will be set by default as 
"_level_:queriedFieldLevel".

_path_: This field will contain the full path, separated by a specific reserved 
char e.g., '.'
for example: "first.second.third".
This will enable users to search for a specific path, or provide a regular 
expression to search for fields sharing the same name in different levels of 
the document, filtering using the _level_ key if needed.

To make this happen at index time, changes have to be made to the JSON loader, 
which will add the above fields, as well as the _root_ field, which holds the 
documents top most level docId. This will only happen when a specified 
parameter is added to the update request, e.g. "nested=true".

The new child doc transformer will be able to either reassemble the whole 
document structure, or do so from a specific level, if specified.
Full hierarchy reconstruction can be done relatively cheaply, using the _root_ 
field to get to the highest level document, and querying the block for its 
children, ordering the query by the _level_ field.


was (Author: moshebla):
Approach: I see [~janhoy]'s proposal as a starting point for this issue, as it 
addresses most of the problems

> Index Full nested document Hierarchy For Queries (umbrella issue)
> -
>
> Key: SOLR-12298
> URL: https://issues.apache.org/jira/browse/SOLR-12298
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Major
>
> Solr ought to have the ability to index deeply nested objects, while storing 
> the original document hierarchy.
> Currently the client has to index the child document's full path and level to 
> manually reconstruct the original document structure, since the children are 
> flattened and returned in the reserved "_childDocuments_" key.
> Ideally you could index a nested document, having Solr transparently add the 
> required fields while providing a document transformer to rebuild the 
> original document's hierarchy.
>  
> This issue is an umbrella issue for the particular tasks that will make it 
> all happen – either subtasks or issue linking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8273) Add a ConditionalTokenFilter

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459643#comment-16459643
 ] 

Alan Woodward commented on LUCENE-8273:
---

bq. it seems you need to make the ConditionalTokenFilterFactory implement the 
resourceloaderaware stuff always

Just spotted Robert's comment here, I've added a new patch which fixes this.  
Thanks!

> Add a ConditionalTokenFilter
> 
>
> Key: LUCENE-8273
> URL: https://issues.apache.org/jira/browse/LUCENE-8273
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch, 
> LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch
>
>
> Spinoff of LUCENE-8265.  It would be useful to be able to wrap a TokenFilter 
> in such a way that it could optionally be bypassed based on the current state 
> of the TokenStream.  This could be used to, for example, only apply 
> WordDelimiterFilter to terms that contain hyphens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8273) Add a ConditionalTokenFilter

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8273:
--
Attachment: LUCENE-8273.patch

> Add a ConditionalTokenFilter
> 
>
> Key: LUCENE-8273
> URL: https://issues.apache.org/jira/browse/LUCENE-8273
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch, 
> LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch
>
>
> Spinoff of LUCENE-8265.  It would be useful to be able to wrap a TokenFilter 
> in such a way that it could optionally be bypassed based on the current state 
> of the TokenStream.  This could be used to, for example, only apply 
> WordDelimiterFilter to terms that contain hyphens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups

[Lucene/Solr QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459634#comment-16459634
 ] 

Lucene/Solr QA commented on SOLR-8998:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 3 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  4m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  4m 43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  4m 43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  4m 43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m 
55s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}136m 55s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-8998 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12921176/SOLR-8998.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP 
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / d92b891 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 8 2015 |
| Default Java | 1.8.0_172 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/70/testReport/ |
| modules | C: solr solr/core U: solr |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/70/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
>  Issue Type: New Feature
>  Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR-8998.patch, SOLR-8998.patch, 
> SOLR_8998.patch, SOLR_8998.patch, SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and 
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on 
> derived values from their children.  The most important part (and the most 
> difficult part) will be the external API.
> [~mkhludnev] says
> {quote}
> The bottom line is to introduce {{uniqueBlock(\_root_)}} aggregation, which 
> is supposed to be faster than {{unique(\_root_)}} and purposed for blocked 
> index only. For now it it supports singlevalue string fields, docValues 
> usually make sense.   
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12298) Index Full nested document Hierarchy For Queries (umbrella issue)

[mosh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459631#comment-16459631
 ] 

mosh edited comment on SOLR-12298 at 5/1/18 11:42 AM:
--

Approach: I see [~janhoy]'s proposal as a starting point for this issue, as it 
addresses most of the problems


was (Author: moshebla):
A similar issue has been brought up in the mailing list

> Index Full nested document Hierarchy For Queries (umbrella issue)
> -
>
> Key: SOLR-12298
> URL: https://issues.apache.org/jira/browse/SOLR-12298
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Major
>
> Solr ought to have the ability to index deeply nested objects, while storing 
> the original document hierarchy.
> Currently the client has to index the child document's full path and level to 
> manually reconstruct the original document structure, since the children are 
> flattened and returned in the reserved "_childDocuments_" key.
> Ideally you could index a nested document, having Solr transparently add the 
> required fields while providing a document transformer to rebuild the 
> original document's hierarchy.
>  
> This issue is an umbrella issue for the particular tasks that will make it 
> all happen – either subtasks or issue linking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12298) Index Full nested document Hierarchy For Queries (umbrella issue)

[mosh (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459631#comment-16459631
 ] 

mosh commented on SOLR-12298:
-

A similar issue has been brought up in the mailing list

> Index Full nested document Hierarchy For Queries (umbrella issue)
> -
>
> Key: SOLR-12298
> URL: https://issues.apache.org/jira/browse/SOLR-12298
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: mosh
>Priority: Major
>
> Solr ought to have the ability to index deeply nested objects, while storing 
> the original document hierarchy.
> Currently the client has to index the child document's full path and level to 
> manually reconstruct the original document structure, since the children are 
> flattened and returned in the reserved "_childDocuments_" key.
> Ideally you could index a nested document, having Solr transparently add the 
> required fields while providing a document transformer to rebuild the 
> original document's hierarchy.
>  
> This issue is an umbrella issue for the particular tasks that will make it 
> all happen – either subtasks or issue linking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12298) Index Full nested document Hierarchy For Queries (umbrella issue)

[mosh (JIRA)]

mosh created SOLR-12298:
---

 Summary: Index Full nested document Hierarchy For Queries 
(umbrella issue)
 Key: SOLR-12298
 URL: https://issues.apache.org/jira/browse/SOLR-12298
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: mosh


Solr ought to have the ability to index deeply nested objects, while storing 
the original document hierarchy.
Currently the client has to index the child document's full path and level to 
manually reconstruct the original document structure, since the children are 
flattened and returned in the reserved "_childDocuments_" key.

Ideally you could index a nested document, having Solr transparently add the 
required fields while providing a document transformer to rebuild the original 
document's hierarchy.

 

This issue is an umbrella issue for the particular tasks that will make it all 
happen – either subtasks or issue linking.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12286) Wrap log instances in "if (LOG.isLevelEnabled) { log... }

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459524#comment-16459524
 ] 

Uwe Schindler edited comment on SOLR-12286 at 5/1/18 10:37 AM:
---

Unfortunately we use slf4j, because log4j has a better feature on top (I hope 
it gets added to slf4j, too): As said before paramters with {{"\{\}"}} make 
much sense, but won't help if you want to pass a complex string generated 
on-they fly. In this case, log4j offers to pass a Java8 lambda:

{code:java}
log.trace("My complex operation took {} and produces no additional objects in 
production when you print a list like {}!",
  () -> Duration.between(startInstant, Instant.now()), () -> String.join("; ", 
list));
{code}

Of course this generates the lambda using the invokedynamic, but it's not 
executed. It is static and compiled one time. I use this now all the time for 
stuff like the above (when the string is complex to be generated, like if you 
print contents of a List or Set).

P.S.: BTW in the initial description of the issue:

{quote}
{code:java}
// "test" is an instance of a class with an overridden toString() method.
// These generate a zillion spurious objects.
logger.info("This is a test {} {} {}", test, rand.nextInt(), "whatever");
{code}
{quote}

The comment is wrong: the method toString() is *not* called! The {{info()}} 
method takes {{Object}} as parameter, so {{test}} is passed as is, with no 
object generated. The problem is more the {{rand.nextInt()}} as it does 
autoboxing (which should be eliminated by Hotspot). Keep in mind: When you run 
this stuff with a debugger it produces more objects than in production, as 
hotspot can't jump in and eliminate those objects. Autoboxing is in those cases 
100% removed by Hotspot, unless you use a debugger!


was (Author: thetaphi):
Unfortunately we use slf4j, because log4j has a better feature on top (I hope 
it gets added to slf4j, too): As said before paramters with {{"\{\}"}} make 
much sense, but won't help if you want to pass a complex string generated 
on-they fly. In this case, log4j offers to pass a Java8 lambda:

{code:java}
log.trace("My complex operation took {} and produces no additional objects in 
production when you print a list!",
  () -> Duration.between(startInstant, Instant.now()), () -> String.join("; ", 
list));
{code}

Of course this generates the lambda using the invokedynamic, but it's not 
executed. It is static and compiled one time. I use this now all the time for 
stuff like the above (when the string is complex to be generated, like if you 
print contents of a List or Set).

P.S.: BTW in the initial description of the issue:

{quote}
{code:java}
// "test" is an instance of a class with an overridden toString() method.
// These generate a zillion spurious objects.
logger.info("This is a test {} {} {}", test, rand.nextInt(), "whatever");
{code}
{quote}

The comment is wrong: the method toString() is *not* called! The {{info()}} 
method takes {{Object}} as parameter, so {{test}} is passed as is, with no 
object generated. The problem is more the {{rand.nextInt()}} as it does 
autoboxing (which should be eliminated by Hotspot). Keep in mind: When you run 
this stuff with a debugger it produces more objects than in production, as 
hotspot can't jump in and eliminate those objects. Autoboxing is in those cases 
100% removed by Hotspot, unless you use a debugger!

> Wrap log instances in "if (LOG.isLevelEnabled) { log... }
> -
>
> Key: SOLR-12286
> URL: https://issues.apache.org/jira/browse/SOLR-12286
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> I've been playing around with the question "are objects generated when 
> logging if I use pattern." and "it depends" (tm). I'll attach a test 
> program for anyone to use. Attach VisualVM to it and sample memory when 
> various patterns are enabled.
> The nub of it is this: with the log level set at ERROR, no messages from any 
> of these are printed, yet the number of objects created is vastly different:
> {code}
>   while (true) {
>   // "test" is an instance of a class with an overridden toString() 
> method.
>   // These generate a zillion spurious objects.
>   logger.info("This is a test {} {} {}", test, rand.nextInt(), 
> "whatever");
>   logger.info("This is a test {}", rand.nextInt());
>   logger.info("This is really bad" + test);
>   
>   // These don't generate spurious objects
>   logger.info("This is a test {} {}", test, "whatever");
>   logger.info("This is really bad" + "whatever");
>   }
> {code}
> Simply generating a new random number doesn't create zillions of 

[jira] [Commented] (LUCENE-8273) Add a ConditionalTokenFilter

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459564#comment-16459564
 ] 

Alan Woodward commented on LUCENE-8273:
---

Updated patch.  {{ConditionalTokenFilterFactory}} is now a top-level class, 
distinct from {{ConditionBuilder}}.  I've added a TermExclusionFilter that 
accepts a list of terms and only runs its child filters if the current token is 
not in its list, and demonstrated how to use it in TestCustomAnalyzer.  At the 
moment it just reads a word file, but we can expand it to accept patterns or a 
directly passed in list of terms in follow ups.  I've also changed the 
CustomAnalyzerBuilder to use {{when}} rather than {{ifXXX}} - thanks for the 
suggestion Steve!

> Add a ConditionalTokenFilter
> 
>
> Key: LUCENE-8273
> URL: https://issues.apache.org/jira/browse/LUCENE-8273
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch, 
> LUCENE-8273.patch, LUCENE-8273.patch
>
>
> Spinoff of LUCENE-8265.  It would be useful to be able to wrap a TokenFilter 
> in such a way that it could optionally be bypassed based on the current state 
> of the TokenStream.  This could be used to, for example, only apply 
> WordDelimiterFilter to terms that contain hyphens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8273) Add a ConditionalTokenFilter

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8273:
--
Attachment: LUCENE-8273.patch

> Add a ConditionalTokenFilter
> 
>
> Key: LUCENE-8273
> URL: https://issues.apache.org/jira/browse/LUCENE-8273
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch, 
> LUCENE-8273.patch, LUCENE-8273.patch
>
>
> Spinoff of LUCENE-8265.  It would be useful to be able to wrap a TokenFilter 
> in such a way that it could optionally be bypassed based on the current state 
> of the TokenStream.  This could be used to, for example, only apply 
> WordDelimiterFilter to terms that contain hyphens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12286) Wrap log instances in "if (LOG.isLevelEnabled) { log... }

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459524#comment-16459524
 ] 

Uwe Schindler commented on SOLR-12286:
--

Unfortunately we use slf4j, because log4j has a better feature on top (I hope 
it gets added to slf4j, too): As said before paramters with {{"\{\}"}} make 
much sense, but won't help if you want to pass a complex string generated 
on-they fly. In this case, log4j offers to pass a Java8 lambda:

{code:java}
log.trace("My complex operation took {} and produces no additional objects in 
production when you print a list!",
  () -> Duration.between(startInstant, Instant.now()), () -> String.join("; ", 
list));
{code}

Of course this generates the lambda using the invokedynamic, but it's not 
executed. It is static and compiled one time. I use this now all the time for 
stuff like the above (when the string is complex to be generated, like if you 
print contents of a List or Set).

P.S.: BTW in the initial description of the issue:

{quote}
{code:java}
// "test" is an instance of a class with an overridden toString() method.
// These generate a zillion spurious objects.
logger.info("This is a test {} {} {}", test, rand.nextInt(), "whatever");
{code}

The comment is wrong: the method toString() is *not* called! The {{info()}} 
method takes {{Object}} as parameter, so {{test}} is passed as is, with no 
object generated. The problem is more the {{rand.nextInt()}} as it does 
autoboxing (which should be eliminated by Hotspot). Keep in mind: When you run 
this stuff with a debugger it produces more objects than in production, as 
hotspot can't jump in and eliminate those objects. Autoboxing is in those cases 
100% removed by Hotspot, unless you use a debugger!

> Wrap log instances in "if (LOG.isLevelEnabled) { log... }
> -
>
> Key: SOLR-12286
> URL: https://issues.apache.org/jira/browse/SOLR-12286
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> I've been playing around with the question "are objects generated when 
> logging if I use pattern." and "it depends" (tm). I'll attach a test 
> program for anyone to use. Attach VisualVM to it and sample memory when 
> various patterns are enabled.
> The nub of it is this: with the log level set at ERROR, no messages from any 
> of these are printed, yet the number of objects created is vastly different:
> {code}
>   while (true) {
>   // "test" is an instance of a class with an overridden toString() 
> method.
>   // These generate a zillion spurious objects.
>   logger.info("This is a test {} {} {}", test, rand.nextInt(), 
> "whatever");
>   logger.info("This is a test {}", rand.nextInt());
>   logger.info("This is really bad" + test);
>   
>   // These don't generate spurious objects
>   logger.info("This is a test {} {}", test, "whatever");
>   logger.info("This is really bad" + "whatever");
>   }
> {code}
> Simply generating a new random number doesn't create zillions of objects.
> I don't particularly care _why_ the differences exist, the take-away is that 
> if we simply establish the rule "wrap log.level() messages with if..." then 
> we'll avoid the problems altogether. That's _much_ easier to both remember 
> and enforce than trying to understand and remember when pattern A is 
> acceptable and pattern B is not.
> Maybe we could even make this a precommit failure?
> Solr has enough "interesting" things happen re: GC that we don't need to 
> needlessly add to GC pressure.
> Parameterizing is still a good idea as in SOLR-10415 (I'll link)
> Here's the full program, there's not a lot of sophistication here:
> {code}
> import org.apache.logging.log4j.Level;
> import org.apache.logging.log4j.Logger;
> import org.apache.logging.log4j.LogManager;
> import org.apache.logging.log4j.core.LoggerContext;
> import org.apache.logging.log4j.core.config.Configuration;
> import org.apache.logging.log4j.core.config.LoggerConfig;
> import java.util.Random;
> public class Log4J2Test {
>   // Define a static logger variable so that it references the
>   // Logger instance named "MyApp".
>   private static final Logger logger = LogManager.getLogger(Log4J2Test.class);
>   static Random rand = new Random();
>   public static void main(final String... args) {
> // Set up a simple configuration that logs on the console.
> logger.trace("Entering application.");
> LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
> Configuration config = ctx.getConfiguration();
> LoggerConfig loggerConfig = 
> config.getLoggerConfig(LogManager.ROOT_LOGGER_NAME);
> loggerConfig.setLevel(Level.ERROR);
> 

[jira] [Comment Edited] (SOLR-12286) Wrap log instances in "if (LOG.isLevelEnabled) { log... }

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459524#comment-16459524
 ] 

Uwe Schindler edited comment on SOLR-12286 at 5/1/18 8:20 AM:
--

Unfortunately we use slf4j, because log4j has a better feature on top (I hope 
it gets added to slf4j, too): As said before paramters with {{"\{\}"}} make 
much sense, but won't help if you want to pass a complex string generated 
on-they fly. In this case, log4j offers to pass a Java8 lambda:

{code:java}
log.trace("My complex operation took {} and produces no additional objects in 
production when you print a list!",
  () -> Duration.between(startInstant, Instant.now()), () -> String.join("; ", 
list));
{code}

Of course this generates the lambda using the invokedynamic, but it's not 
executed. It is static and compiled one time. I use this now all the time for 
stuff like the above (when the string is complex to be generated, like if you 
print contents of a List or Set).

P.S.: BTW in the initial description of the issue:

{quote}
{code:java}
// "test" is an instance of a class with an overridden toString() method.
// These generate a zillion spurious objects.
logger.info("This is a test {} {} {}", test, rand.nextInt(), "whatever");
{code}
{quote}

The comment is wrong: the method toString() is *not* called! The {{info()}} 
method takes {{Object}} as parameter, so {{test}} is passed as is, with no 
object generated. The problem is more the {{rand.nextInt()}} as it does 
autoboxing (which should be eliminated by Hotspot). Keep in mind: When you run 
this stuff with a debugger it produces more objects than in production, as 
hotspot can't jump in and eliminate those objects. Autoboxing is in those cases 
100% removed by Hotspot, unless you use a debugger!


was (Author: thetaphi):
Unfortunately we use slf4j, because log4j has a better feature on top (I hope 
it gets added to slf4j, too): As said before paramters with {{"\{\}"}} make 
much sense, but won't help if you want to pass a complex string generated 
on-they fly. In this case, log4j offers to pass a Java8 lambda:

{code:java}
log.trace("My complex operation took {} and produces no additional objects in 
production when you print a list!",
  () -> Duration.between(startInstant, Instant.now()), () -> String.join("; ", 
list));
{code}

Of course this generates the lambda using the invokedynamic, but it's not 
executed. It is static and compiled one time. I use this now all the time for 
stuff like the above (when the string is complex to be generated, like if you 
print contents of a List or Set).

P.S.: BTW in the initial description of the issue:

{quote}
{code:java}
// "test" is an instance of a class with an overridden toString() method.
// These generate a zillion spurious objects.
logger.info("This is a test {} {} {}", test, rand.nextInt(), "whatever");
{code}

The comment is wrong: the method toString() is *not* called! The {{info()}} 
method takes {{Object}} as parameter, so {{test}} is passed as is, with no 
object generated. The problem is more the {{rand.nextInt()}} as it does 
autoboxing (which should be eliminated by Hotspot). Keep in mind: When you run 
this stuff with a debugger it produces more objects than in production, as 
hotspot can't jump in and eliminate those objects. Autoboxing is in those cases 
100% removed by Hotspot, unless you use a debugger!

> Wrap log instances in "if (LOG.isLevelEnabled) { log... }
> -
>
> Key: SOLR-12286
> URL: https://issues.apache.org/jira/browse/SOLR-12286
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> I've been playing around with the question "are objects generated when 
> logging if I use pattern." and "it depends" (tm). I'll attach a test 
> program for anyone to use. Attach VisualVM to it and sample memory when 
> various patterns are enabled.
> The nub of it is this: with the log level set at ERROR, no messages from any 
> of these are printed, yet the number of objects created is vastly different:
> {code}
>   while (true) {
>   // "test" is an instance of a class with an overridden toString() 
> method.
>   // These generate a zillion spurious objects.
>   logger.info("This is a test {} {} {}", test, rand.nextInt(), 
> "whatever");
>   logger.info("This is a test {}", rand.nextInt());
>   logger.info("This is really bad" + test);
>   
>   // These don't generate spurious objects
>   logger.info("This is a test {} {}", test, "whatever");
>   logger.info("This is really bad" + "whatever");
>   }
> {code}
> Simply generating a new random number doesn't create zillions of objects.
> I don't 

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_162) - Build # 21938 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21938/
Java: 64bit/jdk1.8.0_162 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at 
__randomizedtesting.SeedInfo.seed([6E6F9484BC39052E:DA4A20625F67603]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:187)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13192 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> 

[JENKINS] Lucene-Solr-Tests-7.3 - Build # 54 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.3/54/

2 tests failed.
FAILED:  
org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth

Error Message:
Error from server at https://127.0.0.1:42980/solr: KeeperErrorCode = NoNode for 
/overseer/collection-queue-work/qnr-00

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:42980/solr: KeeperErrorCode = NoNode for 
/overseer/collection-queue-work/qnr-00
at 
__randomizedtesting.SeedInfo.seed([DB9E3CE659F631AB:50CA8D81FB6970D7]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1105)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:885)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:818)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:200)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:172)
at 
org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth(TestSSLRandomization.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at