[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+43) - Build # 1405 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1405/
Java: 64bit/jdk-10-ea+43 -XX:-UseCompressedOops -XX:+UseSerialGC

8 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([C0F57B1E7E34975A]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager

Error Message:
Captured an uncaught exception in thread: Thread[id=1652, name=Thread-1403, 
state=RUNNABLE, group=TGRP-TestSearcherManager]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=1652, name=Thread-1403, state=RUNNABLE, 
group=TGRP-TestSearcherManager]
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is 
closed
at __randomizedtesting.SeedInfo.seed([C0F57B1E7E34975A]:0)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:897)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:911)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1725)
at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1464)
at 
org.apache.lucene.search.TestSearcherManager$8.run(TestSearcherManager.java:574)
Caused by: java.nio.file.FileSystemException: 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/build/core/test/J1/temp/lucene.search.TestSearcherManager_C0F57B1E7E34975A-001/tempDir-001/_4b_TestBloomFilteredLucenePostings_0.tip:
 Too many open files
at 
org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
at 
org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
at 
org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
at 
org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
at java.base/java.nio.file.Files.newOutputStream(Files.java:218)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:413)
at 
org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:409)
at 
org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
at 
org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:665)
at 
org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
at 
org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
at 
org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.(BlockTreeTermsWriter.java:277)
at 
org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsConsumer(Lucene50PostingsFormat.java:427)
at 
org.apache.lucene.codecs.bloom.BloomFilteringPostingsFormat.fieldsConsumer(BloomFilteringPostingsFormat.java:143)
at 
org.apache.lucene.codecs.bloom.TestBloomFilteredLucenePostings.fieldsConsumer(TestBloomFilteredLucenePostings.java:65)
at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:138)
at 
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:108)
at 
org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:162)
at 
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:452)
at 
org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:557)
at 
org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:673)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:453)
at 
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:293)
at 
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:268)
at 
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:258)
at 
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
at 
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:156)
at 
org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:58)
at 
org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:176)
at 
org.apache.lucene.search.ReferenceManager.maybeRefreshBlocking(ReferenceManager.java:253)
at 
org.apache.lucene.search.TestSearcherManager$10.run(TestSearcherManager.java:634)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager

Error Message:
Captured an uncaught exception in thread: Thread[id=1655, name=Thread-1406, 
state=RUNNABLE, 

[JENKINS] Lucene-Solr-SmokeRelease-7.x - Build # 154 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/154/

No tests ran.

Build Log:
[...truncated 28782 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist
 [copy] Copying 491 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 215 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] Java 9 JAVA_HOME=/home/jenkins/tools/java/latest1.9
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (32.6 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-7.3.0-src.tgz...
   [smoker] 31.7 MB in 0.03 sec (1170.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.3.0.tgz...
   [smoker] 73.2 MB in 0.07 sec (1028.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.3.0.zip...
   [smoker] 83.8 MB in 0.08 sec (1037.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-7.3.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.3.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.3.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 217 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker] run tests w/ Java 9 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 9...
   [smoker]   got 217 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.00 sec (257.3 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-7.3.0-src.tgz...
   [smoker] 54.1 MB in 0.05 sec (1037.4 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.3.0.tgz...
   [smoker] 151.0 MB in 0.17 sec (913.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.3.0.zip...
   [smoker] 152.0 MB in 0.16 sec (938.2 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-7.3.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-7.3.0.tgz...
   [smoker]   **WARNING**: skipping check of 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
   [smoker]   **WARNING**: skipping check of 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
   [smoker] copying unpacked distribution for Java 8 ...
   [smoker] test solr example w/ Java 8...
   [smoker]   start Solr instance 
(log=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0-java8/solr-example.log)...
   [smoker] No process found for Solr node running on port 8983
   [smoker]   Running techproducts example on port 8983 from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0-java8
   [smoker] *** [WARN] *** Your open file 

[JENKINS] Lucene-Solr-repro - Build # 83 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/83/

[...truncated 50 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-Tests-7.x/432/consoleText

[repro] Revision: 6d712c5e4b2fc8f4ac6dfec2eac4a17386c978f0

[repro] Repro line:  ant test  -Dtestcase=TriggerIntegrationTest 
-Dtests.method=testEventQueue -Dtests.seed=5776B9C607767CE6 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=et-EE 
-Dtests.timezone=NZ-CHAT -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestAuthenticationFramework 
-Dtests.method=testBasics -Dtests.seed=5776B9C607767CE6 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=ar-LY -Dtests.timezone=Asia/Saigon 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestRankingFeature 
-Dtests.seed=210D9FAD4571C3AF -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=zh-HK -Dtests.timezone=Asia/Katmandu -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
137e647f2c9ab6885e21471f3219488eed512b32
[repro] git checkout 6d712c5e4b2fc8f4ac6dfec2eac4a17386c978f0

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/contrib/ltr
[repro]   TestRankingFeature
[repro]solr/core
[repro]   TestAuthenticationFramework
[repro]   TriggerIntegrationTest
[repro] ant compile-test

[...truncated 2579 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestRankingFeature" -Dtests.showOutput=onerror 
-Dtests.seed=210D9FAD4571C3AF -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=zh-HK -Dtests.timezone=Asia/Katmandu -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[...truncated 68 lines...]
[repro] ant compile-test

[...truncated 1331 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 
-Dtests.class="*.TestAuthenticationFramework|*.TriggerIntegrationTest" 
-Dtests.showOutput=onerror -Dtests.seed=5776B9C607767CE6 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=ar-LY -Dtests.timezone=Asia/Saigon 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 11471 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.TestAuthenticationFramework
[repro]   0/5 failed: org.apache.solr.ltr.feature.TestRankingFeature
[repro]   2/5 failed: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[repro] git checkout 137e647f2c9ab6885e21471f3219488eed512b32

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1692 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1692/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionOnCommitTest.test

Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 
127.0.0.1:65197 within 45000 ms

Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: 
Could not connect to ZooKeeper 127.0.0.1:65197 within 45000 ms
at 
__randomizedtesting.SeedInfo.seed([332666DC6CED4CF9:BB725906C2112101]:0)
at 
org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:183)
at 
org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:120)
at 
org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:110)
at 
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:88)
at 
org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:82)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.distribSetUp(AbstractDistribZkTestBase.java:81)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.distribSetUp(AbstractFullDistribZkTestBase.java:233)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:963)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: Could not connect to 
ZooKeeper 127.0.0.1:65197 within 45000 ms
at 
org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:232)
at 

[jira] [Comment Edited] (SOLR-11267) Add support for "add-distinct" atomic update operation

[Amrit Sarkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372507#comment-16372507
 ] 

Amrit Sarkar edited comment on SOLR-11267 at 2/22/18 7:14 AM:
--

examples on how to use {{add-distinct}}:

pass as list:
{code}
{"id":"mydoc",
 "price":{"set":99},
 "popularity":{"inc":20},
 "categories":{"add":["toys","games"]},
 "sub_categories":{"add-distinct":["children games","PG games"]},
 "promo_ids":{"remove":"a123x"},
 "tags":{"remove":["free_to_try","on_sale"]}
}
{code}
pass as singleton value:
{code}
{"id":"mydoc",
 "sub_categories":{"add-distinct":"V games"}
}
{code}


was (Author: sarkaramr...@gmail.com):
examples on how to use {{add-distinct}}:

pass as list:
{code}
{"id":"mydoc",
 "price":{"set":99},
 "popularity":{"inc":20},
 "categories":{"add":["toys","games"]},
 "sub_categories":{"add-distinct":["children games","PG games"]},
 "promo_ids":{"remove":"a123x"},
 "tags":{"remove":["free_to_try","on_sale"]}
}

pass as singleton value:
{code}
{code}
{"id":"mydoc",
 "sub_categories":{"add-distinct":"V games"}
}
{code}

> Add support for "add-distinct" atomic update operation
> --
>
> Key: SOLR-11267
> URL: https://issues.apache.org/jira/browse/SOLR-11267
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Attachments: SOLR-11267.patch, SOLR-11267.patch
>
>
> Often, a multivalued field is used as a set of values. Since multivalued 
> fields are more like lists than sets, users do two consecutive operations, 
> remove and add, to insert an element into the field and also maintain the 
> set's property of only having unique elements.
> Proposing a new single operation, called "add-distinct" (which essentially 
> means "add-if-doesn't exist") for this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11267) Add support for "add-distinct" atomic update operation

[Amrit Sarkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372507#comment-16372507
 ] 

Amrit Sarkar commented on SOLR-11267:
-

examples on how to use {{add-distinct}}:

pass as list:
{code}
{"id":"mydoc",
 "price":{"set":99},
 "popularity":{"inc":20},
 "categories":{"add":["toys","games"]},
 "sub_categories":{"add-distinct":["children games","PG games"]},
 "promo_ids":{"remove":"a123x"},
 "tags":{"remove":["free_to_try","on_sale"]}
}

pass as singleton value:
{code}
{code}
{"id":"mydoc",
 "sub_categories":{"add-distinct":"V games"}
}
{code}

> Add support for "add-distinct" atomic update operation
> --
>
> Key: SOLR-11267
> URL: https://issues.apache.org/jira/browse/SOLR-11267
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Attachments: SOLR-11267.patch, SOLR-11267.patch
>
>
> Often, a multivalued field is used as a set of values. Since multivalued 
> fields are more like lists than sets, users do two consecutive operations, 
> remove and add, to insert an element into the field and also maintain the 
> set's property of only having unique elements.
> Proposing a new single operation, called "add-distinct" (which essentially 
> means "add-if-doesn't exist") for this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Test failures are out of control......

[Uwe Schindler]

Hi Hoss,

great and very helpful! Does it only contain Solr or are there also Lucene 
tests reported?

Uwe

-
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

> -Original Message-
> From: Chris Hostetter [mailto:hossman_luc...@fucit.org]
> Sent: Thursday, February 22, 2018 1:56 AM
> To: dev@lucene.apache.org
> Subject: Re: Test failures are out of control..
> 
> 
> : * Hoss has worked on aggregating all test failures from the 3 Jenkins
> : systems (ASF, Policeman, and Steve's), downloading the test results & logs,
> : and running some reports/stats on failures. He should be ready to share
> : this more publicly soon.
> 
> I think Steve's linked to some of this before from jira comments, but it
> was only recently I realized i've never explicitly said to the list "Hey
> folks, here's a thing i've been working on" ...
> 
>   http://fucit.org/solr-jenkins-reports/
>   https://github.com/hossman/jenkins-reports/
> 
> The most interesting bit is probably here...
> 
>   http://fucit.org/solr-jenkins-reports/failure-report.html
> 
> ...but there are currently a few caveats:
> 
> 1) there's some noise inthe '7days' data because I wasn't accounting for
> the way jenkins reports some types of failure -- that will gradually clean
> itself up
> 
> 2) I think i've been been blocked by builds.apache.org, so at the moment
> the data seems to just be from the sarowe & policeman jenkins failures.
> 
> 3) allthough the system is archiving the past 7 days worth of jenkins logs
> for any jobs with failures, there is currently no easy way to download
> the relevant log(s) from that failure report -- you currently have to
> download a CSV file like this one to corrolate the test failures to the
> jenkins job, and then go look for that job in the job-data dirs...
> 
>   http://fucit.org/solr-jenkins-reports/reports/7days-method-failures.csv
>   http://fucit.org/solr-jenkins-reports/job-data/
> 
> (My hope is to make #3 trivial from failure-report.html -- so you can say
> "hey weird, this test has failed X times, let's go download those logs."
> right from a single screen in your browser)
> 
> 
> 
> 
> -Hoss
> http://www.lucidworks.com/
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-7.x - Build # 433 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/433/

2 tests failed.
FAILED:  org.apache.solr.cloud.ReplaceNodeNoTargetTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([756CE3BD36E5903:8F02F1E17D9234FB]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testSelectedCollections

Error Message:
The operations computed by ComputePlanAction should not be 
nullSolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], 
BEFORE_ACTION=[compute_plan, null]}

Stack Trace:
java.lang.AssertionError: The operations computed by ComputePlanAction should 
not be 

Re: Test failures are out of control......

[Dawid Weiss]

Don't know, Yonik... If I make a change I am interested in regressions from
the start state; running flaky tests makes it impossible and frustrating
(and pointless in my opinion). I don't think my expectations are that much
off from the average - you may wake up being the only person who has the
defaults enabled, which seems wrong.

Dawid



On Feb 22, 2018 01:55, "Chris Hostetter"  wrote:

>
> : * Hoss has worked on aggregating all test failures from the 3 Jenkins
> : systems (ASF, Policeman, and Steve's), downloading the test results &
> logs,
> : and running some reports/stats on failures. He should be ready to share
> : this more publicly soon.
>
> I think Steve's linked to some of this before from jira comments, but it
> was only recently I realized i've never explicitly said to the list "Hey
> folks, here's a thing i've been working on" ...
>
>   http://fucit.org/solr-jenkins-reports/
>   https://github.com/hossman/jenkins-reports/
>
> The most interesting bit is probably here...
>
>   http://fucit.org/solr-jenkins-reports/failure-report.html
>
> ...but there are currently a few caveats:
>
> 1) there's some noise inthe '7days' data because I wasn't accounting for
> the way jenkins reports some types of failure -- that will gradually clean
> itself up
>
> 2) I think i've been been blocked by builds.apache.org, so at the moment
> the data seems to just be from the sarowe & policeman jenkins failures.
>
> 3) allthough the system is archiving the past 7 days worth of jenkins logs
> for any jobs with failures, there is currently no easy way to download
> the relevant log(s) from that failure report -- you currently have to
> download a CSV file like this one to corrolate the test failures to the
> jenkins job, and then go look for that job in the job-data dirs...
>
>   http://fucit.org/solr-jenkins-reports/reports/7days-method-failures.csv
>   http://fucit.org/solr-jenkins-reports/job-data/
>
> (My hope is to make #3 trivial from failure-report.html -- so you can say
> "hey weird, this test has failed X times, let's go download those logs."
> right from a single screen in your browser)
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk1.8.0) - Build # 467 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/467/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<5> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([856810C964687D3D:E894B434DE20823A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12539 lines...]
   [junit4] Suite: 

[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 155 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/155/

6 tests failed.
FAILED:  org.apache.lucene.analysis.core.TestRandomChains.testRandomChains

Error Message:
startOffset must be non-negative, and endOffset must be >= startOffset, and 
offsets must not go backwards startOffset=6,endOffset=15,lastStartOffset=7 for 
field 'dummy'

Stack Trace:
java.lang.IllegalArgumentException: startOffset must be non-negative, and 
endOffset must be >= startOffset, and offsets must not go backwards 
startOffset=6,endOffset=15,lastStartOffset=7 for field 'dummy'
at 
__randomizedtesting.SeedInfo.seed([540F1928645813E:38A1D8F3C1579CFE]:0)
at 
org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:767)
at 
org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
at 
org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
at 
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:240)
at 
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:497)
at 
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1729)
at 
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1464)
at 
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:672)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:562)
at 
org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:856)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 

[jira] [Updated] (SOLR-11795) Add Solr metrics exporter for Prometheus

[Minoru Osuka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Minoru Osuka updated SOLR-11795:

Attachment: (was: SOLR-11795-9.patch)

> Add Solr metrics exporter for Prometheus
> 
>
> Key: SOLR-11795
> URL: https://issues.apache.org/jira/browse/SOLR-11795
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 7.2
>Reporter: Minoru Osuka
>Assignee: Koji Sekiguchi
>Priority: Minor
> Fix For: 7.3
>
> Attachments: SOLR-11795-2.patch, SOLR-11795-3.patch, 
> SOLR-11795-4.patch, SOLR-11795-5.patch, SOLR-11795-6.patch, 
> SOLR-11795-7.patch, SOLR-11795-8.patch, SOLR-11795-9.patch, SOLR-11795.patch, 
> solr-dashboard.png, solr-exporter-diagram.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I 'd like to monitor Solr using Prometheus and Grafana.
> I've already created Solr metrics exporter for Prometheus. I'd like to 
> contribute to contrib directory if you don't mind.
> !solr-exporter-diagram.png|thumbnail!
> !solr-dashboard.png|thumbnail!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11795) Add Solr metrics exporter for Prometheus

[Minoru Osuka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Minoru Osuka updated SOLR-11795:

Attachment: SOLR-11795-9.patch

> Add Solr metrics exporter for Prometheus
> 
>
> Key: SOLR-11795
> URL: https://issues.apache.org/jira/browse/SOLR-11795
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 7.2
>Reporter: Minoru Osuka
>Assignee: Koji Sekiguchi
>Priority: Minor
> Fix For: 7.3
>
> Attachments: SOLR-11795-2.patch, SOLR-11795-3.patch, 
> SOLR-11795-4.patch, SOLR-11795-5.patch, SOLR-11795-6.patch, 
> SOLR-11795-7.patch, SOLR-11795-8.patch, SOLR-11795-9.patch, SOLR-11795.patch, 
> solr-dashboard.png, solr-exporter-diagram.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I 'd like to monitor Solr using Prometheus and Grafana.
> I've already created Solr metrics exporter for Prometheus. I'd like to 
> contribute to contrib directory if you don't mind.
> !solr-exporter-diagram.png|thumbnail!
> !solr-dashboard.png|thumbnail!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4455 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4455/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest.testSimple

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:51746/solr]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:51746/solr]
at 
__randomizedtesting.SeedInfo.seed([FCEF794CD188444C:C45C5DB2F67B909D]:0)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest.testSimple(AutoAddReplicasPlanActionTest.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[jira] [Commented] (SOLR-11976) TokenizerChain is overwriting, not chaining TokenFilters in normalize()

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372388#comment-16372388
 ] 

David Smiley commented on SOLR-11976:
-

I am interested but waiting to get back from vacation tomorrow 

> TokenizerChain is overwriting, not chaining TokenFilters in normalize()
> ---
>
> Key: SOLR-11976
> URL: https://issues.apache.org/jira/browse/SOLR-11976
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: master (8.0)
>Reporter: Tim Allison
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> TokenizerChain is overwriting, not chaining tokenfilters in {{normalize}}.  
> This doesn't currently break search because {{normalize}} is not being used 
> at the Solr level (AFAICT); rather, TextField has its own 
> {{analyzeMultiTerm()}} that duplicates code from the newer {{normalize}}. 
> Code as is:
> {noformat}
> TokenStream result = in;
> for (TokenFilterFactory filter : filters) {
>   if (filter instanceof MultiTermAwareComponent) {
> filter = (TokenFilterFactory) ((MultiTermAwareComponent) 
> filter).getMultiTermComponent();
> result = filter.create(in);
>   }
> }
> {noformat}
> The fix is simple:
> {noformat}
> -result = filter.create(in);
> +result = filter.create(result);
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11066) Implement a scheduled trigger

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372386#comment-16372386
 ] 

Shalin Shekhar Mangar commented on SOLR-11066:
--

Final patch attached with ref guide changes. I'll commit this to master now.

> Implement a scheduled trigger
> -
>
> Key: SOLR-11066
> URL: https://issues.apache.org/jira/browse/SOLR-11066
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: SOLR-11066.patch, SOLR-11066.patch, SOLR-11066.patch, 
> SOLR-11066.patch, SOLR-11066.patch, SOLR-11066.patch
>
>
> Implement a trigger that runs on a fixed interval say every 1 hour or every 
> 24 hours starting at midnight etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11066) Implement a scheduled trigger

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-11066:
-
Attachment: SOLR-11066.patch

> Implement a scheduled trigger
> -
>
> Key: SOLR-11066
> URL: https://issues.apache.org/jira/browse/SOLR-11066
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: SOLR-11066.patch, SOLR-11066.patch, SOLR-11066.patch, 
> SOLR-11066.patch, SOLR-11066.patch, SOLR-11066.patch
>
>
> Implement a trigger that runs on a fixed interval say every 1 hour or every 
> 24 hours starting at midnight etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_162) - Build # 21509 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21509/
Java: 64bit/jdk1.8.0_162 -XX:+UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  org.apache.solr.cloud.ReplaceNodeNoTargetTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([BBEA183626B44C34:33BE27EC884821CC]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitAfterFailedSplit

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([BBEA183626B44C34:42A78B991AC101BE]:0)
at 

[JENKINS] Lucene-Solr-Tests-master - Build # 2356 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2356/

6 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.ltr.feature.TestExternalValueFeatures

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.ltr.feature.TestExternalValueFeatures: 1) Thread[id=24, 
name=qtp572864418-24, state=TIMED_WAITING, 
group=TGRP-TestExternalValueFeatures] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.ltr.feature.TestExternalValueFeatures: 
   1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, 
group=TGRP-TestExternalValueFeatures]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([BFB4FE6E10EE8A02]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.ltr.feature.TestExternalValueFeatures

Error Message:
There are still zombie threads that couldn't be terminated:1) Thread[id=24, 
name=qtp572864418-24, state=TIMED_WAITING, 
group=TGRP-TestExternalValueFeatures] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, 
group=TGRP-TestExternalValueFeatures]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([BFB4FE6E10EE8A02]:0)


FAILED:  org.apache.solr.cloud.ReplaceNodeNoTargetTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([B8802F5058FD4B33:30D4108AF60126CB]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at 

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk1.8.0_144) - Build # 469 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/469/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseSerialGC

7 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.lucene.index.TestBackwardsCompatibility

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\4.0.0-cfs-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\4.0.0-cfs-001

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\2.3.0-cfs-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\2.3.0-cfs-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\4.0.0-cfs-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\4.0.0-cfs-001
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\2.3.0-cfs-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\test\J0\temp\lucene.index.TestBackwardsCompatibility_682991DFA27B52E8-001\2.3.0-cfs-001

at __randomizedtesting.SeedInfo.seed([682991DFA27B52E8]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueueBlocking

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at 
__randomizedtesting.SeedInfo.seed([2D3449E4708A8B03:689E3B9D34D83777]:0)
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at 
org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.testDistributedQueueBlocking(TestSimDistributedQueue.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[jira] [Commented] (SOLR-11795) Add Solr metrics exporter for Prometheus

[Minoru Osuka (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372333#comment-16372333
 ] 

Minoru Osuka commented on SOLR-11795:
-

I attached new patch file (SOLR-11795-9.patch) that passed `ant validate` task. 
It includes to fix following:

- added SHA1 checksum, LICENSE and NOTICE files into solr/licenses directory.
- fixed invalid source code. (use DefaultSolrThreadFactory and 
InputStreamReader.)
- added JAR version into lucene/ivy-versions.properties.
- fixed duplicate section name for Ref Guide.

> Add Solr metrics exporter for Prometheus
> 
>
> Key: SOLR-11795
> URL: https://issues.apache.org/jira/browse/SOLR-11795
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 7.2
>Reporter: Minoru Osuka
>Assignee: Koji Sekiguchi
>Priority: Minor
> Fix For: 7.3
>
> Attachments: SOLR-11795-2.patch, SOLR-11795-3.patch, 
> SOLR-11795-4.patch, SOLR-11795-5.patch, SOLR-11795-6.patch, 
> SOLR-11795-7.patch, SOLR-11795-8.patch, SOLR-11795-9.patch, SOLR-11795.patch, 
> solr-dashboard.png, solr-exporter-diagram.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I 'd like to monitor Solr using Prometheus and Grafana.
> I've already created Solr metrics exporter for Prometheus. I'd like to 
> contribute to contrib directory if you don't mind.
> !solr-exporter-diagram.png|thumbnail!
> !solr-dashboard.png|thumbnail!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11795) Add Solr metrics exporter for Prometheus

[Minoru Osuka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Minoru Osuka updated SOLR-11795:

Attachment: SOLR-11795-9.patch

> Add Solr metrics exporter for Prometheus
> 
>
> Key: SOLR-11795
> URL: https://issues.apache.org/jira/browse/SOLR-11795
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 7.2
>Reporter: Minoru Osuka
>Assignee: Koji Sekiguchi
>Priority: Minor
> Fix For: 7.3
>
> Attachments: SOLR-11795-2.patch, SOLR-11795-3.patch, 
> SOLR-11795-4.patch, SOLR-11795-5.patch, SOLR-11795-6.patch, 
> SOLR-11795-7.patch, SOLR-11795-8.patch, SOLR-11795-9.patch, SOLR-11795.patch, 
> solr-dashboard.png, solr-exporter-diagram.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I 'd like to monitor Solr using Prometheus and Grafana.
> I've already created Solr metrics exporter for Prometheus. I'd like to 
> contribute to contrib directory if you don't mind.
> !solr-exporter-diagram.png|thumbnail!
> !solr-dashboard.png|thumbnail!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12011) Consistence problem when in-sync replicas are DOWN

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-12011:

Attachment: SOLR-12011.patch

> Consistence problem when in-sync replicas are DOWN
> --
>
> Key: SOLR-12011
> URL: https://issues.apache.org/jira/browse/SOLR-12011
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-12011.patch
>
>
> Currently, we will meet consistency problem when in-sync replicas are DOWN. 
> For example:
> 1. A collection with 1 shard with 1 leader and 2 replicas
> 2. Nodes contain 2 replicas go down
> 3. The leader receives an update A, success
> 4. The node contains the leader goes down
> 5. 2 replicas come back
> 6. One of them become leader --> But they shouldn't become leader since they 
> missed the update A



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10809) Get precommit lint warnings out of Solr core

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372303#comment-16372303
 ] 

Joel Bernstein commented on SOLR-10809:
---

After a quick review things look OK. I'll have some more time tomorrow to 
review a little close.

> Get precommit lint warnings out of Solr core
> 
>
> Key: SOLR-10809
> URL: https://issues.apache.org/jira/browse/SOLR-10809
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-10809.patch, SOLR-10809.patch
>
>
> Changing to fit "the new paradigm" of de-linting a directory at a time. I 
> hope to get precommit to fail on precommit warnings from some point on down 
> the tree, and solr/core is the first unit I've been working on.
> I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10809) Get precommit lint warnings out of Solr core

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372303#comment-16372303
 ] 

Joel Bernstein edited comment on SOLR-10809 at 2/22/18 1:39 AM:


After a quick review things look OK. I'll have some more time tomorrow to 
review a little closer.


was (Author: joel.bernstein):
After a quick review things look OK. I'll have some more time tomorrow to 
review a little close.

> Get precommit lint warnings out of Solr core
> 
>
> Key: SOLR-10809
> URL: https://issues.apache.org/jira/browse/SOLR-10809
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-10809.patch, SOLR-10809.patch
>
>
> Changing to fit "the new paradigm" of de-linting a directory at a time. I 
> hope to get precommit to fail on precommit warnings from some point on down 
> the tree, and solr/core is the first unit I've been working on.
> I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10809) Get precommit lint warnings out of Solr core

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372286#comment-16372286
 ] 

Erick Erickson commented on SOLR-10809:
---

This is getting closer. I ran into some issues on this latest go-round, and I'd 
particularly like [~joel.bernstein][~risdenk][~dpgove] and [~covolution] to 
weigh in. Basically, a number of the close operations on streams had to have 
null checks included to pass the tests. That part seems fairly safe. I didn't 
go very far into trying to figure out _how_ these are null when close is 
called, as I mentioned earlier I'm trying mightily to not rewrite too much.

However, StreamExpressionToExpressionTest.testTopicStream still fails. The root 
of the issue is that TopicStream.close() calls 
TopicStream.persistCheckpoints(), which fails in this try block because 
cloudSolrClient is null. This is about line 469 in my code.
try {
  cloudSolrClient.request(request);
 .
 . 
.
 }

Is it as simple as returning from persistCheckpoints if cloudSolrClient is null 
and is that reasonable? If so, I'll make that happen.

I also changed the following any comments you have welcome.

FeaturesSelectionStream
TopicStream
DeamonStream
FacetStream
FeaturesSelectionStream
TextLogitStream
TopicStream

StreamExpressionTest
StreamExpressionToExplanationTest
StreamingTest
TestJavabinTupleStreamParser



> Get precommit lint warnings out of Solr core
> 
>
> Key: SOLR-10809
> URL: https://issues.apache.org/jira/browse/SOLR-10809
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-10809.patch, SOLR-10809.patch
>
>
> Changing to fit "the new paradigm" of de-linting a directory at a time. I 
> hope to get precommit to fail on precommit warnings from some point on down 
> the tree, and solr/core is the first unit I've been working on.
> I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10809) Get precommit lint warnings out of Solr core

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated SOLR-10809:
--
Attachment: SOLR-10809.patch

> Get precommit lint warnings out of Solr core
> 
>
> Key: SOLR-10809
> URL: https://issues.apache.org/jira/browse/SOLR-10809
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: SOLR-10809.patch, SOLR-10809.patch
>
>
> Changing to fit "the new paradigm" of de-linting a directory at a time. I 
> hope to get precommit to fail on precommit warnings from some point on down 
> the tree, and solr/core is the first unit I've been working on.
> I'll have a patch soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Chris Hostetter]

: * Hoss has worked on aggregating all test failures from the 3 Jenkins
: systems (ASF, Policeman, and Steve's), downloading the test results & logs,
: and running some reports/stats on failures. He should be ready to share
: this more publicly soon.

I think Steve's linked to some of this before from jira comments, but it 
was only recently I realized i've never explicitly said to the list "Hey 
folks, here's a thing i've been working on" ...

  http://fucit.org/solr-jenkins-reports/
  https://github.com/hossman/jenkins-reports/

The most interesting bit is probably here...

  http://fucit.org/solr-jenkins-reports/failure-report.html

...but there are currently a few caveats:

1) there's some noise inthe '7days' data because I wasn't accounting for 
the way jenkins reports some types of failure -- that will gradually clean 
itself up

2) I think i've been been blocked by builds.apache.org, so at the moment 
the data seems to just be from the sarowe & policeman jenkins failures.

3) allthough the system is archiving the past 7 days worth of jenkins logs 
for any jobs with failures, there is currently no easy way to download 
the relevant log(s) from that failure report -- you currently have to 
download a CSV file like this one to corrolate the test failures to the 
jenkins job, and then go look for that job in the job-data dirs...

  http://fucit.org/solr-jenkins-reports/reports/7days-method-failures.csv
  http://fucit.org/solr-jenkins-reports/job-data/

(My hope is to make #3 trivial from failure-report.html -- so you can say 
"hey weird, this test has failed X times, let's go download those logs." 
right from a single screen in your browser)




-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7798) Improve robustness of ExpandComponent

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372251#comment-16372251
 ] 

Joel Bernstein commented on SOLR-7798:
--

Thanks for the patch. If you can setup a pull request I should have time to 
review and commit when its ready.

> Improve robustness of ExpandComponent
> -
>
> Key: SOLR-7798
> URL: https://issues.apache.org/jira/browse/SOLR-7798
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Reporter: Jörg Rathlev
>Assignee: Joel Bernstein
>Priority: Minor
> Attachments: expand-component.patch, expand-npe.patch
>
>
> The {{ExpandComponent}} causes a {{NullPointerException}} if accidentally 
> used without prior collapsing of results.
> If there are multiple documents in the result which have the same term value 
> in the expand field, the size of the {{ordBytes}}/{{groupSet}} differs from 
> the {{count}} value, and the {{getGroupQuery}} method creates an incompletely 
> filled {{bytesRef}} array, which later causes a {{NullPointerException}} when 
> trying to sort the terms.
> The attached patch extends the test to demonstrate the error, and modifies 
> the {{getGroupQuery}} methods to create the array based on the size of the 
> input maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-11988) MockDirectoryFactory.exists() behaves diff then other impls -- can cause FullSolrCloudDistribCmdsTest failures due to SolrCore.initIndex incorrectly thinking index direc

[Hoss Man (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-11988.
-
   Resolution: Fixed
Fix Version/s: 7.3
   master (8.0)

> MockDirectoryFactory.exists() behaves diff then other impls -- can cause 
> FullSolrCloudDistribCmdsTest failures due to SolrCore.initIndex incorrectly 
> thinking index directory for brand new SolrCores already exist?
> 
>
> Key: SOLR-11988
> URL: https://issues.apache.org/jira/browse/SOLR-11988
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: SOLR-11988.patch, SOLR-11988_nocommit_logging.patch, 
> log.txt
>
>
> There's been quite a few jenkins failures from FullSolrCloudDistribCmdsTest 
> that all seem to follow a similar pattern:
>  * Failure manifests as "Could not find collection:collection2"
>  * Failing seeds _frequently_ reproduce, but aren't guaranteed to
>  * Root cause can be traced back to the collection creation failing because 
> one of more replica cores failed due to the brand new (Solr)IndexWriter 
> expects to find an existing segments file
>  ** SolrCore should have already created an (empty) index in 
> {{SolrCore.initIndex(...)}}
>  ** The fact that the {{SolrIndexWrite}} throws this exception in it's 
> constructor suggests that the earlier call to {{SolrCore.initIndex(...)}} is 
> not functioning reliably
>  ** Based on some experimenting i've done, it seems like the underlying 
> problem is that in {{SolrCore.initIndex(...)}} the DirectoryFactory can "lie" 
> about wether a directory already exists.
> More details to follow in comments.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11988) MockDirectoryFactory.exists() behaves diff then other impls -- can cause FullSolrCloudDistribCmdsTest failures due to SolrCore.initIndex incorrectly thinking index dire

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372250#comment-16372250
 ] 

Hoss Man commented on SOLR-11988:
-

Not sure why gitbot didn't record these...
{noformat}
Branch: refs/heads/master
Commit: ee51b658ece5b23431a2200e763f5198b53952fa
Parents: 32f3570

Branch: refs/heads/branch_7x
Commit: 9876e8832ad40288e8a852f1594e520495da2cfa
Parents: 05e9201
{noformat}

> MockDirectoryFactory.exists() behaves diff then other impls -- can cause 
> FullSolrCloudDistribCmdsTest failures due to SolrCore.initIndex incorrectly 
> thinking index directory for brand new SolrCores already exist?
> 
>
> Key: SOLR-11988
> URL: https://issues.apache.org/jira/browse/SOLR-11988
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: SOLR-11988.patch, SOLR-11988_nocommit_logging.patch, 
> log.txt
>
>
> There's been quite a few jenkins failures from FullSolrCloudDistribCmdsTest 
> that all seem to follow a similar pattern:
>  * Failure manifests as "Could not find collection:collection2"
>  * Failing seeds _frequently_ reproduce, but aren't guaranteed to
>  * Root cause can be traced back to the collection creation failing because 
> one of more replica cores failed due to the brand new (Solr)IndexWriter 
> expects to find an existing segments file
>  ** SolrCore should have already created an (empty) index in 
> {{SolrCore.initIndex(...)}}
>  ** The fact that the {{SolrIndexWrite}} throws this exception in it's 
> constructor suggests that the earlier call to {{SolrCore.initIndex(...)}} is 
> not functioning reliably
>  ** Based on some experimenting i've done, it seems like the underlying 
> problem is that in {{SolrCore.initIndex(...)}} the DirectoryFactory can "lie" 
> about wether a directory already exists.
> More details to follow in comments.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-7798) Improve robustness of ExpandComponent

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein reassigned SOLR-7798:


Assignee: Joel Bernstein

> Improve robustness of ExpandComponent
> -
>
> Key: SOLR-7798
> URL: https://issues.apache.org/jira/browse/SOLR-7798
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Reporter: Jörg Rathlev
>Assignee: Joel Bernstein
>Priority: Minor
> Attachments: expand-component.patch, expand-npe.patch
>
>
> The {{ExpandComponent}} causes a {{NullPointerException}} if accidentally 
> used without prior collapsing of results.
> If there are multiple documents in the result which have the same term value 
> in the expand field, the size of the {{ordBytes}}/{{groupSet}} differs from 
> the {{count}} value, and the {{getGroupQuery}} method creates an incompletely 
> filled {{bytesRef}} array, which later causes a {{NullPointerException}} when 
> trying to sort the terms.
> The attached patch extends the test to demonstrate the error, and modifies 
> the {{getGroupQuery}} methods to create the array based on the size of the 
> input maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 959 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/959/

No tests ran.

Build Log:
[...truncated 28738 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist
 [copy] Copying 491 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 215 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] Java 9 JAVA_HOME=/home/jenkins/tools/java/latest1.9
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (25.6 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-8.0.0-src.tgz...
   [smoker] 30.2 MB in 0.03 sec (1101.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-8.0.0.tgz...
   [smoker] 73.2 MB in 0.07 sec (1114.4 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-8.0.0.zip...
   [smoker] 83.7 MB in 0.07 sec (1141.3 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-8.0.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6243 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6243 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-8.0.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6243 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6243 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-8.0.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 212 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker] run tests w/ Java 9 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 9...
   [smoker]   got 212 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (42.1 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-8.0.0-src.tgz...
   [smoker] 52.6 MB in 0.19 sec (271.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-8.0.0.tgz...
   [smoker] 151.0 MB in 0.45 sec (333.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-8.0.0.zip...
   [smoker] 152.0 MB in 0.17 sec (900.1 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-8.0.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-8.0.0.tgz...
   [smoker]   **WARNING**: skipping check of 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-8.0.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
   [smoker]   **WARNING**: skipping check of 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-8.0.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
   [smoker] copying unpacked distribution for Java 8 ...
   [smoker] test solr example w/ Java 8...
   [smoker]   start Solr instance 
(log=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-8.0.0-java8/solr-example.log)...
   [smoker] No process found for Solr node running on port 8983
   [smoker]   Running techproducts example on port 8983 from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-8.0.0-java8
   [smoker] *** 

[jira] [Commented] (SOLR-10261) TestStressCloudBlindAtomicUpdates.test_dv() fail

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372226#comment-16372226
 ] 

Cao Manh Dat commented on SOLR-10261:
-

The fix actually a bug, which causes MigrateCmd and SplitShardCmd silently 
shallow Exception. See the latest patch on SOLR-7034 when I replaced the LIR 
inside SolrCmdDistributor by adding the error to the errors list and handle it 
in DUP.finish().

> TestStressCloudBlindAtomicUpdates.test_dv() fail
> 
>
> Key: SOLR-10261
> URL: https://issues.apache.org/jira/browse/SOLR-10261
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-10261.patch, SOLR-10261.patch
>
>
> I found a reproducing seed that cause 
> TestStressCloudBlindAtomicUpdates.test_dv() fail
> {code}
> [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestStressCloudBlindAtomicUpdates -Dtests.method=test_dv 
> -Dtests.seed=AD8E7B56D53B627F -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.locale=bg -Dtests.timezone=America/La_Paz -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   1.21s J2 | TestStressCloudBlindAtomicUpdates.test_dv <<<
>[junit4]> Throwable #1: java.util.concurrent.ExecutionException: 
> java.lang.RuntimeException: Error from server at 
> http://127.0.0.1:49825/solr/test_col: Async exception during distributed 
> update: Error from server at 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2: Server Error
>[junit4]> request: 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A49825%2Fsolr%2Ftest_col_shard5_replica1%2F=javabin=2
>[junit4]> Remote error message: Failed synchronous update on shard 
> StdNode: http://127.0.0.1:49836/solr/test_col_shard2_replica1/ update: 
> org.apache.solr.client.solrj.request.UpdateRequest@5919dfb3
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([AD8E7B56D53B627F:9B9A19105F66586E]:0)
>[junit4]>  at 
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
>[junit4]>  at 
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
>[junit4]>  at 
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
>[junit4]>  at 
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]> Caused by: java.lang.RuntimeException: Error from server at 
> http://127.0.0.1:49825/solr/test_col: Async exception during distributed 
> update: Error from server at 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2: Server Error
>[junit4]> request: 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A49825%2Fsolr%2Ftest_col_shard5_replica1%2F=javabin=2
>[junit4]> Remote error message: Failed synchronous update on shard 
> StdNode: http://127.0.0.1:49836/solr/test_col_shard2_replica1/ update: 
> org.apache.solr.client.solrj.request.UpdateRequest@5919dfb3
>[junit4]>  at 
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
>[junit4]>  at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>[junit4]>  at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>[junit4]>  at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
>[junit4]>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>[junit4]>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>[junit4]>  ... 1 more
>[junit4]> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://127.0.0.1:49825/solr/test_col: Async exception during 
> distributed update: Error from server at 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-7.x - Build # 432 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/432/

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.ltr.feature.TestRankingFeature

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.ltr.feature.TestRankingFeature: 1) Thread[id=93, 
name=qtp1405911032-93, state=TIMED_WAITING, group=TGRP-TestRankingFeature]  
   at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.ltr.feature.TestRankingFeature: 
   1) Thread[id=93, name=qtp1405911032-93, state=TIMED_WAITING, 
group=TGRP-TestRankingFeature]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([210D9FAD4571C3AF]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.ltr.feature.TestRankingFeature

Error Message:
There are still zombie threads that couldn't be terminated:1) Thread[id=93, 
name=qtp1405911032-93, state=TIMED_WAITING, group=TGRP-TestRankingFeature]  
   at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=93, name=qtp1405911032-93, state=TIMED_WAITING, 
group=TGRP-TestRankingFeature]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([210D9FAD4571C3AF]:0)


FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
Error from server at 
http://127.0.0.1:36194/solr/testcollection_shard1_replica_n2: Expected mime 
type application/octet-stream but got text/html.Error 404 
Can not find: /solr/testcollection_shard1_replica_n2/update  
HTTP ERROR 404 Problem accessing 
/solr/testcollection_shard1_replica_n2/update. Reason: Can not find: 
/solr/testcollection_shard1_replica_n2/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.8.v20171121  
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:36194/solr/testcollection_shard1_replica_n2: 
Expected mime type application/octet-stream but got text/html. 


Error 404 Can not find: 

[jira] [Commented] (SOLR-11525) Add cloud/standalone check to "bin/solr assert"

[Jason Gerlowski (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372210#comment-16372210
 ] 

Jason Gerlowski commented on SOLR-11525:


Planning on re-running tests on this and committing it soon unless anyone has 
any concerns.

> Add cloud/standalone check to "bin/solr assert"
> ---
>
> Key: SOLR-11525
> URL: https://issues.apache.org/jira/browse/SOLR-11525
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: master (8.0)
>Reporter: Jason Gerlowski
>Priority: Minor
>  Labels: patch
> Attachments: SOLR-11525.patch
>
>
> {{bin/solr}}'s AssertTool provides a simple way to check a number of things 
> about a running Solr.  Directory existence/non-existence, running user, 
> exposed URL, etc.
> A natural extension of this is to add options to check whether Solr is 
> running in standalone or cloud mode.
> This JIRA proposes adding {{--cloud }} and {{--not-cloud }} options 
> to the AssertTool.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8175) ICUTokenizer might return corrupt tokens due to concurrency bug in ICU4J

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372206#comment-16372206
 ] 

Robert Muir commented on LUCENE-8175:
-

ICU responded to Adrien's email about release plans: 
https://sourceforge.net/p/icu/mailman/message/36233218/


> ICUTokenizer might return corrupt tokens due to concurrency bug in ICU4J
> 
>
> Key: LUCENE-8175
> URL: https://issues.apache.org/jira/browse/LUCENE-8175
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
>Priority: Critical
>
> I was digging some test failures with {{testRandomHugeStrings}} that occurred 
> since the upgrade to ICU4J 60.2 which happen to boil down to this bug: 
> http://bugs.icu-project.org/trac/ticket/13512 which is fixed but not released 
> yet.
> In short an int[] is shared across several threads while it shouldn't. As a 
> consequence, ICUTokenizer may sometimes return corrupt tokens. I pinged on 
> the issue to know when a release fixing this bug is expected.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Test failures are out of control......

[Uwe Schindler]

Hey Yonik,

Have you read my e-mail? I just said that there is no need to add another 
sysprop as its already there! The default value for the sysprop is just a 
common-build.xml one-line change.

BTW, as I don't care about Solr tests most of the time, I disabled them 
completely on my local machine using lucene.build.properties in my user's home 
directory. Every developer can do the same on his own lucene.build.properties 
file (e.g. enable/disable bad apples). Just the default should be decided here.

Uwe

-
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

> -Original Message-
> From: Yonik Seeley [mailto:ysee...@gmail.com]
> Sent: Thursday, February 22, 2018 12:17 AM
> To: Solr/Lucene Dev 
> Subject: Re: Test failures are out of control..
> 
> On Wed, Feb 21, 2018 at 6:13 PM, Uwe Schindler  wrote:
> >> > There are exactly three
> >> > BadApple annotations in the entire code base at present, is there
> >> > enough value in introducing another annotation to make it worthwhile?
> >>
> >> If we change BadApple tests to be executed by default for "ant test"
> >> (but not for the most frequent jenkins jobs), then that would be fine.
> >> Basically, add a -Dtests.disable-badapples and use that for the
> >> jenkins jobs that email the list all the time.
> >
> > No need for a new sysprop. It's already there, just inverted! Configuring
> Jenkins to enable disable them is trivial.
> 
> The issue is that flakey tests should not be ignored by developers
> running unit tests before committing new changes.  That's the most
> important point in time for test coverage.
> 
> -Yonik
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Yonik Seeley]

On Wed, Feb 21, 2018 at 6:13 PM, Uwe Schindler  wrote:
>> > There are exactly three
>> > BadApple annotations in the entire code base at present, is there
>> > enough value in introducing another annotation to make it worthwhile?
>>
>> If we change BadApple tests to be executed by default for "ant test"
>> (but not for the most frequent jenkins jobs), then that would be fine.
>> Basically, add a -Dtests.disable-badapples and use that for the
>> jenkins jobs that email the list all the time.
>
> No need for a new sysprop. It's already there, just inverted! Configuring 
> Jenkins to enable disable them is trivial.

The issue is that flakey tests should not be ignored by developers
running unit tests before committing new changes.  That's the most
important point in time for test coverage.

-Yonik

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10284) Solr connection to Standalone node in Ensemble causes cluster failure

[Ben DeMott (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ben DeMott updated SOLR-10284:
--
Affects Version/s: 7.0
   7.1
   7.2

> Solr connection to Standalone node in Ensemble causes cluster failure
> -
>
> Key: SOLR-10284
> URL: https://issues.apache.org/jira/browse/SOLR-10284
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.3, 6.4, 7.0, 7.1, 7.2
> Environment: Solrcloud, with Zookeeper 
>Reporter: Ben DeMott
>Priority: Major
>
> I posted this issue on the Dev mailing list and was encouraged to create a 
> Jira ticket.  This isn't a bug per-se.
> Solr connects / reconnects to "Standalone" Zookeeper nodes, within an 
> ensemble cluster, which causes absolute havoc. 
> I work for Dice.com, as one of the core search developers.
> I'm happy to write a patch, as we'll probably do that internally anyways.  I 
> just want to get consensus from the community about how to provide the best 
> solution.
> My original email describing the issue: 
> http://mail-archives.apache.org/mod_mbox/lucene-dev/201703.mbox/raw/%3CCACbtCQ2cSPA8NbnqCbXZE9nZdT40xFHjpUhAOqUnd%3DqZaRMEsA%40mail.gmail.com%3E/2
> Proposed Solution:
> My thought was an explicit setting in solr.in.sh "ZK_STANDALONE" (which would 
> default to TRUE for the solr.in.sh file found next to bin/solr).  Upon 
> connection or reconnection of the Zookeeper Client, it would ask the server 
> "are you standalone", and disconnect if it is and ZK_STANDALONE=false, and 
> try the next host.  If all hosts are in standalone, an error would be shown - 
> "No zookeeper hosts available, that aren't in standalone operation - The 
> setting ZK_STANDALONE=false prevents connecting to a standalone Zookeeper"
> In order to urge users to use the setting, I would possibly also have a 
> warning shown in the logs, if your ZK_HOSTS is set, has multiple hosts in the 
> connection string, and ZK_STANDALONE is not false.
> I can't think of any implicit way to internalize a setting Other than 
>  ZK_HOSTS connection string setting has multiple hosts, there should be no 
> scenario in which any node is standalone, so you could assume there should be 
> no standalone servers.  But maybe an explicit setting is preferable.
> This solution should be:
> 1.) backwards compatible
> 2.) have very little performance impact (1 extra call upon connection to ZK)
> 3.) isolated to one part of the code.
> *Update 6/26/2017:*
> I started working on this, and it occurred to me the same issue exists for 
> *SolrJ* clients.  So SolrJ might be the place to make this change. I'm not 
> sure yet.
> A SolrJ client that has a multi-zk-node connection string that connects (even 
> temporarily) to a zk host that is standalone will believe there are no Solr 
> hosts that can answer the query, and you'll get the following error.  
> {{CloudSolrClient - Request to collection efc-profiles-match-col failed due 
> to (510) org.apache.solr.common.SolrException: Could not find a healthy node 
> to handle the request.}}
> I am not as familiar with the SolrJ codebase ... so I'll have to do some 
> digging.
> Instead of moving onto a different Zookeeper host, the SolrJ client will 
> think everything is fully working, just no Solr Hosts or Collections
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Test failures are out of control......

[Uwe Schindler]

> > There are exactly three
> > BadApple annotations in the entire code base at present, is there
> > enough value in introducing another annotation to make it worthwhile?
> 
> If we change BadApple tests to be executed by default for "ant test"
> (but not for the most frequent jenkins jobs), then that would be fine.
> Basically, add a -Dtests.disable-badapples and use that for the
> jenkins jobs that email the list all the time.

No need for a new sysprop. It's already there, just inverted! Configuring 
Jenkins to enable disable them is trivial.

Uwe


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Cassandra Targett]

This issue is hugely important.

At Lucidworks we have implemented a "Test Confidence" role that focuses on
improving the ability of all members of the community to trust that
reported failures from any of the Jenkins systems are actual failures and
not flakey tests. This role rotates among the committers on our Solr Team,
and a committer is assigned to the role for 2-week periods of time. Our
goal is to have at least one committer on our team focused full-time on
improving test confidence at all times. (Just a note on timing, we started
this last summer, but we only recently reconfirmed our commitment to having
someone assigned to it at all times.)

One of the guidelines we've agreed to is that the person in the role should
not look (only) at tests he has worked on. Instead, he should focus on
tests that fail less than 100% of the time and/or are hard to reproduce
*even if he didn't write the test or the code*.

Another aspect of the Test Confidence role is to try to develop tools that
can help the community overall in improving this situation. Two things have
grown out of this effort so far:

* Steve Rowe's work on a Jenkins job to reproduce test failures
(LUCENE-8106)
* Hoss has worked on aggregating all test failures from the 3 Jenkins
systems (ASF, Policeman, and Steve's), downloading the test results & logs,
and running some reports/stats on failures. He should be ready to share
this more publicly soon.

I think it's important to understand that flakey tests will *never* go
away. There will always be a new flakey test to review/fix. Our goal should
be to make it so most of the time, you can assume the test is broken and
only discover it's flakey as part of digging.

The idea of @BadApple marking (or some other notation) is an OK idea, but
the problem is so bad today I worry it does nothing to find a way to ensure
they get fixed. Lots of JIRAs get filed for problems with tests - I count
about 180 open issues today - and many just sit there forever.

The biggest thing I want to to avoid is making it even easier to
avoid/ignore them. We should try to make it easier to highlight them, and
we need a concerted effort to fix the tests once they've been identified as
flakey.

On Wed, Feb 21, 2018 at 5:03 PM, Uwe Schindler  wrote:

> Hi,
>
> > Flakey Test Problems:
> > a) Flakey tests create so much noise that people no longer pay
> > attention to the automated reporting via email.
> > b) When running unit tests manually before a commit (i.e. "ant test")
> > a flakey test can fail.
> >
> > Solutions:
> > We cloud fix (a) by marking as flakey and having a new target
> > "non-flakey" that is run by the jenkins jobs that are currently run
> > continuously.
>
> We have a solution for this already: Mark all those tests with @AwaitsFix
> or @BadApple
> By default those aren't executed in Jenkins runs and also not for
> developers, but devs can enable/disable them using -Dtests.awaitsfix=true
> and -Dtests.badapples=true:
>
>  [help] # Test groups. --
> --
>  [help] #
>  [help] # test groups can be enabled or disabled (true/false). Default
>  [help] # value provided below in [brackets].
>  [help]
>  [help] ant -Dtests.nightly=[false]   - nightly test group (@Nightly)
>  [help] ant -Dtests.weekly=[false]- weekly tests (@Weekly)
>  [help] ant -Dtests.awaitsfix=[false] - known issue (@AwaitsFix)
>  [help] ant -Dtests.slow=[true]   - slow tests (@Slow)
>
> We can of course also make a weekly jenkins jobs that enables those tests
> on Jenkins only weekly (like nightly stuff). We have "tests.badapples" and
> "tests.awaitsfix" - I don't know what's the difference between both.
>
> So we have 2 options to classify tests, let's choose one and apply it to
> all Flakey tests!
>
> Uwe
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Test failures are out of control......

[Yonik Seeley]

On Wed, Feb 21, 2018 at 5:52 PM, Erick Erickson  wrote:

> There are exactly three
> BadApple annotations in the entire code base at present, is there
> enough value in introducing another annotation to make it worthwhile?

If we change BadApple tests to be executed by default for "ant test"
(but not for the most frequent jenkins jobs), then that would be fine.
Basically, add a -Dtests.disable-badapples and use that for the
jenkins jobs that email the list all the time.

-Yonik

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Jason Gerlowski]

I don't have strong opinions about what we do with our existing flaky
tests.  I think re-running failures before commit might theoretically
catch more bugs than ignoring the test outright, but with all the
noise and how standard it is to need to rerun tests I'd be surprised
if the numbers are all that different.

Where I see some potential common ground though is in preventing new
flaky tests.  And in the long run, I think what we do to prevent new
flakes is going to be much more important than how we handle the
BadApples we have at this particular instant.  If we can put our
finger in the dam, the existing flakiness becomes much easier to put a
dent in.

I'm curious what you guys had in mind when you mentioned preventing
new flaky tests from popping up.  What are our options for "enforcing"
that?  Were you imagining reopening JIRAs and asking the original
committer to investigate?  Or outright reverting commits that
introduce flaky tests?  Or something in between (like disabling
features with flaky tests prior to releases)?

Best,

Jason

On Wed, Feb 21, 2018 at 6:03 PM, Uwe Schindler  wrote:
> Hi,
>
>> Flakey Test Problems:
>> a) Flakey tests create so much noise that people no longer pay
>> attention to the automated reporting via email.
>> b) When running unit tests manually before a commit (i.e. "ant test")
>> a flakey test can fail.
>>
>> Solutions:
>> We cloud fix (a) by marking as flakey and having a new target
>> "non-flakey" that is run by the jenkins jobs that are currently run
>> continuously.
>
> We have a solution for this already: Mark all those tests with @AwaitsFix or 
> @BadApple
> By default those aren't executed in Jenkins runs and also not for developers, 
> but devs can enable/disable them using -Dtests.awaitsfix=true and 
> -Dtests.badapples=true:
>
>  [help] # Test groups. 
> 
>  [help] #
>  [help] # test groups can be enabled or disabled (true/false). Default
>  [help] # value provided below in [brackets].
>  [help]
>  [help] ant -Dtests.nightly=[false]   - nightly test group (@Nightly)
>  [help] ant -Dtests.weekly=[false]- weekly tests (@Weekly)
>  [help] ant -Dtests.awaitsfix=[false] - known issue (@AwaitsFix)
>  [help] ant -Dtests.slow=[true]   - slow tests (@Slow)
>
> We can of course also make a weekly jenkins jobs that enables those tests on 
> Jenkins only weekly (like nightly stuff). We have "tests.badapples" and 
> "tests.awaitsfix" - I don't know what's the difference between both.
>
> So we have 2 options to classify tests, let's choose one and apply it to all 
> Flakey tests!
>
> Uwe
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Test failures are out of control......

[Uwe Schindler]

Hi,

> Flakey Test Problems:
> a) Flakey tests create so much noise that people no longer pay
> attention to the automated reporting via email.
> b) When running unit tests manually before a commit (i.e. "ant test")
> a flakey test can fail.
> 
> Solutions:
> We cloud fix (a) by marking as flakey and having a new target
> "non-flakey" that is run by the jenkins jobs that are currently run
> continuously.

We have a solution for this already: Mark all those tests with @AwaitsFix or 
@BadApple
By default those aren't executed in Jenkins runs and also not for developers, 
but devs can enable/disable them using -Dtests.awaitsfix=true and 
-Dtests.badapples=true:

 [help] # Test groups. 
 [help] #
 [help] # test groups can be enabled or disabled (true/false). Default
 [help] # value provided below in [brackets].
 [help]
 [help] ant -Dtests.nightly=[false]   - nightly test group (@Nightly)
 [help] ant -Dtests.weekly=[false]- weekly tests (@Weekly)
 [help] ant -Dtests.awaitsfix=[false] - known issue (@AwaitsFix)
 [help] ant -Dtests.slow=[true]   - slow tests (@Slow)

We can of course also make a weekly jenkins jobs that enables those tests on 
Jenkins only weekly (like nightly stuff). We have "tests.badapples" and 
"tests.awaitsfix" - I don't know what's the difference between both.

So we have 2 options to classify tests, let's choose one and apply it to all 
Flakey tests!

Uwe


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Erick Erickson]

Yonik:

Good discussion. I'm not wedded to a particular solution, it's just
the current direction is not sustainable.

I'll back up a bit and see if I can state my goals more clearly, it
looks like we're arguing for much the same thing.

> I want e-mail messages with test failures to be worth looking at. When I see 
> that a test fail, I don't want to waste a time trying to figure out whether 
> it's something newly introduced or not. I also want some less painful way to 
> say "this change broke tests" rather than "this change may or may not have 
> broken tests. Could somebody beast the old and new versions 100 times and 
> hope that's enough to make a determination?". This looks like your (a).

> When I make a change, I want to be able to quickly determine whether my 
> changes likely the cause of test failures or not. This looks like your (b). 
> If we annotate all flakey tests that would be a significant help since it 
> would be easy to glance at the test to see if it's a known flakey test or 
> not. Armed with that knowledge I can be more comfortable with having it 
> succeed a few times and chalking it up to flakey tests.

> I want to stop the downward trend we've been experiencing lately with more 
> and more tests failing.

An annotation makes that possible I think, although I'm not clear on
why @Flakey this is superior to @BadApple. There are exactly three
BadApple annotations in the entire code base at present, is there
enough value in introducing another annotation to make it worthwhile?
Or could we just figure out whether any of those three tests that use
@BadApple should be changed to, say, @Ignore and then use @BadApple
for the rest? Perhaps we change the build system to enable BadApple by
default when running locally (or, conversely, enabling BadApple on
Jenkins).

Alternatively would it be possible to turn off e-mail notifications of
failures for @Flakey (or @BadApple, whatever) tests? That would do
too. That probably has the added advantage of allowing some reporting
tools to continue to function.

bq: And we can *always* decide to prevent new flakey tests, regardless
of what we do about the existing flakey tests...

We haven't been doing this though, flakey tests have been
proliferating. Mark's tool hasn't been run since last August unless
there's a newer URL than I'm looking at:
http://solr-tests.bitballoon.com/. I'm not so interested in what we
_could_ do as what we _are_ doing. And even tools such as this require
someone to monitor/complain/whimper. And I don't see volunteers
stepping forward. It's much easier to have a system where any failure
is unusual than count on people to wade through voluminous output.

bq: Just because we are frustrated doesn't mean that *any* change is positive.

Of course not. But nobody else seems to be bringing the topic up so I
thought I would.

On Wed, Feb 21, 2018 at 1:49 PM, Yonik Seeley  wrote:
> On Wed, Feb 21, 2018 at 3:26 PM, Erick Erickson  
> wrote:
>> Yonik:
>>
>> What I'm frustrated by now is that variations on these themes haven't
>> cured the problem, and it's spun out of control and is getting worse.
>
> I understand, but what problem(s) are you trying to solve?  Just
> because we are frustrated doesn't mean that *any* change is positive.
> Some changes can have a definite negative affect on software quality.
>
> You didn't respond to the main thrust of my message, so let me try to
> explain it again more succinctly:
>
> Flakey Test Problems:
> a) Flakey tests create so much noise that people no longer pay
> attention to the automated reporting via email.
> b) When running unit tests manually before a commit (i.e. "ant test")
> a flakey test can fail.
>
> Solutions:
> We cloud fix (a) by marking as flakey and having a new target
> "non-flakey" that is run by the jenkins jobs that are currently run
> continuously.
> For (b) "ant test" should still include the flakey tests since it's
> better to have to re-run a seemingly unrelated test to determine if
> one broke something rather than increase committed bugs due to loss of
> test coverage.  It's a pain, but perhaps it should be.  It's a real
> problem that needs fixing and @Ignoring it won't work as a better
> mechanism to get it fixed.  Sweeping it under the rug would seem to
> ensure that it gets less attention.
>
> And we can *always* decide to prevent new flakey tests, regardless of
> what we do about the existing flakey tests.  Mark's tool is a good way
> to see what the current list of flakey tests is.
>
> -Yonik
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_144) - Build # 7185 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7185/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseParallelGC

8 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.lucene.index.TestBackwardsCompatibility

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\backward-codecs\test\J1\temp\lucene.index.TestBackwardsCompatibility_8CA250ACC9E4E7EA-001\2.9.4-cfs-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\backward-codecs\test\J1\temp\lucene.index.TestBackwardsCompatibility_8CA250ACC9E4E7EA-001\2.9.4-cfs-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\backward-codecs\test\J1\temp\lucene.index.TestBackwardsCompatibility_8CA250ACC9E4E7EA-001\2.9.4-cfs-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\backward-codecs\test\J1\temp\lucene.index.TestBackwardsCompatibility_8CA250ACC9E4E7EA-001\2.9.4-cfs-001

at __randomizedtesting.SeedInfo.seed([8CA250ACC9E4E7EA]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
junit.framework.TestSuite.org.apache.lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001\testPostingsEnumReuse-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001\testPostingsEnumReuse-001

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001\testPostingsEnumReuse-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001\testPostingsEnumReuse-001
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\J1\temp\lucene.codecs.blockterms.TestVarGapFixedIntervalPostingsFormat_BD8CC6CCD6388876-001

at __randomizedtesting.SeedInfo.seed([BD8CC6CCD6388876]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Comment Edited] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372126#comment-16372126
 ] 

Uwe Schindler edited comment on LUCENE-8106 at 2/21/18 10:39 PM:
-

That's now the Jenkins script:

{code:bash}
set -x # Log commands

TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT   # Delete the temp file on SIGEXIT

curl -o $TMPFILE "${BUILD_URL}consoleText"

if grep --quiet 'reproduce with' $TMPFILE ; then

# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig

PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*=" 
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*=" 
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch 
file://$TMPFILE

# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro

# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}


was (Author: thetaphi):
That's now the Jenkins script:

{code:bash}
set -x # Log commands

TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT   # Delete the temp file on SIGEXIT

curl -o $TMPFILE "$BUILD_URL/consoleText"

if grep --quiet 'reproduce with' $TMPFILE ; then

# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig

PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*=" 
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*=" 
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch 
file://$TMPFILE

# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro

# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}

> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch, 
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream 
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
>  python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11968) Multi-words query time synonyms

[Jim Ferenczi (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372127#comment-16372127
 ] 

Jim Ferenczi commented on SOLR-11968:
-

This issue is also described in 
https://issues.apache.org/jira/browse/LUCENE-8137 . 
https://issues.apache.org/jira/browse/LUCENE-7848 is different, it is about 
adding gaps in the span query produced when multi-words synonym occurs. The 
problem here is about broken token stream where the posLength of some 
multi-word synonyms are invalidated by the removal of a token.  The query 
builder in this case will omit some tokens because posLength is broken for some 
tokens. I like the idea of adding a new mode to StopFilter that updates 
posLength and posInc when needed because I don't think we can "fix" a broken 
token stream outside of the token filter that broke it.

> Multi-words query time synonyms
> ---
>
> Key: SOLR-11968
> URL: https://issues.apache.org/jira/browse/SOLR-11968
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers, Schema and Analysis
>Affects Versions: master (8.0), 6.6.2
> Environment: Centos 7.x
>Reporter: Dominique Béjean
>Priority: Major
>
> I am trying multi words query time synonyms with Solr 6.6.2 and 
> SynonymGraphFilterFactory filter as explain in this article
>  
> [https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-support/]
>   
>  My field type is :
> {code:java}
> 
>      
>        
>                      articles="lang/contractions_fr.txt"/>
>        
>        
>         ignoreCase="true"/>
>        
>      
>      
>        
>                      articles="lang/contractions_fr.txt"/>
>        
>                      ignoreCase="true" expand="true"/>
>        
>         ignoreCase="true"/>
>        
>      
>    {code}
>  
>  synonyms.txt contains the line :
> {code:java}
> om, olympique de marseille{code}
>  
>  stopwords.txt contains the word 
> {code:java}
> de{code}
>  
>  The order of words in my query has an impact on the generated query in 
> edismax
> {code:java}
> q={!edismax qf='name_text_gp' v=$qq}
>  =false
>  =...{code}
> with "qq=om maillot" or "qq=olympique de marseille maillot", I can see the 
> synonyms expansion. It is working as expected.
> {code:java}
> "parsedquery_toString":"+(((+name_text_gp:olympiqu +name_text_gp:marseil 
> +name_text_gp:maillot) name_text_gp:om))",
>  "parsedquery_toString":"+((name_text_gp:om (+name_text_gp:olympiqu 
> +name_text_gp:marseil +name_text_gp:maillot)))",{code}
> with "qq=maillot om" or "qq=maillot olympique de marseille", I can see the 
> same generated query 
> {code:java}
> "parsedquery_toString":"+((name_text_gp:maillot) (name_text_gp:om))",
>  "parsedquery_toString":"+((name_text_gp:maillot) (name_text_gp:om))",{code}
> I don't understand these generated queries. The first one looks like the 
> synonym expansion is ignored, but the second one shows it is not ignored and 
> only the synonym term is used.
>   
>  When I test the analisys for the field type the synonyms are correctly 
> expanded for both expressions
> {code:java}
> om maillot  
>  maillot om
>  olympique de marseille maillot
>  maillot olympique de marseille{code}
> resulting outputs always include the following terms (obvioulsly not always 
> in the same order)
> {code:java}
> olympiqu om marseil maillot {code}
>  
>  So, i suspect an issue with edismax query parser.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372126#comment-16372126
 ] 

Uwe Schindler commented on LUCENE-8106:
---

That's now the Jenkins script:

{code:bash}
set -x # Log commands

TMPFILE=`mktemp`
trap "rm -f $TMPFILE" EXIT   # Delete the temp file on SIGEXIT

curl -o $TMPFILE "$BUILD_URL/consoleText"

if grep --quiet 'reproduce with' $TMPFILE ; then

# Preserve original build output
mv lucene/build lucene/build.orig
mv solr/build solr/build.orig

PYTHON32_EXE=`grep "^[[:space:]]*python32\.exe[[:space:]]*=" 
~/lucene.build.properties | cut -d'=' -f2`
[ -z $PYTHON32_EXE ] && PYTHON32_EXE=python3
GIT_EXE=`grep "^[[:space:]]*git\.exe[[:space:]]*=" 
~/lucene.build.properties | cut -d'=' -f2`
[ -n $GIT_EXE ] && export PATH=$GIT_EXE:$PATH
export ANT_HOME=$ANT_1_8_2_HOME
export PATH=$ANT_HOME/bin:$PATH
$PYTHON32_EXE -u dev-tools/scripts/reproduceJenkinsFailures.py --no-fetch 
file://$TMPFILE

# Preserve repro build output
mv lucene/build lucene/build.repro
mv solr/build solr/build.repro

# Restore original build output
mv lucene/build.orig lucene/build
mv solr/build.orig solr/build
fi
{code}

> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch, 
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream 
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
>  python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (LUCENE-8106) Add script to attempt to reproduce failing tests from a Jenkins log

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler reopened LUCENE-8106:
---

I changed the script on Policeman Jenkins a bit to use the "Tools Environment" 
plugin to define an env var with ANT in correct version. I used this env var 
instaed of the hardcoded path added by Steve.

It seems to work now. There is still the issue, that GIT is not installed and 
not even used by Jenkins (Jenkins uses JGit for checkouts). This is currently 
no issue on the Linux node (it has cmd-line git), but all other nodes - 
including Windows - do not have a running Git command line. I will definitely 
not install one, becaus ethat makes maintenance even harder (4 operating 
systems, millions of Java versions,...).

So I have one small question to [~steve_rowe]: Can't we make a variant of the 
script for Jenkins that does not deal with Git at all? For reproduing the 
failure directly after running the main build job there is no need to change 
branches or merge in changes. We just leave the checkout as-is. As there are no 
external forces changing the checkout, why would we need GIT?

So let's add a "replacement" option "--no-git" instead of "--no-fetch", that 
completely disables GIT in the python script. After doing this it's easy to 
setup with Jenkins and it would also possibly work with Windows Jenkins. I can 
help with adding a n Ant target to call the reproducer instead of doing it 
directly from Jenkins (like "ant nighly-smoke").

For committers that want to reproduce failures locally, the script does all 
maintenance and Git magic to reproduce Jenkins behaviour.

> Add script to attempt to reproduce failing tests from a Jenkins log
> ---
>
> Key: LUCENE-8106
> URL: https://issues.apache.org/jira/browse/LUCENE-8106
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: master (8.0), 7.3
>
> Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch, 
> LUCENE-8106.patch
>
>
> This script will be runnable from a downstream job triggered by an upstream 
> failing Jenkins job, passing log location info between the two.
> The script will also be runnable manually from a developer's cmdline.
> From the script help:
> {noformat}
> Usage:
>  python3 -u reproduceJenkinsFailures.py URL
> Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> log pointed to by the given URL, parses it for Git revision and failed
> Lucene/Solr tests, checks out the Git revision in the local workspace,
> groups the failed tests by module, then runs
> 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> in each module of interest, failing at the end if any of the runs fails.
> To control the maximum number of concurrent JVMs used for each module's
> test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_162) - Build # 21508 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21508/
Java: 64bit/jdk1.8.0_162 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testSelectedCollections

Error Message:
The operations computed by ComputePlanAction should not be 
nullSolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], 
BEFORE_ACTION=[compute_plan, null]}

Stack Trace:
java.lang.AssertionError: The operations computed by ComputePlanAction should 
not be nullSolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], 
BEFORE_ACTION=[compute_plan, null]}
at 
__randomizedtesting.SeedInfo.seed([61E95F1543B816F7:5B47BACC7DDCCF99]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testSelectedCollections(ComputePlanActionTest.java:469)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-repro - Build # 81 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/81/

[...truncated 37 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-Tests-7.x/431/consoleText

[repro] Revision: 9cbdcf85e7b2647fac7021901475cca6f107961a

[repro] Repro line:  ant test  -Dtestcase=AutoscalingHistoryHandlerTest 
-Dtests.method=testHistory -Dtests.seed=872762B6A3FA9E4D -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=tr-TR 
-Dtests.timezone=America/Indiana/Tell_City -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII

[repro] Repro line:  ant test  -Dtestcase=DeleteShardTest -Dtests.method=test 
-Dtests.seed=872762B6A3FA9E4D -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=lv -Dtests.timezone=Pacific/Easter -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII

[repro] Repro line:  ant test  -Dtestcase=TriggerIntegrationTest 
-Dtests.method=testMetricTrigger -Dtests.seed=872762B6A3FA9E4D 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar-LY 
-Dtests.timezone=Brazil/Acre -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] Repro line:  ant test  -Dtestcase=TestLargeCluster 
-Dtests.method=testSearchRate -Dtests.seed=872762B6A3FA9E4D 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-CL 
-Dtests.timezone=Asia/Aqtobe -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] Repro line:  ant test  -Dtestcase=CollectionsAPIDistributedZkTest 
-Dtests.method=testCollectionsAPI -Dtests.seed=872762B6A3FA9E4D 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=zh-CN 
-Dtests.timezone=Asia/Krasnoyarsk -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII

[repro] Repro line:  ant test  -Dtestcase=LegacyFieldFacetExtrasCloudTest 
-Dtests.seed=A1827EC393726B9 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=el-CY -Dtests.timezone=CNT -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII

[repro] Repro line:  ant test  -Dtestcase=TestModelManager 
-Dtests.seed=8969BA0947740778 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=en-ZA -Dtests.timezone=CET -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
82a99840718bb7e696ef4af114eee976c776a251
[repro] git checkout 9cbdcf85e7b2647fac7021901475cca6f107961a

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/contrib/analytics
[repro]   LegacyFieldFacetExtrasCloudTest
[repro]solr/core
[repro]   AutoscalingHistoryHandlerTest
[repro]   DeleteShardTest
[repro]   CollectionsAPIDistributedZkTest
[repro]   TriggerIntegrationTest
[repro]   TestLargeCluster
[repro]solr/contrib/ltr
[repro]   TestModelManager
[repro] ant compile-test

[...truncated 2589 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.LegacyFieldFacetExtrasCloudTest" -Dtests.showOutput=onerror 
-Dtests.seed=A1827EC393726B9 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=el-CY -Dtests.timezone=CNT -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII

[...truncated 78 lines...]
[repro] ant compile-test

[...truncated 1331 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=25 
-Dtests.class="*.AutoscalingHistoryHandlerTest|*.DeleteShardTest|*.CollectionsAPIDistributedZkTest|*.TriggerIntegrationTest|*.TestLargeCluster"
 -Dtests.showOutput=onerror -Dtests.seed=872762B6A3FA9E4D -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=tr-TR 
-Dtests.timezone=America/Indiana/Tell_City -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII

[...truncated 40266 lines...]
[repro] Setting last failure code to 256

[repro] ant compile-test

[...truncated 566 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestModelManager" -Dtests.showOutput=onerror 
-Dtests.seed=8969BA0947740778 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=en-ZA -Dtests.timezone=CET -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[...truncated 78 lines...]
[repro] Failures:
[repro]   0/5 failed: 
org.apache.solr.analytics.legacy.facet.LegacyFieldFacetExtrasCloudTest
[repro]   0/5 failed: org.apache.solr.cloud.DeleteShardTest
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[repro]   0/5 failed: 
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[repro]   0/5 failed: org.apache.solr.ltr.store.rest.TestModelManager
[repro]   4/5 failed: 
org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest

[repro] Re-testing 100% failures at the tip of branch_7x
[repro] git checkout branch_7x

[...truncated 4 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 8 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   TriggerIntegrationTest
[repro] 

[jira] [Commented] (LUCENE-4065) FilteringTokenFilter should never corrupt the tokenstream graph

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372072#comment-16372072
 ] 

Robert Muir commented on LUCENE-4065:
-

Yeah, you've got it. I really prefer your {{enableGaps}} name. Sorry, the issue 
is just confusing and I was struggling to try to explain it.

Today {{enableGaps}} is always true, which makes deletions pretty simple for 
FilteringTokenFilter. We just have to track an int variable! 

But I think we can potentially support enableGaps=false, and adjust 
positionIncrements/positionLengths so that the result is sane. That's the idea 
of this issue. I think no user _really_ wanted to disable position increments 
entirely before, nobody wants to move synonyms to the incorrect words or 
anything like that. They just want control over whether there are gaps or not: 
it impacts things like phrase queries.

> FilteringTokenFilter should never corrupt the tokenstream graph
> ---
>
> Key: LUCENE-4065
> URL: https://issues.apache.org/jira/browse/LUCENE-4065
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/analysis
>Reporter: Robert Muir
>Priority: Major
> Attachments: LUCENE-4065_test.patch
>
>
> Currently removers like stopfilter have an option (true/false) to enable 
> position increments.
> If its true: it both inserts gaps where necessary AND propagates gaps down 
> the stream.
> If its false: it does neither, which can totally mess up the tokenstream 
> graph (e.g. move synonyms to another word).
> There are totally valid natural usecases for false, where you don't want gaps 
> because you want phrasequeries to act as if the word was never actually there.
> But 'not inserting gaps' is separate from proper propagation of existing gaps.
> So I think we should provide an option (either fix 'false' or make it an 
> enum), where you still get a legit tokenstream and dont totally screw it up, 
> but you simply omit gaps.
> See LUCENE-3848 for more information (Where we at least fixed this case to 
> not begin the tokenstream with posinc=0)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Yonik Seeley]

On Wed, Feb 21, 2018 at 3:26 PM, Erick Erickson  wrote:
> Yonik:
>
> What I'm frustrated by now is that variations on these themes haven't
> cured the problem, and it's spun out of control and is getting worse.

I understand, but what problem(s) are you trying to solve?  Just
because we are frustrated doesn't mean that *any* change is positive.
Some changes can have a definite negative affect on software quality.

You didn't respond to the main thrust of my message, so let me try to
explain it again more succinctly:

Flakey Test Problems:
a) Flakey tests create so much noise that people no longer pay
attention to the automated reporting via email.
b) When running unit tests manually before a commit (i.e. "ant test")
a flakey test can fail.

Solutions:
We cloud fix (a) by marking as flakey and having a new target
"non-flakey" that is run by the jenkins jobs that are currently run
continuously.
For (b) "ant test" should still include the flakey tests since it's
better to have to re-run a seemingly unrelated test to determine if
one broke something rather than increase committed bugs due to loss of
test coverage.  It's a pain, but perhaps it should be.  It's a real
problem that needs fixing and @Ignoring it won't work as a better
mechanism to get it fixed.  Sweeping it under the rug would seem to
ensure that it gets less attention.

And we can *always* decide to prevent new flakey tests, regardless of
what we do about the existing flakey tests.  Mark's tool is a good way
to see what the current list of flakey tests is.

-Yonik

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-11597) Implement RankNet.

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372062#comment-16372062
 ] 

Christine Poerschke edited comment on SOLR-11597 at 2/21/18 9:47 PM:
-

Thanks [~malcorn_redhat]] for reviewing and merging in my [tweaks to fix two 
test failures|https://github.com/airalcorn2/lucene-solr/pull/3], [1x javadocs + 
2x test tweaks|https://github.com/airalcorn2/lucene-solr/pull/4] and [explain 
test, (custom) activation 
(bits)|https://github.com/airalcorn2/lucene-solr/pull/5] pull requests!

Attached SOLR-11597.patch is a snapshot of the changes as of now on top of 
current master:
* {{ant precommit}} passes
* {{ant beast -Dtestcase=TestNeuralNetworkModel -Dbeast.iters=100}} passes
* javadocs outbound links manually checked

I think we're good to go here and if no one has any additional comments, 
concerns, etc. then I'll proceed to commit the changes in around a week or so 
from today.


was (Author: cpoerschke):
Thanks [~malcorn_redhat]] for reviewing and merging in my [tweaks to fix two 
test failures|https://github.com/airalcorn2/lucene-solr/pull/3], [1x javadocs + 
2x test tweaks|https://github.com/airalcorn2/lucene-solr/pull/4] and [1x 
javadocs + 2x test tweaks|https://github.com/airalcorn2/lucene-solr/pull/5] 
pull requests!

Attached SOLR-11597.patch is a snapshot of the changes as of now on top of 
current master:
* {{ant precommit}} passes
* {{ant beast -Dtestcase=TestNeuralNetworkModel -Dbeast.iters=100}} passes
* javadocs outbound links manually checked

I think we're good to go here and if no one has any additional comments, 
concerns, etc. then I'll proceed to commit the changes in around a week or so 
from today.

> Implement RankNet.
> --
>
> Key: SOLR-11597
> URL: https://issues.apache.org/jira/browse/SOLR-11597
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Reporter: Michael A. Alcorn
>Assignee: Christine Poerschke
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11597.patch
>
>
> Implement RankNet as described in [this 
> tutorial|https://github.com/airalcorn2/Solr-LTR].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-11597) Implement RankNet.

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372062#comment-16372062
 ] 

Christine Poerschke edited comment on SOLR-11597 at 2/21/18 9:47 PM:
-

Thanks [~malcorn_redhat] for reviewing and merging in my [tweaks to fix two 
test failures|https://github.com/airalcorn2/lucene-solr/pull/3], [1x javadocs + 
2x test tweaks|https://github.com/airalcorn2/lucene-solr/pull/4] and [explain 
test, (custom) activation 
(bits)|https://github.com/airalcorn2/lucene-solr/pull/5] pull requests!

Attached SOLR-11597.patch is a snapshot of the changes as of now on top of 
current master:
 * {{ant precommit}} passes
 * {{ant beast -Dtestcase=TestNeuralNetworkModel -Dbeast.iters=100}} passes
 * javadocs outbound links manually checked

I think we're good to go here and if no one has any additional comments, 
concerns, etc. then I'll proceed to commit the changes in around a week or so 
from today.


was (Author: cpoerschke):
Thanks [~malcorn_redhat]] for reviewing and merging in my [tweaks to fix two 
test failures|https://github.com/airalcorn2/lucene-solr/pull/3], [1x javadocs + 
2x test tweaks|https://github.com/airalcorn2/lucene-solr/pull/4] and [explain 
test, (custom) activation 
(bits)|https://github.com/airalcorn2/lucene-solr/pull/5] pull requests!

Attached SOLR-11597.patch is a snapshot of the changes as of now on top of 
current master:
* {{ant precommit}} passes
* {{ant beast -Dtestcase=TestNeuralNetworkModel -Dbeast.iters=100}} passes
* javadocs outbound links manually checked

I think we're good to go here and if no one has any additional comments, 
concerns, etc. then I'll proceed to commit the changes in around a week or so 
from today.

> Implement RankNet.
> --
>
> Key: SOLR-11597
> URL: https://issues.apache.org/jira/browse/SOLR-11597
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Reporter: Michael A. Alcorn
>Assignee: Christine Poerschke
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11597.patch
>
>
> Implement RankNet as described in [this 
> tutorial|https://github.com/airalcorn2/Solr-LTR].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11597) Implement RankNet.

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372062#comment-16372062
 ] 

Christine Poerschke commented on SOLR-11597:


Thanks [~malcorn_redhat]] for reviewing and merging in my [tweaks to fix two 
test failures|https://github.com/airalcorn2/lucene-solr/pull/3], [1x javadocs + 
2x test tweaks|https://github.com/airalcorn2/lucene-solr/pull/4] and [1x 
javadocs + 2x test tweaks|https://github.com/airalcorn2/lucene-solr/pull/5] 
pull requests!

Attached SOLR-11597.patch is a snapshot of the changes as of now on top of 
current master:
* {{ant precommit}} passes
* {{ant beast -Dtestcase=TestNeuralNetworkModel -Dbeast.iters=100}} passes
* javadocs outbound links manually checked

I think we're good to go here and if no one has any additional comments, 
concerns, etc. then I'll proceed to commit the changes in around a week or so 
from today.

> Implement RankNet.
> --
>
> Key: SOLR-11597
> URL: https://issues.apache.org/jira/browse/SOLR-11597
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Reporter: Michael A. Alcorn
>Assignee: Christine Poerschke
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11597.patch
>
>
> Implement RankNet as described in [this 
> tutorial|https://github.com/airalcorn2/Solr-LTR].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11597) Implement RankNet.

[Christine Poerschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke updated SOLR-11597:
---
Attachment: SOLR-11597.patch

> Implement RankNet.
> --
>
> Key: SOLR-11597
> URL: https://issues.apache.org/jira/browse/SOLR-11597
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
>Reporter: Michael A. Alcorn
>Assignee: Christine Poerschke
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-11597.patch
>
>
> Implement RankNet as described in [this 
> tutorial|https://github.com/airalcorn2/Solr-LTR].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4065) FilteringTokenFilter should never corrupt the tokenstream graph

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372046#comment-16372046
 ] 

Steve Rowe commented on LUCENE-4065:


[~rcmuir] commented over on 
[https://issues.apache.org/jira/browse/SOLR-11968?focusedCommentId=16370916=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16370916|SOLR-11968]
 about this issue:

{quote}
I think the issue is still valid, its a little more complex now because of 
positionLength (means more buffering when you see posLength > 1, because you'll 
need to adjust if you remove something in its path), but the idea is the same: 
give the user a choice between "insert mode" and "replace mode". But this new 
"insert mode" would actually work correctly, correcting posLengths before and 
posIncs after as appropriate. similar to how your editor might have to 
recompute some line breaks/word wrapping and so on.

If you have baseball (length=2), base(length=1), ball(length=1), and you delete 
"base" in this case, you need to change baseball's length to 1 before you omit 
it, because you deleted base. Thats the "buffering before" that would be 
required for posLength. And you still need the same buffering described on the 
issue for posInc=0 that might occur after the fact, so you don't wrongly 
transfer synonyms to different words entirely.

It would be slower than "replace mode" that we have today, but only because of 
the buffering, and I think its pretty contained, but I haven't fully thought it 
thru or tried to write any code.
{quote}

 +1, though I find the nomenclature confusing; in your proposed "insert mode", 
token deletions would not leave any trace of the deleted tokens -- in posinc 
and poslen -- right?  (I get that you mean "insert mode" and "replace mode" as 
a metaphoric for editor operations.)  Isn't the issue just whether to leave 
gaps (as indicated by posinc and poslen) where deleted tokens were?

> FilteringTokenFilter should never corrupt the tokenstream graph
> ---
>
> Key: LUCENE-4065
> URL: https://issues.apache.org/jira/browse/LUCENE-4065
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/analysis
>Reporter: Robert Muir
>Priority: Major
> Attachments: LUCENE-4065_test.patch
>
>
> Currently removers like stopfilter have an option (true/false) to enable 
> position increments.
> If its true: it both inserts gaps where necessary AND propagates gaps down 
> the stream.
> If its false: it does neither, which can totally mess up the tokenstream 
> graph (e.g. move synonyms to another word).
> There are totally valid natural usecases for false, where you don't want gaps 
> because you want phrasequeries to act as if the word was never actually there.
> But 'not inserting gaps' is separate from proper propagation of existing gaps.
> So I think we should provide an option (either fix 'false' or make it an 
> enum), where you still get a legit tokenstream and dont totally screw it up, 
> but you simply omit gaps.
> See LUCENE-3848 for more information (Where we at least fixed this case to 
> not begin the tokenstream with posinc=0)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11968) Multi-words query time synonyms

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372030#comment-16372030
 ] 

Steve Rowe commented on SOLR-11968:
---

FYI [~rcmuir] I'm going to copy your comment ^^ over to LUCENE-4065 and comment 
on it there.

> Multi-words query time synonyms
> ---
>
> Key: SOLR-11968
> URL: https://issues.apache.org/jira/browse/SOLR-11968
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers, Schema and Analysis
>Affects Versions: master (8.0), 6.6.2
> Environment: Centos 7.x
>Reporter: Dominique Béjean
>Priority: Major
>
> I am trying multi words query time synonyms with Solr 6.6.2 and 
> SynonymGraphFilterFactory filter as explain in this article
>  
> [https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-support/]
>   
>  My field type is :
> {code:java}
> 
>      
>        
>                      articles="lang/contractions_fr.txt"/>
>        
>        
>         ignoreCase="true"/>
>        
>      
>      
>        
>                      articles="lang/contractions_fr.txt"/>
>        
>                      ignoreCase="true" expand="true"/>
>        
>         ignoreCase="true"/>
>        
>      
>    {code}
>  
>  synonyms.txt contains the line :
> {code:java}
> om, olympique de marseille{code}
>  
>  stopwords.txt contains the word 
> {code:java}
> de{code}
>  
>  The order of words in my query has an impact on the generated query in 
> edismax
> {code:java}
> q={!edismax qf='name_text_gp' v=$qq}
>  =false
>  =...{code}
> with "qq=om maillot" or "qq=olympique de marseille maillot", I can see the 
> synonyms expansion. It is working as expected.
> {code:java}
> "parsedquery_toString":"+(((+name_text_gp:olympiqu +name_text_gp:marseil 
> +name_text_gp:maillot) name_text_gp:om))",
>  "parsedquery_toString":"+((name_text_gp:om (+name_text_gp:olympiqu 
> +name_text_gp:marseil +name_text_gp:maillot)))",{code}
> with "qq=maillot om" or "qq=maillot olympique de marseille", I can see the 
> same generated query 
> {code:java}
> "parsedquery_toString":"+((name_text_gp:maillot) (name_text_gp:om))",
>  "parsedquery_toString":"+((name_text_gp:maillot) (name_text_gp:om))",{code}
> I don't understand these generated queries. The first one looks like the 
> synonym expansion is ignored, but the second one shows it is not ignored and 
> only the synonym term is used.
>   
>  When I test the analisys for the field type the synonyms are correctly 
> expanded for both expressions
> {code:java}
> om maillot  
>  maillot om
>  olympique de marseille maillot
>  maillot olympique de marseille{code}
> resulting outputs always include the following terms (obvioulsly not always 
> in the same order)
> {code:java}
> olympiqu om marseil maillot {code}
>  
>  So, i suspect an issue with edismax query parser.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11976) TokenizerChain is overwriting, not chaining TokenFilters in normalize()

[Tim Allison (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372022#comment-16372022
 ] 

Tim Allison commented on SOLR-11976:


Ping...any committer interested in this or a larger PR to swap out 
{{TokenizerChain}} for {{CustomAnalyzer}}?

> TokenizerChain is overwriting, not chaining TokenFilters in normalize()
> ---
>
> Key: SOLR-11976
> URL: https://issues.apache.org/jira/browse/SOLR-11976
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: master (8.0)
>Reporter: Tim Allison
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> TokenizerChain is overwriting, not chaining tokenfilters in {{normalize}}.  
> This doesn't currently break search because {{normalize}} is not being used 
> at the Solr level (AFAICT); rather, TextField has its own 
> {{analyzeMultiTerm()}} that duplicates code from the newer {{normalize}}. 
> Code as is:
> {noformat}
> TokenStream result = in;
> for (TokenFilterFactory filter : filters) {
>   if (filter instanceof MultiTermAwareComponent) {
> filter = (TokenFilterFactory) ((MultiTermAwareComponent) 
> filter).getMultiTermComponent();
> result = filter.create(in);
>   }
> }
> {noformat}
> The fix is simple:
> {noformat}
> -result = filter.create(in);
> +result = filter.create(result);
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12014) Cryptic error message when creating a collection with sharding that violates autoscaling policies

[Jerry Bao (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Bao updated SOLR-12014:
-
Description: 
When creating a collection with sharding a replication factors that are 
impossible because it will violate autoscaling policies, Solr raises a cryptic 
exception that is unrelated to the issue. 


{code:java}
{
"responseHeader":{
"status":500,
"QTime":629},
"Operation create caused 
exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
 Error closing CloudSolrClient",
"exception":{
"msg":"Error closing CloudSolrClient",
"rspCode":500},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":"Error closing CloudSolrClient",
"trace":"org.apache.solr.common.SolrException: Error closing 
CloudSolrClient\n\tat 
org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:309)\n\tat
 
org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)\n\tat
 
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)\n\tat
 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)\n\tat
 org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)\n\tat 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)\n\tat
 org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)\n\tat 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)\n\tat
 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)\n\tat
 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
 org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat
 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)\n\tat
 org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)\n\tat 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat
 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat
 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat
 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat
 java.lang.Thread.run(Thread.java:748)\n",
"code":500}}{code}

  was:When creating a collection with sharding a replication factors that are 
impossible because it will violate autoscaling policies, Solr raises a cryptic 
exception that is unrelated to the issue. 


> Cryptic error message when creating a collection with sharding that violates 
> autoscaling policies
> -
>
> Key: SOLR-12014
> URL: https://issues.apache.org/jira/browse/SOLR-12014
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> When creating a collection with sharding a replication factors that 

[jira] [Created] (SOLR-12014) Cryptic error message when creating a collection with sharding that violates autoscaling policies

[Jerry Bao (JIRA)]

Jerry Bao created SOLR-12014:


 Summary: Cryptic error message when creating a collection with 
sharding that violates autoscaling policies
 Key: SOLR-12014
 URL: https://issues.apache.org/jira/browse/SOLR-12014
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: AutoScaling
Affects Versions: 7.2
Reporter: Jerry Bao


When creating a collection with sharding a replication factors that are 
impossible because it will violate autoscaling policies, Solr raises a cryptic 
exception that is unrelated to the issue. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12013) collections API CUSTERSTATUS command fails when collections have errors

[Ben DeMott (JIRA)]

Ben DeMott created SOLR-12013:
-

 Summary: collections API CUSTERSTATUS command fails when 
collections have errors
 Key: SOLR-12013
 URL: https://issues.apache.org/jira/browse/SOLR-12013
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.2, 7.1, 7.0, 6.0
Reporter: Ben DeMott


CLUSTERSTATUS command can be given independent of a given collection.

http://localhost:8983/solr/admin/collections?action=CLUSTERSTATUS

I would expect that you can still inspect the status of a cluster even if a 
single collection has failed, or is missing its configuration.

*Expected behavior*: all healthy collections status is returned, unhealthy 
collections are either reported with a stacktrace in the response, reported in 
a failure state, or are not present from the response.

For example, CLUSTERSTATUS fails when a collection config-set is missing from 
ZooKeeper with:

{{*org.apache.solr.common.cloud.ZooKeeperException: Specified config does not 
exist in ZooKeeper: config-set-name*}}
{{ *at 
org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:189)*}}
{{ at 
org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(ClusterStatus.java:141)}}
{{ at 
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.lambda$static$19(CollectionsHandler.java:649)}}
{{ at 
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.execute(CollectionsHandler.java:888)}}
{{ at 
org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:226)}}
{{ at 
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:213)}}
{{ at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)}}
{{ at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:748)}}
{{ at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:729)}}
{{ at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)}}
{{ at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)}}
{{ at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)}}
{{ at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)}}
{{ at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)}}
{{ at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)}}
{{ at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)}}
{{ at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)}}
{{ at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)}}
{{ at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)}}
{{ at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)}}
{{ at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)}}
{{ at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)}}
{{ at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)}}
{{ at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)}}
{{ at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)}}
{{ at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)}}
{{ at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)}}
{{ at org.eclipse.jetty.server.Server.handle(Server.java:534)}}
{{ at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)}}
{{ at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)}}
{{ at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)}}
{{ at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)}}
{{ at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)}}
{{ at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)}}
{{ at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)}}
{{ at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)}}
{{ at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)}}
{{ at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)}}
{{ at java.lang.Thread.run(Thread.java:745)}}

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 466 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/466/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ReplaceNodeNoTargetTest.test

Error Message:
org/apache/solr/cloud/ReplaceNodeTest

Stack Trace:
java.lang.NoClassDefFoundError: org/apache/solr/cloud/ReplaceNodeTest
at 
__randomizedtesting.SeedInfo.seed([EC147D1D9061EF67:644042C73E9D829F]:0)
at 
org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:77)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.ClassNotFoundException: 
org.apache.solr.cloud.ReplaceNodeTest
at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:185)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
... 39 more




Build Log:
[...truncated 13698 lines...]
 

[JENKINS] Lucene-Solr-SmokeRelease-7.x - Build # 153 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/153/

No tests ran.

Build Log:
[...truncated 28782 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist
 [copy] Copying 491 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 215 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] Java 9 JAVA_HOME=/home/jenkins/tools/java/latest1.9
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (30.6 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-7.3.0-src.tgz...
   [smoker] 31.7 MB in 0.03 sec (1186.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.3.0.tgz...
   [smoker] 73.2 MB in 0.06 sec (1180.7 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.3.0.zip...
   [smoker] 83.8 MB in 0.07 sec (1189.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-7.3.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.3.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] test demo with 9...
   [smoker]   got 6290 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.3.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 217 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker] run tests w/ Java 9 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 9...
   [smoker]   got 217 hits for query "lucene"
   [smoker] checkindex with 9...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.00 sec (248.0 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-7.3.0-src.tgz...
   [smoker] 54.1 MB in 0.05 sec (1075.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.3.0.tgz...
   [smoker] 151.0 MB in 0.17 sec (909.1 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.3.0.zip...
   [smoker] 152.0 MB in 0.16 sec (932.8 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-7.3.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-7.3.0.tgz...
   [smoker]   **WARNING**: skipping check of 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
   [smoker]   **WARNING**: skipping check of 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
   [smoker] copying unpacked distribution for Java 8 ...
   [smoker] test solr example w/ Java 8...
   [smoker]   start Solr instance 
(log=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0-java8/solr-example.log)...
   [smoker] No process found for Solr node running on port 8983
   [smoker]   Running techproducts example on port 8983 from 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/build/smokeTestRelease/tmp/unpack/solr-7.3.0-java8
   [smoker] *** [WARN] *** Your open file 

[jira] [Resolved] (SOLR-10079) TestInPlaceUpdates(Distrib|Standalone) failures

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya resolved SOLR-10079.
-
Resolution: Fixed

> TestInPlaceUpdates(Distrib|Standalone) failures
> ---
>
> Key: SOLR-10079
> URL: https://issues.apache.org/jira/browse/SOLR-10079
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Cao Manh Dat
>Priority: Major
> Fix For: 6.7, 7.0
>
> Attachments: SOLR-10079.patch, SOLR-10079.patch, stdout, 
> tests-failures.txt
>
>
> From [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18881/], 
> reproduces for me:
> {noformat}
> Checking out Revision d8d61ff61d1d798f5e3853ef66bc485d0d403f18 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestInPlaceUpdatesDistrib -Dtests.method=test 
> -Dtests.seed=E1BB56269B8215B0 -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=sr-Latn-RS -Dtests.timezone=America/Grand_Turk 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 77.7s J2 | TestInPlaceUpdatesDistrib.test <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Earlier: [79, 79, 
> 79], now: [78, 78, 78]
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([E1BB56269B8215B0:69EF69FC357E7848]:0)
>[junit4]>  at 
> org.apache.solr.update.TestInPlaceUpdatesDistrib.ensureRtgWorksWithPartialUpdatesTest(TestInPlaceUpdatesDistrib.java:425)
>[junit4]>  at 
> org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:142)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>[junit4]>  at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>[junit4]>  at 
> java.base/java.lang.reflect.Method.invoke(Method.java:543)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
>[junit4]>  at java.base/java.lang.Thread.run(Thread.java:844)
> [...]
>[junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
> {id_i=PostingsFormat(name=LuceneFixedGap), title_s=FSTOrd50, 
> id=PostingsFormat(name=Asserting), 
> id_field_copy_that_does_not_support_in_place_update_s=FSTOrd50}, 
> docValues:{inplace_updatable_float=DocValuesFormat(name=Asserting), 
> id_i=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Asserting), 
> title_s=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), 
> id_field_copy_that_does_not_support_in_place_update_s=DocValuesFormat(name=Lucene70),
>  inplace_updatable_int_with_default=DocValuesFormat(name=Asserting), 
> inplace_updatable_int=DocValuesFormat(name=Direct), 
> inplace_updatable_float_with_default=DocValuesFormat(name=Direct)}, 
> maxPointsInLeafNode=1342, maxMBSortInHeap=6.368734895089348, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=sr-Latn-RS, 
> timezone=America/Grand_Turk
>[junit4]   2> NOTE: Linux 4.4.0-53-generic i386/Oracle Corporation 9-ea 
> (32-bit)/cpus=12,threads=1,free=107734480,total=518979584
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10079) TestInPlaceUpdates(Distrib|Standalone) failures

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371969#comment-16371969
 ] 

Ishan Chattopadhyaya commented on SOLR-10079:
-

This hasn't failed in a while. The above seed doesn't reproduce for me anymore:
{code}
ant test  -Dtestcase=TestInPlaceUpdatesDistrib -Dtests.method=test 
-Dtests.seed=F02AC8BA5333D665 -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=pt -Dtests.timezone=America/Indianapolis -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
{code}

> TestInPlaceUpdates(Distrib|Standalone) failures
> ---
>
> Key: SOLR-10079
> URL: https://issues.apache.org/jira/browse/SOLR-10079
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Cao Manh Dat
>Priority: Major
> Fix For: 6.7, 7.0
>
> Attachments: SOLR-10079.patch, SOLR-10079.patch, stdout, 
> tests-failures.txt
>
>
> From [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18881/], 
> reproduces for me:
> {noformat}
> Checking out Revision d8d61ff61d1d798f5e3853ef66bc485d0d403f18 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestInPlaceUpdatesDistrib -Dtests.method=test 
> -Dtests.seed=E1BB56269B8215B0 -Dtests.multiplier=3 -Dtests.slow=true 
> -Dtests.locale=sr-Latn-RS -Dtests.timezone=America/Grand_Turk 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 77.7s J2 | TestInPlaceUpdatesDistrib.test <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Earlier: [79, 79, 
> 79], now: [78, 78, 78]
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([E1BB56269B8215B0:69EF69FC357E7848]:0)
>[junit4]>  at 
> org.apache.solr.update.TestInPlaceUpdatesDistrib.ensureRtgWorksWithPartialUpdatesTest(TestInPlaceUpdatesDistrib.java:425)
>[junit4]>  at 
> org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:142)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>[junit4]>  at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>[junit4]>  at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>[junit4]>  at 
> java.base/java.lang.reflect.Method.invoke(Method.java:543)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
>[junit4]>  at java.base/java.lang.Thread.run(Thread.java:844)
> [...]
>[junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
> {id_i=PostingsFormat(name=LuceneFixedGap), title_s=FSTOrd50, 
> id=PostingsFormat(name=Asserting), 
> id_field_copy_that_does_not_support_in_place_update_s=FSTOrd50}, 
> docValues:{inplace_updatable_float=DocValuesFormat(name=Asserting), 
> id_i=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Asserting), 
> title_s=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), 
> id_field_copy_that_does_not_support_in_place_update_s=DocValuesFormat(name=Lucene70),
>  inplace_updatable_int_with_default=DocValuesFormat(name=Asserting), 
> inplace_updatable_int=DocValuesFormat(name=Direct), 
> inplace_updatable_float_with_default=DocValuesFormat(name=Direct)}, 
> maxPointsInLeafNode=1342, maxMBSortInHeap=6.368734895089348, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=sr-Latn-RS, 
> timezone=America/Grand_Turk
>[junit4]   2> NOTE: Linux 4.4.0-53-generic i386/Oracle Corporation 9-ea 
> (32-bit)/cpus=12,threads=1,free=107734480,total=518979584
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10261) TestStressCloudBlindAtomicUpdates.test_dv() fail

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-10261:

Fix Version/s: 7.3

> TestStressCloudBlindAtomicUpdates.test_dv() fail
> 
>
> Key: SOLR-10261
> URL: https://issues.apache.org/jira/browse/SOLR-10261
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Fix For: 7.3
>
> Attachments: SOLR-10261.patch, SOLR-10261.patch
>
>
> I found a reproducing seed that cause 
> TestStressCloudBlindAtomicUpdates.test_dv() fail
> {code}
> [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestStressCloudBlindAtomicUpdates -Dtests.method=test_dv 
> -Dtests.seed=AD8E7B56D53B627F -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.locale=bg -Dtests.timezone=America/La_Paz -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   1.21s J2 | TestStressCloudBlindAtomicUpdates.test_dv <<<
>[junit4]> Throwable #1: java.util.concurrent.ExecutionException: 
> java.lang.RuntimeException: Error from server at 
> http://127.0.0.1:49825/solr/test_col: Async exception during distributed 
> update: Error from server at 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2: Server Error
>[junit4]> request: 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A49825%2Fsolr%2Ftest_col_shard5_replica1%2F=javabin=2
>[junit4]> Remote error message: Failed synchronous update on shard 
> StdNode: http://127.0.0.1:49836/solr/test_col_shard2_replica1/ update: 
> org.apache.solr.client.solrj.request.UpdateRequest@5919dfb3
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([AD8E7B56D53B627F:9B9A19105F66586E]:0)
>[junit4]>  at 
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
>[junit4]>  at 
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
>[junit4]>  at 
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
>[junit4]>  at 
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]> Caused by: java.lang.RuntimeException: Error from server at 
> http://127.0.0.1:49825/solr/test_col: Async exception during distributed 
> update: Error from server at 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2: Server Error
>[junit4]> request: 
> http://127.0.0.1:49824/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A49825%2Fsolr%2Ftest_col_shard5_replica1%2F=javabin=2
>[junit4]> Remote error message: Failed synchronous update on shard 
> StdNode: http://127.0.0.1:49836/solr/test_col_shard2_replica1/ update: 
> org.apache.solr.client.solrj.request.UpdateRequest@5919dfb3
>[junit4]>  at 
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
>[junit4]>  at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>[junit4]>  at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>[junit4]>  at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
>[junit4]>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>[junit4]>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>[junit4]>  ... 1 more
>[junit4]> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://127.0.0.1:49825/solr/test_col: Async exception during 
> distributed update: Error from server at 
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8182) BoostingQuery applies the wrong boost to the query score

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371832#comment-16371832
 ] 

Adrien Grand commented on LUCENE-8182:
--

+1 Maybe also run QueryUtils.check(Random,Query,IndexSearcher) in the test? It 
makes sure that explanations are sane, that they give the same score as the 
Scorer, etc.

> BoostingQuery applies the wrong boost to the query score
> 
>
> Key: LUCENE-8182
> URL: https://issues.apache.org/jira/browse/LUCENE-8182
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.0, 7.1, 7.2
>Reporter: Jim Ferenczi
>Priority: Major
> Attachments: LUCENE-8182.patch, LUCENE-8182.patch, LUCENE-8182.patch
>
>
> BoostingQuery applies the parent query boost instead of the boost set on the 
> query due to a name clash in the anonymous class created by the createWeight 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1483 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1483/

No tests ran.

Build Log:
[...truncated 511 lines...]
ERROR: command execution failed.
ERROR: Step ‘Archive the artifacts’ failed: no workspace for 
Lucene-Solr-NightlyTests-master #1483
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for 
Lucene-Solr-NightlyTests-master #1483
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)
ERROR: lucene2 is offline; cannot locate JDK 1.8 (latest)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-9) - Build # 4454 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4454/
Java: 64bit/jdk-9 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.cloud.ReplaceNodeNoTargetTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([979EE4A4CC8EA76C:1FCADB7E6272CA94]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<5> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([979EE4A4CC8EA76C:FA62405976C6586B]:0)
at org.junit.Assert.fail(Assert.java:93)
at 

[JENKINS] Lucene-Solr-Tests-master - Build # 2355 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2355/

No tests ran.

Build Log:
[...truncated 130 lines...]
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files 
were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Erick Erickson]

Yonik:

What I'm frustrated by now is that variations on these themes haven't
cured the problem, and it's spun out of control and is getting worse.
It's the "getting worse" part that is most disturbing. Continuing as
we have in the past isn't working, it's time to try something else.

There are 17 open JIRAs for tests right now. Some as far back as 2010,
listed below. Since last night I've collected 49 distinct failures
from the dev e-mails (haven't triaged them completely to see if some
are contained in others, but "sort < all_of_them | uniq > my_file"
results in a file 49 lines long).

What I'm after here is a way to keep from backsliding further, and a
path to getting better. That's what's behind the straw-man proposal
that we get the tests to be clean, even if that means disabling the
ones that are flakey. I should have emphasized more strongly that the
corollary to disabling flakey tests is that we need to get aggressive
about not tolerating _new_ flaky tests. I'm volunteering (I guess) to
be the enforcer here, as much as public comments can be construed to
be "enforcing" ;)

If someone has a favorite test that they think adds value even if it
fails fairly frequently, we can un-BadApple it provided someone is
actively working on it. Or it can be un-BadAppled locally and/or
temporarily. I'm perfectly fine with flakey tests being run and
reported _if_ that helps resolve it.

Also I'm volunteering to produce a "weekly BadApple" list so people
can work on them as they see fit expressly to keep them from getting
lost.

(1) I have  no idea how to do this, or even if it's possible. What do
you have in mind?

(2) doesn't seem to be working based on the open JIRAs below and the
number of failing tests that are accumulating.

(3a) Well, the first part is what my "enforcing" comment is about
above I think ;)

(3b) I'd argue that the part about "without losing test coverage" is
partly addressed by the notion of running periodically with BadApple
disabled. Which each individual can also do at their discretion if
they care to. More importantly though, the test coverage isn't very
useful when failures are ignored anyway.

(4) I thoroughly applaud that one long term, but I'll settle in the
short term for doing something to keep even _more_ from accumulating.

I should also emphasize that disabling tests is certainly _NOT_ my
preference, fixing them all is. I doubt that declaring a moratorium on
all commits until all the tests were fixed is practical though ;) And
without something changing in our approach, I don't see much progress
being made.

SOLR-10053
SOLR-10070
SOLR-10071
SOLR-10139
SOLR-10287
SOLR-10815
SOLR-11911
SOLR-2175
SOLR-4147
SOLR-5880
SOLR-6423
SOLR-6944
SOLR-6961
SOLR-6974
SOLR-8122
SOLR-8182
SOLR-9869


On Wed, Feb 21, 2018 at 11:12 AM, Yonik Seeley  wrote:
> We should be careful not to conflate running of unit tests with
> automated reporting, and the differing roles that flakey tests play in
> different scenarios.
> For example, I no longer pay attention to automated failure reports,
> esp if I haven't committed anything recently.
> However, when I'm making code changes and do "ant test", I certainly
> pay attention to failures and re-run any failing tests.  It sucks to
> have to re-run a test just because it's flakey, but it's better than
> accidentally committing a bug because test coverage was reduced.
>
> I'd suggest:
> 1) fix/tweak automated test reporting to increase relevancy for developers
> 2) open a JIRA for each flakey test and evaluate impact of removal on
> test coverage
> 3) If a new feature is added, and the test turns out to be flakey,
> then the feature itself should be disabled before release.  This
> prevents both new flakey tests without resulting in loss of test
> coverage, as well as motivates those who care about the feature to fix
> the tests.
> 4) fix flakey tests ;-)
>
> -Yonik
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 80 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/80/

[...truncated 38 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-Tests-master/2354/consoleText

[repro] Revision: a9f0272380438df88d29ed7c41572136f999f8db

[repro] Repro line:  ant test  -Dtestcase=TestUtilizeNode -Dtests.method=test 
-Dtests.seed=232BF95961C8BCC -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=sr-CS -Dtests.timezone=Australia/ACT -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestLargeCluster 
-Dtests.method=testSearchRate -Dtests.seed=232BF95961C8BCC -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=ms-MY -Dtests.timezone=America/El_Salvador 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=ComputePlanActionTest 
-Dtests.method=testNodeWithMultipleReplicasLost -Dtests.seed=232BF95961C8BCC 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=cs 
-Dtests.timezone=Australia/LHI -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestLBHttpSolrClient 
-Dtests.seed=CFA3DBE1EAE1E10A -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=zh-HK -Dtests.timezone=PST8PDT -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
82a99840718bb7e696ef4af114eee976c776a251
[repro] git checkout a9f0272380438df88d29ed7c41572136f999f8db

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/solrj
[repro]   TestLBHttpSolrClient
[repro]solr/core
[repro]   TestLargeCluster
[repro]   TestUtilizeNode
[repro]   ComputePlanActionTest
[repro] ant compile-test

[...truncated 2444 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestLBHttpSolrClient" -Dtests.showOutput=onerror 
-Dtests.seed=CFA3DBE1EAE1E10A -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=zh-HK -Dtests.timezone=PST8PDT -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[...truncated 78 lines...]
[repro] ant compile-test

[...truncated 1329 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 
-Dtests.class="*.TestLargeCluster|*.TestUtilizeNode|*.ComputePlanActionTest" 
-Dtests.showOutput=onerror -Dtests.seed=232BF95961C8BCC -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=ms-MY -Dtests.timezone=America/El_Salvador 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 33641 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.client.solrj.TestLBHttpSolrClient
[repro]   0/5 failed: org.apache.solr.cloud.TestUtilizeNode
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.ComputePlanActionTest

[repro] Re-testing 100% failures at the tip of master
[repro] git checkout master

[...truncated 4 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 8 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   ComputePlanActionTest
[repro] ant compile-test

[...truncated 3293 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.ComputePlanActionTest" -Dtests.showOutput=onerror 
-Dtests.seed=232BF95961C8BCC -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=cs -Dtests.timezone=Australia/LHI -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[...truncated 20086 lines...]
[repro] Setting last failure code to 256

[repro] Failures at the tip of master:
[repro]   3/5 failed: org.apache.solr.cloud.autoscaling.ComputePlanActionTest
[repro] git checkout 82a99840718bb7e696ef4af114eee976c776a251

[...truncated 8 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-8182) BoostingQuery applies the wrong boost to the query score

[Jim Ferenczi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Ferenczi resolved LUCENE-8182.
--
   Resolution: Fixed
Fix Version/s: 7.3

> BoostingQuery applies the wrong boost to the query score
> 
>
> Key: LUCENE-8182
> URL: https://issues.apache.org/jira/browse/LUCENE-8182
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.0, 7.1, 7.2
>Reporter: Jim Ferenczi
>Priority: Major
> Fix For: 7.3
>
> Attachments: LUCENE-8182.patch, LUCENE-8182.patch, LUCENE-8182.patch
>
>
> BoostingQuery applies the parent query boost instead of the boost set on the 
> query due to a name clash in the anonymous class created by the createWeight 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8182) BoostingQuery applies the wrong boost to the query score

[Jim Ferenczi (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371939#comment-16371939
 ] 

Jim Ferenczi commented on LUCENE-8182:
--

I pushed the fix with the additional tests (QueryUtils.check). Thanks 
[~jpountz] and [~romseygeek] !

> BoostingQuery applies the wrong boost to the query score
> 
>
> Key: LUCENE-8182
> URL: https://issues.apache.org/jira/browse/LUCENE-8182
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.0, 7.1, 7.2
>Reporter: Jim Ferenczi
>Priority: Major
> Attachments: LUCENE-8182.patch, LUCENE-8182.patch, LUCENE-8182.patch
>
>
> BoostingQuery applies the parent query boost instead of the boost set on the 
> query due to a name clash in the anonymous class created by the createWeight 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Yonik Seeley]

We should be careful not to conflate running of unit tests with
automated reporting, and the differing roles that flakey tests play in
different scenarios.
For example, I no longer pay attention to automated failure reports,
esp if I haven't committed anything recently.
However, when I'm making code changes and do "ant test", I certainly
pay attention to failures and re-run any failing tests.  It sucks to
have to re-run a test just because it's flakey, but it's better than
accidentally committing a bug because test coverage was reduced.

I'd suggest:
1) fix/tweak automated test reporting to increase relevancy for developers
2) open a JIRA for each flakey test and evaluate impact of removal on
test coverage
3) If a new feature is added, and the test turns out to be flakey,
then the feature itself should be disabled before release.  This
prevents both new flakey tests without resulting in loss of test
coverage, as well as motivates those who care about the feature to fix
the tests.
4) fix flakey tests ;-)

-Yonik

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12012) Replicas should skip doing recovery on startup

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-12012:
-
Component/s: SolrCloud

> Replicas should skip doing recovery on startup
> --
>
> Key: SOLR-12012
> URL: https://issues.apache.org/jira/browse/SOLR-12012
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> Right now a replica when first loaded always does recovery (except the 
> replica is the leader). This will lead to several problems, for example: 
> - leaderless if at the same time the replica is doing recovery, the current 
> leader goes down.
> - the recovery process is not necessary if the replica is already in-sync 
> with the leader
> By using term value introduced in SOLR-11702 we can skip the recovery process 
> if replica's term equals to leader's term.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8182) BoostingQuery applies the wrong boost to the query score

[Jim Ferenczi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Ferenczi updated LUCENE-8182:
-
Attachment: LUCENE-8182.patch

> BoostingQuery applies the wrong boost to the query score
> 
>
> Key: LUCENE-8182
> URL: https://issues.apache.org/jira/browse/LUCENE-8182
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.0, 7.1, 7.2
>Reporter: Jim Ferenczi
>Priority: Major
> Attachments: LUCENE-8182.patch, LUCENE-8182.patch, LUCENE-8182.patch
>
>
> BoostingQuery applies the parent query boost instead of the boost set on the 
> query due to a name clash in the anonymous class created by the createWeight 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-11911) TestLargeCluster.testSearchRate() failure

[Andrzej Bialecki (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki  reassigned SOLR-11911:


Assignee: Andrzej Bialecki 

> TestLargeCluster.testSearchRate() failure
> -
>
> Key: SOLR-11911
> URL: https://issues.apache.org/jira/browse/SOLR-11911
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Andrzej Bialecki 
>Priority: Major
>
> My Jenkins found a branch_7x seed that reproduced 4/5 times for me:
> {noformat}
> Checking out Revision af9706cb89335a5aa04f9bcae0c2558a61803b50 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLargeCluster 
> -Dtests.method=testSearchRate -Dtests.seed=2D7724685882A83D -Dtests.slow=true 
> -Dtests.locale=be-BY -Dtests.timezone=Africa/Ouagadougou -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.24s J0  | TestLargeCluster.testSearchRate <<<
>[junit4]> Throwable #1: java.lang.AssertionError: The trigger did not 
> fire at all
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([2D7724685882A83D:703F3AE197440E72]:0)
>[junit4]>  at 
> org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
>[junit4]>  at java.lang.Thread.run(Thread.java:748)
> [...]
>[junit4]   2> NOTE: test params are: codec=CheapBastard, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=be-BY, 
> timezone=Africa/Ouagadougou
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_151 (64-bit)/cpus=16,threads=1,free=388243840,total=502267904
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8182) BoostingQuery applies the wrong boost to the query score

[Jim Ferenczi (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371823#comment-16371823
 ] 

Jim Ferenczi commented on LUCENE-8182:
--

Thanks for checking [~jpountz] ! I attached a new patch with the renaming.

> BoostingQuery applies the wrong boost to the query score
> 
>
> Key: LUCENE-8182
> URL: https://issues.apache.org/jira/browse/LUCENE-8182
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.0, 7.1, 7.2
>Reporter: Jim Ferenczi
>Priority: Major
> Attachments: LUCENE-8182.patch, LUCENE-8182.patch
>
>
> BoostingQuery applies the parent query boost instead of the boost set on the 
> query due to a name clash in the anonymous class created by the createWeight 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8182) BoostingQuery applies the wrong boost to the query score

[Jim Ferenczi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jim Ferenczi updated LUCENE-8182:
-
Attachment: LUCENE-8182.patch

> BoostingQuery applies the wrong boost to the query score
> 
>
> Key: LUCENE-8182
> URL: https://issues.apache.org/jira/browse/LUCENE-8182
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.0, 7.1, 7.2
>Reporter: Jim Ferenczi
>Priority: Major
> Attachments: LUCENE-8182.patch, LUCENE-8182.patch
>
>
> BoostingQuery applies the parent query boost instead of the boost set on the 
> query due to a name clash in the anonymous class created by the createWeight 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Test failures are out of control......

[Erick Erickson]

Dawid:
Yep, definitely a recurring theme. But this time I may actually, you
know, do something about it ;)

Mark is one of the advocates of this theme, perhaps he got exhausted
trying to push that stone up the hill ;). Maybe it's my turn to pick
up the baton Comments about there being value to seeing these is
well taken, but outweighed IMO by the harm in there being so much
noise that failures that _should_ get attention are so easy to
overlook.

bq: The noise in Solr tests have increased to a degree that I stopped looking.

Exactly. To one degree or another I think this has happened to a _lot_
of people, myself certainly included.

And you've certainly done more than your share of fixing things in the
infrastructure, many thanks!



I'm not sure blanket @BadApple-ing these is The Right Thing To Do for
_all_ of them though as I know lots of active work is being done in
some areas. I'd hate for someone to be working in some area and
currently trying to fix something and have the  failures disappear and
think they were fixed when in reality they just weren't run.

Straw-man proposal:

> I'll volunteer to gather failing tests through the next few days from the dev 
> e-mails. I'll create yet another umbrella JIRA that proposes to @BadApple 
> _all_ of them unless someone steps up and volunteers to actively work on a 
> particular test failure. Since I brought it up I'll get aggressive about 
> @BadApple-ing failing tests in future. I'll link the current JIRAs for 
> failing tests in as well (on a cursory glance there are 16 open ones)...

> If someone objects to @BadApple-ing a particular test, they should create a 
> JIRA, assign it to themselves and actively work on it. Short shrift given to 
> "I don't think we should @BadApple that test because someday someone might 
> want to try to fix it" In this proposal, it's perfectly acceptable to 
> remove the @BadApple notation and push it, as long as it's being actively 
> worked on.

> Would someone who knows the test infrastructure better than me be willing to 
> volunteer to set up a run periodically with BadApple annotations disabled. 
> Perhaps weekly? Even nightly? That way interested parties can see these but 
> the rest of us would only have _one_ e-mail to ignore, not 10-20 a day. It'd 
> be great if the subject mentioned something allowing the WithBadApple runs to 
> be identified just by glancing at the subject. Then errors that didn't 
> have BadApple annotations would stand out from the noise since they would be 
> in _other_ emails.

> It's easy enough to find all the BadApple-labeled tests, I'll also volunteer 
> to post a weekly list.

Getting e-mails for flakey tests is acceptable IMO only if people are
working on them. I've certainly been in situations where I can't get
something to fail locally and have to rely on Jenkins etc to gather
logging info or see if my fixes really work. I _do_ care that we are
accumulating more and more failures and it's getting harder and harder
to know when failures are a function of new code or not.

WDYT?
Erick

On Wed, Feb 21, 2018 at 8:36 AM, Tommaso Teofili
 wrote:
> +1, agree with Adrien, thanks for bringing this up Erick!
>
>
>
> Il giorno mer 21 feb 2018 alle ore 17:15 Adrien Grand  ha
> scritto:
>>
>> Thanks for bringing this up Erick. I agree with you we should silence
>> those frequent failures. Like you said, the side-effects of not silencing
>> them are even worse. I'll add that these flaky tests also make releasing
>> harder, it took me three runs last time (Lucene/Solr 7.2) for the release
>> build to succeed because of failed tests.
>>
>> Le mer. 21 févr. 2018 à 16:52, Erick Erickson  a
>> écrit :
>>>
>>> There's an elephant in the room, and it's that failing tests are being
>>> ignored. Mind you, Solr and Lucene are progressing at a furious pace
>>> with lots of great functionality being added. That said, we're
>>> building up a considerable "technical debt" when it comes to testing.
>>>
>>> And I should say up front that major new functionality is expected to
>>> take a while to shake out (e.g. autoscaling, streaming, V2 API etc.),
>>> and noise from tests of new functionality is expected while things
>>> bake.
>>>
>>> Below is a list of tests that have failed at least once since just
>>> last night. This has been getting worse as time passes, the broken
>>> window problem. Some e-mails have 10 failing tests (+/-) so unless I
>>> go through each and every one I don't know whether something I've done
>>> is causing a problem or not.
>>>
>>> I'm as guilty of letting things slide as anyone else, there's been a
>>> long-standing issue with TestLazyCores that I work on sporadically for
>>> instance that's _probably_ "something in the test framework"
>>>
>>> Several folks have spent some time digging into test failures and
>>> identifying at least some of the causes, kudos to them. It seems
>>> 

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_162) - Build # 21505 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21505/
Java: 32bit/jdk1.8.0_162 -client -XX:+UseParallelGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.search.join.TestCloudNestedDocsSort

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.search.join.TestCloudNestedDocsSort: 1) Thread[id=8035, 
name=qtp10177888-8035, state=TIMED_WAITING, group=TGRP-TestCloudNestedDocsSort] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.search.join.TestCloudNestedDocsSort: 
   1) Thread[id=8035, name=qtp10177888-8035, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([DB3A8ADC0CE6F582]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.search.join.TestCloudNestedDocsSort

Error Message:
There are still zombie threads that couldn't be terminated:1) 
Thread[id=8035, name=qtp10177888-8035, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
 at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=8035, name=qtp10177888-8035, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([DB3A8ADC0CE6F582]:0)




Build Log:
[...truncated 12441 lines...]
   [junit4] Suite: org.apache.solr.search.join.TestCloudNestedDocsSort
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/init-core-data-001
   [junit4]   2> 423391 WARN  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
   [junit4]   2> 423391 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 423393 INFO 

[jira] [Updated] (SOLR-12011) Consistence problem when in-sync replicas are DOWN

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-12011:
-
Summary: Consistence problem when in-sync replicas are DOWN  (was: 
Consitence problem when in-sync replicas are DOWN)

> Consistence problem when in-sync replicas are DOWN
> --
>
> Key: SOLR-12011
> URL: https://issues.apache.org/jira/browse/SOLR-12011
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> Currently, we will meet consistency problem when in-sync replicas are DOWN. 
> For example:
> 1. A collection with 1 shard with 1 leader and 2 replicas
> 2. Nodes contain 2 replicas go down
> 3. The leader receives an update A, success
> 4. The node contains the leader goes down
> 5. 2 replicas come back
> 6. One of them become leader --> But they shouldn't become leader since they 
> missed the update A



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-7034) Consider allowing any node to become leader, regardless of their last published state.

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-7034:

Component/s: SolrCloud

> Consider allowing any node to become leader, regardless of their last 
> published state.
> --
>
> Key: SOLR-7034
> URL: https://issues.apache.org/jira/browse/SOLR-7034
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: 5.2, 6.0
>
> Attachments: SOLR-7034.patch, SOLR-7034.patch
>
>
> Now that we allow a min replication param for updates, I think it's time to 
> loosen this up. Currently, you can end up in a state where no one in a shard 
> thinks they can be leader and you so do this fast ugly infinite loop trying 
> to pick the leader.
> We should let anyone that is able to properly sync with the available 
> replicas to become leader if that process succeeds.
> The previous strategy was to account for the case of not having enough 
> replicas after a machine loss to ensure you don't lose the data. The idea was 
> that you should stop the cluster to avoid losing data and repair and get all 
> your replicas involved in a leadership election. Instead, we should favor 
> carrying on, and those that want to ensure they don't lose data due to major 
> replica loss should use the min replication update param.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12011) Consistence problem when in-sync replicas are DOWN

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-12011:
-
Component/s: SolrCloud

> Consistence problem when in-sync replicas are DOWN
> --
>
> Key: SOLR-12011
> URL: https://issues.apache.org/jira/browse/SOLR-12011
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> Currently, we will meet consistency problem when in-sync replicas are DOWN. 
> For example:
> 1. A collection with 1 shard with 1 leader and 2 replicas
> 2. Nodes contain 2 replicas go down
> 3. The leader receives an update A, success
> 4. The node contains the leader goes down
> 5. 2 replicas come back
> 6. One of them become leader --> But they shouldn't become leader since they 
> missed the update A



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

FINAL REMINDER: CFP for Apache EU Roadshow Closes 25th February

[Sharan F]

Hello Apache Supporters and Enthusiasts

This is your FINAL reminder that the Call for Papers (CFP) for the 
Apache EU Roadshow is closing soon. Our Apache EU Roadshow will focus on 
Cloud, IoT, Apache Tomcat, Apache Http and will run from 13-14 June 2018 
in Berlin.
Note that the CFP deadline has been extended to *25*^*th* *February *and 
it will be your final opportunity to submit a talk for thisevent.


Please make your submissions at http://apachecon.com/euroadshow18/

Also note that early bird ticket registrations to attend FOSS Backstage 
including the Apache EU Roadshow, have also been extended and will be 
available until 23^rd February. Please register at 
https://foss-backstage.de/tickets


We look forward to seeing you in Berlin!

Thanks
Sharan Foga, VP Apache Community Development

PLEASE NOTE: You are receiving this message because you are subscribed 
to a user@ or dev@ list of one or more Apache Software Foundation projects.

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk-9.0.1) - Build # 468 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/468/
Java: 64bit/jdk-9.0.1 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

8 tests failed.
FAILED:  
org.apache.lucene.replicator.IndexAndTaxonomyReplicationClientTest.testRestart

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1\index:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1\index

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1\index:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1\index
   
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\replicator\test\J1\temp\lucene.replicator.IndexAndTaxonomyReplicationClientTest_3F53BE143601E3D1-001\replicationClientTest-003\1

at 
__randomizedtesting.SeedInfo.seed([3F53BE143601E3D1:AADE5275B3E957C4]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at 
org.apache.lucene.replicator.PerSessionDirectoryFactory.cleanupSession(PerSessionDirectoryFactory.java:58)
at 
org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:259)
at 
org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:401)
at 
org.apache.lucene.replicator.IndexAndTaxonomyReplicationClientTest.testRestart(IndexAndTaxonomyReplicationClientTest.java:244)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 

Re: Test failures are out of control......

[Dawid Weiss]

It's a recurring theme, huh, Erick? :)

http://markmail.org/message/7eykbuyyaxbxn364

I agree with your opinion and I have expressed it more than once -- a
test that is failing for longer while and cannot be identified or
fixed is a candidate for removal. The noise in Solr tests have
increased to a degree that I stopped looking (a long time ago), unless
somebody explicitly pings me about something. I tried to fix some of
those tests, but it's beyond my capabilities in many cases (and my
time budget in others).

I also recall some folks had a different take on the subject; see Mark
Miller's opinion in the thread above, for example (there were other
threads too, but I can't find them now).

Dawid


On Wed, Feb 21, 2018 at 5:36 PM, Tommaso Teofili
 wrote:
> +1, agree with Adrien, thanks for bringing this up Erick!
>
>
>
> Il giorno mer 21 feb 2018 alle ore 17:15 Adrien Grand  ha
> scritto:
>>
>> Thanks for bringing this up Erick. I agree with you we should silence
>> those frequent failures. Like you said, the side-effects of not silencing
>> them are even worse. I'll add that these flaky tests also make releasing
>> harder, it took me three runs last time (Lucene/Solr 7.2) for the release
>> build to succeed because of failed tests.
>>
>> Le mer. 21 févr. 2018 à 16:52, Erick Erickson  a
>> écrit :
>>>
>>> There's an elephant in the room, and it's that failing tests are being
>>> ignored. Mind you, Solr and Lucene are progressing at a furious pace
>>> with lots of great functionality being added. That said, we're
>>> building up a considerable "technical debt" when it comes to testing.
>>>
>>> And I should say up front that major new functionality is expected to
>>> take a while to shake out (e.g. autoscaling, streaming, V2 API etc.),
>>> and noise from tests of new functionality is expected while things
>>> bake.
>>>
>>> Below is a list of tests that have failed at least once since just
>>> last night. This has been getting worse as time passes, the broken
>>> window problem. Some e-mails have 10 failing tests (+/-) so unless I
>>> go through each and every one I don't know whether something I've done
>>> is causing a problem or not.
>>>
>>> I'm as guilty of letting things slide as anyone else, there's been a
>>> long-standing issue with TestLazyCores that I work on sporadically for
>>> instance that's _probably_ "something in the test framework"
>>>
>>> Several folks have spent some time digging into test failures and
>>> identifying at least some of the causes, kudos to them. It seems
>>> they're voices crying out in the wilderness though.
>>>
>>> There is so much noise at this point that tests are becoming
>>> irrelevant. I'm trying to work on SOLR-10809 for instance, where
>>> there's a pretty good possibility that I'll close at least one thing
>>> that shouldn't be closed. So I ran the full suite 10 times and
>>> gathered all the failures. Now I have to try to separate the failures
>>> caused by that JIRA from the ones that aren't related to it so I beast
>>> each of the failing tests 100 times against master. If I get a failure
>>> on master too for a particular test, I'll assume it's "not my problem"
>>> and drive on.
>>>
>>> I freely acknowledge that this is poor practice. It's driven by
>>> frustration and the desire to make progress. While it's poor practice,
>>> it's not as bad as only looking at tests that I _think_ are related or
>>> ignoring all tests failures I can't instantly recognize as "my fault".
>>>
>>> So what's our stance on this? Mark Miller had a terrific program at
>>> one point allowing categorization of tests that failed at a glance,
>>> but it hasn't been updated in a while.  Steve Rowe is working on the
>>> problem too. Hoss and Cassandra have both added to the efforts as
>>> well. And I'm sure I'm leaving out others.
>>>
>>> Then there's the @Ignore and @BadApple annotations
>>>
>>> So, as a community, are we going to devote some energy to this? Or
>>> shall we just start ignoring all of the frequently failing tests?
>>> Frankly we'd be farther ahead at this point marking failing tests that
>>> aren't getting any work with @Ignore or @BadApple and getting
>>> compulsive about not letting any _new_ tests fail than continuing our
>>> current path. I don't _like_ this option mind you, but it's better
>>> than letting these accumulate forever and tests become more and more
>>> difficult to use. As tests become more difficult to use, they're used
>>> less and the problem gets worse.
>>>
>>> Note, I made no effort to separate suite .vs. individual reports
>>> here.
>>>
>>> Erick
>>>
>>> FAILED:
>>> junit.framework.TestSuite.org.apache.lucene.index.TestBagOfPositions
>>> FAILED:
>>> junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterDeleteByQuery
>>> FAILED:
>>> junit.framework.TestSuite.org.apache.lucene.store.TestSleepingLockWrapper
>>> FAILED:
>>> 

Re: Test failures are out of control......

[Tommaso Teofili]

+1, agree with Adrien, thanks for bringing this up Erick!



Il giorno mer 21 feb 2018 alle ore 17:15 Adrien Grand 
ha scritto:

> Thanks for bringing this up Erick. I agree with you we should silence
> those frequent failures. Like you said, the side-effects of not silencing
> them are even worse. I'll add that these flaky tests also make releasing
> harder, it took me three runs last time (Lucene/Solr 7.2) for the release
> build to succeed because of failed tests.
>
> Le mer. 21 févr. 2018 à 16:52, Erick Erickson  a
> écrit :
>
>> There's an elephant in the room, and it's that failing tests are being
>> ignored. Mind you, Solr and Lucene are progressing at a furious pace
>> with lots of great functionality being added. That said, we're
>> building up a considerable "technical debt" when it comes to testing.
>>
>> And I should say up front that major new functionality is expected to
>> take a while to shake out (e.g. autoscaling, streaming, V2 API etc.),
>> and noise from tests of new functionality is expected while things
>> bake.
>>
>> Below is a list of tests that have failed at least once since just
>> last night. This has been getting worse as time passes, the broken
>> window problem. Some e-mails have 10 failing tests (+/-) so unless I
>> go through each and every one I don't know whether something I've done
>> is causing a problem or not.
>>
>> I'm as guilty of letting things slide as anyone else, there's been a
>> long-standing issue with TestLazyCores that I work on sporadically for
>> instance that's _probably_ "something in the test framework"
>>
>> Several folks have spent some time digging into test failures and
>> identifying at least some of the causes, kudos to them. It seems
>> they're voices crying out in the wilderness though.
>>
>> There is so much noise at this point that tests are becoming
>> irrelevant. I'm trying to work on SOLR-10809 for instance, where
>> there's a pretty good possibility that I'll close at least one thing
>> that shouldn't be closed. So I ran the full suite 10 times and
>> gathered all the failures. Now I have to try to separate the failures
>> caused by that JIRA from the ones that aren't related to it so I beast
>> each of the failing tests 100 times against master. If I get a failure
>> on master too for a particular test, I'll assume it's "not my problem"
>> and drive on.
>>
>> I freely acknowledge that this is poor practice. It's driven by
>> frustration and the desire to make progress. While it's poor practice,
>> it's not as bad as only looking at tests that I _think_ are related or
>> ignoring all tests failures I can't instantly recognize as "my fault".
>>
>> So what's our stance on this? Mark Miller had a terrific program at
>> one point allowing categorization of tests that failed at a glance,
>> but it hasn't been updated in a while.  Steve Rowe is working on the
>> problem too. Hoss and Cassandra have both added to the efforts as
>> well. And I'm sure I'm leaving out others.
>>
>> Then there's the @Ignore and @BadApple annotations
>>
>> So, as a community, are we going to devote some energy to this? Or
>> shall we just start ignoring all of the frequently failing tests?
>> Frankly we'd be farther ahead at this point marking failing tests that
>> aren't getting any work with @Ignore or @BadApple and getting
>> compulsive about not letting any _new_ tests fail than continuing our
>> current path. I don't _like_ this option mind you, but it's better
>> than letting these accumulate forever and tests become more and more
>> difficult to use. As tests become more difficult to use, they're used
>> less and the problem gets worse.
>>
>> Note, I made no effort to separate suite .vs. individual reports here.
>>
>> Erick
>>
>> FAILED:  junit.framework.TestSuite.org
>> .apache.lucene.index.TestBagOfPositions
>> FAILED:  junit.framework.TestSuite.org
>> .apache.lucene.index.TestIndexWriterDeleteByQuery
>> FAILED:  junit.framework.TestSuite.org
>> .apache.lucene.store.TestSleepingLockWrapper
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.analytics.legacy.facet.LegacyFieldFacetCloudTest
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.analytics.legacy.facet.LegacyFieldFacetExtrasCloudTest
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.analytics.legacy.facet.LegacyQueryFacetCloudTest
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.client.solrj.TestLBHttpSolrClient
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.cloud.TestSolrCloudWithSecureImpersonation
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.core.AlternateDirectoryTest
>> FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
>> FAILED:  junit.framework.TestSuite.org
>> .apache.solr.handler.component.DistributedFacetPivotSmallAdvancedTest
>> FAILED:  

Re: Test failures are out of control......

[Adrien Grand]

Thanks for bringing this up Erick. I agree with you we should silence those
frequent failures. Like you said, the side-effects of not silencing them
are even worse. I'll add that these flaky tests also make releasing harder,
it took me three runs last time (Lucene/Solr 7.2) for the release build to
succeed because of failed tests.

Le mer. 21 févr. 2018 à 16:52, Erick Erickson  a
écrit :

> There's an elephant in the room, and it's that failing tests are being
> ignored. Mind you, Solr and Lucene are progressing at a furious pace
> with lots of great functionality being added. That said, we're
> building up a considerable "technical debt" when it comes to testing.
>
> And I should say up front that major new functionality is expected to
> take a while to shake out (e.g. autoscaling, streaming, V2 API etc.),
> and noise from tests of new functionality is expected while things
> bake.
>
> Below is a list of tests that have failed at least once since just
> last night. This has been getting worse as time passes, the broken
> window problem. Some e-mails have 10 failing tests (+/-) so unless I
> go through each and every one I don't know whether something I've done
> is causing a problem or not.
>
> I'm as guilty of letting things slide as anyone else, there's been a
> long-standing issue with TestLazyCores that I work on sporadically for
> instance that's _probably_ "something in the test framework"
>
> Several folks have spent some time digging into test failures and
> identifying at least some of the causes, kudos to them. It seems
> they're voices crying out in the wilderness though.
>
> There is so much noise at this point that tests are becoming
> irrelevant. I'm trying to work on SOLR-10809 for instance, where
> there's a pretty good possibility that I'll close at least one thing
> that shouldn't be closed. So I ran the full suite 10 times and
> gathered all the failures. Now I have to try to separate the failures
> caused by that JIRA from the ones that aren't related to it so I beast
> each of the failing tests 100 times against master. If I get a failure
> on master too for a particular test, I'll assume it's "not my problem"
> and drive on.
>
> I freely acknowledge that this is poor practice. It's driven by
> frustration and the desire to make progress. While it's poor practice,
> it's not as bad as only looking at tests that I _think_ are related or
> ignoring all tests failures I can't instantly recognize as "my fault".
>
> So what's our stance on this? Mark Miller had a terrific program at
> one point allowing categorization of tests that failed at a glance,
> but it hasn't been updated in a while.  Steve Rowe is working on the
> problem too. Hoss and Cassandra have both added to the efforts as
> well. And I'm sure I'm leaving out others.
>
> Then there's the @Ignore and @BadApple annotations
>
> So, as a community, are we going to devote some energy to this? Or
> shall we just start ignoring all of the frequently failing tests?
> Frankly we'd be farther ahead at this point marking failing tests that
> aren't getting any work with @Ignore or @BadApple and getting
> compulsive about not letting any _new_ tests fail than continuing our
> current path. I don't _like_ this option mind you, but it's better
> than letting these accumulate forever and tests become more and more
> difficult to use. As tests become more difficult to use, they're used
> less and the problem gets worse.
>
> Note, I made no effort to separate suite .vs. individual reports here.
>
> Erick
>
> FAILED:  junit.framework.TestSuite.org
> .apache.lucene.index.TestBagOfPositions
> FAILED:  junit.framework.TestSuite.org
> .apache.lucene.index.TestIndexWriterDeleteByQuery
> FAILED:  junit.framework.TestSuite.org
> .apache.lucene.store.TestSleepingLockWrapper
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.analytics.legacy.facet.LegacyFieldFacetCloudTest
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.analytics.legacy.facet.LegacyFieldFacetExtrasCloudTest
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.analytics.legacy.facet.LegacyQueryFacetCloudTest
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.client.solrj.TestLBHttpSolrClient
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.cloud.TestSolrCloudWithSecureImpersonation
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.core.AlternateDirectoryTest
> FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.handler.component.DistributedFacetPivotSmallAdvancedTest
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.ltr.TestSelectiveWeightCreation
> FAILED:  junit.framework.TestSuite.org
> .apache.solr.ltr.store.rest.TestModelManager
> FAILED:  junit.framework.TestSuite.org
> 

[jira] [Commented] (SOLR-7798) Improve robustness of ExpandComponent

[Michael Gibney (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371599#comment-16371599
 ] 

Michael Gibney commented on SOLR-7798:
--

Although [~joergr]'s initial description mentions an NPE in ExpandComponent "if 
_accidentally_ used without prior collapsing of results" (italics mine), there 
are applications of ExpandComponent that _intentionally_ do not involve prior 
collapsing of results on the expand field. For example, I'm using cached Join 
queries to implement tiered deduplication of the search domain across multiple 
document sources, but do not wish to deduplicate documents against other 
documents from the same source (and specifically wish to deduplicate the search 
domain, as opposed to the set of results). The approach is described in a bit 
more detail [here|https://github.com/upenn-libraries/solr-source-deduplication] 
(bullet points 3, 4, and 7 are particularly relevant).

[^expand-component.patch] looks good to me, as I can't see a reason why 
{{count}} is being tracked separately, rather than relying on 
{{ordBytes.size()}}. The only potential issue I see with it is that where 
{{count}} is used to determine whether {{groupQuery}} is initialized, {{count}} 
now represents a different concept than {{ordBytes.size()}}. I'm not sure what 
the desired behavior would be (or for that matter, what the explanation is for 
the magic "200" ceiling on {{count)}}.

I've uploaded an alternative, [^expand-npe.patch] , which differs only in that 
it leaves the separate tracking of {{count}} in place (though I don't think it 
should have to), and also in that it checks for duplication on addition of ord 
to groupBits/groupSet, thereby avoiding unnecessary {{BytesRef.deepCopyOf()}} 
in the (normally rare) case where duplicate terms are encountered.

> Improve robustness of ExpandComponent
> -
>
> Key: SOLR-7798
> URL: https://issues.apache.org/jira/browse/SOLR-7798
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Reporter: Jörg Rathlev
>Priority: Minor
> Attachments: expand-component.patch, expand-npe.patch
>
>
> The {{ExpandComponent}} causes a {{NullPointerException}} if accidentally 
> used without prior collapsing of results.
> If there are multiple documents in the result which have the same term value 
> in the expand field, the size of the {{ordBytes}}/{{groupSet}} differs from 
> the {{count}} value, and the {{getGroupQuery}} method creates an incompletely 
> filled {{bytesRef}} array, which later causes a {{NullPointerException}} when 
> trying to sort the terms.
> The attached patch extends the test to demonstrate the error, and modifies 
> the {{getGroupQuery}} methods to create the array based on the size of the 
> input maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+43) - Build # 1401 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1401/
Java: 64bit/jdk-10-ea+43 -XX:-UseCompressedOops -XX:+UseG1GC

6 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeWithPullReplicasTest

Error Message:
9 threads leaked from SUITE scope at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeWithPullReplicasTest: 1) 
Thread[id=25586, name=zkCallback-7428-thread-4, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeWithPullReplicasTest] at 
java.base@10/jdk.internal.misc.Unsafe.park(Native Method) at 
java.base@10/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
 at 
java.base@10/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
 at 
java.base@10/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
 at 
java.base@10/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
 at java.base@10/java.lang.Thread.run(Thread.java:844)2) 
Thread[id=25410, name=zkConnectionManagerCallback-7429-thread-1, state=WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeWithPullReplicasTest] at 
java.base@10/jdk.internal.misc.Unsafe.park(Native Method) at 
java.base@10/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)  
   at 
java.base@10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
 at 
java.base@10/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
 at java.base@10/java.lang.Thread.run(Thread.java:844)3) 
Thread[id=25577, name=zkCallback-7428-thread-3, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeWithPullReplicasTest] at 
java.base@10/jdk.internal.misc.Unsafe.park(Native Method) at 
java.base@10/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
 at 
java.base@10/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
 at 
java.base@10/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
 at 
java.base@10/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1060)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
 at 
java.base@10/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
 at java.base@10/java.lang.Thread.run(Thread.java:844)4) 
Thread[id=25407, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeWithPullReplicasTest] at 
java.base@10/java.lang.Thread.sleep(Native Method) at 
app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
 at java.base@10/java.lang.Thread.run(Thread.java:844)5) 
Thread[id=25409, 
name=TEST-ChaosMonkeyNothingIsSafeWithPullReplicasTest.test-seed#[C7DEEB492DC5DC66]-EventThread,
 state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeWithPullReplicasTest]
 at java.base@10/jdk.internal.misc.Unsafe.park(Native Method) at 
java.base@10/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)  
   at 
java.base@10/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2075)
 at 
java.base@10/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
 at 
app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)6) 
Thread[id=25609, name=zkCallback-7428-thread-5, state=TIMED_WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeWithPullReplicasTest] at 
java.base@10/jdk.internal.misc.Unsafe.park(Native Method) at 
java.base@10/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
 at 
java.base@10/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
 at 
java.base@10/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
 at 
java.base@10/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
 at 

Test failures are out of control......

[Erick Erickson]

There's an elephant in the room, and it's that failing tests are being
ignored. Mind you, Solr and Lucene are progressing at a furious pace
with lots of great functionality being added. That said, we're
building up a considerable "technical debt" when it comes to testing.

And I should say up front that major new functionality is expected to
take a while to shake out (e.g. autoscaling, streaming, V2 API etc.),
and noise from tests of new functionality is expected while things
bake.

Below is a list of tests that have failed at least once since just
last night. This has been getting worse as time passes, the broken
window problem. Some e-mails have 10 failing tests (+/-) so unless I
go through each and every one I don't know whether something I've done
is causing a problem or not.

I'm as guilty of letting things slide as anyone else, there's been a
long-standing issue with TestLazyCores that I work on sporadically for
instance that's _probably_ "something in the test framework"

Several folks have spent some time digging into test failures and
identifying at least some of the causes, kudos to them. It seems
they're voices crying out in the wilderness though.

There is so much noise at this point that tests are becoming
irrelevant. I'm trying to work on SOLR-10809 for instance, where
there's a pretty good possibility that I'll close at least one thing
that shouldn't be closed. So I ran the full suite 10 times and
gathered all the failures. Now I have to try to separate the failures
caused by that JIRA from the ones that aren't related to it so I beast
each of the failing tests 100 times against master. If I get a failure
on master too for a particular test, I'll assume it's "not my problem"
and drive on.

I freely acknowledge that this is poor practice. It's driven by
frustration and the desire to make progress. While it's poor practice,
it's not as bad as only looking at tests that I _think_ are related or
ignoring all tests failures I can't instantly recognize as "my fault".

So what's our stance on this? Mark Miller had a terrific program at
one point allowing categorization of tests that failed at a glance,
but it hasn't been updated in a while.  Steve Rowe is working on the
problem too. Hoss and Cassandra have both added to the efforts as
well. And I'm sure I'm leaving out others.

Then there's the @Ignore and @BadApple annotations

So, as a community, are we going to devote some energy to this? Or
shall we just start ignoring all of the frequently failing tests?
Frankly we'd be farther ahead at this point marking failing tests that
aren't getting any work with @Ignore or @BadApple and getting
compulsive about not letting any _new_ tests fail than continuing our
current path. I don't _like_ this option mind you, but it's better
than letting these accumulate forever and tests become more and more
difficult to use. As tests become more difficult to use, they're used
less and the problem gets worse.

Note, I made no effort to separate suite .vs. individual reports here.

Erick

FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestBagOfPositions
FAILED:  
junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterDeleteByQuery
FAILED:  
junit.framework.TestSuite.org.apache.lucene.store.TestSleepingLockWrapper
FAILED:  
junit.framework.TestSuite.org.apache.solr.analytics.legacy.facet.LegacyFieldFacetCloudTest
FAILED:  
junit.framework.TestSuite.org.apache.solr.analytics.legacy.facet.LegacyFieldFacetExtrasCloudTest
FAILED:  
junit.framework.TestSuite.org.apache.solr.analytics.legacy.facet.LegacyQueryFacetCloudTest
FAILED:  
junit.framework.TestSuite.org.apache.solr.client.solrj.TestLBHttpSolrClient
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest
FAILED:  junit.framework.TestSuite.org.apache.solr.core.AlternateDirectoryTest
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.component.DistributedFacetPivotSmallAdvancedTest
FAILED:  
junit.framework.TestSuite.org.apache.solr.ltr.TestSelectiveWeightCreation
FAILED:  
junit.framework.TestSuite.org.apache.solr.ltr.store.rest.TestModelManager
FAILED:  
junit.framework.TestSuite.org.apache.solr.rest.schema.analysis.TestManagedSynonymFilterFactory
FAILED:  
junit.framework.TestSuite.org.apache.solr.search.join.BlockJoinFacetDistribTest
FAILED:  
junit.framework.TestSuite.org.apache.solr.security.TestAuthorizationFramework
FAILED:  
junit.framework.TestSuite.org.apache.solr.update.processor.TestOpenNLPExtractNamedEntitiesUpdateProcessorFactory
FAILED:  org.apache.lucene.index.TestStressNRT.test
FAILED:  org.apache.solr.cloud.AddReplicaTest.test
FAILED:  org.apache.solr.cloud.DeleteShardTest.test
FAILED:  org.apache.solr.cloud.PeerSyncReplicationTest.test
FAILED:  org.apache.solr.cloud.ReplaceNodeNoTargetTest.test

[jira] [Updated] (SOLR-7798) Improve robustness of ExpandComponent

[Michael Gibney (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-7798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Gibney updated SOLR-7798:
-
Attachment: expand-npe.patch

> Improve robustness of ExpandComponent
> -
>
> Key: SOLR-7798
> URL: https://issues.apache.org/jira/browse/SOLR-7798
> Project: Solr
>  Issue Type: Improvement
>  Components: SearchComponents - other
>Reporter: Jörg Rathlev
>Priority: Minor
> Attachments: expand-component.patch, expand-npe.patch
>
>
> The {{ExpandComponent}} causes a {{NullPointerException}} if accidentally 
> used without prior collapsing of results.
> If there are multiple documents in the result which have the same term value 
> in the expand field, the size of the {{ordBytes}}/{{groupSet}} differs from 
> the {{count}} value, and the {{getGroupQuery}} method creates an incompletely 
> filled {{bytesRef}} array, which later causes a {{NullPointerException}} when 
> trying to sort the terms.
> The attached patch extends the test to demonstrate the error, and modifies 
> the {{getGroupQuery}} methods to create the array based on the size of the 
> input maps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: silent "ASF subversion and git services" bot?

[Cassandra Targett]

It looks like someone filed an issue with INFRA for another project:
https://issues.apache.org/jira/browse/INFRA-16067 (sort of hard to tell,
but that looks like the same bot?)

On Tue, Feb 20, 2018 at 11:54 AM, Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> Has anyone else also noticed or already followed up (with INFRA?) on the
> lack of "ASF subversion and git services" updates on the LUCENE and SOLR
> tickets?
>
> This one seems to be the most recent one.
>
> Christine
>
> - Original Message -
> From: dev@lucene.apache.org
> To: dev@lucene.apache.org
> At: 02/16/18 15:47:09
>
>
> [ https://issues.apache.org/jira/browse/LUCENE-8106?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel=16367489#comment-16367489 ]
>
> ASF subversion and git services commented on LUCENE-8106:
> -
>
> Commit 1a6d896dfc1bdafff3067a513bade205f6e1ad11 in lucene-solr's branch
> refs/heads/branch_7x from [~steve_rowe]
> [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1a6d896 ]
>
> LUCENE-8106: always fast-forward merge after checkout
>
>
> > Add script to attempt to reproduce failing tests from a Jenkins log
> > ---
> >
> > Key: LUCENE-8106
> > URL: https://issues.apache.org/jira/browse/LUCENE-8106
> > Project: Lucene - Core
> >  Issue Type: Improvement
> >Reporter: Steve Rowe
> >Assignee: Steve Rowe
> >Priority: Major
> > Fix For: master (8.0), 7.3
> >
> > Attachments: LUCENE-8106-part2.patch, LUCENE-8106.patch,
> LUCENE-8106.patch
> >
> >
> > This script will be runnable from a downstream job triggered by an
> upstream failing Jenkins job, passing log location info between the two.
> > The script will also be runnable manually from a developer's cmdline.
> > From the script help:
> > {noformat}
> > Usage:
> >  python3 -u reproduceJenkinsFailures.py URL
> > Must be run from a Lucene/Solr git workspace. Downloads the Jenkins
> > log pointed to by the given URL, parses it for Git revision and failed
> > Lucene/Solr tests, checks out the Git revision in the local workspace,
> > groups the failed tests by module, then runs
> > 'ant test -Dtest.dups=5 -Dtests.class="*.test1[|*.test2[...]]" ...'
> > in each module of interest, failing at the end if any of the runs fails.
> > To control the maximum number of concurrent JVMs used for each module's
> > test run, set 'tests.jvms', e.g. in ~/lucene.build.properties
> > {noformat}
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
>

[jira] [Commented] (SOLR-11934) Visit Solr logging, it's too noisy.

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371524#comment-16371524
 ] 

Gus Heck commented on SOLR-11934:
-

Forgot slf4j didn't have that level (though note it can be added in... sort 
of..: https://www.slf4j.org/faq.html#fatal). Without that, yeah error has to 
include the panic button. I tend to use log4j2 in most of my projects :), so of 
course I would not mind it here too, but yeah master only, and as much as I 
like it, such a change would probably be no minor thing and need clear a clear 
cost/benefit analysis. 

> Visit Solr logging, it's too noisy.
> ---
>
> Key: SOLR-11934
> URL: https://issues.apache.org/jira/browse/SOLR-11934
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> I think we have way too much INFO level logging. Or, perhaps more correctly, 
> Solr logging needs to be examined and messages logged at an appropriate level.
> We log every update at an INFO level for instance. But I think we log LIR at 
> INFO as well. As a sysadmin I don't care to have my logs polluted with a 
> message for every update, but if I'm trying to keep my system healthy I want 
> to see LIR messages and try to understand why.
> Plus, in large installations logging at INFO level is creating a _LOT_ of 
> files.
> What I want to discuss on this JIRA is
> 1> What kinds of messages do we want log at WARN, INFO, DEBUG, and TRACE 
> levels?
> 2> Who's the audience at each level? For a running system that's functioning, 
> sysops folks would really like WARN messages that mean something need 
> attention for instance. If I'm troubleshooting should I turn on INFO? DEBUG? 
> TRACE?
> So let's say we get some kind of agreement as to the above. Then I propose 
> three things
> 1> Someone (and probably me but all help gratefully accepted) needs to go 
> through our logging and assign appropriate levels. This will take quite a 
> while, I intend to work on it in small chunks.
> 2> Actually answer whether unnecessary objects are created when something 
> like log.info("whatever {}", someObjectOrMethodCall); is invoked. Is this 
> independent on the logging implementation used? The SLF4J and log4j seem a 
> bit contradictory.
> 3> Maybe regularize log, logger, LOG as variable names, but that's a nit.
> As a tactical approach, I suggest we tag each LoggerFactory.getLogger in 
> files we work on with //SOLR-(whatever number is assigned when I create 
> this). We can remove them all later, but since I expect to approach this 
> piecemeal it'd be nice to keep track of which files have been done already.
> Finally, I really really really don't want to do this all at once. There are 
> 5-6 thousand log messages. Even at 1,000 a week that's 6 weeks, even starting 
> now it would probably span the 7.3 release.
> This will probably be an umbrella issue so we can keep all the commits 
> straight and people can volunteer to "fix the files in core" as a separate 
> piece of work (hint).
> There are several existing JIRAs about logging in general, let's link them in 
> here as well.
> Let the discussion begin!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org