[
https://issues.apache.org/jira/browse/SOLR-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16098949#comment-16098949
]
Steve Rowe edited comment on SOLR-10372 at 7/24/17 6:32 PM:
------------------------------------------------------------
This fails every day or two on ASF & Policeman Jenkins. Here's another
reproducible one from my Jenkins:
{noformat}
Checking out Revision d5c9ed0b99d5abf56d8f30ebaf3177b4c67bd4a7
(refs/remotes/origin/branch_7_0)
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestStressCloudBlindAtomicUpdates -Dtests.method=test_dv
-Dtests.seed=1DBA6C72BF3E1122 -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=it -Dtests.timezone=America/Detroit -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] ERROR 4.73s J2 | TestStressCloudBlindAtomicUpdates.test_dv <<<
[junit4] > Throwable #1: java.util.concurrent.ExecutionException:
java.lang.RuntimeException: Error from server at
http://127.0.0.1:54596/solr/test_col: Async exception during distributed
update: Error from server at
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1: Server Error
[junit4] > request:
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54596%2Fsolr%2Ftest_col_shard2_replica_n2%2F&wt=javabin&version=2
[junit4] > Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:54596/solr/test_col_shard2_replica_n2/ update:
org.apache.solr.client.solrj.request.UpdateRequest@69f635bb
[junit4] > at
__randomizedtesting.SeedInfo.seed([1DBA6C72BF3E1122:2BAE0E3435632B33]:0)
[junit4] > at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] > at
java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: java.lang.RuntimeException: Error from server at
http://127.0.0.1:54596/solr/test_col: Async exception during distributed
update: Error from server at
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1: Server Error
[junit4] > request:
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54596%2Fsolr%2Ftest_col_shard2_replica_n2%2F&wt=javabin&version=2
[junit4] > Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:54596/solr/test_col_shard2_replica_n2/ update:
org.apache.solr.client.solrj.request.UpdateRequest@69f635bb
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
[junit4] > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] > at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > ... 1 more
[junit4] > Caused by:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://127.0.0.1:54596/solr/test_col: Async exception during
distributed update: Error from server at
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1: Server Error
[junit4] > request:
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54596%2Fsolr%2Ftest_col_shard2_replica_n2%2F&wt=javabin&version=2
[junit4] > Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:54596/solr/test_col_shard2_replica_n2/ update:
org.apache.solr.client.solrj.request.UpdateRequest@69f635bb
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:626)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
[junit4] > ... 6 more
[...]
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
{long_stored_idx=Lucene50(blocksize=128),long_dv_idx=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
long_dv_stored_idx=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))},
docValues:{_version_=DocValuesFormat(name=Lucene70),
long_dv_idx=DocValuesFormat(name=Lucene70),
long_dv_stored=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Lucene70),
long_dv=DocValuesFormat(name=Direct),
long_dv_stored_idx=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1783,
maxMBSortInHeap=7.379701091693216, sim=RandomSimilarity(queryNorm=true): {},
locale=it, timezone=America/Detroit
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
1.8.0_77 (64-bit)/cpus=16,threads=1,free=145440720,total=504889344
{noformat}
was (Author: steve_rowe):
This fails every day or two on ASF & Policeman Jenkins. Here's another
reproducible one from my Jenkins:
{noformat}
Checking out Revision d5c9ed0b99d5abf56d8f30ebaf3177b4c67bd4a7 (refs/remotes/ori
gin/branch_7_0)
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestStressCloudBlindAtomicUpdates -Dtests.method=test_dv
-Dtests.seed=1DBA6C72BF3E1122 -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=it -Dtests.timezone=America/Detroit -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] ERROR 4.73s J2 | TestStressCloudBlindAtomicUpdates.test_dv <<<
[junit4] > Throwable #1: java.util.concurrent.ExecutionException:
java.lang.RuntimeException: Error from server at
http://127.0.0.1:54596/solr/test_col: Async exception during distributed
update: Error from server at
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1: Server Error
[junit4] > request:
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54596%2Fsolr%2Ftest_col_shard2_replica_n2%2F&wt=javabin&version=2
[junit4] > Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:54596/solr/test_col_shard2_replica_n2/ update:
org.apache.solr.client.solrj.request.UpdateRequest@69f635bb
[junit4] > at
__randomizedtesting.SeedInfo.seed([1DBA6C72BF3E1122:2BAE0E3435632B33]:0)
[junit4] > at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] > at
java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: java.lang.RuntimeException: Error from server at
http://127.0.0.1:54596/solr/test_col: Async exception during distributed
update: Error from server at
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1: Server Error
[junit4] > request:
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54596%2Fsolr%2Ftest_col_shard2_replica_n2%2F&wt=javabin&version=2
[junit4] > Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:54596/solr/test_col_shard2_replica_n2/ update:
org.apache.solr.client.solrj.request.UpdateRequest@69f635bb
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
[junit4] > at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] > at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > ... 1 more
[junit4] > Caused by:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://127.0.0.1:54596/solr/test_col: Async exception during
distributed update: Error from server at
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1: Server Error
[junit4] > request:
http://127.0.0.1:41137/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A54596%2Fsolr%2Ftest_col_shard2_replica_n2%2F&wt=javabin&version=2
[junit4] > Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:54596/solr/test_col_shard2_replica_n2/ update:
org.apache.solr.client.solrj.request.UpdateRequest@69f635bb
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:626)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
[junit4] > at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
[junit4] > at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
[junit4] > ... 6 more
[...]
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70):
{long_stored_idx=Lucene50(blocksize=128),long_dv_idx=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
long_dv_stored_idx=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))},
docValues:{_version_=DocValuesFormat(name=Lucene70),
long_dv_idx=DocValuesFormat(name=Lucene70),
long_dv_stored=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Lucene70),
long_dv=DocValuesFormat(name=Direct),
long_dv_stored_idx=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1783,
maxMBSortInHeap=7.379701091693216, sim=RandomSimilarity(queryNorm=true): {},
locale=it, timezone=America/Detroit
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
1.8.0_77 (64-bit)/cpus=16,threads=1,free=145440720,total=504889344
{noformat}
> TestStressCloudBlindAtomicUpdates.test_dv() failure: Failed synchronous update
> ------------------------------------------------------------------------------
>
> Key: SOLR-10372
> URL: https://issues.apache.org/jira/browse/SOLR-10372
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Steve Rowe
>
> Policeman Jenkins found a reproducing branch_6x seed
> [https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/3157/]:
> {noformat}
> Checking out Revision 337612259a928ab61f1380470e6e0fbe4e71bca4
> (refs/remotes/origin/branch_6x)
> [...]
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestStressCloudBlindAtomicUpdates -Dtests.method=test_dv
> -Dtests.seed=72B0B386CE823E83 -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=vi-VN -Dtests.timezone=Etc/GMT-14 -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> [junit4] ERROR 0.66s J1 | TestStressCloudBlindAtomicUpdates.test_dv <<<
> [junit4] > Throwable #1: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: Error from server at
> http://127.0.0.1:34879/solr/test_col: Async exception during distributed
> update: Error from server at
> http://127.0.0.1:46166/solr/test_col_shard1_replica1: Server Error
> [junit4] > request:
> http://127.0.0.1:46166/solr/test_col_shard1_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A34879%2Fsolr%2Ftest_col_shard2_replica2%2F&wt=javabin&version=2
> [junit4] > Remote error message: Failed synchronous update on shard
> StdNode: http://127.0.0.1:35050/solr/test_col_shard1_replica2/ update:
> org.apache.solr.client.solrj.request.UpdateRequest@11acebf
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([72B0B386CE823E83:44A4D1C044DF0492]:0)
> [junit4] > at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [junit4] > at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [junit4] > at
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
> [junit4] > at
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
> [junit4] > at java.lang.Thread.run(Thread.java:745)
> [junit4] > Caused by: java.lang.RuntimeException: Error from server at
> http://127.0.0.1:34879/solr/test_col: Async exception during distributed
> update: Error from server at
> http://127.0.0.1:46166/solr/test_col_shard1_replica1: Server Error
> [junit4] > request:
> http://127.0.0.1:46166/solr/test_col_shard1_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A34879%2Fsolr%2Ftest_col_shard2_replica2%2F&wt=javabin&version=2
> [junit4] > Remote error message: Failed synchronous update on shard
> StdNode: http://127.0.0.1:35050/solr/test_col_shard1_replica2/ update:
> org.apache.solr.client.solrj.request.UpdateRequest@11acebf
> [junit4] > at
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
> [junit4] > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [junit4] > at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [junit4] > at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
> [junit4] > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [junit4] > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [junit4] > ... 1 more
> [junit4] > Caused by:
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://127.0.0.1:34879/solr/test_col: Async exception during
> distributed update: Error from server at
> http://127.0.0.1:46166/solr/test_col_shard1_replica1: Server Error
> [junit4] > request:
> http://127.0.0.1:46166/solr/test_col_shard1_replica1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A34879%2Fsolr%2Ftest_col_shard2_replica2%2F&wt=javabin&version=2
> [junit4] > Remote error message: Failed synchronous update on shard
> StdNode: http://127.0.0.1:35050/solr/test_col_shard1_replica2/ update:
> org.apache.solr.client.solrj.request.UpdateRequest@11acebf
> [junit4] > at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:610)
> [junit4] > at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
> [junit4] > at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
> [junit4] > at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
> [junit4] > at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
> [junit4] > at
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:368)
> [junit4] > at
> org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:397)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
