Build: https://builds.apache.org/job/Lucene-Solr-repro/2997/
[...truncated 29 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1786/consoleText [repro] Revision: 20de3d2ee0d234d04bbcf7c2cddc18ff67a09f8b [repro] Ant options: -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt [repro] Repro line: ant test -Dtestcase=HdfsCollectionsAPIDistributedZkTest -Dtests.method=testCollectionsAPI -Dtests.seed=A23D00C6F401CC94 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-NZ -Dtests.timezone=Asia/Dili -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=HdfsCollectionsAPIDistributedZkTest -Dtests.seed=A23D00C6F401CC94 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-NZ -Dtests.timezone=Asia/Dili -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 27df824a4b62512f64c885a3d213827a315109e9 [repro] git fetch [repro] git checkout 20de3d2ee0d234d04bbcf7c2cddc18ff67a09f8b [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro] solr/core [repro] HdfsCollectionsAPIDistributedZkTest [repro] ant compile-test [...truncated 3565 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.HdfsCollectionsAPIDistributedZkTest" -Dtests.showOutput=onerror -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.seed=A23D00C6F401CC94 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-NZ -Dtests.timezone=Asia/Dili -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 33847 lines...] [junit4] 2> 734411 ERROR (Finalizer) [ ] o.a.s.c.SolrCore REFCOUNT ERROR: unreferenced org.apache.solr.core.SolrCore@7f9da14 (awhollynewcollection_0_shard4_replica_n30) has a reference count of -1 [junit4] 2> 734457 INFO (parallelCoreAdminExecutor-1381-thread-4-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_0_shard4_replica_n24 84794f3a-f0c9-452d-ba11-b39e7006622013293759349382291 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_0 s:shard4 r:core_node27 x:awhollynewcollection_0_shard4_replica_n24] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 734457 INFO (parallelCoreAdminExecutor-1381-thread-4-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_0_shard4_replica_n24 84794f3a-f0c9-452d-ba11-b39e7006622013293759349382291 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_0 s:shard4 r:core_node27 x:awhollynewcollection_0_shard4_replica_n24] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 734457 INFO (parallelCoreAdminExecutor-1381-thread-4-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_0_shard4_replica_n24 84794f3a-f0c9-452d-ba11-b39e7006622013293759349382291 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_0 s:shard4 r:core_node27 x:awhollynewcollection_0_shard4_replica_n24] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 734480 INFO (qtp17673574-5720) [n:127.0.0.1:41782_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 7f0c2109-f29b-4239-bec2-63619b60094613293759585445216 [junit4] 2> 734480 INFO (qtp17673574-5720) [n:127.0.0.1:41782_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=7f0c2109-f29b-4239-bec2-63619b60094613293759585445216&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 734484 INFO (parallelCoreAdminExecutor-1367-thread-6-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_2_shard4_replica_n12 7f0c2109-f29b-4239-bec2-63619b60094613293759562386343 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_2 s:shard4 r:core_node18 x:awhollynewcollection_2_shard4_replica_n12] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 734484 INFO (parallelCoreAdminExecutor-1367-thread-6-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_2_shard4_replica_n12 7f0c2109-f29b-4239-bec2-63619b60094613293759562386343 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_2 s:shard4 r:core_node18 x:awhollynewcollection_2_shard4_replica_n12] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 734491 INFO (parallelCoreAdminExecutor-1372-thread-1-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_0_shard1_replica_n4 84794f3a-f0c9-452d-ba11-b39e7006622013293758827839790 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_0 s:shard1 r:core_node7 x:awhollynewcollection_0_shard1_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 734491 INFO (parallelCoreAdminExecutor-1372-thread-1-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_0_shard1_replica_n4 84794f3a-f0c9-452d-ba11-b39e7006622013293758827839790 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_0 s:shard1 r:core_node7 x:awhollynewcollection_0_shard1_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 734496 INFO (parallelCoreAdminExecutor-1381-thread-6-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_2_shard2_replica_n4 7f0c2109-f29b-4239-bec2-63619b60094613293759526686105 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_2 s:shard2 r:core_node8 x:awhollynewcollection_2_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 734496 INFO (parallelCoreAdminExecutor-1381-thread-6-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_2_shard2_replica_n4 7f0c2109-f29b-4239-bec2-63619b60094613293759526686105 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_2 s:shard2 r:core_node8 x:awhollynewcollection_2_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 734499 INFO (qtp17673574-5722) [n:127.0.0.1:41782_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 04d3a707-afe6-4268-9713-b467f42d412313293759631993147 [junit4] 2> 734499 INFO (qtp17673574-5722) [n:127.0.0.1:41782_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=04d3a707-afe6-4268-9713-b467f42d412313293759631993147&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 734500 INFO (qtp813816882-5948) [n:127.0.0.1:36729_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 84794f3a-f0c9-452d-ba11-b39e7006622013293759352783795 [junit4] 2> 734500 INFO (qtp813816882-5948) [n:127.0.0.1:36729_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=84794f3a-f0c9-452d-ba11-b39e7006622013293759352783795&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 734503 INFO (parallelCoreAdminExecutor-1381-thread-4-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_0_shard4_replica_n24 84794f3a-f0c9-452d-ba11-b39e7006622013293759349382291 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_0 s:shard4 r:core_node27 x:awhollynewcollection_0_shard4_replica_n24] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 734503 INFO (parallelCoreAdminExecutor-1381-thread-4-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_0_shard4_replica_n24 84794f3a-f0c9-452d-ba11-b39e7006622013293759349382291 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_0 s:shard4 r:core_node27 x:awhollynewcollection_0_shard4_replica_n24] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 734513 INFO (parallelCoreAdminExecutor-1372-thread-7-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard3_replica_n7 7f0c2109-f29b-4239-bec2-63619b60094613293759536827994 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard3 r:core_node13 x:awhollynewcollection_2_shard3_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 734513 INFO (parallelCoreAdminExecutor-1372-thread-7-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard3_replica_n7 7f0c2109-f29b-4239-bec2-63619b60094613293759536827994 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard3 r:core_node13 x:awhollynewcollection_2_shard3_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 734513 INFO (parallelCoreAdminExecutor-1372-thread-7-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard3_replica_n7 7f0c2109-f29b-4239-bec2-63619b60094613293759536827994 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard3 r:core_node13 x:awhollynewcollection_2_shard3_replica_n7] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 734531 INFO (parallelCoreAdminExecutor-1372-thread-7-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard3_replica_n7 7f0c2109-f29b-4239-bec2-63619b60094613293759536827994 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard3 r:core_node13 x:awhollynewcollection_2_shard3_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 734531 INFO (parallelCoreAdminExecutor-1372-thread-7-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard3_replica_n7 7f0c2109-f29b-4239-bec2-63619b60094613293759536827994 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard3 r:core_node13 x:awhollynewcollection_2_shard3_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 734537 INFO (parallelCoreAdminExecutor-1381-thread-8-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_1_shard2_replica_n12 04d3a707-afe6-4268-9713-b467f42d412313293759584802633 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_1 s:shard2 r:core_node19 x:awhollynewcollection_1_shard2_replica_n12] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 734537 INFO (parallelCoreAdminExecutor-1381-thread-8-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_1_shard2_replica_n12 04d3a707-afe6-4268-9713-b467f42d412313293759584802633 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_1 s:shard2 r:core_node19 x:awhollynewcollection_1_shard2_replica_n12] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 734537 INFO (parallelCoreAdminExecutor-1381-thread-8-processing-n:127.0.0.1:42222_solr x:awhollynewcollection_1_shard2_replica_n12 04d3a707-afe6-4268-9713-b467f42d412313293759584802633 CREATE) [n:127.0.0.1:42222_solr c:awhollynewcollection_1 s:shard2 r:core_node19 x:awhollynewcollection_1_shard2_replica_n12] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737852 INFO (parallelCoreAdminExecutor-1372-thread-11-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard4_replica_n22 04d3a707-afe6-4268-9713-b467f42d412313293759604180541 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard4 r:core_node28 x:awhollynewcollection_1_shard4_replica_n22] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 737852 INFO (parallelCoreAdminExecutor-1372-thread-11-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard4_replica_n22 04d3a707-afe6-4268-9713-b467f42d412313293759604180541 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard4 r:core_node28 x:awhollynewcollection_1_shard4_replica_n22] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 737874 INFO (parallelCoreAdminExecutor-1372-thread-5-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard1_replica_n1 7f0c2109-f29b-4239-bec2-63619b60094613293759518554468 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard1 r:core_node3 x:awhollynewcollection_2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737874 INFO (parallelCoreAdminExecutor-1372-thread-5-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard1_replica_n1 7f0c2109-f29b-4239-bec2-63619b60094613293759518554468 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard1 r:core_node3 x:awhollynewcollection_2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737874 INFO (parallelCoreAdminExecutor-1372-thread-5-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard1_replica_n1 7f0c2109-f29b-4239-bec2-63619b60094613293759518554468 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard1 r:core_node3 x:awhollynewcollection_2_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737877 INFO (parallelCoreAdminExecutor-1372-thread-10-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard3_replica_n15 04d3a707-afe6-4268-9713-b467f42d412313293759589286959 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard3 r:core_node21 x:awhollynewcollection_1_shard3_replica_n15] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737877 INFO (parallelCoreAdminExecutor-1372-thread-10-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard3_replica_n15 04d3a707-afe6-4268-9713-b467f42d412313293759589286959 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard3 r:core_node21 x:awhollynewcollection_1_shard3_replica_n15] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737877 INFO (parallelCoreAdminExecutor-1372-thread-10-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard3_replica_n15 04d3a707-afe6-4268-9713-b467f42d412313293759589286959 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard3 r:core_node21 x:awhollynewcollection_1_shard3_replica_n15] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737879 INFO (parallelCoreAdminExecutor-1363-thread-9-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_1_shard2_replica_n9 04d3a707-afe6-4268-9713-b467f42d412313293759550787195 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_1 s:shard2 r:core_node14 x:awhollynewcollection_1_shard2_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737879 INFO (parallelCoreAdminExecutor-1363-thread-9-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_1_shard2_replica_n9 04d3a707-afe6-4268-9713-b467f42d412313293759550787195 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_1 s:shard2 r:core_node14 x:awhollynewcollection_1_shard2_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737879 INFO (parallelCoreAdminExecutor-1363-thread-9-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_1_shard2_replica_n9 04d3a707-afe6-4268-9713-b467f42d412313293759550787195 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_1 s:shard2 r:core_node14 x:awhollynewcollection_1_shard2_replica_n9] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737885 INFO (parallelCoreAdminExecutor-1367-thread-3-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard3_replica_n22 84794f3a-f0c9-452d-ba11-b39e7006622013293759344822401 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard3 r:core_node25 x:awhollynewcollection_0_shard3_replica_n22] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737885 INFO (parallelCoreAdminExecutor-1367-thread-3-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard3_replica_n22 84794f3a-f0c9-452d-ba11-b39e7006622013293759344822401 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard3 r:core_node25 x:awhollynewcollection_0_shard3_replica_n22] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737885 INFO (parallelCoreAdminExecutor-1367-thread-3-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard3_replica_n22 84794f3a-f0c9-452d-ba11-b39e7006622013293759344822401 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard3 r:core_node25 x:awhollynewcollection_0_shard3_replica_n22] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737889 INFO (parallelCoreAdminExecutor-1372-thread-8-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard1_replica_n1 04d3a707-afe6-4268-9713-b467f42d412313293759518979274 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard1 r:core_node3 x:awhollynewcollection_1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737889 INFO (parallelCoreAdminExecutor-1372-thread-8-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard1_replica_n1 04d3a707-afe6-4268-9713-b467f42d412313293759518979274 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard1 r:core_node3 x:awhollynewcollection_1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737889 INFO (parallelCoreAdminExecutor-1372-thread-8-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_1_shard1_replica_n1 04d3a707-afe6-4268-9713-b467f42d412313293759518979274 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_1 s:shard1 r:core_node3 x:awhollynewcollection_1_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737896 INFO (parallelCoreAdminExecutor-1363-thread-1-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_0_shard1_replica_n2 84794f3a-f0c9-452d-ba11-b39e7006622013293758824668761 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_0 s:shard1 r:core_node5 x:awhollynewcollection_0_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737896 INFO (parallelCoreAdminExecutor-1363-thread-1-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_0_shard1_replica_n2 84794f3a-f0c9-452d-ba11-b39e7006622013293758824668761 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_0 s:shard1 r:core_node5 x:awhollynewcollection_0_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737896 INFO (parallelCoreAdminExecutor-1363-thread-1-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_0_shard1_replica_n2 84794f3a-f0c9-452d-ba11-b39e7006622013293758824668761 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_0 s:shard1 r:core_node5 x:awhollynewcollection_0_shard1_replica_n2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737900 INFO (parallelCoreAdminExecutor-1363-thread-7-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_2_shard3_replica_n9 7f0c2109-f29b-4239-bec2-63619b60094613293759540903707 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_2 s:shard3 r:core_node15 x:awhollynewcollection_2_shard3_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737900 INFO (parallelCoreAdminExecutor-1363-thread-7-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_2_shard3_replica_n9 7f0c2109-f29b-4239-bec2-63619b60094613293759540903707 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_2 s:shard3 r:core_node15 x:awhollynewcollection_2_shard3_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737900 INFO (parallelCoreAdminExecutor-1363-thread-7-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_2_shard3_replica_n9 7f0c2109-f29b-4239-bec2-63619b60094613293759540903707 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_2 s:shard3 r:core_node15 x:awhollynewcollection_2_shard3_replica_n9] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737907 INFO (parallelCoreAdminExecutor-1367-thread-2-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard2_replica_n14 84794f3a-f0c9-452d-ba11-b39e7006622013293759327757475 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard2 r:core_node17 x:awhollynewcollection_0_shard2_replica_n14] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 737907 INFO (parallelCoreAdminExecutor-1367-thread-2-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard2_replica_n14 84794f3a-f0c9-452d-ba11-b39e7006622013293759327757475 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard2 r:core_node17 x:awhollynewcollection_0_shard2_replica_n14] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 737907 INFO (parallelCoreAdminExecutor-1367-thread-2-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard2_replica_n14 84794f3a-f0c9-452d-ba11-b39e7006622013293759327757475 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard2 r:core_node17 x:awhollynewcollection_0_shard2_replica_n14] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3 [junit4] 2> 737932 INFO (qtp813816882-5739) [n:127.0.0.1:36729_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 84794f3a-f0c9-452d-ba11-b39e7006622013293759352783795 [junit4] 2> 737932 INFO (qtp813816882-5739) [n:127.0.0.1:36729_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=84794f3a-f0c9-452d-ba11-b39e7006622013293759352783795&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 737933 INFO (qtp17673574-5724) [n:127.0.0.1:41782_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 04d3a707-afe6-4268-9713-b467f42d412313293759631993147 [junit4] 2> 737933 INFO (qtp17673574-5724) [n:127.0.0.1:41782_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=04d3a707-afe6-4268-9713-b467f42d412313293759631993147&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 737934 INFO (qtp17673574-5724) [n:127.0.0.1:41782_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 7f0c2109-f29b-4239-bec2-63619b60094613293759585445216 [junit4] 2> 737934 INFO (qtp17673574-5724) [n:127.0.0.1:41782_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=7f0c2109-f29b-4239-bec2-63619b60094613293759585445216&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 741271 INFO (parallelCoreAdminExecutor-1372-thread-5-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard1_replica_n1 7f0c2109-f29b-4239-bec2-63619b60094613293759518554468 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard1 r:core_node3 x:awhollynewcollection_2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 741271 INFO (parallelCoreAdminExecutor-1372-thread-5-processing-n:127.0.0.1:41782_solr x:awhollynewcollection_2_shard1_replica_n1 7f0c2109-f29b-4239-bec2-63619b60094613293759518554468 CREATE) [n:127.0.0.1:41782_solr c:awhollynewcollection_2 s:shard1 r:core_node3 x:awhollynewcollection_2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 741314 INFO (parallelCoreAdminExecutor-1367-thread-3-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard3_replica_n22 84794f3a-f0c9-452d-ba11-b39e7006622013293759344822401 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard3 r:core_node25 x:awhollynewcollection_0_shard3_replica_n22] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 741314 INFO (parallelCoreAdminExecutor-1367-thread-3-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_0_shard3_replica_n22 84794f3a-f0c9-452d-ba11-b39e7006622013293759344822401 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_0 s:shard3 r:core_node25 x:awhollynewcollection_0_shard3_replica_n22] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 830802 INFO (parallelCoreAdminExecutor-1363-thread-9-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_1_shard2_replica_n9 04d3a707-afe6-4268-9713-b467f42d412313293759550787195 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_1 s:shard2 r:core_node14 x:awhollynewcollection_1_shard2_replica_n9] o.a.s.c.SolrCore [awhollynewcollection_1_shard2_replica_n9] CLOSING SolrCore org.apache.solr.core.SolrCore@3cd8845b [junit4] 2> 832340 WARN (jetty-launcher-2130-thread-3-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 94410ms for sessionid 0x103185e9eb8000b [junit4] 2> 832341 WARN (jetty-launcher-2130-thread-1-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 94412ms for sessionid 0x103185e9eb80011 [junit4] 2> 832341 WARN (jetty-launcher-2130-thread-4-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 94407ms for sessionid 0x103185e9eb8000e [junit4] 2> 832341 ERROR (OverseerThreadFactory-1362-thread-3-processing-n:127.0.0.1:36729_solr) [n:127.0.0.1:36729_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:41782/solr [junit4] 2> => org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41782/solr/admin/cores [junit4] 2> at org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:406) [junit4] 2> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41782/solr/admin/cores [junit4] 2> at org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:406) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:739) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1274) ~[java/:?] [junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.request(HttpShardHandler.java:227) ~[java/:?] [junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:188) ~[java/:?] [junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_191] [junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6] [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> Caused by: java.util.concurrent.TimeoutException [junit4] 2> at org.eclipse.jetty.client.util.InputStreamResponseListener.get(InputStreamResponseListener.java:216) ~[jetty-client-9.4.14.v20181114.jar:9.4.14.v20181114] [junit4] 2> at org.apache.solr.client.solrj.impl.Http2SolrClient.request(Http2SolrClient.java:397) ~[java/:?] [junit4] 2> ... 12 more [junit4] 2> 832343 INFO (qtp813816882-5925) [n:127.0.0.1:36729_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 84794f3a-f0c9-452d-ba11-b39e7006622013293759352783795 [junit4] 2> 832343 INFO (qtp17673574-5926) [n:127.0.0.1:41782_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 04d3a707-afe6-4268-9713-b467f42d412313293759631993147 [junit4] 2> 832343 INFO (qtp17673574-5721) [n:127.0.0.1:41782_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 7f0c2109-f29b-4239-bec2-63619b60094613293759585445216 [junit4] 2> 832353 WARN (jetty-launcher-2130-thread-4-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 98909ms for sessionid 0x103185e9eb8000f [junit4] 2> 832353 WARN (parallelCoreAdminExecutor-1367-thread-11-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_1_shard5_replica_n36 04d3a707-afe6-4268-9713-b467f42d412313293759650720921 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_1 s:shard5 r:core_node38 x:awhollynewcollection_1_shard5_replica_n36] o.a.h.h.c.i.BlockReaderFactory I/O error constructing remote block reader. [junit4] 2> => java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:55234 remote=/127.0.0.1:36323] [junit4] 2> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164) [junit4] 2> java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:55234 remote=/127.0.0.1:36323] [junit4] 2> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at java.io.FilterInputStream.read(FilterInputStream.java:83) ~[?:1.8.0_191] [junit4] 2> at org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:547) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderRemote.newBlockReader(BlockReaderRemote.java:407) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.getRemoteBlockReader(BlockReaderFactory.java:853) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:749) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.build(BlockReaderFactory.java:379) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:644) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.actualGetFromOneDataNode(DFSInputStream.java:1049) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.fetchBlockByteRange(DFSInputStream.java:1001) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1360) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1324) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:121) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:111) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.solr.store.hdfs.HdfsDirectory$HdfsIndexInput.readInternal(HdfsDirectory.java:220) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.refill(CustomBufferedIndexInput.java:191) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.readBytes(CustomBufferedIndexInput.java:93) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.readBytes(CustomBufferedIndexInput.java:67) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.BlockDirectory$CachedIndexInput.readIntoCacheAndResult(BlockDirectory.java:221) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.BlockDirectory$CachedIndexInput.fetchBlock(BlockDirectory.java:208) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.BlockDirectory$CachedIndexInput.readInternal(BlockDirectory.java:192) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.refill(CustomBufferedIndexInput.java:191) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.readByte(CustomBufferedIndexInput.java:45) ~[java/:?] [junit4] 2> at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41) ~[java/:?] [junit4] 2> at org.apache.lucene.store.DataInput.readInt(DataInput.java:101) ~[java/:?] [junit4] 2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:301) ~[java/:?] [junit4] 2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289) ~[java/:?] [junit4] 2> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:846) ~[java/:?] [junit4] 2> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:124) ~[java/:?] [junit4] 2> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:97) ~[java/:?] [junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:259) ~[java/:?] [junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:133) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2113) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2272) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1115) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1192) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1102) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) ~[java/:?] [junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6] [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 832354 WARN (parallelCoreAdminExecutor-1367-thread-11-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_1_shard5_replica_n36 04d3a707-afe6-4268-9713-b467f42d412313293759650720921 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_1 s:shard5 r:core_node38 x:awhollynewcollection_1_shard5_replica_n36] o.a.h.h.DFSClient Connection failure: Failed to connect to /127.0.0.1:36323 for file /solr_hdfs_home/awhollynewcollection_1/core_node38/data/index/segments_1 for block BP-1665621557-10.41.0.5-1552158982407:blk_1073741864_1040:java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:55234 remote=/127.0.0.1:36323] [junit4] 2> => java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:55234 remote=/127.0.0.1:36323] [junit4] 2> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164) [junit4] 2> java.net.SocketTimeoutException: 60000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=localhost/127.0.0.1:55234 remote=/127.0.0.1:36323] [junit4] 2> at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at java.io.FilterInputStream.read(FilterInputStream.java:83) ~[?:1.8.0_191] [junit4] 2> at org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:547) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderRemote.newBlockReader(BlockReaderRemote.java:407) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.getRemoteBlockReader(BlockReaderFactory.java:853) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:749) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.client.impl.BlockReaderFactory.build(BlockReaderFactory.java:379) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:644) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.actualGetFromOneDataNode(DFSInputStream.java:1049) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.fetchBlockByteRange(DFSInputStream.java:1001) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1360) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1324) ~[hadoop-hdfs-client-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:121) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:111) ~[hadoop-common-3.2.0.jar:?] [junit4] 2> at org.apache.solr.store.hdfs.HdfsDirectory$HdfsIndexInput.readInternal(HdfsDirectory.java:220) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.refill(CustomBufferedIndexInput.java:191) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.readBytes(CustomBufferedIndexInput.java:93) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.readBytes(CustomBufferedIndexInput.java:67) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.BlockDirectory$CachedIndexInput.readIntoCacheAndResult(BlockDirectory.java:221) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.BlockDirectory$CachedIndexInput.fetchBlock(BlockDirectory.java:208) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.BlockDirectory$CachedIndexInput.readInternal(BlockDirectory.java:192) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.refill(CustomBufferedIndexInput.java:191) ~[java/:?] [junit4] 2> at org.apache.solr.store.blockcache.CustomBufferedIndexInput.readByte(CustomBufferedIndexInput.java:45) ~[java/:?] [junit4] 2> at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41) ~[java/:?] [junit4] 2> at org.apache.lucene.store.DataInput.readInt(DataInput.java:101) ~[java/:?] [junit4] 2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:301) ~[java/:?] [junit4] 2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289) ~[java/:?] [junit4] 2> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:846) ~[java/:?] [junit4] 2> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:124) ~[java/:?] [junit4] 2> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:97) ~[java/:?] [junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:259) ~[java/:?] [junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:133) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2113) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2272) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1115) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002) ~[java/:?] [junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1192) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1102) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) ~[java/:?] [junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6] [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 832354 WARN (jetty-launcher-2130-thread-2-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 91080ms for sessionid 0x103185e9eb80008 [junit4] 2> 832354 WARN (parallelCoreAdminExecutor-1367-thread-11-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_1_shard5_replica_n36 04d3a707-afe6-4268-9713-b467f42d412313293759650720921 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_1 s:shard5 r:core_node38 x:awhollynewcollection_1_shard5_replica_n36] o.a.h.h.DFSClient No live nodes contain block BP-1665621557-10.41.0.5-1552158982407:blk_1073741864_1040 after checking nodes = [DatanodeInfoWithStorage[127.0.0.1:36323,DS-8f247f7a-8238-4030-8aeb-c72f50bfe74c,DISK]], ignoredNodes = null [junit4] 2> 832354 WARN (parallelCoreAdminExecutor-1367-thread-11-processing-n:127.0.0.1:46218_solr x:awhollynewcollection_1_shard5_replica_n36 04d3a707-afe6-4268-9713-b467f42d412313293759650720921 CREATE) [n:127.0.0.1:46218_solr c:awhollynewcollection_1 s:shard5 r:core_node38 x:awhollynewcollection_1_shard5_replica_n36] o.a.h.h.DFSClient DFS chooseDataNode: got # 1 IOException, will wait for 1983.8318348438095 msec. [junit4] 2> AsyncLogger error handling event seq=12695, value='Logger=org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler Level=ERROR Message=Error from shard: http://127.0.0.1:36729/solr': java.lang.OutOfMemoryError: GC overhead limit exceeded [junit4] 2> java.lang.OutOfMemoryError: GC overhead limit exceeded [junit4] 2> 832354 ERROR (OverseerThreadFactory-1362-thread-1-processing-n:127.0.0.1:36729_solr) [n:127.0.0.1:36729_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error from shard 127.0.0.1:36729_solr: {STATUS=failed} [junit4] 2> 832355 WARN (jetty-launcher-2130-thread-2-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 34589ms for sessionid 0x103185e9eb80009 [junit4] 2> 832355 WARN (TEST-HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[A23D00C6F401CC94]-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 30474ms for sessionid 0x103185e9eb80000 [junit4] 2> 832355 WARN (TEST-HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[A23D00C6F401CC94]-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 30473ms for sessionid 0x103185e9eb80001 [junit4] 2> 835669 WARN (zkConnectionManagerCallback-2127-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8406173 name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None [junit4] 2> 835669 WARN (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a43faf2 name: ZooKeeperConnection Watcher:127.0.0.1:44351 got event WatchedEvent state:Disconnected type:None path:null path: null type: None [junit4] 2> 835669 WARN (zkConnectionManagerCallback-2127-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected [junit4] 2> 835669 WARN (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected [junit4] 2> 835678 WARN (zkConnectionManagerCallback-2170-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10571384 name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None [junit4] 2> 835678 WARN (zkConnectionManagerCallback-2170-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected [junit4] 2> 839002 INFO (qtp17673574-5723) [n:127.0.0.1:41782_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 84794f3a-f0c9-452d-ba11-b39e7006622013293759365823486 [junit4] 2> 839002 INFO (qtp17673574-5723) [n:127.0.0.1:41782_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=84794f3a-f0c9-452d-ba11-b39e7006622013293759365823486&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 839009 INFO (qtp17673574-5721) [n:127.0.0.1:41782_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=7f0c2109-f29b-4239-bec2-63619b60094613293759585445216&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6665 [junit4] 2> 839010 INFO (qtp17673574-5926) [n:127.0.0.1:41782_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=04d3a707-afe6-4268-9713-b467f42d412313293759631993147&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6667 [junit4] 2> 839010 INFO (qtp813816882-5925) [n:127.0.0.1:36729_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=84794f3a-f0c9-452d-ba11-b39e7006622013293759352783795&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6667 [junit4] 2> 839010 ERROR (OverseerThreadFactory-1362-thread-3-processing-n:127.0.0.1:36729_solr) [n:127.0.0.1:36729_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error from shard 127.0.0.1:41782_solr: {STATUS=failed} [junit4] 2> 841828 WARN (jetty-launcher-2130-thread-2-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x103185e9eb80009 has expired [junit4] 2> 841828 WARN (jetty-launcher-2130-thread-4-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x103185e9eb8000f has expired [junit4] 2> 841830 WARN (jetty-launcher-2130-thread-2-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x103185e9eb80008 has expired [junit4] 2> 841830 WARN (jetty-launcher-2130-thread-3-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x103185e9eb8000c has expired [junit4] 2> 841830 WARN (jetty-launcher-2130-thread-1-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x103185e9eb80012 has expired [junit4] 2> 841830 WARN (TEST-HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[A23D00C6F401CC94]-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x103185e9eb80000 has expired [junit4] 2> 841830 WARN (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4a43faf2 name: ZooKeeperConnection Watcher:127.0.0.1:44351 got event WatchedEvent state:Expired type:None path:null path: null type: None [junit4] 2> 841831 WARN (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper... [junit4] 2> 841831 WARN (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one... [junit4] 2> 841831 WARN (TEST-HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[A23D00C6F401CC94]-SendThread(localhost:44351)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x103185e9eb80001 has expired [junit4] 2> 841831 WARN (zkConnectionManagerCallback-2127-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8406173 name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Expired type:None path:null path: null type: None [junit4] 2> 841831 WARN (zkConnectionManagerCallback-2127-thread-1) [ ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper... [junit4] 2> 841831 WARN (zkConnectionManagerCallback-2127-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one... [junit4] 2> 841841 WARN (zkConnectionManagerCallback-2170-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10571384 name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Expired type:None path:null path: null type: None [junit4] 2> 841841 WARN (zkConnectionManagerCallback-2170-thread-1) [ ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper... [junit4] 2> 841841 WARN (zkConnectionManagerCallback-2170-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one... [junit4] 2> Mar 10, 2019 4:21:49 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException [junit4] 2> WARNING: Uncaught exception in thread: Thread[parallelCoreAdminExecutor-1381-thread-2,5,TGRP-HdfsCollectionsAPIDistributedZkTest] [junit4] 2> java.lang.OutOfMemoryError: GC overhead limit exceeded [junit4] 2> [junit4] 2> Mar 10, 2019 4:21:49 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException [junit4] 2> WARNING: Uncaught exception in thread: Thread[parallelCoreAdminExecutor-1367-thread-2,5,TGRP-HdfsCollectionsAPIDistributedZkTest] [junit4] 2> java.lang.OutOfMemoryError: GC overhead limit exceeded [junit4] 2> [junit4] 2> Mar 10, 2019 4:21:49 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException [junit4] 2> WARNING: Uncaught exception in thread: Thread[OverseerThreadFactory-1362-thread-1,5,Overseer collection creation process.] [junit4] 2> java.lang.OutOfMemoryError: GC overhead limit exceeded [junit4] 2> [junit4] 2> Mar 10, 2019 4:21:49 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException [junit4] 2> WARNING: Uncaught exception in thread: Thread[parallelCoreAdminExecutor-1372-thread-2,5,TGRP-HdfsCollectionsAPIDistributedZkTest] [junit4] 2> java.lang.OutOfMemoryError: GC overhead limit exceeded [junit4] 2> [junit4] 2> 841928 WARN (MetricsHistoryHandler-1369-thread-1) [ ] o.a.s.h.a.MetricsHistoryHandler Could not obtain overseer's address, skipping. [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getData(ZkDistribStateManager.java:87) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.cloud.DistribStateManager.getData(DistribStateManager.java:55) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.getOverseerLeader(MetricsHistoryHandler.java:317) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:352) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:347) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:443) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:371) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:231) ~[java/:?] [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 841928 WARN (zkConnectionManagerCallback-2194-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@525d8c74 name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None [junit4] 2> 841928 WARN (zkConnectionManagerCallback-2194-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected [junit4] 2> 841929 INFO (jetty-closer-2254-thread-4) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1129119647 [junit4] 2> 841929 INFO (jetty-closer-2254-thread-4) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42222_solr [junit4] 2> 841929 INFO (parallelCoreAdminExecutor-1363-thread-9-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_1_shard2_replica_n9 04d3a707-afe6-4268-9713-b467f42d412313293759550787195 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_1 s:shard2 r:core_node14 x:awhollynewcollection_1_shard2_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.awhollynewcollection_1.shard2.replica_n9, tag=3cd8845b [junit4] 2> 841929 WARN (MetricsHistoryHandler-1376-thread-1) [ ] o.a.s.h.a.MetricsHistoryHandler Could not obtain overseer's address, skipping. [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getData(ZkDistribStateManager.java:87) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.cloud.DistribStateManager.getData(DistribStateManager.java:55) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.getOverseerLeader(MetricsHistoryHandler.java:317) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:352) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:347) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:443) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:371) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:231) ~[java/:?] [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 841929 WARN (zkConnectionManagerCallback-2177-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4306b761 name: ZooKeeperConnection Watcher:127.0.0.1:44351/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None [junit4] 2> 841929 WARN (zkConnectionManagerCallback-2177-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected [junit4] 2> 841930 INFO (parallelCoreAdminExecutor-1363-thread-9-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_1_shard2_replica_n9 04d3a707-afe6-4268-9713-b467f42d412313293759550787195 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_1 s:shard2 r:core_node14 x:awhollynewcollection_1_shard2_replica_n9] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6673191d: rootName = null, domain = solr.core.awhollynewcollection_1.shard2.replica_n9, service url = null, agent id = null] for registry solr.core.awhollynewcollection_1.shard2.replica_n9 / com.codahale.metrics.MetricRegistry@168f7ff9 [junit4] 2> 841930 INFO (OverseerStateUpdate-72928813636911112-127.0.0.1:36729_solr-n_0000000000) [n:127.0.0.1:36729_solr ] o.a.s.c.Overseer According to ZK I (id=72928813636911112-127.0.0.1:36729_solr-n_0000000000) am no longer a leader. propsId=null [junit4] 2> 841930 INFO (OverseerStateUpdate-72928813636911112-127.0.0.1:36729_solr-n_0000000000) [n:127.0.0.1:36729_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36729_solr [junit4] 2> 841930 INFO (OverseerCollectionConfigSetProcessor-72928813636911112-127.0.0.1:36729_solr-n_0000000000) [n:127.0.0.1:36729_solr ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=72928813636911112-127.0.0.1:36729_solr-n_0000000000) am no longer a leader. propsId=null [junit4] 2> 841930 ERROR (parallelCoreAdminExecutor-1363-thread-2-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_0_shard2_replica_n10 84794f3a-f0c9-452d-ba11-b39e7006622013293758836224621 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_0 s:shard2 r:core_node13 x:awhollynewcollection_0_shard2_replica_n10] o.a.s.c.ZkContainer [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn- [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn- [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueue.java:326) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1573) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1486) ~[java/:?] [junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:202) ~[java/:?] [junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:222) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1040) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1202) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1102) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) ~[java/:?] [junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6] [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 841931 ERROR (parallelCoreAdminExecutor-1363-thread-2-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_0_shard2_replica_n10 84794f3a-f0c9-452d-ba11-b39e7006622013293758836224621 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_0 s:shard2 r:core_node13 x:awhollynewcollection_0_shard2_replica_n10] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Could not load collection from ZK: awhollynewcollection_0 [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1366) [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:748) [junit4] 2> at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148) [junit4] 2> at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131) [junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:639) [junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:341) [junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:171) [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:136) [junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:308) [junit4] 2> at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1440) [junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1182) [junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1134) [junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:190) [junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:222) [junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1040) [junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1202) [junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1102) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) [junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [junit4] 2> at java.lang.Thread.run(Thread.java:748) [junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awhollynewcollection_0/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1378) [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1364) [junit4] 2> ... 25 more [junit4] 2> [junit4] 2> 841932 WARN (MetricsHistoryHandler-1383-thread-1) [ ] o.a.s.h.a.MetricsHistoryHandler Could not obtain overseer's address, skipping. [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getData(ZkDistribStateManager.java:87) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.cloud.DistribStateManager.getData(DistribStateManager.java:55) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.getOverseerLeader(MetricsHistoryHandler.java:317) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:352) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:347) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:443) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:371) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:231) ~[java/:?] [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 841932 WARN (MetricsHistoryHandler-1371-thread-1) [ ] o.a.s.h.a.MetricsHistoryHandler Could not obtain overseer's address, skipping. [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getData(ZkDistribStateManager.java:87) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.cloud.DistribStateManager.getData(DistribStateManager.java:55) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.getOverseerLeader(MetricsHistoryHandler.java:317) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:352) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.amIOverseerLeader(MetricsHistoryHandler.java:347) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:443) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:371) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:231) ~[java/:?] [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 841933 INFO (zkConnectionManagerCallback-2125-thread-1-EventThread) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 841933 INFO (jetty-closer-2254-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1724671618 [junit4] 2> 841933 INFO (jetty-closer-2254-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41782_solr [junit4] 2> 841937 ERROR (parallelCoreAdminExecutor-1363-thread-5-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_2_shard1_replica_n2 7f0c2109-f29b-4239-bec2-63619b60094613293759523043461 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_2 s:shard1 r:core_node5 x:awhollynewcollection_2_shard1_replica_n2] o.a.s.c.ZkContainer [junit4] 2> => org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn- [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn- [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) ~[java/:?] [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:415) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueue.java:326) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1573) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1486) ~[java/:?] [junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:202) ~[java/:?] [junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:222) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1040) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1202) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1102) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) ~[java/:?] [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) ~[java/:?] [junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6] [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> 841937 ERROR (parallelCoreAdminExecutor-1363-thread-5-processing-n:127.0.0.1:36729_solr x:awhollynewcollection_2_shard1_replica_n2 7f0c2109-f29b-4239-bec2-63619b60094613293759523043461 CREATE) [n:127.0.0.1:36729_solr c:awhollynewcollection_2 s:shard1 r:core_node5 x:awhollynewcollection_2_shard1_replica_n2] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Could not load collection from ZK: awhollynewcollection_2 [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1366) [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:748) [junit4] 2> at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148) [junit4] 2> at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131) [junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.waitForReplicasToComeUp(ElectionContext.java:639) [junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:341) [junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:171) [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:136) [junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:308) [junit4] 2> at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1440) [junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1182) [junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1134) [junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:190) [junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:222) [junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1040) [junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1202) [junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1102) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) [junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) [junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [junit4] 2> at java.lang.Thread.run(Thread.java:748) [junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awhollynewcollection_2/state.json [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1378) [junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1364) [junit4] 2> ... 25 more [junit4] 2> [junit4] 2> 841938 INFO (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished. [junit4] 2> 841938 INFO (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper [junit4] 2> 841938 INFO (zkConnectionManagerCallback-2125-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient Connected:true [junit4] 2> 841938 INFO (jetty-closer-2254-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1625202163 [junit4] 2> 841938 INFO (jetty-closer-2254-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36729_solr [junit4] 2> 841938 INFO (jetty-closer-2254-thread-2) [ ] o.a.s.c.CoreContainer Publish this node as DOWN... [junit4] 2> 841938 INFO (jetty-closer-2254-thread-2) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:36729_solr as DOWN [junit4] 2> 841938 WARN (jetty-closer-2254-thread-2) [ ] o.a.s.c.CoreContainer Error publishing nodes as down. Continuing to close CoreContainer [junit4] 2> => org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue [junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) [junit4] 2> org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue [junit4] 2> at org.apache.solr.cloud.ZkDistributedQueue.<init>(ZkDistributedQueue.java:129) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:690) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:678) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.ZkController.publishNodeAsDown(ZkController.java:2599) ~[java/:?] [junit4] 2> at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:862) ~[java/:?] [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.close(SolrDispatchFilter.java:333) ~[java/:?] [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:623) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.MiniSolrCloudCluster.stopJettySolrRunner(MiniSolrCloudCluster.java:512) ~[java/:?] [junit4] 2> at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$shutdown$5(MiniSolrCloudCluster.java:606) ~[java/:?] [junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_191] [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] [junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] [junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130 [...truncated too long message...] e/Lucene-Solr-repro/solr/solrj/lib/jetty-http-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/solrj/lib/jetty-io-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/solrj/lib/jetty-util-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/solrj/lib/noggit-0.8.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/solrj/lib/slf4j-api-1.7.24.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/solrj/lib/stax2-api-3.1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/solrj/lib/zookeeper-3.4.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/gmetric4j-1.0.7.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/http2-common-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/http2-hpack-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/http2-server-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/javax.servlet-api-3.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-alpn-java-server-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-alpn-server-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-continuation-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-deploy-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-http-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-io-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-jmx-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-rewrite-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-security-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-server-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-servlet-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-servlets-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-util-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-webapp-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/jetty-xml-9.4.14.v20181114.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/metrics-core-3.2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/metrics-ganglia-3.2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/metrics-graphite-3.2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/metrics-jetty9-3.2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/server/lib/metrics-jvm-3.2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/example/example-DIH/solr/db/lib/hsqldb-2.4.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/test-framework/lib/hamcrest-core-1.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/test-framework/lib/junit-4.12.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/test-framework/lib/randomizedtesting-runner-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/antlr4-runtime-4.5.1-1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/byte-buddy-1.9.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/commons-logging-1.1.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/commons-text-1.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/hadoop-common-3.2.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/hadoop-hdfs-3.2.0-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/hadoop-hdfs-3.2.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/hadoop-minicluster-3.2.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/hadoop-minikdc-3.2.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/jersey-core-1.19.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/jersey-server-1.19.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/jersey-servlet-1.19.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerb-admin-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerb-client-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerb-common-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerb-crypto-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerb-identity-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerb-server-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerb-simplekdc-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerby-config-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerby-kdc-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/kerby-util-1.0.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/mockito-core-2.23.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/netty-all-4.0.52.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/core/test-lib/objenesis-2.6.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/build/analysis/icu/lucene-analyzers-icu-9.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/contrib/analysis-extras/lib/icu4j-62.1.jar:/usr/share/ant/lib/ant-launcher.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/xercesImpl.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/usr/share/ant/lib/ant-junit.jar:/usr/share/ant/lib/ant-apache-log4j.jar:/usr/share/ant/lib/ant-junit4.jar:/usr/share/ant/lib/ant-javamail.jar:/usr/share/ant/lib/ant-apache-bsf.jar:/usr/share/ant/lib/ant-commons-net.jar:/usr/share/ant/lib/ant-antlr.jar:/usr/share/ant/lib/ant-jsch.jar:/usr/share/ant/lib/ant-apache-oro.jar:/usr/share/ant/lib/ant-commons-logging.jar:/usr/share/ant/lib/ant.jar:/usr/share/ant/lib/ant-testutil.jar:/usr/share/ant/lib/ant-jdepend.jar:/usr/share/ant/lib/ant-apache-bcel.jar:/usr/share/ant/lib/ant-apache-xalan2.jar:/usr/share/ant/lib/ant-apache-resolver.jar:/usr/share/ant/lib/ant-jmf.jar:/usr/share/ant/lib/ant-apache-regexp.jar:/usr/share/ant/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_191/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.7.2.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/temp/junit4-J2-20190309_190450_7857708240789016186312.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/temp/junit4-J2-20190309_190450_7854466393364959383005.suites -stdin [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs. [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1563) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:123) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:997) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:994) [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [junit4] at java.lang.Thread.run(Thread.java:748) BUILD FAILED /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/common-build.xml:1572: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/common-build.xml:1099: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs. Total time: 71 minutes 55 seconds [repro] Setting last failure code to 256 [repro] Failures: [repro] 3/5 failed: org.apache.solr.cloud.api.collections.HdfsCollectionsAPIDistributedZkTest [repro] git checkout 27df824a4b62512f64c885a3d213827a315109e9 Previous HEAD position was 20de3d2... SOLR-12923: harden TestSimLargeCluster HEAD is now at 27df824... SOLR-13292: Provide extended per-segment status of a collection. [repro] Exiting with code 256 Archiving artifacts Recording test results Build step 'Publish JUnit test result report' changed build result to UNSTABLE Email was triggered for: Unstable (Test Failures) Sending email for trigger: Unstable (Test Failures)
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
