Build: https://builds.apache.org/job/Lucene-Solr-repro/1727/
[...truncated 28 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1671/consoleText [repro] Revision: fd9164801e703b278922dae6cc3c53e0578fa1d6 [repro] Ant options: -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt [repro] Repro line: ant test -Dtestcase=DeleteNodeTest -Dtests.method=test -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=fr-FR -Dtests.timezone=Europe/Moscow -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=CdcrReplicationHandlerTest -Dtests.method=testReplicationWithBufferedUpdates -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=da-DK -Dtests.timezone=Asia/Samarkand -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=HdfsRestartWhileUpdatingTest -Dtests.method=test -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-NZ -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=LIROnShardRestartTest -Dtests.method=testSeveralReplicasInLIR -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=pl -Dtests.timezone=Antarctica/Rothera -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=LIROnShardRestartTest -Dtests.method=testAllReplicasInLIR -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=pl -Dtests.timezone=Antarctica/Rothera -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: e1da5f953731b4e2990e054d09ec0bcb2e5146b8 [repro] git fetch [repro] git checkout fd9164801e703b278922dae6cc3c53e0578fa1d6 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro] solr/core [repro] CdcrReplicationHandlerTest [repro] HdfsRestartWhileUpdatingTest [repro] DeleteNodeTest [repro] LIROnShardRestartTest [repro] ant compile-test [...truncated 3423 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=20 -Dtests.class="*.CdcrReplicationHandlerTest|*.HdfsRestartWhileUpdatingTest|*.DeleteNodeTest|*.LIROnShardRestartTest" -Dtests.showOutput=onerror -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=da-DK -Dtests.timezone=Asia/Samarkand -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [...truncated 7150 lines...] [junit4] 2> 553804 ERROR (Finalizer) [ ] o.a.s.c.SolrCore REFCOUNT ERROR: unreferenced org.apache.solr.core.SolrCore@eace80b (collection1_shard1_replica_n21) has a reference count of -1 [junit4] 2> 553943 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id [junit4] 2> 554022 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0 [junit4] 2> 554022 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0 [junit4] 2> 554022 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/extraction/lib [junit4] 2> 554023 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist [junit4] 2> 554023 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/clustering/lib [junit4] 2> 554024 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist [junit4] 2> 554024 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/langid/lib [junit4] 2> 554024 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist [junit4] 2> 554024 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../contrib/velocity/lib [junit4] 2> 554024 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist [junit4] 2> 554024 WARN (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-ltr-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1/../../../../dist [junit4] 2> 554048 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0 [junit4] 2> 554093 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.s.IndexSchema [severalReplicasInLIR_shard1_replica_n1] Schema name=default-config [junit4] 2> 554386 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id [junit4] 2> 554412 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'severalReplicasInLIR_shard1_replica_n1' using configuration from collection severalReplicasInLIR, trusted=true [junit4] 2> 554413 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42513.solr.core.severalReplicasInLIR.shard1.replica_n1' (registry 'solr.core.severalReplicasInLIR.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aa9b56 [junit4] 2> 554413 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrCore [[severalReplicasInLIR_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/severalReplicasInLIR_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node1/./severalReplicasInLIR_shard1_replica_n1/data/] [junit4] 2> 554422 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5 [junit4] 2> 554579 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog [junit4] 2> 554579 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 554580 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; [junit4] 2> 554580 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 554596 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b15d8ff[severalReplicasInLIR_shard1_replica_n1] main] [junit4] 2> 554597 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/severalReplicasInLIR.AUTOCREATED [junit4] 2> 554598 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/severalReplicasInLIR.AUTOCREATED [junit4] 2> 554598 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/severalReplicasInLIR.AUTOCREATED/managed-schema [junit4] 2> 554598 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest [junit4] 2> 554598 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers [junit4] 2> 554599 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01} [junit4] 2> 554600 INFO (coreLoadExecutor-878-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms. [junit4] 2> 554600 INFO (searcherExecutor-879-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@1b15d8ff[severalReplicasInLIR_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(_0(8.0.0):C2))} [junit4] 2> 554600 INFO (searcherExecutor-879-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener done. [junit4] 2> 554600 INFO (searcherExecutor-879-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default [junit4] 2> 554602 INFO (searcherExecutor-879-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrCore [severalReplicasInLIR_shard1_replica_n1] Registered new searcher Searcher@1b15d8ff[severalReplicasInLIR_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(_0(8.0.0):C2))} [junit4] 2> 554617 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/severalReplicasInLIR/terms/shard1 to Terms{values={core_node3=0}, version=7} [junit4] 2> 554620 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=3 found=1 timeoutin=9999ms [junit4] 2> 555106 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11 [junit4] 2> 555107 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 555107 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 555107 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.e.j.s.session node0 Scavenging every 600000ms [junit4] 2> 555108 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5da64{/solr,null,AVAILABLE} [junit4] 2> 555108 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3ab3e216{HTTP/1.1,[http/1.1]}{127.0.0.1:33707} [junit4] 2> 555108 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.e.j.s.Server Started @555163ms [junit4] 2> 555109 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33707} [junit4] 2> 555109 ERROR (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 555109 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory [junit4] 2> 555109 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0 [junit4] 2> 555109 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 555109 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 555109 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-10-20T01:20:08.725Z [junit4] 2> 555111 INFO (zkConnectionManagerCallback-1376-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 555131 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 555233 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44687/solr [junit4] 2> 555234 INFO (zkConnectionManagerCallback-1380-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 555237 INFO (zkConnectionManagerCallback-1382-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 555242 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 555245 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 555245 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33707_solr [junit4] 2> 555246 INFO (zkCallback-1372-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 555246 INFO (zkCallback-1365-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 555251 INFO (zkCallback-1357-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 555268 INFO (zkCallback-1381-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 555319 INFO (zkConnectionManagerCallback-1389-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 555327 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) [junit4] 2> 555328 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44687/solr ready [junit4] 2> 555353 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 555368 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33707.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aa9b56 [junit4] 2> 555388 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33707.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aa9b56 [junit4] 2> 555389 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33707.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aa9b56 [junit4] 2> 555410 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/. [junit4] 2> 555410 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [severalReplicasInLIR_shard1_replica_n2] [junit4] 2> 555443 INFO (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.RequestParams conf resource params.json loaded . version : 0 [junit4] 2> 555443 INFO (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.RequestParams request params refreshed to version 0 [junit4] 2> 555444 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/extraction/lib [junit4] 2> 555444 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555444 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/clustering/lib [junit4] 2> 555444 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555444 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/langid/lib [junit4] 2> 555444 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555445 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/velocity/lib [junit4] 2> 555445 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555445 WARN (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-ltr-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555456 INFO (coreLoadExecutor-892-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0 [junit4] 2> 555533 INFO (coreLoadExecutor-892-thread-1) [ ] o.a.s.s.IndexSchema [severalReplicasInLIR_shard1_replica_n2] Schema name=default-config [junit4] 2> 555797 INFO (coreLoadExecutor-892-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id [junit4] 2> 555844 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0 [junit4] 2> 555844 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0 [junit4] 2> 555844 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/extraction/lib [junit4] 2> 555844 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555845 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/clustering/lib [junit4] 2> 555845 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555845 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/langid/lib [junit4] 2> 555845 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555845 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../contrib/velocity/lib [junit4] 2> 555845 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555845 WARN (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-ltr-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2/../../../../dist [junit4] 2> 555852 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0 [junit4] 2> 555872 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.s.IndexSchema [severalReplicasInLIR_shard1_replica_n2] Schema name=default-config [junit4] 2> 556014 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id [junit4] 2> 556038 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'severalReplicasInLIR_shard1_replica_n2' using configuration from collection severalReplicasInLIR, trusted=true [junit4] 2> 556038 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33707.solr.core.severalReplicasInLIR.shard1.replica_n2' (registry 'solr.core.severalReplicasInLIR.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aa9b56 [junit4] 2> 556039 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrCore [[severalReplicasInLIR_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/severalReplicasInLIR_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J2/temp/solr.cloud.LIROnShardRestartTest_BBDF5DA8FB090389-002/tempDir-001/node3/./severalReplicasInLIR_shard1_replica_n2/data/] [junit4] 2> 556053 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5 [junit4] 2> 556114 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog [junit4] 2> 556114 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 556115 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; [junit4] 2> 556115 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 556119 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ee535fe[severalReplicasInLIR_shard1_replica_n2] main] [junit4] 2> 556120 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/severalReplicasInLIR.AUTOCREATED [junit4] 2> 556120 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/severalReplicasInLIR.AUTOCREATED [junit4] 2> 556120 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/severalReplicasInLIR.AUTOCREATED/managed-schema [junit4] 2> 556121 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest [junit4] 2> 556121 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Initializing spell checkers [junit4] 2> 556121 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01} [junit4] 2> 556122 INFO (coreLoadExecutor-892-thread-1) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms. [junit4] 2> 556123 INFO (searcherExecutor-893-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@1ee535fe[severalReplicasInLIR_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader(_0(8.0.0):C2))} [junit4] 2> 556123 INFO (searcherExecutor-893-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.QuerySenderListener QuerySenderListener done. [junit4] 2> 556123 INFO (searcherExecutor-893-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default [junit4] 2> 556128 INFO (searcherExecutor-893-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrCore [severalReplicasInLIR_shard1_replica_n2] Registered new searcher Searcher@1ee535fe[severalReplicasInLIR_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader(_0(8.0.0):C2))} [junit4] 2> 556132 INFO (coreZkRegister-884-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/severalReplicasInLIR/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=8} [junit4] 2> 564730 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Was waiting for replicas to come up, but they are taking too long - assuming they won't come back till later [junit4] 2> 564730 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync [junit4] 2> 564730 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/ [junit4] 2> 564731 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=severalReplicasInLIR_shard1_replica_n1 url=http://127.0.0.1:42513/solr START replicas=[http://127.0.0.1:33707/solr/severalReplicasInLIR_shard1_replica_n2/] nUpdates=100 [junit4] 2> 564779 INFO (qtp234054712-3185) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.S.Request [severalReplicasInLIR_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=9 [junit4] 2> 564780 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=severalReplicasInLIR_shard1_replica_n1 url=http://127.0.0.1:42513/solr Received 2 versions from http://127.0.0.1:33707/solr/severalReplicasInLIR_shard1_replica_n2/ fingerprint:null [junit4] 2> 564792 INFO (qtp234054712-3181) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.S.Request [severalReplicasInLIR_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&checkCanHandleVersionRanges=false&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 564794 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=severalReplicasInLIR_shard1_replica_n1 url=http://127.0.0.1:42513/solr No additional versions requested. ourHighThreshold=1614805360071147520 otherLowThreshold=1614805359791177728 ourHighest=1614805360071147520 otherHighest=1614805360071147520 [junit4] 2> 564794 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=severalReplicasInLIR_shard1_replica_n1 url=http://127.0.0.1:42513/solr DONE. sync succeeded [junit4] 2> 564794 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me [junit4] 2> 564795 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/: try and ask http://127.0.0.1:33707/solr/severalReplicasInLIR_shard1_replica_n2/ to sync [junit4] 2> 564802 INFO (qtp234054712-3180) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=severalReplicasInLIR_shard1_replica_n2 url=http://127.0.0.1:33707/solr START replicas=[http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/] nUpdates=100 [junit4] 2> 564835 INFO (qtp43116373-3142) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.IndexFingerprint IndexFingerprint millis:7.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=1614805360071147520, maxInHash=1614805360071147520, versionsHash=-7927489911433321150, numVersions=2, numDocs=2, maxDoc=2} [junit4] 2> 564835 INFO (qtp43116373-3142) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.S.Request [severalReplicasInLIR_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=7 [junit4] 2> 564851 INFO (qtp234054712-3180) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.IndexFingerprint IndexFingerprint millis:15.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=1614805360071147520, maxInHash=1614805360071147520, versionsHash=-7927489911433321150, numVersions=2, numDocs=2, maxDoc=2} [junit4] 2> 564851 INFO (qtp234054712-3180) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync [junit4] 2> 564851 INFO (qtp234054712-3180) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.S.Request [severalReplicasInLIR_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&getVersions=100&sync=http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/&wt=javabin&version=2} status=0 QTime=49 [junit4] 2> 564852 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/: sync completed with http://127.0.0.1:33707/solr/severalReplicasInLIR_shard1_replica_n2/ [junit4] 2> 564854 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/ shard1 [junit4] 2> 564906 INFO (coreZkRegister-866-thread-1-processing-n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1 c:severalReplicasInLIR s:shard1 r:core_node3) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary [junit4] 2> 565009 INFO (zkCallback-1365-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/severalReplicasInLIR/state.json] for collection [severalReplicasInLIR] has occurred - updating... (live nodes size: [2]) [junit4] 2> 565168 INFO (coreZkRegister-884-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.ZkController Core needs to recover:severalReplicasInLIR_shard1_replica_n2 [junit4] 2> 565168 INFO (updateExecutor-1377-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.DefaultSolrCoreState Running recovery [junit4] 2> 565179 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true [junit4] 2> 565180 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy startupVersions size=2 range=[1614805360071147520 to 1614805359791177728] [junit4] 2> 565197 INFO (qtp43116373-3137) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.S.Request [severalReplicasInLIR_shard1_replica_n1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=2 status=0 QTime=1 [junit4] 2> 565197 INFO (qtp43116373-3137) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.S.Request [severalReplicasInLIR_shard1_replica_n1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=1 [junit4] 2> 565198 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[severalReplicasInLIR_shard1_replica_n2] [junit4] 2> 565198 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} [junit4] 2> 565198 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy Publishing state of core [severalReplicasInLIR_shard1_replica_n2] as recovering, leader is [http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/] and I am [http://127.0.0.1:33707/solr/severalReplicasInLIR_shard1_replica_n2/] [junit4] 2> 565202 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:42513/solr]; [WaitForState: action=PREPRECOVERY&core=severalReplicasInLIR_shard1_replica_n1&nodeName=127.0.0.1:33707_solr&coreNodeName=core_node5&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] [junit4] 2> 565209 INFO (qtp43116373-3138) [n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node5, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s [junit4] 2> 565213 INFO (qtp43116373-3138) [n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=severalReplicasInLIR, shard=shard1, thisCore=severalReplicasInLIR_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:33707_solr, coreNodeName=core_node5, onlyIfActiveCheckResult=false, nodeProps: core_node5:{"core":"severalReplicasInLIR_shard1_replica_n2","base_url":"http://127.0.0.1:33707/solr","node_name":"127.0.0.1:33707_solr","state":"down","type":"NRT","force_set_state":"false"} [junit4] 2> 565303 INFO (zkCallback-1381-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/severalReplicasInLIR/state.json] for collection [severalReplicasInLIR] has occurred - updating... (live nodes size: [2]) [junit4] 2> 565308 INFO (zkCallback-1365-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/severalReplicasInLIR/state.json] for collection [severalReplicasInLIR] has occurred - updating... (live nodes size: [2]) [junit4] 2> 566214 INFO (qtp43116373-3138) [n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=severalReplicasInLIR, shard=shard1, thisCore=severalReplicasInLIR_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:33707_solr, coreNodeName=core_node5, onlyIfActiveCheckResult=false, nodeProps: core_node5:{"core":"severalReplicasInLIR_shard1_replica_n2","base_url":"http://127.0.0.1:33707/solr","node_name":"127.0.0.1:33707_solr","state":"recovering","type":"NRT","force_set_state":"false"} [junit4] 2> 566214 INFO (qtp43116373-3138) [n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds. [junit4] 2> 566214 INFO (qtp43116373-3138) [n:127.0.0.1:42513_solr x:severalReplicasInLIR_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:33707_solr&onlyIfLeaderActive=true&core=severalReplicasInLIR_shard1_replica_n1&coreNodeName=core_node5&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1004 [junit4] 2> 566715 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/] - recoveringAfterStartup=[true] [junit4] 2> 566715 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.PeerSyncWithLeader PeerSync: core=severalReplicasInLIR_shard1_replica_n2 url=http://127.0.0.1:33707/solr START leader=http://127.0.0.1:42513/solr/severalReplicasInLIR_shard1_replica_n1/ nUpdates=100 [junit4] 2> 566717 INFO (qtp43116373-3141) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=1614805360071147520, maxInHash=1614805360071147520, versionsHash=-7927489911433321150, numVersions=2, numDocs=2, maxDoc=2} [junit4] 2> 566717 INFO (qtp43116373-3141) [n:127.0.0.1:42513_solr c:severalReplicasInLIR s:shard1 r:core_node3 x:severalReplicasInLIR_shard1_replica_n1] o.a.s.c.S.Request [severalReplicasInLIR_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 566718 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=1614805360071147520, maxInHash=1614805360071147520, versionsHash=-7927489911433321150, numVersions=2, numDocs=2, maxDoc=2} [junit4] 2> 566718 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.PeerSyncWithLeader Fingerprint comparison result: 0 [junit4] 2> 566718 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 566718 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit. [junit4] 2> 566719 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush [junit4] 2> 566719 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful. [junit4] 2> 566719 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync. [junit4] 2> 566719 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy No replay needed. [junit4] 2> 566719 INFO (recoveryExecutor-1378-thread-1-processing-n:127.0.0.1:33707_solr x:severalReplicasInLIR_shard1_replica_n2 c:severalReplicasInLIR s:shard1 r:core_node5) [n:127.0.0.1:33707_solr c:severalReplicasInLIR s:shard1 r:core_node5 x:severalReplicasInLIR_shard1_replica_n2] o.a.s.c.RecoveryStrategy Registering as Active after recovery. [junit4] 2> 566821 INFO (zkCallback-1381-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/severalReplicasInLIR/state.json] for collection [severalReplicasInLIR] has occurred - updating... (live nodes size: [2]) [junit4] 2> 566821 INFO (zkCallback-1365-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/severalReplicasInLIR/state.json] for collection [severalReplicasInLIR] has occurred - updating... (live nodes size: [2]) [junit4] 2> 612655 INFO (qtp234054712-3181) [n:127.0.0.1:33707_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.severalReplicasInLIR.shard1.replica_n2:UPDATE./update.requests&key=solr.core.severalReplicasInLIR.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.severalReplicasInLIR.shard1.replica_n2:QUERY./select.requests} status=0 QTime=1 [junit4] 2> 612666 INFO (qtp234054712-3186) [n:127.0.0.1:33707_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} status=0 QTime=10 [junit4] 2> 612669 INFO (qtp43116373-3137) [n:127.0.0.1:42513_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.severalReplicasInLIR.shard1.replica_n1:QUERY./select.requests&key=solr.core.severalReplicasInLIR.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.severalReplicasInLIR.shard1.replica_n1:UPDATE./update.requests} status=0 QTime=1 [junit4] 2> 612704 INFO (qtp43116373-3137) [n:127.0.0.1:42513_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} status=0 QTime=34 [junit4] 2> 616139 INFO (TEST-LIROnShardRestartTest.testSeveralReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSeveralReplicasInLIR [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory. [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=LIROnShardRestartTest -Dtests.method=testSeveralReplicasInLIR -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=da-DK -Dtests.timezone=Asia/Samarkand -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] FAILURE 86.5s J2 | LIROnShardRestartTest.testSeveralReplicasInLIR <<< [junit4] > Throwable #1: java.lang.AssertionError: Timeout waiting for replicas rejoin election [junit4] > at __randomizedtesting.SeedInfo.seed([BBDF5DA8FB090389:3D85CA9CDB534ED0]:0) [junit4] > at org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR(LIROnShardRestartTest.java:242) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] 2> 616272 INFO (TEST-LIROnShardRestartTest.testAllReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAllReplicasInLIR [junit4] 2> 616295 INFO (qtp234054712-3183) [n:127.0.0.1:33707_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&name=allReplicasInLIR&nrtReplicas=3&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 616313 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection allReplicasInLIR [junit4] 2> 616331 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/allReplicasInLIR.AUTOCREATED/protwords.txt [junit4] 2> 616333 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/allReplicasInLIR.AUTOCREATED/managed-schema [junit4] 2> 616360 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/allReplicasInLIR.AUTOCREATED/solrconfig.xml [junit4] 2> 616379 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/allReplicasInLIR.AUTOCREATED/synonyms.txt [junit4] 2> 616381 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/allReplicasInLIR.AUTOCREATED/stopwords.txt [junit4] 2> 616404 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/allReplicasInLIR.AUTOCREATED/params.json [junit4] 2> 616420 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_it.txt [junit4] 2> 616431 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_no.txt [junit4] 2> 616433 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_hy.txt [junit4] 2> 616452 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stemdict_nl.txt [junit4] 2> 616455 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_ca.txt [junit4] 2> 616470 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_id.txt [junit4] 2> 616484 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ga.txt [junit4] 2> 616487 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_da.txt [junit4] 2> 616502 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_hi.txt [junit4] 2> 616517 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ja.txt [junit4] 2> 616519 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_pt.txt [junit4] 2> 616543 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_hu.txt [junit4] 2> 616558 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_el.txt [junit4] 2> 616573 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ru.txt [junit4] 2> 616576 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_tr.txt [junit4] 2> 616591 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ar.txt [junit4] 2> 616606 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/userdict_ja.txt [junit4] 2> 616608 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_it.txt [junit4] 2> 616623 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_eu.txt [junit4] 2> 616639 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_cz.txt [junit4] 2> 616642 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stoptags_ja.txt [junit4] 2> 616657 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_fr.txt [junit4] 2> 616675 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_fa.txt [junit4] 2> 616678 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_de.txt [junit4] 2> 616696 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_ga.txt [junit4] 2> 616698 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ca.txt [junit4] 2> 616725 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_nl.txt [junit4] 2> 616756 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_bg.txt [junit4] 2> 616759 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_es.txt [junit4] 2> 616761 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_en.txt [junit4] 2> 616762 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_lv.txt [junit4] 2> 616764 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_th.txt [junit4] 2> 616779 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ro.txt [junit4] 2> 616793 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_fr.txt [junit4] 2> 616796 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_sv.txt [junit4] 2> 616811 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_fi.txt [junit4] 2> 616826 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/hyphenations_ga.txt [junit4] 2> 616829 INFO (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_gl.txt [junit4] 2> 616976 WARN (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 3 on collection allReplicasInLIR is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance. [junit4] 2> 616977 ERROR (OverseerThreadFactory-874-thread-1-processing-n:127.0.0.1:42513_solr) [n:127.0.0.1:42513_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: allReplicasInLIR operation: create failed:org.apache.solr.common.SolrException: Cannot create collection allReplicasInLIR. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 2. This allows a maximum of 2 to be created. Value of numShards is 1, value of nrtReplicas is 3, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 3 shards to be created (higher than the allowed number) [junit4] 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.buildReplicaPositions(CreateCollectionCmd.java:375) [junit4] 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:178) [junit4] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:259) [junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:478) [junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [junit4] 2> at java.lang.Thread.run(Thread.java:748) [junit4] 2> [junit4] 2> 616995 INFO (qtp234054712-3183) [n:127.0.0.1:33707_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&name=allReplicasInLIR&nrtReplicas=3&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=400 QTime=699 [junit4] 2> 616996 INFO (TEST-LIROnShardRestartTest.testAllReplicasInLIR-seed#[BBDF5DA8FB090389]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testAllReplicasInLIR [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory. [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=LIROnShardRestartTest -Dtests.method=testAllReplicasInLIR -Dtests.seed=BBDF5DA8FB090389 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.badapples=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=da-DK -Dtests.timezone=Asia/Samarkand -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.86s J2 | LIROnShardRestartTest.testAllReplicasInLIR <<< [junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33707/solr: Cannot create collection allReplicasInLIR. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 2. This allows a maximum of 2 to be created. Value of numShards is 1, value of nrtReplicas is 3, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 3 shards to be created (higher than the allowed number) [junit4] > at __randomizedtesting.SeedInfo.seed([BBDF5DA8FB090389:E147676E8589646E]:0) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817) [junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194) [junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211) [junit4] > at org.apache.solr.cloud.LIROnShardRestartTest.testAllReplicasInLIR(LIROnShardRestartTest.java:78) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] 2> 617021 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1053350eb0f000e, likely client has closed socket [junit4] 2> 617043 INFO (jetty-closer-1301-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@742f062c{HTTP/1.1,[http/1.1]}{127.0.0.1:42513} [junit4] 2> 617065 INFO (jetty-closer-1301-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2121684770 [junit4] 2> 617065 INFO (jetty-closer-1301-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null [junit4] 2> [...truncated too long message...] t4] 2> 334440 INFO (zkConnectionManagerCallback-211-thread-1) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46145_solr [junit4] 2> 334442 INFO (zkCallback-217-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 334442 INFO (zkCallback-210-thread-9) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 334442 INFO (zkConnectionManagerCallback-211-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper [junit4] 2> 334442 INFO (zkConnectionManagerCallback-211-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient Connected:true [junit4] 2> 334443 INFO (coreZkRegister-222-thread-3) [ ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/allReplicasInLIR.AUTOCREATED/managed-schema [junit4] 2> 334443 INFO (Thread-160) [ ] o.a.s.c.SolrCore config update listener called for core allReplicasInLIR_shard1_replica_n1 [junit4] 2> 334443 INFO (coreZkRegister-222-thread-3) [ ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest [junit4] 2> 337441 INFO (recoveryExecutor-207-thread-1-processing-n:127.0.0.1:46145_solr x:allReplicasInLIR_shard1_replica_n1 c:allReplicasInLIR s:shard1 r:core_node4) [n:127.0.0.1:46145_solr c:allReplicasInLIR s:shard1 r:core_node4 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.RecoveryStrategy RecoveryStrategy has been closed [junit4] 2> 337441 INFO (recoveryExecutor-207-thread-1-processing-n:127.0.0.1:46145_solr x:allReplicasInLIR_shard1_replica_n1 c:allReplicasInLIR s:shard1 r:core_node4) [n:127.0.0.1:46145_solr c:allReplicasInLIR s:shard1 r:core_node4 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[false] [junit4] 2> 337441 INFO (recoveryExecutor-207-thread-1-processing-n:127.0.0.1:46145_solr x:allReplicasInLIR_shard1_replica_n1 c:allReplicasInLIR s:shard1 r:core_node4) [n:127.0.0.1:46145_solr c:allReplicasInLIR s:shard1 r:core_node4 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrCore [allReplicasInLIR_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@316f50b6 [junit4] 2> 337441 INFO (recoveryExecutor-207-thread-1-processing-n:127.0.0.1:46145_solr x:allReplicasInLIR_shard1_replica_n1 c:allReplicasInLIR s:shard1 r:core_node4) [n:127.0.0.1:46145_solr c:allReplicasInLIR s:shard1 r:core_node4 x:allReplicasInLIR_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.allReplicasInLIR.shard1.replica_n1, tag=316f50b6 [junit4] 2> 337442 INFO (recoveryExecutor-207-thread-1-processing-n:127.0.0.1:46145_solr x:allReplicasInLIR_shard1_replica_n1 c:allReplicasInLIR s:shard1 r:core_node4) [n:127.0.0.1:46145_solr c:allReplicasInLIR s:shard1 r:core_node4 x:allReplicasInLIR_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7e0098f6: rootName = solr_46145, domain = solr.core.allReplicasInLIR.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.allReplicasInLIR.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@28338897 [junit4] 2> 337451 INFO (recoveryExecutor-207-thread-1-processing-n:127.0.0.1:46145_solr x:allReplicasInLIR_shard1_replica_n1 c:allReplicasInLIR s:shard1 r:core_node4) [n:127.0.0.1:46145_solr c:allReplicasInLIR s:shard1 r:core_node4 x:allReplicasInLIR_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.allReplicasInLIR.shard1.leader, tag=316f50b6 [junit4] 2> 337452 WARN (recoveryExecutor-207-thread-1-processing-n:127.0.0.1:46145_solr x:allReplicasInLIR_shard1_replica_n1 c:allReplicasInLIR s:shard1 r:core_node4) [n:127.0.0.1:46145_solr c:allReplicasInLIR s:shard1 r:core_node4 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.RecoveryStrategy Stopping recovery for core=[allReplicasInLIR_shard1_replica_n1] coreNodeName=[core_node4] [junit4] 2> 338959 WARN (zkCallback-210-thread-9) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes] [junit4] 2> 338959 WARN (zkCallback-210-thread-10) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes] [junit4] 2> 338960 INFO (jetty-closer-146-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@683ac8c1{/solr,null,UNAVAILABLE} [junit4] 2> 338960 INFO (jetty-closer-146-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging [junit4] 2> 338961 ERROR (SUITE-LIROnShardRestartTest-seed#[A9F6D52D8CBD27D5]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes [junit4] 2> 338962 INFO (SUITE-LIROnShardRestartTest-seed#[A9F6D52D8CBD27D5]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41429 41429 [junit4] 2> 339005 INFO (Thread-92) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41429 41429 [junit4] 2> 339006 WARN (Thread-92) [ ] o.a.s.c.ZkTestServer Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 28 /solr/collections/allReplicasInLIR/terms/shard1 [junit4] 2> 16 /solr/aliases.json [junit4] 2> 9 /solr/security.json [junit4] 2> 9 /solr/configs/allReplicasInLIR.AUTOCREATED/managed-schema [junit4] 2> 9 /solr/configs/allReplicasInLIR.AUTOCREATED [junit4] 2> [junit4] 2> Maximum concurrent data watches above limit: [junit4] 2> [junit4] 2> 22 /solr/collections/allReplicasInLIR/state.json [junit4] 2> 16 /solr/clusterprops.json [junit4] 2> 16 /solr/clusterstate.json [junit4] 2> 4 /solr/autoscaling.json [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 23 /solr/live_nodes [junit4] 2> 16 /solr/collections [junit4] 2> 4 /solr/overseer/queue [junit4] 2> 4 /solr/autoscaling/events/.scheduled_maintenance [junit4] 2> 4 /solr/autoscaling/events/.auto_add_replicas [junit4] 2> 4 /solr/overseer/collection-queue-work [junit4] 2> 4 /solr/overseer/queue-work [junit4] 2> [junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/solr/build/solr-core/test/J1/temp/solr.cloud.LIROnShardRestartTest_A9F6D52D8CBD27D5-002 [junit4] 2> Oct 20, 2018 1:41:10 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {}, docValues:{}, maxPointsInLeafNode=624, maxMBSortInHeap=6.855475308817198, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@53b9faed), locale=pl, timezone=Antarctica/Rothera [junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=218550264,total=383254528 [junit4] 2> NOTE: All tests run in this JVM: [LIROnShardRestartTest, LIROnShardRestartTest] [junit4] Completed [5/5 (5!)] on J1 in 146.13s, 2 tests, 2 errors <<< FAILURES! [junit4] [junit4] [junit4] Tests with failures [seed: A9F6D52D8CBD27D5]: [junit4] - org.apache.solr.cloud.LIROnShardRestartTest.testAllReplicasInLIR [junit4] - org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR [junit4] - org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR [junit4] - org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR [junit4] - org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR [junit4] - org.apache.solr.cloud.LIROnShardRestartTest.testAllReplicasInLIR [junit4] - org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR [junit4] [junit4] [junit4] JVM J0: 0.71 .. 256.65 = 255.94s [junit4] JVM J1: 0.75 .. 340.68 = 339.92s [junit4] JVM J2: 0.74 .. 197.04 = 196.30s [junit4] Execution time total: 5 minutes 40 seconds [junit4] Tests summary: 5 suites, 10 tests, 4 errors, 3 failures BUILD FAILED /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/common-build.xml:1567: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-repro/lucene/common-build.xml:1092: There were test failures: 5 suites, 10 tests, 4 errors, 3 failures [seed: A9F6D52D8CBD27D5] Total time: 5 minutes 42 seconds [repro] Setting last failure code to 256 [repro] Failures at the tip of master without a seed: [repro] 5/5 failed: org.apache.solr.cloud.LIROnShardRestartTest [repro] git checkout e1da5f953731b4e2990e054d09ec0bcb2e5146b8 Note: checking out 'e1da5f953731b4e2990e054d09ec0bcb2e5146b8'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name> HEAD is now at e1da5f9... LUCENE-8531: QueryBuilder#analyzeGraphPhrase now creates one phrase query per finite strings in the graph if the slop is greater than 0. Span queries cannot be used in this case because they don't handle slop the same way than phrase queries. [repro] Exiting with code 256 Archiving artifacts Recording test results Build step 'Publish JUnit test result report' changed build result to UNSTABLE Email was triggered for: Unstable (Test Failures) Sending email for trigger: Unstable (Test Failures)
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
