Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/451/
2 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.search.join.BlockJoinFacetDistribTest Error Message: 1 thread leaked from SUITE scope at org.apache.solr.search.join.BlockJoinFacetDistribTest: 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(Thread.java:745) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.search.join.BlockJoinFacetDistribTest: 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(Thread.java:745) at __randomizedtesting.SeedInfo.seed([CBAC70BF30C091A6]:0) FAILED: junit.framework.TestSuite.org.apache.solr.search.join.BlockJoinFacetDistribTest Error Message: There are still zombie threads that couldn't be terminated: 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(Thread.java:745) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(Thread.java:745) at __randomizedtesting.SeedInfo.seed([CBAC70BF30C091A6]:0) Build Log: [...truncated 11624 lines...] [junit4] Suite: org.apache.solr.search.join.BlockJoinFacetDistribTest [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/init-core-data-001 [junit4] 2> 1596172 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN) [junit4] 2> 1596173 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 1596174 INFO (Thread-3287) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 1596174 INFO (Thread-3287) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 1596276 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46345 [junit4] 2> 1596276 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596284 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596300 INFO (zkCallback-1251-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@348ee128 name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596300 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596300 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596301 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml [junit4] 2> 1596305 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /solr/clusterprops.json [junit4] 2> 1596318 INFO (jetty-launcher-1250-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1596320 INFO (jetty-launcher-1250-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1596322 INFO (jetty-launcher-1250-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1596337 INFO (jetty-launcher-1250-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1596339 INFO (jetty-launcher-1250-thread-5) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1596349 INFO (jetty-launcher-1250-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19993617{/solr,null,AVAILABLE} [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.e.j.s.ServerConnector Started ServerConnector@4c30a8fd{SSL,[ssl, http/1.1]}{127.0.0.1:33541} [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.e.j.s.Server Started @1603350ms [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33541} [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node5' [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1596351 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596352 INFO (jetty-launcher-1250-thread-6) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1596353 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596354 INFO (jetty-launcher-1250-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5331eafa{/solr,null,AVAILABLE} [junit4] 2> 1596355 INFO (jetty-launcher-1250-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@31471517{SSL,[ssl, http/1.1]}{127.0.0.1:35634} [junit4] 2> 1596355 INFO (jetty-launcher-1250-thread-4) [ ] o.e.j.s.Server Started @1603354ms [junit4] 2> 1596355 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35634} [junit4] 2> 1596355 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1596355 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node4' [junit4] 2> 1596355 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1596356 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1596356 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596385 INFO (jetty-launcher-1250-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f72a7b8{/solr,null,AVAILABLE} [junit4] 2> 1596386 INFO (jetty-launcher-1250-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@461cc8d2{SSL,[ssl, http/1.1]}{127.0.0.1:48178} [junit4] 2> 1596386 INFO (jetty-launcher-1250-thread-3) [ ] o.e.j.s.Server Started @1603386ms [junit4] 2> 1596386 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48178} [junit4] 2> 1596386 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1596387 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node3' [junit4] 2> 1596387 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1596387 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1596396 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596403 INFO (jetty-launcher-1250-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2099200e{/solr,null,AVAILABLE} [junit4] 2> 1596404 INFO (jetty-launcher-1250-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@2d34ea8a{SSL,[ssl, http/1.1]}{127.0.0.1:58944} [junit4] 2> 1596404 INFO (jetty-launcher-1250-thread-2) [ ] o.e.j.s.Server Started @1603403ms [junit4] 2> 1596404 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58944} [junit4] 2> 1596404 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1596404 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node2' [junit4] 2> 1596404 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1596404 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1596412 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596414 INFO (jetty-launcher-1250-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d06992{/solr,null,AVAILABLE} [junit4] 2> 1596415 INFO (jetty-launcher-1250-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@511cfbee{SSL,[ssl, http/1.1]}{127.0.0.1:54209} [junit4] 2> 1596415 INFO (jetty-launcher-1250-thread-1) [ ] o.e.j.s.Server Started @1603414ms [junit4] 2> 1596415 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54209} [junit4] 2> 1596415 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1596415 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node1' [junit4] 2> 1596415 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1596415 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1596424 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596428 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596458 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596460 INFO (zkCallback-1252-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@699b1734 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596460 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596460 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596461 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1596587 INFO (jetty-launcher-1250-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@768f7840{/solr,null,AVAILABLE} [junit4] 2> 1596588 INFO (jetty-launcher-1250-thread-6) [ ] o.e.j.s.ServerConnector Started ServerConnector@7c28f7d{SSL,[ssl, http/1.1]}{127.0.0.1:41141} [junit4] 2> 1596588 INFO (jetty-launcher-1250-thread-6) [ ] o.e.j.s.Server Started @1603587ms [junit4] 2> 1596588 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41141} [junit4] 2> 1596588 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1596588 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node6' [junit4] 2> 1596588 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1596589 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1596589 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596592 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596596 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596597 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596597 INFO (zkCallback-1253-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21b8897f name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596600 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596600 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596601 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1596598 INFO (zkCallback-1254-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77098ec3 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596597 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node5/. [junit4] 2> 1596602 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.CoreContainer New CoreContainer 1504186182 [junit4] 2> 1596602 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node5] [junit4] 2> 1596602 WARN (jetty-launcher-1250-thread-5) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node5/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node5/lib [junit4] 2> 1596602 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000, [junit4] 2> 1596603 INFO (zkCallback-1255-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1769e989 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596603 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596603 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596604 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596604 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1596604 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596612 INFO (zkCallback-1257-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@579cc9c6 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596613 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596613 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node2/. [junit4] 2> 1596613 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596613 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.CoreContainer New CoreContainer 2003494909 [junit4] 2> 1596613 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node2] [junit4] 2> 1596613 WARN (jetty-launcher-1250-thread-2) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node2/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node2/lib [junit4] 2> 1596613 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1596613 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000, [junit4] 2> 1596614 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1596614 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1596615 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1596615 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46345/solr [junit4] 2> 1596615 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1596615 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596615 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596615 INFO (zkCallback-1256-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69169acb name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596620 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596621 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596621 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node4/. [junit4] 2> 1596621 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.CoreContainer New CoreContainer 1571211722 [junit4] 2> 1596621 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1596621 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node4] [junit4] 2> 1596621 WARN (jetty-launcher-1250-thread-4) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node4/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node4/lib [junit4] 2> 1596621 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000, [junit4] 2> 1596622 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1596622 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1596622 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46345/solr [junit4] 2> 1596622 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1596622 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596622 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1596624 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596624 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1596632 INFO (zkCallback-1260-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3589f6e3 name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596626 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node6/. [junit4] 2> 1596633 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.CoreContainer New CoreContainer 249252340 [junit4] 2> 1596633 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node6] [junit4] 2> 1596633 WARN (jetty-launcher-1250-thread-6) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node6/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node6/lib [junit4] 2> 1596636 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000, [junit4] 2> 1596626 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1596637 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1596637 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46345/solr [junit4] 2> 1596637 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1596637 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596626 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 1596640 INFO (zkCallback-1263-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2a26ed9b name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596637 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1596633 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596641 INFO (jetty-launcher-1250-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596642 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596642 INFO (jetty-launcher-1250-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596642 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596645 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1596645 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1596645 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46345/solr [junit4] 2> 1596645 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1596645 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596646 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node1/. [junit4] 2> 1596651 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.CoreContainer New CoreContainer 506728159 [junit4] 2> 1596651 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node1] [junit4] 2> 1596652 WARN (jetty-launcher-1250-thread-1) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node1/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node1/lib [junit4] 2> 1596651 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/829565923@5ec39bbe [junit4] 2> 1596652 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000, [junit4] 2> 1596653 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596650 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596647 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x157300b0d7d0007, likely client has closed socket [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 1596653 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1596657 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x157300b0d7d0008, likely client has closed socket [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 1596657 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x157300b0d7d0002, likely client has closed socket [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 1596658 INFO (zkCallback-1266-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65213ba0 name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596658 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596658 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1596658 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1596658 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46345/solr [junit4] 2> 1596658 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596658 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/829565923@20414782 [junit4] 2> 1596658 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1596659 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596660 INFO (zkCallback-1269-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6a69a63a name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596660 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596660 INFO (jetty-launcher-1250-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596660 INFO (zkCallback-1270-thread-1-processing-n:127.0.0.1:33541_solr) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5f804d73 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596660 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596661 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/829565923@12cbd53a [junit4] 2> 1596661 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue [junit4] 2> 1596663 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596664 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/829565923@45e59088 [junit4] 2> 1596665 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596665 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node3/. [junit4] 2> 1596665 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.CoreContainer New CoreContainer 316081885 [junit4] 2> 1596665 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node3] [junit4] 2> 1596666 WARN (jetty-launcher-1250-thread-3) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node3/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node3/lib [junit4] 2> 1596666 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000, [junit4] 2> 1596667 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596667 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596668 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work [junit4] 2> 1596669 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1596669 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1596669 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1596669 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46345/solr [junit4] 2> 1596669 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1596669 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1596670 INFO (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65b7202c name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596671 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596672 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596672 INFO (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1ae00268 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596672 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596673 INFO (zkCallback-1279-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1556e12 name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596677 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596677 INFO (jetty-launcher-1250-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596677 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running [junit4] 2> 1596677 INFO (zkCallback-1273-thread-1-processing-n:127.0.0.1:58944_solr) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@219d3d99 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596678 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596678 INFO (zkCallback-1274-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@26abcd3c name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596678 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596678 INFO (jetty-launcher-1250-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1596683 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x157300b0d7d0010, likely client has closed socket [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 1596684 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/829565923@4145dbfe [junit4] 2> 1596692 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed [junit4] 2> 1596692 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed [junit4] 2> 1596692 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed [junit4] 2> 1596693 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596694 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/829565923@89f4c79 [junit4] 2> 1596694 INFO (zkCallback-1280-thread-1-processing-n:127.0.0.1:48178_solr) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d9b0568 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596694 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596695 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1596696 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure [junit4] 2> 1596697 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure [junit4] 2> 1596697 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure [junit4] 2> 1596703 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes [junit4] 2> 1596703 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes [junit4] 2> 1596704 INFO (zkCallback-1281-thread-1-processing-n:127.0.0.1:54209_solr) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@52e16494 name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1596704 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1596704 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes [junit4] 2> 1596704 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes [junit4] 2> 1596706 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /collections [junit4] 2> 1596706 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /collections [junit4] 2> 1596706 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /collections [junit4] 2> 1596706 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /collections [junit4] 2> 1596708 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json [junit4] 2> 1596709 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json [junit4] 2> 1596709 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json [junit4] 2> 1596709 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json [junit4] 2> 1596711 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json [junit4] 2> 1596711 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json [junit4] 2> 1596711 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json [junit4] 2> 1596711 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json [junit4] 2> 1596713 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json [junit4] 2> 1596713 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json [junit4] 2> 1596713 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json [junit4] 2> 1596713 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json [junit4] 2> 1596714 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json [junit4] 2> 1596715 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1596715 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1596715 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1596715 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1596715 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1596715 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1596716 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1596716 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1596717 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1596718 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1596718 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1596723 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 1596723 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 1596723 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 1596724 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 1596724 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 1596728 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 1596730 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 1596730 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 1596731 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 1596731 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 1596731 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 1596731 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 1596733 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1596733 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1596733 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1596733 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96598741040824332-127.0.0.1:41141_solr-n_0000000000 [junit4] 2> 1596734 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96598741040824331-127.0.0.1:33541_solr-n_0000000001 [junit4] 2> 1596734 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1596734 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96598741040824338-127.0.0.1:54209_solr-n_0000000002 [junit4] 2> 1596734 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1596734 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41141_solr [junit4] 2> 1596734 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader [junit4] 2> 1596734 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1596735 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96598741040824335-127.0.0.1:35634_solr-n_0000000003 [junit4] 2> 1596735 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96598741040824334-127.0.0.1:58944_solr-n_0000000004 [junit4] 2> 1596736 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96598741040824337-127.0.0.1:48178_solr-n_0000000005 [junit4] 2> 1596736 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96598741040824331-127.0.0.1:33541_solr-n_0000000001 to know if I could be the leader [junit4] 2> 1596736 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96598741040824332-127.0.0.1:41141_solr-n_0000000000 to know if I could be the leader [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54209_solr [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54209_solr [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96598741040824338-127.0.0.1:54209_solr-n_0000000002 to know if I could be the leader [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96598741040824335-127.0.0.1:35634_solr-n_0000000003 to know if I could be the leader [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33541_solr [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33541_solr [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96598741040824334-127.0.0.1:58944_solr-n_0000000004 to know if I could be the leader [junit4] 2> 1596737 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35634_solr [junit4] 2> 1596738 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35634_solr [junit4] 2> 1596738 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58944_solr [junit4] 2> 1596738 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58944_solr [junit4] 2> 1596738 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.Overseer Overseer (id=96598741040824332-127.0.0.1:41141_solr-n_0000000000) starting [junit4] 2> 1596738 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48178_solr [junit4] 2> 1596738 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48178_solr [junit4] 2> 1596739 INFO (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0]) [junit4] 2> 1596739 INFO (zkCallback-1273-thread-1-processing-n:127.0.0.1:58944_solr) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0]) [junit4] 2> 1596739 INFO (zkCallback-1270-thread-1-processing-n:127.0.0.1:33541_solr) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0]) [junit4] 2> 1596740 INFO (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0]) [junit4] 2> 1596740 INFO (zkCallback-1280-thread-1-processing-n:127.0.0.1:48178_solr) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0]) [junit4] 2> 1596740 INFO (zkCallback-1281-thread-1-processing-n:127.0.0.1:54209_solr) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0]) [junit4] 2> 1596741 INFO (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) [junit4] 2> 1596746 INFO (zkCallback-1273-thread-1-processing-n:127.0.0.1:58944_solr) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) [junit4] 2> 1596747 INFO (zkCallback-1270-thread-1-processing-n:127.0.0.1:33541_solr) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) [junit4] 2> 1596748 INFO (zkCallback-1281-thread-1-processing-n:127.0.0.1:54209_solr) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) [junit4] 2> 1596753 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1596753 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1596754 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node1/. [junit4] 2> 1596754 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions [junit4] 2> 1596755 INFO (zkCallback-1281-thread-1-processing-n:127.0.0.1:54209_solr) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3]) [junit4] 2> 1596755 INFO (zkCallback-1273-thread-1-processing-n:127.0.0.1:58944_solr) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3]) [junit4] 2> 1596756 INFO (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3]) [junit4] 2> 1596780 INFO (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) [junit4] 2> 1596780 INFO (zkCallback-1280-thread-1-processing-n:127.0.0.1:48178_solr) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) [junit4] 2> 1596792 INFO (zkCallback-1280-thread-2-processing-n:127.0.0.1:48178_solr) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [4]) [junit4] 2> 1596813 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1596813 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1596813 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node4/. [junit4] 2> 1596813 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions [junit4] 2> 1596833 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1596833 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1596833 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node5/. [junit4] 2> 1596833 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions [junit4] 2> 1596900 INFO (zkCallback-1270-thread-1-processing-n:127.0.0.1:33541_solr) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3]) [junit4] 2> 1596900 INFO (zkCallback-1280-thread-2-processing-n:127.0.0.1:48178_solr) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) [junit4] 2> 1596900 INFO (zkCallback-1281-thread-1-processing-n:127.0.0.1:54209_solr) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) [junit4] 2> 1596900 INFO (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [4]) [junit4] 2> 1596901 INFO (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) [junit4] 2> 1596901 INFO (zkCallback-1273-thread-1-processing-n:127.0.0.1:58944_solr) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) [junit4] 2> 1596904 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0 [junit4] 2> 1596904 INFO (jetty-launcher-1250-thread-5) [n:127.0.0.1:33541_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1596904 INFO (zkCallback-1270-thread-1-processing-n:127.0.0.1:33541_solr) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) [junit4] 2> 1596908 INFO (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) [junit4] 2> 1596908 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0 [junit4] 2> 1596908 INFO (jetty-launcher-1250-thread-4) [n:127.0.0.1:35634_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1596914 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1596914 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1596916 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work [junit4] 2> 1596925 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node2/. [junit4] 2> 1596928 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0 [junit4] 2> 1596928 INFO (jetty-launcher-1250-thread-1) [n:127.0.0.1:54209_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1596936 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions [junit4] 2> 1596937 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1596937 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1596944 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node3/. [junit4] 2> 1596944 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions [junit4] 2> 1596945 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0 [junit4] 2> 1596945 INFO (jetty-launcher-1250-thread-2) [n:127.0.0.1:58944_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1596952 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0 [junit4] 2> 1596952 INFO (jetty-launcher-1250-thread-3) [n:127.0.0.1:48178_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1596960 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 1596961 INFO (OverseerCollectionConfigSetProcessor-96598741040824332-127.0.0.1:41141_solr-n_0000000000) [n:127.0.0.1:41141_solr ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations [junit4] 2> 1596962 INFO (OverseerStateUpdate-96598741040824332-127.0.0.1:41141_solr-n_0000000000) [n:127.0.0.1:41141_solr ] o.a.s.c.Overseer Starting to work on the main queue [junit4] 2> 1596962 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:41141_solr as DOWN [junit4] 2> 1597048 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41141_solr [junit4] 2> 1597048 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41141_solr [junit4] 2> 1597049 INFO (OverseerStateUpdate-96598741040824332-127.0.0.1:41141_solr-n_0000000000) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (5) [junit4] 2> 1597049 INFO (zkCallback-1270-thread-1-processing-n:127.0.0.1:33541_solr) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [5]) [junit4] 2> 1597049 INFO (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [5]) [junit4] 2> 1597049 INFO (zkCallback-1273-thread-1-processing-n:127.0.0.1:58944_solr) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [5]) [junit4] 2> 1597050 INFO (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [5]) [junit4] 2> 1597050 INFO (zkCallback-1281-thread-1-processing-n:127.0.0.1:54209_solr) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [5]) [junit4] 2> 1597050 INFO (zkCallback-1280-thread-2-processing-n:127.0.0.1:48178_solr) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [5]) [junit4] 2> 1597050 INFO (zkCallback-1281-thread-1-processing-n:127.0.0.1:54209_solr) [n:127.0.0.1:54209_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) [junit4] 2> 1597050 INFO (zkCallback-1273-thread-1-processing-n:127.0.0.1:58944_solr) [n:127.0.0.1:58944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) [junit4] 2> 1597050 INFO (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) [junit4] 2> 1597051 INFO (zkCallback-1280-thread-2-processing-n:127.0.0.1:48178_solr) [n:127.0.0.1:48178_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) [junit4] 2> 1597052 INFO (zkCallback-1270-thread-1-processing-n:127.0.0.1:33541_solr) [n:127.0.0.1:33541_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) [junit4] 2> 1597065 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1597066 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1597066 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0/temp/solr.search.join.BlockJoinFacetDistribTest_CBAC70BF30C091A6-001/tempDir-001/node6/. [junit4] 2> 1597066 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions [junit4] 2> 1597067 INFO (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) [junit4] 2> 1597092 INFO (OverseerStateUpdate-96598741040824332-127.0.0.1:41141_solr-n_0000000000) [n:127.0.0.1:41141_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "operation":"downnode", [junit4] 2> "node_name":"127.0.0.1:41141_solr"} current state version: 0 [junit4] 2> 1597092 INFO (OverseerStateUpdate-96598741040824332-127.0.0.1:41141_solr-n_0000000000) [n:127.0.0.1:41141_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:41141_solr [junit4] 2> 1597120 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J0 [junit4] 2> 1597120 INFO (jetty-launcher-1250-thread-6) [n:127.0.0.1:41141_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1597120 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1597128 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1597140 INFO (zkCallback-1288-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d9a7399 name:ZooKeeperConnection Watcher:127.0.0.1:46345 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1597142 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1597142 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1597155 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1597157 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1597157 INFO (zkCallback-1291-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58b829ff name:ZooKeeperConnection Watcher:127.0.0.1:46345/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1597157 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1597157 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1597157 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1597158 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1597158 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6) [junit4] 2> 1597160 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-spellcheckcomponent.xml [junit4] 2> 1597166 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-null-charfilters-analyzer.xml [junit4] 2> 1597167 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/regex-boost-processor-test.txt [junit4] 2> 1597172 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-spellchecker.xml [junit4] 2> 1597173 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-schema-multiple-asterisk-copyfield-dest-should-fail-test.xml [junit4] 2> 1597175 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-docValuesFaceting.xml [junit4] 2> 1597180 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-tlog-with-delayingcomponent.xml [junit4] 2> 1597181 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-schema-currency-ft-bogus-default-code.xml [junit4] 2> 1597182 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-numeric.xml [junit4] 2> 1597184 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig_codec.xml [junit4] 2> 1597189 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-schema-multiple-asterisk-copyfield-source-should-fail-test.xml [junit4] 2> 1597190 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema_latest.xml [junit4] 2> 1597192 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-sorts.xml [junit4] 2> 1597260 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-headers.xml [junit4] 2> 1597328 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-implicitproperties.xml [junit4] 2> 1597360 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-solrconfig-bogus-scriptengine-name.xml [junit4] 2> 1597376 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-snippet-types.incl [junit4] 2> 1597381 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-parsing-update-processor-chains.xml [junit4] 2> 1597389 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-docValuesMissing.xml [junit4] 2> 1597393 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-noopregen.xml [junit4] 2> 1597397 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-one-field-no-dynamic-field-unique-key.xml [junit4] 2> 1597401 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-script-updateprocessor.xml [junit4] 2> 1597412 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-error-solrconfig.xml [junit4] 2> 1597428 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-suggestercomponent.xml [junit4] 2> 1597445 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-currency.xml [junit4] 2> 1597460 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-query-parser-init.xml [junit4] 2> 1597480 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-caching.xml [junit4] 2> 1597497 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/conditional.updateprocessor.js [junit4] 2> 1597585 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-indexconfig-mergepolicyfactory.xml [junit4] 2> 1597601 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/compoundDictionary.txt [junit4] 2> 1597617 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-tieredmergepolicyfactory.xml [junit4] 2> 1597633 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/phrasesuggest.txt [junit4] 2> 1597648 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-transformers.xml [junit4] 2> 1597661 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-schema-not-indexed-but-norms.xml [junit4] 2> 1597680 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/blendedInfixSuggest.txt [junit4] 2> 1597693 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-phrasesuggest.xml [junit4] 2> 1597712 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/hunspell-test.dic [junit4] 2> 1597732 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-HighlighterMaxOffsetTest.xml [junit4] 2> 1597748 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-rest-lucene-match-version.xml [junit4] 2> 1597768 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-functionquery.xml [junit4] 2> 1597775 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-externalversionconstraint.xml [junit4] 2> 1597776 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/bad-schema-codec-global-vs-ft-mismatch.xml [junit4] 2> 1597778 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-sim.xml [junit4] 2> 1597779 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/schema-docValuesMulti.xml [junit4] 2> 1597781 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-logmergepolicyfactory.xml [junit4] 2> 1597782 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/solrCloudCollectionConfig/solrconfig-collapseqparser.xml [junit4] 2> 1597784 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /configs [...truncated too long message...] or Update replica state for core_node2:{"core":"facetcollection_shard1_replica2","base_url":"https://127.0.0.1:33541/solr","node_name":"127.0.0.1:33541_solr","state":"down","leader":"true"} to down [junit4] 2> 1612818 INFO (jetty-launcher-1250-thread-4) [ ] o.a.s.c.Overseer Overseer (id=96598741040824337-127.0.0.1:48178_solr-n_0000000005) closing [junit4] 2> 1612819 INFO (OverseerStateUpdate-96598741040824337-127.0.0.1:48178_solr-n_0000000005) [n:127.0.0.1:48178_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:48178_solr [junit4] 2> 1612822 INFO (jetty-launcher-1250-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4f72a7b8{/solr,null,UNAVAILABLE} [junit4] 2> 1613680 WARN (zkCallback-1275-thread-1-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /collections/facetcollection/state.json] [junit4] 2> 1613684 WARN (zkCallback-1275-thread-5-processing-n:127.0.0.1:41141_solr) [n:127.0.0.1:41141_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes] [junit4] 2> 1613696 INFO (jetty-launcher-1250-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@768f7840{/solr,null,UNAVAILABLE} [junit4] 2> 1614124 WARN (zkCallback-1276-thread-1-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /collections/facetcollection/state.json] [junit4] 2> 1614129 WARN (zkCallback-1276-thread-3-processing-n:127.0.0.1:35634_solr) [n:127.0.0.1:35634_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes] [junit4] 2> 1614132 INFO (jetty-launcher-1250-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5331eafa{/solr,null,UNAVAILABLE} [junit4] 2> 1614137 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46345 46345 [junit4] 2> 1614304 INFO (Thread-3287) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46345 46345 [junit4] 2> 1614308 WARN (Thread-3287) [ ] o.a.s.c.ZkTestServer Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 7 /solr/aliases.json [junit4] 2> 6 /solr/security.json [junit4] 2> 6 /solr/configs/solrCloudCollectionConfig [junit4] 2> [junit4] 2> Maximum concurrent data watches above limit: [junit4] 2> [junit4] 2> 7 /solr/clusterstate.json [junit4] 2> 7 /solr/clusterprops.json [junit4] 2> 6 /solr/collections/facetcollection/state.json [junit4] 2> 2 /solr/overseer_elect/election/96598741040824331-127.0.0.1:33541_solr-n_0000000001 [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 18 /solr/overseer/queue [junit4] 2> 13 /solr/overseer/collection-queue-work [junit4] 2> 7 /solr/live_nodes [junit4] 2> 7 /solr/collections [junit4] 2> 4 /solr/overseer/queue-work [junit4] 2> [junit4] 2> 1614312 INFO (SUITE-BlockJoinFacetDistribTest-seed#[CBAC70BF30C091A6]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore [junit4] 2> Sep 15, 2016 10:50:31 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> WARNING: Will linger awaiting termination of 3 leaked thread(s). [junit4] 2> 1614660 WARN (OverseerCollectionConfigSetProcessor-96598741040824334-127.0.0.1:58944_solr-n_0000000004) [n:127.0.0.1:58944_solr ] o.a.s.c.OverseerTaskProcessor Overseer cannot talk to ZK [junit4] 2> Sep 15, 2016 10:50:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.search.join.BlockJoinFacetDistribTest: [junit4] 2> 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] [junit4] 2> at java.lang.Thread.sleep(Native Method) [junit4] 2> at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> Sep 15, 2016 10:50:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll [junit4] 2> INFO: Starting to interrupt leaked threads: [junit4] 2> 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] [junit4] 2> Sep 15, 2016 10:50:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll [junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated: [junit4] 2> 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] [junit4] 2> at java.lang.Thread.sleep(Native Method) [junit4] 2> at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=1, maxDocsPerChunk=5, blockSize=677), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=1, blockSize=677)), sim=ClassicSimilarity, locale=sq, timezone=Europe/London [junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=96053408,total=519045120 [junit4] 2> NOTE: All tests run in this JVM: [SortSpecParsingTest, SignatureUpdateProcessorFactoryTest, TestInfoStreamLogging, SolrCoreTest, DocumentAnalysisRequestHandlerTest, TestSolrCloudWithDelegationTokens, XsltUpdateRequestHandlerTest, UUIDFieldTest, TestNumericTerms64, TestClassicSimilarityFactory, TestStressRecovery, TestCloudPseudoReturnFields, SaslZkACLProviderTest, TestCodecSupport, TestSolrCoreProperties, GraphQueryTest, CdcrBootstrapTest, StandardRequestHandlerTest, TestFieldCacheSort, HdfsRecoverLeaseTest, TestPerFieldSimilarityClassic, TestBinaryResponseWriter, TestCloudSchemaless, OutputWriterTest, TestNamedUpdateProcessors, TestManagedResource, TestHighlightDedupGrouping, ConnectionManagerTest, TestReplicationHandler, CurrencyFieldOpenExchangeTest, TestSolrDeletionPolicy2, TestDocBasedVersionConstraints, TestFastLRUCache, BlobRepositoryCloudTest, HdfsSyncSliceTest, UniqFieldsUpdateProcessorFactoryTest, RestartWhileUpdatingTest, TestOrdValues, TestDocSet, TestSSLRandomization, TestSolrConfigHandlerConcurrent, TestConfigOverlay, TestFieldCollectionResource, DistributedQueueTest, TestSolrDynamicMBean, ReturnFieldsTest, TestSolr4Spatial2, NumericFieldsTest, PolyFieldTest, HttpPartitionTest, QueryEqualityTest, TestCloudDeleteByQuery, TestJsonRequest, LeaderElectionIntegrationTest, FastVectorHighlighterTest, ActionThrottleTest, ScriptEngineTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, RequestLoggingTest, TestMacros, TestIBSimilarityFactory, TestExactStatsCache, TestFieldTypeResource, TestFoldingMultitermQuery, TestGraphMLResponseWriter, TestConfigSets, HighlighterConfigTest, BasicDistributedZkTest, TestBulkSchemaConcurrent, TestHdfsBackupRestoreCore, VersionInfoTest, CoreSorterTest, PKIAuthenticationIntegrationTest, CheckHdfsIndexTest, TestValueSourceCache, TestJoin, OverseerTaskQueueTest, TestFileDictionaryLookup, TestFieldSortValues, SynonymTokenizerTest, AlternateDirectoryTest, TestMissingGroups, TestReplicaProperties, BlockJoinFacetDistribTest] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BlockJoinFacetDistribTest -Dtests.seed=CBAC70BF30C091A6 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sq -Dtests.timezone=Europe/London -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.00s J0 | BlockJoinFacetDistribTest (suite) <<< [junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.search.join.BlockJoinFacetDistribTest: [junit4] > 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > at __randomizedtesting.SeedInfo.seed([CBAC70BF30C091A6]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: [junit4] > 1) Thread[id=9448, name=OverseerHdfsCoreFailoverThread-96598741040824334-127.0.0.1:58944_solr-n_0000000004, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > at __randomizedtesting.SeedInfo.seed([CBAC70BF30C091A6]:0) [junit4] Completed [303/630 (1!)] on J0 in 41.42s, 1 test, 2 errors <<< FAILURES! [...truncated 61249 lines...]
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org