Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/734/ Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed. FAILED: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir Error Message: Captured an uncaught exception in thread: Thread[id=4037, name=cdcr-replicator-1532-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest] Stack Trace: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4037, name=cdcr-replicator-1532-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest] Caused by: java.lang.AssertionError: 1608410588425224192 != 1608410588418932736 at __randomizedtesting.SeedInfo.seed([67529FD0B46CDB3E]:0) at org.apache.solr.update.CdcrUpdateLog$CdcrLogReader.forwardSeek(CdcrUpdateLog.java:611) at org.apache.solr.handler.CdcrReplicator.run(CdcrReplicator.java:125) at org.apache.solr.handler.CdcrReplicatorScheduler.lambda$start$0(CdcrReplicatorScheduler.java:81) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base/java.lang.Thread.run(Thread.java:844) Build Log: [...truncated 12629 lines...] [junit4] Suite: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest [junit4] 2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\init-core-data-001 [junit4] 2> 304299 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster2-001 [junit4] 2> 304299 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 304300 INFO (Thread-527) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 304300 INFO (Thread-527) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 304303 ERROR (Thread-527) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes [junit4] 2> 304400 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.ZkTestServer start zk server on port:59362 [junit4] 2> 304403 INFO (zkConnectionManagerCallback-760-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304407 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100105a0dd30000, likely client has closed socket [junit4] 2> 304412 INFO (jetty-launcher-757-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11 [junit4] 2> 304413 INFO (jetty-launcher-757-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 304413 INFO (jetty-launcher-757-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 304413 INFO (jetty-launcher-757-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms [junit4] 2> 304413 INFO (jetty-launcher-757-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30c2777f{/solr,null,AVAILABLE} [junit4] 2> 304419 INFO (jetty-launcher-757-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@68a0016e{SSL,[ssl, http/1.1]}{127.0.0.1:59366} [junit4] 2> 304419 INFO (jetty-launcher-757-thread-1) [ ] o.e.j.s.Server Started @304476ms [junit4] 2> 304419 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59366} [junit4] 2> 304419 ERROR (jetty-launcher-757-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 304419 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory [junit4] 2> 304420 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0 [junit4] 2> 304420 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 304420 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 304420 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-10T11:17:16.648184Z [junit4] 2> 304424 INFO (zkConnectionManagerCallback-762-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304424 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 304430 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100105a0dd30001, likely client has closed socket [junit4] 2> 304586 INFO (jetty-launcher-757-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59362/solr [junit4] 2> 304589 INFO (zkConnectionManagerCallback-766-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304592 INFO (zkConnectionManagerCallback-768-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304691 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59366_solr [junit4] 2> 304692 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.c.Overseer Overseer (id=72075573002960899-127.0.0.1:59366_solr-n_0000000000) starting [junit4] 2> 304700 INFO (zkConnectionManagerCallback-775-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304702 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59362/solr ready [junit4] 2> 304703 INFO (OverseerStateUpdate-72075573002960899-127.0.0.1:59366_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59366_solr [junit4] 2> 304703 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59366_solr [junit4] 2> 304709 INFO (OverseerStateUpdate-72075573002960899-127.0.0.1:59366_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 304716 INFO (zkCallback-774-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 304736 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 304782 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59366.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 304786 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59366.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 304786 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59366.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 304787 INFO (jetty-launcher-757-thread-1) [n:127.0.0.1:59366_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster2-001\node1\. [junit4] 2> 304807 INFO (zkConnectionManagerCallback-778-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304808 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100105a0dd30005, likely client has closed socket [junit4] 2> 304811 INFO (zkConnectionManagerCallback-781-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304812 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100105a0dd30006, likely client has closed socket [junit4] 2> 304813 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster1-001 [junit4] 2> 304814 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 304814 INFO (Thread-537) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 304814 INFO (Thread-537) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 304818 ERROR (Thread-537) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes [junit4] 2> 304914 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.ZkTestServer start zk server on port:59395 [junit4] 2> 304918 INFO (zkConnectionManagerCallback-785-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304930 INFO (jetty-launcher-782-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9.0.4+11 [junit4] 2> 304931 INFO (jetty-launcher-782-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0 [junit4] 2> 304931 INFO (jetty-launcher-782-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults [junit4] 2> 304931 INFO (jetty-launcher-782-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms [junit4] 2> 304932 INFO (jetty-launcher-782-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16ec127b{/solr,null,AVAILABLE} [junit4] 2> 304934 INFO (jetty-launcher-782-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6c28516b{SSL,[ssl, http/1.1]}{127.0.0.1:59408} [junit4] 2> 304934 INFO (jetty-launcher-782-thread-1) [ ] o.e.j.s.Server Started @304992ms [junit4] 2> 304934 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59408} [junit4] 2> 304934 ERROR (jetty-launcher-782-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. [junit4] 2> 304934 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory [junit4] 2> 304934 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0 [junit4] 2> 304934 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null [junit4] 2> 304935 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null [junit4] 2> 304935 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-08-10T11:17:17.163784200Z [junit4] 2> 304942 INFO (zkConnectionManagerCallback-787-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 304943 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading... [junit4] 2> 305187 INFO (jetty-launcher-782-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59395/solr [junit4] 2> 305189 INFO (zkConnectionManagerCallback-791-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 305192 INFO (zkConnectionManagerCallback-793-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 305302 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59408_solr [junit4] 2> 305303 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.c.Overseer Overseer (id=72075573036646403-127.0.0.1:59408_solr-n_0000000000) starting [junit4] 2> 305312 INFO (zkConnectionManagerCallback-800-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 305315 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59395/solr ready [junit4] 2> 305315 INFO (OverseerStateUpdate-72075573036646403-127.0.0.1:59408_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59408_solr [junit4] 2> 305316 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59408_solr [junit4] 2> 305317 INFO (OverseerStateUpdate-72075573036646403-127.0.0.1:59408_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 305319 INFO (zkCallback-799-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 305340 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory. [junit4] 2> 305356 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59408.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 305361 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59408.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 305361 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59408.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 305362 INFO (jetty-launcher-782-thread-1) [n:127.0.0.1:59408_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster1-001\node1\. [junit4] 2> 305384 INFO (zkConnectionManagerCallback-803-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 305389 INFO (zkConnectionManagerCallback-806-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 305390 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.c.CdcrBidirectionalTest cluster2 zkHost = 127.0.0.1:59362/solr [junit4] 2> 305390 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.c.CdcrBidirectionalTest cluster1 zkHost = 127.0.0.1:59395/solr [junit4] 2> 305393 INFO (zkConnectionManagerCallback-808-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 305400 INFO (zkConnectionManagerCallback-812-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 305401 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 305402 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59395/solr ready [junit4] 2> 305423 INFO (qtp799279249-3904) [n:127.0.0.1:59408_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 305426 INFO (OverseerThreadFactory-1493-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster1 [junit4] 2> 305535 INFO (OverseerStateUpdate-72075573036646403-127.0.0.1:59408_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() { [junit4] 2> "operation":"ADDREPLICA", [junit4] 2> "collection":"cdcr-cluster1", [junit4] 2> "shard":"shard1", [junit4] 2> "core":"cdcr-cluster1_shard1_replica_n1", [junit4] 2> "state":"down", [junit4] 2> "base_url":"https://127.0.0.1:59408/solr", [junit4] 2> "type":"NRT", [junit4] 2> "waitForFinalState":"false"} [junit4] 2> 305538 INFO (OverseerStateUpdate-72075573036646403-127.0.0.1:59408_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() { [junit4] 2> "operation":"ADDREPLICA", [junit4] 2> "collection":"cdcr-cluster1", [junit4] 2> "shard":"shard2", [junit4] 2> "core":"cdcr-cluster1_shard2_replica_n2", [junit4] 2> "state":"down", [junit4] 2> "base_url":"https://127.0.0.1:59408/solr", [junit4] 2> "type":"NRT", [junit4] 2> "waitForFinalState":"false"} [junit4] 2> 305750 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin [junit4] 2> 305751 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 305754 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin [junit4] 2> 306790 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0 [junit4] 2> 306790 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0 [junit4] 2> 306797 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster1_shard1_replica_n1] Schema name=minimal [junit4] 2> 306797 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema [cdcr-cluster1_shard2_replica_n2] Schema name=minimal [junit4] 2> 306798 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id [junit4] 2> 306798 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id [junit4] 2> 306798 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard1_replica_n1' using configuration from collection cdcr-cluster1, trusted=true [junit4] 2> 306798 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard2_replica_n2' using configuration from collection cdcr-cluster1, trusted=true [junit4] 2> 306799 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59408.solr.core.cdcr-cluster1.shard1.replica_n1' (registry 'solr.core.cdcr-cluster1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 306799 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59408.solr.core.cdcr-cluster1.shard2.replica_n2' (registry 'solr.core.cdcr-cluster1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 306802 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder [junit4] 2> 306802 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster1_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster1-001\node1\cdcr-cluster1_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster1-001\node1\.\cdcr-cluster1_shard1_replica_n1\data\] [junit4] 2> 306807 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder [junit4] 2> 306807 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [[cdcr-cluster1_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster1-001\node1\cdcr-cluster1_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster1-001\node1\.\cdcr-cluster1_shard2_replica_n2\data\] [junit4] 2> 306993 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog [junit4] 2> 306993 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 306995 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 306995 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 307000 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6c20aa1c[cdcr-cluster1_shard2_replica_n2] main] [junit4] 2> 307001 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1 [junit4] 2> 307001 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1 [junit4] 2> 307001 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema [junit4] 2> 307001 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest [junit4] 2> 307002 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog [junit4] 2> 307002 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 307002 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms. [junit4] 2> 307004 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 307004 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 307007 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster1/cdcr/state/buffer [junit4] 2> 307007 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3dca9d43[cdcr-cluster1_shard1_replica_n1] main] [junit4] 2> 307007 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1 [junit4] 2> 307007 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1 [junit4] 2> 307007 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema [junit4] 2> 307011 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest [junit4] 2> 307012 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms. [junit4] 2> 307014 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster1/cdcr/state/process [junit4] 2> 307040 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1608410557860282368 [junit4] 2> 307041 INFO (searcherExecutor-1499-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster1_shard2_replica_n2] Registered new searcher Searcher@6c20aa1c[cdcr-cluster1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 307041 INFO (searcherExecutor-1498-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster1_shard1_replica_n1] Registered new searcher Searcher@3dca9d43[cdcr-cluster1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 307040 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1608410557860282368 [junit4] 2> 307050 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=0}, version=0} [junit4] 2> 307053 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=0}, version=0} [junit4] 2> 307059 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. [junit4] 2> 307059 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync [junit4] 2> 307059 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:59408/solr/cdcr-cluster1_shard2_replica_n2/ [junit4] 2> 307059 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me [junit4] 2> 307060 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. [junit4] 2> 307060 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync [junit4] 2> 307060 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:59408/solr/cdcr-cluster1_shard1_replica_n1/ [junit4] 2> 307060 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:59408/solr/cdcr-cluster1_shard2_replica_n2/ has no replicas [junit4] 2> 307060 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR [junit4] 2> 307060 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me [junit4] 2> 307060 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:59408/solr/cdcr-cluster1_shard1_replica_n1/ has no replicas [junit4] 2> 307060 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR [junit4] 2> 307064 INFO (zkCallback-792-thread-1) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard2 [junit4] 2> 307065 INFO (zkCallback-792-thread-2) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard1 [junit4] 2> 307068 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:59408/solr/cdcr-cluster1_shard2_replica_n2/ shard2 [junit4] 2> 307068 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:59408/solr/cdcr-cluster1_shard1_replica_n1/ shard1 [junit4] 2> 307171 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary [junit4] 2> 307174 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1424 [junit4] 2> 307220 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary [junit4] 2> 307223 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1469 [junit4] 2> 307226 INFO (qtp799279249-3904) [n:127.0.0.1:59408_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas [junit4] 2> 307323 INFO (zkCallback-792-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/state.json] for collection [cdcr-cluster1] has occurred - updating... (live nodes size: [1]) [junit4] 2> 307429 INFO (OverseerCollectionConfigSetProcessor-72075573036646403-127.0.0.1:59408_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper [junit4] 2> 308227 INFO (qtp799279249-3904) [n:127.0.0.1:59408_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2803 [junit4] 2> 308230 INFO (zkConnectionManagerCallback-816-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 308239 INFO (zkConnectionManagerCallback-820-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 308240 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 308241 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59362/solr ready [junit4] 2> 308279 INFO (qtp947469995-3843) [n:127.0.0.1:59366_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true [junit4] 2> 308284 INFO (OverseerThreadFactory-1480-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster2 [junit4] 2> 308396 INFO (OverseerStateUpdate-72075573002960899-127.0.0.1:59366_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() { [junit4] 2> "operation":"ADDREPLICA", [junit4] 2> "collection":"cdcr-cluster2", [junit4] 2> "shard":"shard1", [junit4] 2> "core":"cdcr-cluster2_shard1_replica_n1", [junit4] 2> "state":"down", [junit4] 2> "base_url":"https://127.0.0.1:59366/solr", [junit4] 2> "type":"NRT", [junit4] 2> "waitForFinalState":"false"} [junit4] 2> 308397 INFO (OverseerStateUpdate-72075573002960899-127.0.0.1:59366_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() { [junit4] 2> "operation":"ADDREPLICA", [junit4] 2> "collection":"cdcr-cluster2", [junit4] 2> "shard":"shard2", [junit4] 2> "core":"cdcr-cluster2_shard2_replica_n2", [junit4] 2> "state":"down", [junit4] 2> "base_url":"https://127.0.0.1:59366/solr", [junit4] 2> "type":"NRT", [junit4] 2> "waitForFinalState":"false"} [junit4] 2> 308677 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin [junit4] 2> 308677 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores [junit4] 2> 308691 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin [junit4] 2> 309711 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0 [junit4] 2> 309711 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0 [junit4] 2> 309718 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster2_shard1_replica_n1] Schema name=minimal [junit4] 2> 309718 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.IndexSchema [cdcr-cluster2_shard2_replica_n2] Schema name=minimal [junit4] 2> 309733 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id [junit4] 2> 309733 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id [junit4] 2> 309733 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard1_replica_n1' using configuration from collection cdcr-cluster2, trusted=true [junit4] 2> 309733 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard2_replica_n2' using configuration from collection cdcr-cluster2, trusted=true [junit4] 2> 309733 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59366.solr.core.cdcr-cluster2.shard2.replica_n2' (registry 'solr.core.cdcr-cluster2.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 309733 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59366.solr.core.cdcr-cluster2.shard1.replica_n1' (registry 'solr.core.cdcr-cluster2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2133ee67 [junit4] 2> 309733 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder [junit4] 2> 309733 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder [junit4] 2> 309733 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [[cdcr-cluster2_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster2-001\node1\cdcr-cluster2_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster2-001\node1\.\cdcr-cluster2_shard2_replica_n2\data\] [junit4] 2> 309733 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster2_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster2-001\node1\cdcr-cluster2_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_67529FD0B46CDB3E-001\cdcr-cluster2-001\node1\.\cdcr-cluster2_shard1_replica_n1\data\] [junit4] 2> 309874 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog [junit4] 2> 309874 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 309875 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog [junit4] 2> 309875 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 309876 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 309876 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 309877 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled [junit4] 2> 309877 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 309879 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@53763f2e[cdcr-cluster2_shard1_replica_n1] main] [junit4] 2> 309880 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@387df205[cdcr-cluster2_shard2_replica_n2] main] [junit4] 2> 309881 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2 [junit4] 2> 309881 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2 [junit4] 2> 309881 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema [junit4] 2> 309881 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2 [junit4] 2> 309882 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2 [junit4] 2> 309882 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest [junit4] 2> 309882 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema [junit4] 2> 309882 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest [junit4] 2> 309883 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms. [junit4] 2> 309883 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms. [junit4] 2> 309886 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster2/cdcr/state/buffer [junit4] 2> 309887 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster2/cdcr/state/process [junit4] 2> 309890 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1608410560848723968 [junit4] 2> 309890 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1608410560848723968 [junit4] 2> 309891 INFO (searcherExecutor-1510-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster2_shard1_replica_n1] Registered new searcher Searcher@53763f2e[cdcr-cluster2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 309891 INFO (searcherExecutor-1511-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster2_shard2_replica_n2] Registered new searcher Searcher@387df205[cdcr-cluster2_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 309898 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard1 to Terms{values={core_node3=0}, version=0} [junit4] 2> 309900 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard2 to Terms{values={core_node4=0}, version=0} [junit4] 2> 309903 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. [junit4] 2> 309903 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync [junit4] 2> 309903 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:59366/solr/cdcr-cluster2_shard1_replica_n1/ [junit4] 2> 309904 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me [junit4] 2> 309904 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:59366/solr/cdcr-cluster2_shard1_replica_n1/ has no replicas [junit4] 2> 309904 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR [junit4] 2> 309905 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. [junit4] 2> 309905 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync [junit4] 2> 309905 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:59366/solr/cdcr-cluster2_shard2_replica_n2/ [junit4] 2> 309906 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me [junit4] 2> 309906 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:59366/solr/cdcr-cluster2_shard2_replica_n2/ has no replicas [junit4] 2> 309906 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR [junit4] 2> 309908 INFO (zkCallback-767-thread-1) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard1 [junit4] 2> 309912 INFO (zkCallback-767-thread-1) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard2 [junit4] 2> 309915 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:59366/solr/cdcr-cluster2_shard1_replica_n1/ shard1 [junit4] 2> 309916 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:59366/solr/cdcr-cluster2_shard2_replica_n2/ shard2 [junit4] 2> 310068 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary [junit4] 2> 310083 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary [junit4] 2> 310086 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1410 [junit4] 2> 310087 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1397 [junit4] 2> 310091 INFO (qtp947469995-3843) [n:127.0.0.1:59366_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas [junit4] 2> 310188 INFO (zkCallback-767-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster2/state.json] for collection [cdcr-cluster2] has occurred - updating... (live nodes size: [1]) [junit4] 2> 310286 INFO (OverseerCollectionConfigSetProcessor-72075573002960899-127.0.0.1:59366_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper [junit4] 2> 311092 INFO (qtp947469995-3843) [n:127.0.0.1:59366_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2812 [junit4] 2> 311101 INFO (zkConnectionManagerCallback-827-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 311102 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 311103 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59362/solr ready [junit4] 2> 311118 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 311118 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 311126 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 311126 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 311127 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=18 [junit4] 2> 311128 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard1 [junit4] 2> 311128 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2, shard: shard1 [junit4] 2> 311130 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor [junit4] 2> 311131 INFO (cdcr-bootstrap-status-823-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2 shard: shard1 leader: https://127.0.0.1:59366/solr/cdcr-cluster2_shard1_replica_n1/ [junit4] 2> 311133 INFO (zkCallback-792-thread-2) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard2 [junit4] 2> 311133 INFO (zkCallback-792-thread-1) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard1 [junit4] 2> 311134 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-cluster1:4&action=start&wt=javabin&version=2} status=0 QTime=37 [junit4] 2> 311136 INFO (zkCallback-792-thread-2) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard2 [junit4] 2> 311136 INFO (zkCallback-792-thread-1) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard1 [junit4] 2> 311137 INFO (qtp947469995-3846) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=https://127.0.0.1:59408/solr/cdcr-cluster1_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=1 [junit4] 2> 311138 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 311139 INFO (cdcr-bootstrap-status-823-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms [junit4] 2> 311140 INFO (zkConnectionManagerCallback-832-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected [junit4] 2> 311141 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} [junit4] 2> 311141 INFO (zkCallback-792-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 311143 INFO (zkCallback-792-thread-2) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59362/solr ready [junit4] 2> 311151 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1608410562170978304,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 311152 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit. [junit4] 2> 311155 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush [junit4] 2> 311155 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=7 [junit4] 2> 311178 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0 [junit4] 2> 311179 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's generation: 1 [junit4] 2> 311179 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's version: 0 [junit4] 2> 311179 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's generation: 1 [junit4] 2> 311179 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's version: 0 [junit4] 2> 311179 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher New index in Master. Deleting mine... [junit4] 2> 311184 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a65086a[cdcr-cluster2_shard1_replica_n1] main] [junit4] 2> 311186 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 311186 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 311187 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 311187 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=13 [junit4] 2> 311189 INFO (searcherExecutor-1510-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster2_shard1_replica_n1] Registered new searcher Searcher@4a65086a[cdcr-cluster2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 311189 INFO (qtp947469995-3845) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=31 [junit4] 2> 311190 INFO (zkCallback-792-thread-2) [ ] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard2 [junit4] 2> 311190 INFO (zkCallback-792-thread-2) [ ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2, shard: shard2 [junit4] 2> 311190 INFO (zkCallback-792-thread-2) [ ] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor [junit4] 2> 311190 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler No replay needed. [junit4] 2> 311191 INFO (cdcr-bootstrap-status-828-thread-1) [ ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2 shard: shard2 leader: https://127.0.0.1:59366/solr/cdcr-cluster2_shard2_replica_n2/ [junit4] 2> 311194 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=https://127.0.0.1:59408/solr/cdcr-cluster1_shard2_replica_n2/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 311194 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} [junit4] 2> 311196 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 311196 INFO (cdcr-bootstrap-status-828-thread-1) [ ] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms [junit4] 2> 311201 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1608410562223407104,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 311202 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit. [junit4] 2> 311203 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush [junit4] 2> 311203 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=2 [junit4] 2> 311205 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0 [junit4] 2> 311206 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Master's generation: 1 [junit4] 2> 311206 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Master's version: 0 [junit4] 2> 311206 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Slave's generation: 1 [junit4] 2> 311206 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Slave's version: 0 [junit4] 2> 311206 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher New index in Master. Deleting mine... [junit4] 2> 311208 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7bf4c0db[cdcr-cluster2_shard2_replica_n2] main] [junit4] 2> 311209 INFO (searcherExecutor-1511-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster2_shard2_replica_n2] Registered new searcher Searcher@7bf4c0db[cdcr-cluster2_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 311209 INFO (recoveryExecutor-764-thread-1-processing-n:127.0.0.1:59366_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler No replay needed. [junit4] 2> 313135 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.c.CdcrBidirectionalTest Adding 10 docs with commit=true, numDocs=100 [junit4] 2> 313142 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 313145 INFO (cdcr-bootstrap-status-823-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap successful in 3 seconds [junit4] 2> 313158 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=1}, version=1} [junit4] 2> 313159 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=20 [junit4] 2> 313160 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=1}, version=1} [junit4] 2> 313160 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=12 [junit4] 2> 313171 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 313171 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 313177 INFO (qtp947469995-3846) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 313177 INFO (qtp947469995-3846) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 313179 INFO (qtp947469995-3848) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=31 [junit4] 2> 313180 INFO (cdcr-bootstrap-status-823-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard1 [junit4] 2> 313181 INFO (cdcr-bootstrap-status-823-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Bootstrap successful, giving the go-ahead to replicator [junit4] 2> 313191 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1608410564310073344,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 313191 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5f42a1a2 commitCommandVersion:1608410564310073344 [junit4] 2> 313199 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1608410564318461952,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 313199 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@44f2f2b4 commitCommandVersion:1608410564318461952 [junit4] 2> 313203 INFO (qtp947469995-3846) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 313203 INFO (cdcr-bootstrap-status-828-thread-1) [ ] o.a.s.h.CdcrReplicatorManager CDCR bootstrap successful in 3 seconds [junit4] 2> 313219 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 313219 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 313230 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0 [junit4] 2> 313230 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0 [junit4] 2> 313231 INFO (qtp947469995-3846) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=25 [junit4] 2> 313231 INFO (cdcr-bootstrap-status-828-thread-1) [ ] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard2 [junit4] 2> 313231 INFO (cdcr-bootstrap-status-828-thread-1) [ ] o.a.s.h.CdcrReplicatorManager Bootstrap successful, giving the go-ahead to replicator [junit4] 2> 314149 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard1 to Terms{values={core_node3=1}, version=1} [junit4] 2> 314201 INFO (qtp947469995-3846) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard2 to Terms{values={core_node4=1}, version=1} [junit4] 2> 314219 INFO (qtp947469995-3846) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/update params={_stateVer_=cdcr-cluster2:4&cdcr.update=&wt=javabin&version=2} status=0 QTime=25 [junit4] 2> 314220 INFO (qtp947469995-3844) [n:127.0.0.1:59366_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-cluster2:4&cdcr.update=&wt=javabin&version=2} status=0 QTime=78 [junit4] 2> 314220 INFO (cdcr-replicator-1527-thread-1) [ ] o.a.s.h.CdcrReplicator Forwarded 57 updates to target cdcr-cluster2 [junit4] 2> 314220 INFO (cdcr-replicator-1524-thread-1) [ ] o.a.s.h.CdcrReplicator Forwarded 43 updates to target cdcr-cluster2 [junit4] 2> 314404 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5b88efa1[cdcr-cluster1_shard2_replica_n2] main] [junit4] 2> 314405 INFO (searcherExecutor-1499-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster1_shard2_replica_n2] Registered new searcher Searcher@5b88efa1[cdcr-cluster1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.5.0):C57)))} [junit4] 2> 314406 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush [junit4] 2> 314406 INFO (qtp799279249-3900) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&update.chain=cdcr-processor-chain&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:59408/solr/cdcr-cluster1_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=1215 [junit4] 2> 314407 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d242991[cdcr-cluster1_shard1_replica_n1] main] [junit4] 2> 314409 INFO (searcherExecutor-1498-thread-1-processing-n:127.0.0.1:59408_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster1_shard1_replica_n1] Registered new searcher Searcher@6d242991[cdcr-cluster1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.5.0):C43)))} [junit4] 2> 314409 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush [junit4] 2> 314410 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&update.chain=cdcr-processor-chain&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:59408/solr/cdcr-cluster1_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=1211 [junit4] 2> 314411 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update params={_stateVer_=cdcr-cluster1:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=1248 [junit4] 2> 314411 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[67529FD0B46CDB3E]) [ ] o.a.s.c.c.CdcrBidirectionalTest Adding 10 docs with commit=true, numDocs=200 [junit4] 2> 314420 INFO (qtp799279249-3898) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=7 [junit4] 2> 314463 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=47 [junit4] 2> 314497 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1608410565679513600,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 314497 INFO (qtp799279249-3899) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@44f2f2b4 commitCommandVersion:1608410565679513600 [junit4] 2> 314508 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1608410565686853632,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4] 2> 314509 INFO (qtp799279249-3901) [n:127.0.0.1:59408_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_ [...truncated too long message...] .0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-authz-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-changelog-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-collective-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-event-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-exception-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-journal-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-normalization-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-operational-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-referral-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-schema-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-subtree-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-interceptors-trigger-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-jdbm-partition-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-jdbm1-2.0.0-M2.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-kerberos-codec-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-ldif-partition-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-mavibot-partition-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-protocol-kerberos-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-protocol-ldap-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-protocol-shared-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\apacheds-xdbm-partition-2.0.0-M15.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\api-all-1.0.0-M20.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\bcprov-jdk15on-1.54.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\byte-buddy-1.6.2.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\commons-collections-3.2.2.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\commons-math3-3.6.1.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\ehcache-core-2.4.4.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\hadoop-common-2.7.4-tests.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\hadoop-hdfs-2.7.4-tests.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\hadoop-minikdc-2.7.4.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\htrace-core-3.2.0-incubating.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\jersey-core-1.9.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\jersey-server-1.9.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\jetty-6.1.26.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\jetty-sslengine-6.1.26.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\jetty-util-6.1.26.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\mina-core-2.0.0-M5.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\mockito-core-2.6.2.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\netty-all-4.0.36.Final.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\test-lib\objenesis-2.5.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\analysis\icu\lucene-analyzers-icu-7.5.0-SNAPSHOT.jar;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\contrib\solr-analysis-extras\classes\java;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\queryparser\classes\test;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\build\backward-codecs\classes\test;C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\contrib\analysis-extras\lib\icu4j-62.1.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-launcher.jar;C:\Users\jenkins\.ant\lib\ivy-2.4.0.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-antlr.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-apache-bcel.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-apache-bsf.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-apache-log4j.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-apache-oro.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-apache-regexp.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-apache-resolver.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-apache-xalan2.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-commons-logging.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-commons-net.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-jai.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-javamail.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-jdepend.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-jmf.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-jsch.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-junit.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-junit4.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-netrexx.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-swing.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant-testutil.jar;C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2\lib\ant.jar;C:\Users\jenkins\.ivy2\cache\com.carrotsearch.randomizedtesting\junit4-ant\jars\junit4-ant-2.6.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\temp\junit4-J1-20180810_111212_0627756157758630469735.events @C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\temp\junit4-J1-20180810_111212_0626588834800454581433.suites -stdin [junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\temp\junit4-J1-20180810_111212_06212484793949768514676.sysout [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979) [junit4] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [junit4] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) [junit4] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) [junit4] at java.base/java.lang.Thread.run(Thread.java:844) BUILD FAILED C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\build.xml:633: The following error occurred while executing this line: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\build.xml:577: The following error occurred while executing this line: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\build.xml:59: The following error occurred while executing this line: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build.xml:267: The following error occurred while executing this line: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\common-build.xml:556: The following error occurred while executing this line: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\common-build.xml:1568: The following error occurred while executing this line: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\common-build.xml:1092: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\temp\junit4-J1-20180810_111212_06212484793949768514676.sysout Total time: 79 minutes 0 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2 [WARNINGS] Skipping publisher since build result is FAILURE Recording test results Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2 Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2 Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2 Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2 Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org