Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11085/
Java: 32bit/jdk1.7.0_67 -client -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration
node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because
creating the ephemeral registration node in ZooKeeper failed
at
__randomizedtesting.SeedInfo.seed([6001E52C352F5EF5:64096ADF278AB1D4]:0)
at
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:150)
at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at
org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
at
org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException:
KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at
org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:462)
at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:459)
at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:416)
at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
at
org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:142)
at
org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
at
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:137)
... 47 more
Build Log:
[...truncated 11517 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.OverseerTest-6001E52C352F5EF5-001/init-core-data-001
[junit4] 2> 1281815 T4012 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl
(false) and clientAuth (true)
[junit4] 2> 1281816 T4012 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 1281816 T4012 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 1281821 T4012 oas.SolrTestCaseJ4.setUp ###Starting testReplay
[junit4] 2> 1281822 T4012 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1281823 T4013 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 1281922 T4012 oasc.ZkTestServer.run start zk server on
port:33410
[junit4] 2> 1281924 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1281926 T4019 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@14872fc name:ZooKeeperConnection
Watcher:127.0.0.1:33410/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1281926 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1281927 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1281928 T4021 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@13e368a name:ZooKeeperConnection
Watcher:127.0.0.1:33410 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1281928 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1281930 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1281931 T4023 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@11ad4bf name:ZooKeeperConnection
Watcher:127.0.0.1:33410 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1281931 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1281931 T4012 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1281934 T4012 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1281935 T4012 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 1281936 T4012 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 1281937 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1281939 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 1281941 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1281942 T4025 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@61605f name:ZooKeeperConnection
Watcher:127.0.0.1:33410/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1281942 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1281942 T4012 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1281943 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 1281945 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 1281946 T4012 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1281947 T4012 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:33410_solr
[junit4] 2> 1281948 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1281949 T4012 oasc.Overseer.start Overseer
(id=92326161521115139-127.0.0.1:33410_solr-n_0000000000) starting
[junit4] 2> 1281950 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1281952 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 1281954 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 1281956 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 1281958 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 1281960 T4012 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1281960 T4027 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1281962 T4026 oasc.Overseer$ClusterStateUpdater.run Replaying
operations from work queue.
[junit4] 2> 1281962 T4026 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr";,
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1281963 T4029 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1281965 T4026 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr";,
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1281965 T4029 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1281966 T4026 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1281968 T4026 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr";,
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1281969 T4030 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1281970 T4029 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1282064 T4012 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:33410 33410
[junit4] 2> 1282637 T4013 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:33410 33410
[junit4] 2> 1282638 T4012 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
[junit4] 2> 1282639 T4012 oasc.Overseer.close Overseer
(id=92326161521115139-127.0.0.1:33410_solr-n_0000000000) closing
[junit4] 2> 1282639 T4026 oasc.Overseer$ClusterStateUpdater.run Overseer
Loop exiting : 127.0.0.1:33410_solr
[junit4] 2> 1282645 T4031
oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read
the data org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1282646 T4012 oas.SolrTestCaseJ4.setUp ###Starting
testPlaceholders
[junit4] 2> 1282647 T4012 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1282648 T4032 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 1282747 T4012 oasc.ZkTestServer.run start zk server on
port:50393
[junit4] 2> 1282750 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1282752 T4038 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@185ca01 name:ZooKeeperConnection
Watcher:127.0.0.1:50393/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1282752 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1282753 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1282754 T4040 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@a9d3e6 name:ZooKeeperConnection
Watcher:127.0.0.1:50393 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1282755 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1282757 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1282760 T4042 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1be5223 name:ZooKeeperConnection
Watcher:127.0.0.1:50393 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1282761 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1282761 T4012 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1282764 T4012 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1282767 T4012 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 1282769 T4012 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 1282770 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1282773 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1282774 T4044 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@7dcdd5 name:ZooKeeperConnection
Watcher:127.0.0.1:50393/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1282775 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1282776 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1282777 T4012 oascc.SolrZkClient.makePath makePath:
/live_nodes/node1
[junit4] 2> 1282787 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1282790 T4048 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1b0942e name:ZooKeeperConnection
Watcher:127.0.0.1:50393/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1282790 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1282790 T4012 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1282792 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 1282793 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 1282795 T4012 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1282797 T4012 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:50393_solr
[junit4] 2> 1282797 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1282799 T4012 oasc.Overseer.start Overseer
(id=92326161575116804-127.0.0.1:50393_solr-n_0000000000) starting
[junit4] 2> 1282800 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1282801 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 1282803 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 1282804 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 1282806 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 1282808 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 1282812 T4012 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1282813 T4050 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1282813 T4049 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1282815 T4049 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=12 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"12",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1282816 T4049
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6,
shard7, shard8, shard9, shard10, shard11, shard12]
[junit4] 2> 1282816 T4049 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard3
[junit4] 2> 1282817 T4052 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1282818 T4046 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1282818 T4045 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1283314 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard3/election
[junit4] 2> 1283322 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard3
[junit4] 2> 1283327 T4052 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1283331 T4012 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:50393 50393
[junit4] 2> 1283881 T4032 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:50393 50393
[junit4] 2> 1283882 T4012 oas.SolrTestCaseJ4.tearDown ###Ending
testPlaceholders
[junit4] 2> 1283882 T4012 oasc.Overseer.close Overseer
(id=92326161575116804-127.0.0.1:50393_solr-n_0000000000) closing
[junit4] 2> 1283883 T4049 oasc.Overseer$ClusterStateUpdater.run Overseer
Loop exiting : 127.0.0.1:50393_solr
[junit4] 2> 1283884 T4053
oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read
the data org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1283887 T4012 oas.SolrTestCaseJ4.setUp ###Starting
testBadQueueItem
[junit4] 2> 1283888 T4012 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1283888 T4054 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 1283988 T4012 oasc.ZkTestServer.run start zk server on
port:38098
[junit4] 2> 1283989 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1283991 T4060 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@aa9b3d name:ZooKeeperConnection
Watcher:127.0.0.1:38098 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1283991 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1283993 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1283994 T4062 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@e6a6b0 name:ZooKeeperConnection
Watcher:127.0.0.1:38098 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1283994 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1283994 T4012 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1283997 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1283998 T4064 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1fd7556 name:ZooKeeperConnection
Watcher:127.0.0.1:38098/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1283998 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1283998 T4012 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1284000 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1284001 T4066 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1a72b9a name:ZooKeeperConnection
Watcher:127.0.0.1:38098/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1284001 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1284001 T4012 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1284002 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 1284003 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 1284005 T4012 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1284006 T4012 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:38098_solr
[junit4] 2> 1284007 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1284008 T4012 oasc.Overseer.start Overseer
(id=92326161656381443-127.0.0.1:38098_solr-n_0000000000) starting
[junit4] 2> 1284009 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1284010 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 1284012 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 1284014 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 1284015 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 1284017 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 1284021 T4012 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1284021 T4068 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1284022 T4067 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1284022 T4012 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 1284024 T4012 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 1284028 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1284031 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1284033 T4071 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@16d1a33 name:ZooKeeperConnection
Watcher:127.0.0.1:38098/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1284033 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1284034 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1284037 T4012 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1
[junit4] 2> 1284042 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1284044 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1284044 T4067
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 1284044 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard3
[junit4] 2> 1284047 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1284047 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1284542 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard3/election
[junit4] 2> 1284548 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard3
[junit4] 2> 1284550 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1284553 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1284553 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1284553 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard2
[junit4] 2> 1284554 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1284555 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1284555 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1285051 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard2/election
[junit4] 2> 1285061 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard2
[junit4] 2> 1285067 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1285072 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1285073 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1285073 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1285076 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1285077 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1285077 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1285569 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 1285580 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 1285586 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1285691 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1285692 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1285739 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1285743 T4067
oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid
Overseer message because it has no collection specified: {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1285747 T4067
oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid
Overseer message because it has no collection specified: {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":""}
[junit4] 2> 1285751 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1285752 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1285752 T4067
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
collection2 with shards [shard1, shard2, shard3]
[junit4] 2> 1285753 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard3
[junit4] 2> 1285858 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1285858 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1286245 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection2/leader_elect/shard3/election
[junit4] 2> 1286250 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection2/leaders/shard3
[junit4] 2> 1286255 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1286258 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1286258 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1286258 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard2
[junit4] 2> 1286259 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1286260 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1286260 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1286756 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection2/leader_elect/shard2/election
[junit4] 2> 1286760 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection2/leaders/shard2
[junit4] 2> 1286763 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1286765 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1286766 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1286766 T4067 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1286767 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1286768 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1286768 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1287264 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection2/leader_elect/shard1/election
[junit4] 2> 1287270 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection2/leaders/shard1
[junit4] 2> 1287274 T4074 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1287378 T4073 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1287378 T4072 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1287428 T4012 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:38098 38098
[junit4] 2> 1288025 T4068 oasc.OverseerCollectionProcessor.amILeader
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:513)
[junit4] 2> at
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:271)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1288146 T4068 oasc.OverseerCollectionProcessor.amILeader
According to ZK I (id=92326161656381443-127.0.0.1:38098_solr-n_0000000000) am
no longer a leader.
[junit4] 2> 1288146 T4054 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:38098 38098
[junit4] 2> 1288148 T4012 oas.SolrTestCaseJ4.tearDown ###Ending
testBadQueueItem
[junit4] 2> 1288149 T4012 oasc.Overseer.close Overseer
(id=92326161656381443-127.0.0.1:38098_solr-n_0000000000) closing
[junit4] 2> 1288149 T4067 oasc.Overseer$ClusterStateUpdater.run Overseer
Loop exiting : 127.0.0.1:38098_solr
[junit4] 2> 1288151 T4075
oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read
the data org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1288155 T4012 oas.SolrTestCaseJ4.setUp ###Starting
testShardAssignment
[junit4] 2> 1288155 T4012 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1288156 T4076 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 1288256 T4012 oasc.ZkTestServer.run start zk server on
port:50654
[junit4] 2> 1288257 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1288259 T4082 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@965bd6 name:ZooKeeperConnection
Watcher:127.0.0.1:50654 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1288259 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1288261 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1288262 T4084 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@b3a774 name:ZooKeeperConnection
Watcher:127.0.0.1:50654 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1288266 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1288266 T4012 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1288270 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1288272 T4086 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@150477a name:ZooKeeperConnection
Watcher:127.0.0.1:50654/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1288272 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1288272 T4012 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1288275 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1288276 T4088 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@981cae name:ZooKeeperConnection
Watcher:127.0.0.1:50654/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1288276 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1288277 T4012 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1288278 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 1288280 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 1288281 T4012 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1288283 T4012 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:50654_solr
[junit4] 2> 1288283 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1288285 T4012 oasc.Overseer.start Overseer
(id=92326161936285699-127.0.0.1:50654_solr-n_0000000000) starting
[junit4] 2> 1288286 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1288290 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 1288292 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 1288294 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 1288296 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 1288298 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 1288302 T4012 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1288303 T4089 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1288303 T4090 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1288310 T4012 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 1288314 T4012 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 1288316 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1288330 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1288330 T4093 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@181ec36 name:ZooKeeperConnection
Watcher:127.0.0.1:50654/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1288331 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1288332 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1288334 T4012 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1
[junit4] 2> 1288337 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1288338 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1288338 T4089
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 1288339 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard3
[junit4] 2> 1288341 T4094 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1288341 T4095 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1288837 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard3/election
[junit4] 2> 1288842 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard3
[junit4] 2> 1288846 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1288849 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1288849 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1288850 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard2
[junit4] 2> 1288851 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1288852 T4095 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1288853 T4094 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1289347 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard2/election
[junit4] 2> 1289352 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard2
[junit4] 2> 1289354 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1289356 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1289357 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1289357 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1289358 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1289358 T4094 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1289358 T4095 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1289855 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 1289862 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 1289866 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1289871 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1289872 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1289872 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard3
[junit4] 2> 1289875 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1289876 T4095 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1289876 T4094 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1290368 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard3/election
[junit4] 2> 1290378 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1290380 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1290380 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1290381 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard2
[junit4] 2> 1290485 T4095 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1290485 T4094 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1290878 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard2/election
[junit4] 2> 1290883 T4096 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1290884 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1290885 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1290885 T4089 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1290988 T4094 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1290989 T4095 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1291383 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 1291390 T4093 oaz.ClientCnxn$EventThread.processEvent ERROR
Error while calling watcher java.util.concurrent.RejectedExecutionException:
Task java.util.concurrent.FutureTask@117ea69 rejected from
java.util.concurrent.ThreadPoolExecutor@11eb0c7[Shutting down, pool size = 2,
active threads = 2, queued tasks = 0, completed tasks = 8]
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:206)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1291390 T4097 oasc.LeaderElector$ElectionWatcher.process WARN
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /collections/collection1/leader_elect/shard3/election
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1292890 T4094 oasc.LeaderElector$ElectionWatcher.process WARN
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /collections/collection1/leader_elect/shard2/election
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at
java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1292894 T4012 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:50654 50654
[junit4] 2> 1293443 T4076 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:50654 50654
[junit4] 2> 1293444 T4012 oas.SolrTestCaseJ4.tearDown ###Ending
testShardAssignment
[junit4] 2> 1293445 T4012 oasc.Overseer.close Overseer
(id=92326161936285699-127.0.0.1:50654_solr-n_0000000000) closing
[junit4] 2> 1293445 T4089 oasc.Overseer$ClusterStateUpdater.run Overseer
Loop exiting : 127.0.0.1:50654_solr
[junit4] 2> 1293447 T4099
oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read
the data org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1293451 T4012 oas.SolrTestCaseJ4.setUp ###Starting
testDoubleAssignment
[junit4] 2> 1293452 T4012 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1293452 T4100 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 1293552 T4012 oasc.ZkTestServer.run start zk server on
port:51410
[junit4] 2> 1293553 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1293556 T4106 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@16a9062 name:ZooKeeperConnection
Watcher:127.0.0.1:51410/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1293556 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1293557 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1293559 T4108 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@19eef99 name:ZooKeeperConnection
Watcher:127.0.0.1:51410 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1293559 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1293562 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1293563 T4110 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@bbbfc5 name:ZooKeeperConnection
Watcher:127.0.0.1:51410 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1293563 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1293564 T4012 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1293569 T4012 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1293573 T4012 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 1293576 T4012 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 1293578 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1293581 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1293583 T4112 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1eed4b name:ZooKeeperConnection
Watcher:127.0.0.1:51410/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1293583 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1293585 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1293587 T4012 oascc.SolrZkClient.makePath makePath:
/live_nodes/node1
[junit4] 2> 1293589 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1293590 T4116 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@68fc3a name:ZooKeeperConnection
Watcher:127.0.0.1:51410/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1293590 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1293590 T4012 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1293591 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 1293593 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 1293594 T4012 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1293596 T4012 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:51410_solr
[junit4] 2> 1293596 T4012 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1293597 T4012 oasc.Overseer.start Overseer
(id=92326162283167748-127.0.0.1:51410_solr-n_0000000000) starting
[junit4] 2> 1293599 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1293600 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 1293602 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 1293604 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 1293606 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 1293608 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 1293611 T4012 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1293612 T4118 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1293612 T4117 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1293614 T4117 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1293614 T4117
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
collection1 with shards [shard1]
[junit4] 2> 1293615 T4117 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1293617 T4120 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1293617 T4113 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1293617 T4114 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1294113 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 1294125 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 1294132 T4120 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1294137 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1294138 T4122 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@181bf68 name:ZooKeeperConnection
Watcher:127.0.0.1:51410/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1294138 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1294140 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1294144 T4012 oascc.SolrZkClient.makePath makePath:
/live_nodes/node1
[junit4] 2> 1294152 T4120 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1294152 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 1294154 T4117 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1294154 T4117 oasc.Overseer$ClusterStateUpdater.updateState
shard=shard1 is already registered
[junit4] 2> 1294159 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 1294164 T4120 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1294269 T4114 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1294269 T4123 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1294274 T4012 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:51410 51410
[junit4] 2> 1294951 T4100 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:51410 51410
[junit4] 2> 1294954 T4012 oas.SolrTestCaseJ4.tearDown ###Ending
testDoubleAssignment
[junit4] 2> 1294954 T4012 oasc.Overseer.close Overseer
(id=92326162283167748-127.0.0.1:51410_solr-n_0000000000) closing
[junit4] 2> 1294955 T4117 oasc.Overseer$ClusterStateUpdater.run Overseer
Loop exiting : 127.0.0.1:51410_solr
[junit4] 2> 1294958 T4124
oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read
the data org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1294965 T4012 oas.SolrTestCaseJ4.setUp ###Starting
testShardLeaderChange
[junit4] 2> 1294966 T4012 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1294967 T4125 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 1295066 T4012 oasc.ZkTestServer.run start zk server on
port:37925
[junit4] 2> 1295067 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1295070 T4131 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@2bd309 name:ZooKeeperConnection
Watcher:127.0.0.1:37925/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1295071 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1295072 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1295073 T4133 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1d437f5 name:ZooKeeperConnection
Watcher:127.0.0.1:37925 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1295074 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1295078 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1295079 T4135 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@50b04f name:ZooKeeperConnection
Watcher:127.0.0.1:37925 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1295080 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1295081 T4012 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1295086 T4012 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1295090 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1295090 T4136 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1295092 T4139 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@888cdf name:ZooKeeperConnection
Watcher:127.0.0.1:37925/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1295092 T4140 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@d8c24b name:ZooKeeperConnection
Watcher:127.0.0.1:37925/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1295092 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1295093 T4136 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1295094 T4136 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1295094 T4012 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 1295096 T4136 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 1295098 T4012 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 1295098 T4136 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 1295100 T4012
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1295101 T4136 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1295104 T4012 oascc.SolrZkClient.makePath makePath:
/live_nodes/node1
[junit4] 2> 1295104 T4136 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:37925_solr
[junit4] 2> 1295105 T4136 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1295108 T4136 oasc.Overseer.start Overseer
(id=92326162382520324-127.0.0.1:37925_solr-n_0000000000) starting
[junit4] 2> 1295108 T4012 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1295109 T4136 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1295112 T4136 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 1295115 T4136 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 1295118 T4136 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 1295121 T4136 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 1295123 T4136 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 1295127 T4136 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1295128 T4143 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1295128 T4136 oasc.OverseerTest$OverseerRestarter.run Killing
overseer.
[junit4] 2> 1295129 T4142 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1295129 T4142 oasc.Overseer$ClusterStateUpdater.amILeader
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:606)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:211)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1295130 T4136 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1295130 T4142 oasc.Overseer$ClusterStateUpdater.amILeader
According to ZK I (id=92326162382520324-127.0.0.1:37925_solr-n_0000000000) am
no longer a leader.
[junit4] 2> 1295130 T4142 oasc.Overseer$ClusterStateUpdater.run Overseer
Loop exiting : 127.0.0.1:37925_solr
[junit4] 2> 1295131 T4146 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@bcd91 name:ZooKeeperConnection
Watcher:127.0.0.1:37925/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1295131 T4136 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1295132 T4147
oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read
the data org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1295132 T4136 oasc.Overseer.close Overseer
(id=92326162382520324-127.0.0.1:37925_solr-n_0000000000) closing
[junit4] 2> 1295133 T4143 oasc.OverseerCollectionProcessor.run ERROR
Unable to prioritize overseer
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode =
Session expired for /roles.json
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:256)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:253)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:253)
[junit4] 2> at
org.apache.solr.cloud.OverseerCollectionProcessor.prioritizeOverseerNodes(OverseerCollectionProcessor.java:411)
[junit4] 2> at
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:258)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1295134 T4136 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1295137 T4136 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1295140 T4136 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:37925_solr
[junit4] 2> 1295140 T4136 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1295142 T4136 oasc.Overseer.start Overseer
(id=92326162382520325-127.0.0.1:37925_solr-n_0000000001) starting
[junit4] 2> 1295156 T4136 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1295156 T4149 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1295157 T4148 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1295160 T4148 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1295161 T4148
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
collection1 with shards [shard1]
[junit4] 2> 1295161 T4148 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1295164 T4151 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1295165 T4141 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1295611 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 1295620 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 1295626 T4151 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1295627 T4012 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 1295630 T4148 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1295631 T4148 oasc.Overseer$ClusterStateUpdater.updateState
shard=shard1 is already registered
[junit4] 2> 1295633 T4012 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1295633 T4151 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 1295634 T4153 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1492b92 name:ZooKeeperConnection
Watcher:127.0.0.1:37925/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1295634 T4012 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 129
[...truncated too long message...]
java:783)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:462)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:459)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:416)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
[junit4] 2> at
org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:142)
[junit4] 2> at
org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
[junit4] 2> at
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:137)
[junit4] 2> ... 10 more
[junit4] 2>
[junit4] 2> 1355522 T4012 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:57519 57519
[junit4] 2> 1356103 T4814 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:57519 57519
[junit4] 2> 1356105 T4012 oas.SolrTestCaseJ4.tearDown ###Ending
testOverseerFailure
[junit4] 2> 1356106 T4012 oasc.Overseer.close Overseer
(id=92326165173174277-127.0.0.1:57519_solr-n_0000000001) closing
[junit4] 2> 1356106 T4838 oasc.Overseer$ClusterStateUpdater.run Overseer
Loop exiting : 127.0.0.1:57519_solr
[junit4] 2> 1356107 T4843
oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read
the data org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest
-Dtests.method=testOverseerFailure -Dtests.seed=6001E52C352F5EF5
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_VE
-Dtests.timezone=Africa/Djibouti -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 18.6s J1 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: Could not
register as the leader because creating the ephemeral registration node in
ZooKeeper failed
[junit4] > at
__randomizedtesting.SeedInfo.seed([6001E52C352F5EF5:64096ADF278AB1D4]:0)
[junit4] > at
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:150)
[junit4] > at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
[junit4] > at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
[junit4] > at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
[junit4] > at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] > at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] > at
org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
[junit4] > at
org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
NodeExists for /collections/collection1/leaders/shard1
[junit4] > at
org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] > at
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at
org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:462)
[junit4] > at
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:459)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:416)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
[junit4] > at
org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:142)
[junit4] > at
org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
[junit4] > at
org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:137)
[junit4] > ... 47 more
[junit4] 2> 1359121 T4012 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.OverseerTest-6001E52C352F5EF5-001
[junit4] 2> NOTE: test params are: codec=Lucene410: {}, docValues:{},
sim=DefaultSimilarity, locale=es_VE, timezone=Africa/Djibouti
[junit4] 2> NOTE: Linux 3.13.0-34-generic i386/Oracle Corporation 1.7.0_67
(32-bit)/cpus=8,threads=1,free=168530072,total=400728064
[junit4] 2> NOTE: All tests run in this JVM: [TestRecovery,
CoreAdminRequestStatusTest, ScriptEngineTest, SolrPluginUtilsTest,
ZkSolrClientTest, TestRealTimeGet, TestHighlightDedupGrouping, DateFieldTest,
TermVectorComponentTest, DistribDocExpirationUpdateProcessorTest,
SolrInfoMBeanTest, FileBasedSpellCheckerTest, StatsComponentTest,
TestMultiCoreConfBootstrap, SoftAutoCommitTest, TestQueryTypes,
TestPostingsSolrHighlighter, TestMissingGroups, SortByFunctionTest,
TestFieldSortValues, TestReversedWildcardFilterFactory, QueryParsingTest,
TestCSVResponseWriter, HighlighterMaxOffsetTest, TestLFUCache,
TestDefaultSearchFieldResource, CollectionsAPIDistributedZkTest,
TestReRankQParserPlugin, LegacyHTMLStripCharFilterTest, SuggestComponentTest,
TestIndexingPerformance, TestQueryUtils, TestUpdate, TestDistributedGrouping,
TestCopyFieldCollectionResource, CoreMergeIndexesAdminHandlerTest,
TermVectorComponentDistributedTest, ConvertedLegacyTest, DocValuesTest,
URLClassifyProcessorTest, TestHighFrequencyDictionaryFactory,
TestPHPSerializedResponseWriter, TestPivotHelperCode, CopyFieldTest,
TestNonDefinedSimilarityFactory, TestPerFieldSimilarity, RequestHandlersTest,
TestExtendedDismaxParser, TestSolrQueryParser, ChaosMonkeySafeLeaderTest,
TestManagedResourceStorage, TestSolrQueryParserResource,
DistributedDebugComponentTest, TestCollationFieldDocValues,
TestSolrQueryParserDefaultOperatorResource,
UniqFieldsUpdateProcessorFactoryTest, TestLeaderElectionZkExpiry,
CurrencyFieldXmlFileTest, CoreContainerCoreInitFailuresTest,
TestValueSourceCache, TestOrdValues, TestDocBasedVersionConstraints,
TestCollationField, TestStandardQParsers, TestWriterPerf, PreAnalyzedFieldTest,
FunctionTest, MigrateRouteKeyTest, DistanceFunctionTest, SolrCoreTest,
TestAnalyzeInfixSuggestions, TestMergePolicyConfig, CustomCollectionTest,
AnalysisAfterCoreReloadTest, AddSchemaFieldsUpdateProcessorFactoryTest,
SchemaVersionSpecificBehaviorTest, DebugComponentTest, TestNoOpRegenerator,
DistributedFacetPivotSmallTest, AnalysisErrorHandlingTest, TestLazyCores,
SolrRequestParserTest, TestOverriddenPrefixQueryForCustomFieldType, TestUtils,
HdfsWriteToMultipleCollectionsTest, AliasIntegrationTest,
TestSweetSpotSimilarityFactory, MultiTermTest, TestCloudInspectUtil,
QueryElevationComponentTest, TestCSVLoader, TestCustomSort,
TestAtomicUpdateErrorCases, ChangedSchemaMergeTest, ShardRoutingCustomTest,
SearchHandlerTest, AutoCommitTest, TestInitQParser, HdfsBasicDistributedZkTest,
TestZkChroot, TestReplicationHandlerBackup, DirectUpdateHandlerOptimizeTest,
TestAddFieldRealTimeGet, BasicDistributedZk2Test, SyncSliceTest,
OpenCloseCoreStressTest, OverseerTest]
[junit4] Completed on J1 in 77.34s, 9 tests, 1 error <<< FAILURES!
[...truncated 739 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:485: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:465: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:48: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:494:
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1342:
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:957:
There were test failures: 430 suites, 1783 tests, 1 error, 40 ignored (19
assumptions)
Total time: 96 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_67 -client
-XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
