Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11580/
Java: 64bit/jdk1.8.0_40-ea-b20 -XX:-UseCompressedOops -XX:+UseParallelGC
2 tests failed.
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([273BBFE4963E4923]:0)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([273BBFE4963E4923]:0)
Build Log:
[...truncated 10184 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/init-core-data-001
[junit4] 2> 881492 T6375 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl
(true) and clientAuth (false)
[junit4] 2> 881493 T6375 oas.BaseDistributedSearchTestCase.initHostContext
Setting hostContext system property: /
[junit4] 2> 881495 T6375 oas.SolrTestCaseJ4.setUp ###Starting
testDistribSearch
[junit4] 2> 881495 T6375 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 881496 T6376 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 881596 T6375 oasc.ZkTestServer.run start zk server on
port:54337
[junit4] 2> 881596 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 881597 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 881599 T6383 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@7057aa1 name:ZooKeeperConnection
Watcher:127.0.0.1:54337 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 881599 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 881600 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 881600 T6375 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 881602 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 881603 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 881604 T6386 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@3639c58f
name:ZooKeeperConnection Watcher:127.0.0.1:54337/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 881604 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 881604 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 881605 T6375 oascc.SolrZkClient.makePath makePath:
/collections/collection1
[junit4] 2> 881606 T6375 oascc.SolrZkClient.makePath makePath:
/collections/collection1/shards
[junit4] 2> 881607 T6375 oascc.SolrZkClient.makePath makePath:
/collections/control_collection
[junit4] 2> 881607 T6375 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/shards
[junit4] 2> 881608 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
to /configs/conf1/solrconfig.xml
[junit4] 2> 881609 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.xml
[junit4] 2> 881610 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml
to /configs/conf1/schema.xml
[junit4] 2> 881610 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/schema.xml
[junit4] 2> 881611 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 881612 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 881612 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
to /configs/conf1/stopwords.txt
[junit4] 2> 881613 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/stopwords.txt
[junit4] 2> 881614 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
to /configs/conf1/protwords.txt
[junit4] 2> 881614 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/protwords.txt
[junit4] 2> 881615 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
to /configs/conf1/currency.xml
[junit4] 2> 881615 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/currency.xml
[junit4] 2> 881616 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
to /configs/conf1/enumsConfig.xml
[junit4] 2> 881616 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/enumsConfig.xml
[junit4] 2> 881617 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
to /configs/conf1/open-exchange-rates.json
[junit4] 2> 881617 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/open-exchange-rates.json
[junit4] 2> 881618 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 881618 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 881619 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
to /configs/conf1/old_synonyms.txt
[junit4] 2> 881620 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/old_synonyms.txt
[junit4] 2> 881621 T6375 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
to /configs/conf1/synonyms.txt
[junit4] 2> 881621 T6375 oascc.SolrZkClient.makePath makePath:
/configs/conf1/synonyms.txt
[junit4] 2> 881622 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 881622 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 881623 T6389 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@6007bbc4
name:ZooKeeperConnection Watcher:127.0.0.1:54337/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 881623 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 881624 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 881859 T6375 oejs.Server.doStart jetty-9.2.6.v20141205
[junit4] 2> 881877 T6375 oejsh.ContextHandler.doStart Started
o.e.j.s.ServletContextHandler@3e9bc11c{/,null,AVAILABLE}
[junit4] 2> 881887 T6375 oejs.AbstractConnector.doStart Started
ServerConnector@3e52f865{SSL-http/1.1}{127.0.0.1:39766}
[junit4] 2> 881888 T6375 oejs.Server.doStart Started @882916ms
[junit4] 2> 881889 T6375 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
[junit4] 2> 881890 T6375 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 881890 T6375 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-002
[junit4] 2> 881890 T6375 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-002/'
[junit4] 2> 881902 T6375 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-002/solr.xml
[junit4] 2> 881910 T6375 oasc.CoreContainer.<init> New CoreContainer
943786978
[junit4] 2> 881910 T6375 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-002/]
[junit4] 2> 881911 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 881911 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 881912 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 881912 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 881912 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnections to: 10000
[junit4] 2> 881912 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 881913 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 881913 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 881913 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 881914 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 881915 T6375 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 881916 T6375 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 881916 T6375 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 881916 T6375 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 881916 T6375 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:54337/solr
[junit4] 2> 881917 T6375 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 881917 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 881917 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 881919 T6403 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@54717786
name:ZooKeeperConnection Watcher:127.0.0.1:54337 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 881919 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 881920 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 881921 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 881922 T6406 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@3735015e
name:ZooKeeperConnection Watcher:127.0.0.1:54337/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 881922 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 881923 T6375 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 881925 T6375 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 881926 T6375 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 881928 T6375 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 881929 T6375 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 881930 T6375 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 881931 T6375 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39766_
[junit4] 2> 881932 T6375 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:39766_
[junit4] 2> 881933 T6375 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 881934 T6375 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 881935 T6375 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 881935 T6375 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:39766_
[junit4] 2> 881936 T6375 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 881936 T6375 oasc.Overseer.start Overseer
(id=93130563923804164-127.0.0.1:39766_-n_0000000000) starting
[junit4] 2> 881938 T6375 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 881941 T6375 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 881942 T6408 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 881942 T6375 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 881943 T6375 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 881944 T6375
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 881945 T6407 oasc.Overseer$ClusterStateUpdater.run Starting to
work on the main queue
[junit4] 2> 881946 T6410 oasc.ZkController.publish publishing
core=collection1 state=down collection=control_collection
[junit4] 2> 881947 T6410 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 881947 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 881947 T6410 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 881949 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 0, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:39766";,
[junit4] 2> "node_name":"127.0.0.1:39766_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 881949 T6407 oasco.ReplicaMutator.updateState Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:39766";,
[junit4] 2> "node_name":"127.0.0.1:39766_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 881949 T6407 oasco.ClusterStateMutator.createCollection
building a new cName: control_collection
[junit4] 2> 881950 T6407 oasco.ReplicaMutator.updateState Assigning new
node to shard shard=shard1
[junit4] 2> 881950 T6406 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 882948 T6410 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 882948 T6410 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:control_collection
[junit4] 2> 882949 T6410 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 882949 T6410 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/control_collection
[junit4] 2> 882949 T6410 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 882950 T6410 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-002/collection1/'
[junit4] 2> 882950 T6410 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-002/collection1/lib/.svn/'
to classloader
[junit4] 2> 882950 T6410 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-002/collection1/lib/classes/'
to classloader
[junit4] 2> 882950 T6410 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-002/collection1/lib/README'
to classloader
[junit4] 2> 882959 T6410 oasc.ZkController.watchZKConfDir watch zkdir
/configs/conf1
[junit4] 2> 882961 T6410 oasc.Config.<init> loaded config solrconfig.xml
with version 0
[junit4] 2> 882964 T6410 oasc.SolrConfig.refreshRequestParams current
version of requestparams : -1
[junit4] 2> 882968 T6410 oasc.SolrConfig.<init> Using Lucene MatchVersion:
6.0.0
[junit4] 2> 882977 T6410 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 882978 T6410 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 882981 T6410 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 883063 T6410 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 883065 T6410 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 883066 T6410 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 883068 T6410 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 883076 T6410 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection control_collection
[junit4] 2> 883076 T6410 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 883077 T6410 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-002/collection1/,
dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/control/data/
[junit4] 2> 883077 T6410 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@615e5c33
[junit4] 2> 883078 T6410 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/control/data
[junit4] 2> 883078 T6410 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/control/data/index/
[junit4] 2> 883078 T6410 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/control/data/index' doesn't exist. Creating
new index...
[junit4] 2> 883078 T6410 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/control/data/index
[junit4] 2> 883079 T6410 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=19, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=4.6474609375,
floorSegmentMB=2.1572265625, forceMergeDeletesPctAllowed=9.000088723067844,
segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7718117633583595
[junit4] 2> 883157 T6410 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 883157 T6410 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 883159 T6410 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 883159 T6410 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 883159 T6410 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 883159 T6410 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 883160 T6410 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 883160 T6410 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 883160 T6410 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 883160 T6410 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 883160 T6410 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 883161 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 883161 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 883161 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 883161 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 883161 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 883162 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 883162 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 883162 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 883162 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 883163 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 883163 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 883163 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 883163 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 883164 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/properties:
org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 883164 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 883164 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 883164 T6410 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 883165 T6410 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 883166 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 883166 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 883167 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 883168 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 883168 T6410 oasc.SolrCore.initStatsCache Using default
statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 883169 T6410 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 883169 T6410 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 883169 T6410 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 883170 T6410 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 883170 T6410 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 883170 T6410 oass.SolrIndexSearcher.<init> Opening
Searcher@5ab02aac[collection1] main
[junit4] 2> 883170 T6410 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/control_collection
[junit4] 2> 883171 T6410 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 883171 T6410 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 883172 T6410
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 883172 T6410 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 883172 T6410 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 883173 T6410
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 883173 T6410 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 883173 T6410 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 883173 T6410 oash.ReplicationHandler.inform Commits will be
reserved for 10000
[junit4] 2> 883174 T6411 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@5ab02aac[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 883174 T6410 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 883174 T6414 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:39766 collection:control_collection
shard:shard1
[junit4] 2> 883175 T6375 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 883175 T6375 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 883175 T6414 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leader_elect/shard1/election
[junit4] 2> 883177 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 883177 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 883178 T6414 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard1
[junit4] 2> 883178 T6417 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@581db93b
name:ZooKeeperConnection Watcher:127.0.0.1:54337/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 883179 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 883179 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 883179 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 883179 T6414
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 883179 T6414 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1006 name=collection1
org.apache.solr.core.SolrCore@25a03f08 url=https://127.0.0.1:39766/collection1
node=127.0.0.1:39766_ C1006_STATE=coll:control_collection core:collection1
props:{core=collection1, base_url=https://127.0.0.1:39766,
node_name=127.0.0.1:39766_, state=down}
[junit4] 2> 883180 T6414 C1006 P39766 oasc.SyncStrategy.sync Sync replicas
to https://127.0.0.1:39766/collection1/
[junit4] 2> 883180 T6414 C1006 P39766 oasc.SyncStrategy.syncReplicas Sync
Success - now sync replicas to me
[junit4] 2> 883180 T6414 C1006 P39766 oasc.SyncStrategy.syncToMe
https://127.0.0.1:39766/collection1/ has no replicas
[junit4] 2> 883180 T6414 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: https://127.0.0.1:39766/collection1/ shard1
[junit4] 2> 883180 T6414 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leaders/shard1
[junit4] 2> 883180 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state
version: 1
[junit4] 2> 883181 T6375
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 883182 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 883183 T6375 oasc.ChaosMonkey.monkeyLog monkey: init - expire
sessions:false cause connection loss:false
[junit4] 2> 883183 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"https://127.0.0.1:39766";,
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 1
[junit4] 2> 883183 T6375 oasc.AbstractFullDistribZkTestBase.createJettys
Creating collection1 with stateFormat=2
[junit4] 2> 883183 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 883184 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 883185 T6420 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@391af239
name:ZooKeeperConnection Watcher:127.0.0.1:54337/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 883185 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 883185 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 883186 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 883187 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "stateFormat":2} current state version: 1
[junit4] 2> 883187 T6407 oasco.ClusterStateMutator.createCollection
building a new cName: collection1
[junit4] 2> 883187 T6417 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 883187 T6406 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 883291 T6407 oasco.ZkStateWriter.writePendingUpdates going to
create_collection /collections/collection1/state.json
[junit4] 2> 883291 T6417 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 883333 T6414 oasc.ZkController.register We are
https://127.0.0.1:39766/collection1/ and leader is
https://127.0.0.1:39766/collection1/
[junit4] 2> 883334 T6414 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=https://127.0.0.1:39766
[junit4] 2> 883334 T6414 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 883334 T6414 oasc.ZkController.publish publishing
core=collection1 state=active collection=control_collection
[junit4] 2> 883334 T6414 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 883335 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 883336 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:39766";,
[junit4] 2> "node_name":"127.0.0.1:39766_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"} current state version: 3
[junit4] 2> 883336 T6407 oasco.ReplicaMutator.updateState Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:39766";,
[junit4] 2> "node_name":"127.0.0.1:39766_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 883424 T6375 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 1
[junit4] 2> 883425 T6375 oejs.Server.doStart jetty-9.2.6.v20141205
[junit4] 2> 883426 T6375 oejsh.ContextHandler.doStart Started
o.e.j.s.ServletContextHandler@5e88206a{/,null,AVAILABLE}
[junit4] 2> 883429 T6375 oejs.AbstractConnector.doStart Started
ServerConnector@51925adc{SSL-http/1.1}{127.0.0.1:43219}
[junit4] 2> 883429 T6375 oejs.Server.doStart Started @884457ms
[junit4] 2> 883430 T6375 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
[junit4] 2> 883430 T6375 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 883430 T6375 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-003
[junit4] 2> 883431 T6375 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-003/'
[junit4] 2> 883438 T6406 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 883438 T6417 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 883442 T6375 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-003/solr.xml
[junit4] 2> 883456 T6375 oasc.CoreContainer.<init> New CoreContainer
236040776
[junit4] 2> 883457 T6375 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-003/]
[junit4] 2> 883457 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 883457 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 883457 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 883458 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 883458 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnections to: 10000
[junit4] 2> 883458 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 883458 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 883458 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 883459 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 883459 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 883460 T6375 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 883461 T6375 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 883461 T6375 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 883461 T6375 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 883461 T6375 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:54337/solr
[junit4] 2> 883462 T6375 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 883462 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 883462 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 883464 T6434 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@34990251
name:ZooKeeperConnection Watcher:127.0.0.1:54337 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 883464 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 883464 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 883465 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 883466 T6437 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@28f5d3de
name:ZooKeeperConnection Watcher:127.0.0.1:54337/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 883467 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 883470 T6375
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 884473 T6375 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43219_
[junit4] 2> 884474 T6375 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:43219_
[junit4] 2> 884476 T6375 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 884478 T6438 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 884478 T6438 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 884479 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 884479 T6438 oasc.ZkController.preRegister Registering watch
for external collection collection1
[junit4] 2> 884480 T6438 oascc.ZkStateReader.addZkWatch addZkWatch
collection1
[junit4] 2> 884480 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43219";,
[junit4] 2> "node_name":"127.0.0.1:43219_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 884480 T6438 oascc.ZkStateReader.addZkWatch Updating
collection state at /collections/collection1/state.json from ZooKeeper...
[junit4] 2> 884480 T6407 oasco.ReplicaMutator.updateState Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43219";,
[junit4] 2> "node_name":"127.0.0.1:43219_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 884480 T6407 oasco.ReplicaMutator.updateState Collection
already exists with numShards=1
[junit4] 2> 884481 T6407 oasco.ReplicaMutator.updateState Assigning new
node to shard shard=shard1
[junit4] 2> 884481 T6438 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 0
[junit4] 2> 884481 T6438 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 884582 T6407 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 0
[junit4] 2> 884583 T6437 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 884584 T6437 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 1
[junit4] 2> 885481 T6438 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 885482 T6438 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:collection1
[junit4] 2> 885482 T6438 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 885482 T6438 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 885483 T6438 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 885483 T6438 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-003/collection1/'
[junit4] 2> 885483 T6438 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-003/collection1/lib/.svn/'
to classloader
[junit4] 2> 885484 T6438 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-003/collection1/lib/classes/'
to classloader
[junit4] 2> 885484 T6438 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-003/collection1/lib/README'
to classloader
[junit4] 2> 885493 T6438 oasc.ZkController.watchZKConfDir watch zkdir
/configs/conf1
[junit4] 2> 885494 T6438 oasc.Config.<init> loaded config solrconfig.xml
with version 0
[junit4] 2> 885499 T6438 oasc.SolrConfig.refreshRequestParams current
version of requestparams : -1
[junit4] 2> 885504 T6438 oasc.SolrConfig.<init> Using Lucene MatchVersion:
6.0.0
[junit4] 2> 885516 T6438 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 885517 T6438 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 885521 T6438 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 885607 T6438 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 885609 T6438 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 885609 T6438 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 885611 T6438 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 885621 T6438 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 885621 T6438 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 885621 T6438 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-003/collection1/,
dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty1/
[junit4] 2> 885622 T6438 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@615e5c33
[junit4] 2> 885622 T6438 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty1
[junit4] 2> 885622 T6438 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty1/index/
[junit4] 2> 885623 T6438 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty1/index' doesn't exist. Creating new
index...
[junit4] 2> 885623 T6438 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty1/index
[junit4] 2> 885623 T6438 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=19, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=4.6474609375,
floorSegmentMB=2.1572265625, forceMergeDeletesPctAllowed=9.000088723067844,
segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7718117633583595
[junit4] 2> 885630 T6438 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 885631 T6438 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 885633 T6438 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 885633 T6438 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 885633 T6438 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 885633 T6438 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 885633 T6438 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 885634 T6438 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 885634 T6438 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 885634 T6438 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 885634 T6438 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 885634 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 885635 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 885635 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 885635 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 885635 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 885636 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 885637 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 885637 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 885638 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 885638 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 885639 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 885639 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 885640 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 885640 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/properties:
org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 885641 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 885641 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 885641 T6438 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 885642 T6438 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 885644 T6438 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 885645 T6438 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 885646 T6438 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 885647 T6438 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 885647 T6438 oasc.SolrCore.initStatsCache Using default
statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 885648 T6438 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 885648 T6438 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 885650 T6438 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 885650 T6438 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 885651 T6438 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 885651 T6438 oass.SolrIndexSearcher.<init> Opening
Searcher@69be616b[collection1] main
[junit4] 2> 885651 T6438 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 885652 T6438 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 885652 T6438 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 885652 T6438
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 885652 T6438 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 885653 T6438 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 885653 T6438
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 885653 T6438 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 885653 T6438 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 885653 T6438 oash.ReplicationHandler.inform Commits will be
reserved for 10000
[junit4] 2> 885654 T6439 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@69be616b[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 885654 T6438 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 885655 T6442 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:43219 collection:collection1
shard:shard1
[junit4] 2> 885655 T6375 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 885656 T6375 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 885656 T6442 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 885663 T6442 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard1
[junit4] 2> 885664 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 885664 T6442
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 885664 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1"} current state version: 4
[junit4] 2> 885664 T6442 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1007 name=collection1
org.apache.solr.core.SolrCore@3713496d url=https://127.0.0.1:43219/collection1
node=127.0.0.1:43219_ C1007_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:43219,
node_name=127.0.0.1:43219_, state=down}
[junit4] 2> 885664 T6442 C1007 P43219 oasc.SyncStrategy.sync Sync replicas
to https://127.0.0.1:43219/collection1/
[junit4] 2> 885665 T6442 C1007 P43219 oasc.SyncStrategy.syncReplicas Sync
Success - now sync replicas to me
[junit4] 2> 885665 T6442 C1007 P43219 oasc.SyncStrategy.syncToMe
https://127.0.0.1:43219/collection1/ has no replicas
[junit4] 2> 885665 T6442 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: https://127.0.0.1:43219/collection1/ shard1
[junit4] 2> 885665 T6407 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 1
[junit4] 2> 885665 T6442 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 885665 T6437 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 885666 T6437 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 2
[junit4] 2> 885668 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 885669 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:43219";,
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 885670 T6407 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 2
[junit4] 2> 885670 T6437 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 885671 T6437 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 3
[junit4] 2> 885719 T6442 oasc.ZkController.register We are
https://127.0.0.1:43219/collection1/ and leader is
https://127.0.0.1:43219/collection1/
[junit4] 2> 885719 T6442 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=https://127.0.0.1:43219
[junit4] 2> 885719 T6442 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 885719 T6442 oasc.ZkController.publish publishing
core=collection1 state=active collection=collection1
[junit4] 2> 885720 T6442 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 885720 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 885721 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43219";,
[junit4] 2> "node_name":"127.0.0.1:43219_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 885722 T6407 oasco.ReplicaMutator.updateState Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43219";,
[junit4] 2> "node_name":"127.0.0.1:43219_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 885722 T6407 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 3
[junit4] 2> 885723 T6437 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 885723 T6442 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 4
[junit4] 2> 885724 T6437 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 4
[junit4] 2> 885899 T6375 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 2
[junit4] 2> 885900 T6375 oejs.Server.doStart jetty-9.2.6.v20141205
[junit4] 2> 885901 T6375 oejsh.ContextHandler.doStart Started
o.e.j.s.ServletContextHandler@3fbf9092{/,null,AVAILABLE}
[junit4] 2> 885904 T6375 oejs.AbstractConnector.doStart Started
ServerConnector@5ddd5673{SSL-http/1.1}{127.0.0.1:59232}
[junit4] 2> 885904 T6375 oejs.Server.doStart Started @886932ms
[junit4] 2> 885906 T6375 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
[junit4] 2> 885906 T6375 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 885906 T6375 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-004
[junit4] 2> 885907 T6375 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-004/'
[junit4] 2> 885921 T6375 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-004/solr.xml
[junit4] 2> 885929 T6375 oasc.CoreContainer.<init> New CoreContainer
1539605580
[junit4] 2> 885929 T6375 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-004/]
[junit4] 2> 885930 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 885930 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 885930 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 885930 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 885930 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnections to: 10000
[junit4] 2> 885931 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 885931 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 885931 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 885931 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 885931 T6375 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 885932 T6375 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 885933 T6375 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 885933 T6375 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 885934 T6375 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 885934 T6375 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:54337/solr
[junit4] 2> 885934 T6375 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 885934 T6375
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 885935 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 885936 T6456 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@6a721be0
name:ZooKeeperConnection Watcher:127.0.0.1:54337 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 885936 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 885937 T6375 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 885938 T6375 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 885939 T6459 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@4eac343b
name:ZooKeeperConnection Watcher:127.0.0.1:54337/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 885939 T6375 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 885942 T6375
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 886945 T6375 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59232_
[junit4] 2> 886946 T6375 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:59232_
[junit4] 2> 886948 T6375 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 886951 T6460 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 886952 T6460 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 886952 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 886953 T6460 oasc.ZkController.preRegister Registering watch
for external collection collection1
[junit4] 2> 886953 T6460 oascc.ZkStateReader.addZkWatch addZkWatch
collection1
[junit4] 2> 886953 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:59232";,
[junit4] 2> "node_name":"127.0.0.1:59232_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 886953 T6460 oascc.ZkStateReader.addZkWatch Updating
collection state at /collections/collection1/state.json from ZooKeeper...
[junit4] 2> 886954 T6407 oasco.ReplicaMutator.updateState Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:59232";,
[junit4] 2> "node_name":"127.0.0.1:59232_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 886954 T6407 oasco.ReplicaMutator.updateState Collection
already exists with numShards=1
[junit4] 2> 886954 T6460 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 4
[junit4] 2> 886954 T6407 oasco.ReplicaMutator.updateState Assigning new
node to shard shard=shard1
[junit4] 2> 886954 T6460 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 886955 T6407 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 4
[junit4] 2> 886955 T6437 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 886955 T6459 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 886956 T6437 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 5
[junit4] 2> 886958 T6459 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 5
[junit4] 2> 887955 T6460 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 887955 T6460 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:collection1
[junit4] 2> 887955 T6460 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 887955 T6460 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 887956 T6460 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 887956 T6460 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-004/collection1/'
[junit4] 2> 887957 T6460 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-004/collection1/lib/.svn/'
to classloader
[junit4] 2> 887957 T6460 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-004/collection1/lib/classes/'
to classloader
[junit4] 2> 887957 T6460 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest%20273BBFE4963E4923-001/tempDir-004/collection1/lib/README'
to classloader
[junit4] 2> 887966 T6460 oasc.ZkController.watchZKConfDir watch zkdir
/configs/conf1
[junit4] 2> 887968 T6460 oasc.Config.<init> loaded config solrconfig.xml
with version 0
[junit4] 2> 887971 T6460 oasc.SolrConfig.refreshRequestParams current
version of requestparams : -1
[junit4] 2> 887984 T6460 oasc.SolrConfig.<init> Using Lucene MatchVersion:
6.0.0
[junit4] 2> 887993 T6460 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 887994 T6460 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 887998 T6460 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 888087 T6460 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 888088 T6460 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 888089 T6460 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 888091 T6460 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 888101 T6460 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 888101 T6460 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 888102 T6460 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-004/collection1/,
dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/
[junit4] 2> 888102 T6460 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@615e5c33
[junit4] 2> 888102 T6460 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2
[junit4] 2> 888103 T6460 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index/
[junit4] 2> 888103 T6460 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index' doesn't exist. Creating new
index...
[junit4] 2> 888103 T6460 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index
[junit4] 2> 888104 T6460 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=19, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=4.6474609375,
floorSegmentMB=2.1572265625, forceMergeDeletesPctAllowed=9.000088723067844,
segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.7718117633583595
[junit4] 2> 888112 T6460 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 888112 T6460 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 888116 T6460 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 888117 T6460 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 888117 T6460 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 888117 T6460 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 888117 T6460 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 888118 T6460 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 888118 T6460 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 888118 T6460 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 888118 T6460 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 888119 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 888119 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 888120 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 888120 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 888121 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 888121 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 888123 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 888123 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 888124 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 888124 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 888125 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 888125 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 888125 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 888126 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/properties:
org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 888126 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 888127 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 888127 T6460 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 888128 T6460 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 888130 T6460 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 888131 T6460 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 888132 T6460 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 888132 T6460 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 888133 T6460 oasc.SolrCore.initStatsCache Using default
statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 888134 T6460 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 888134 T6460 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 888136 T6460 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 888136 T6460 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 888137 T6460 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 888137 T6460 oass.SolrIndexSearcher.<init> Opening
Searcher@51e873c7[collection1] main
[junit4] 2> 888137 T6460 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 888138 T6460 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 888138 T6460 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 888138 T6460
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 888139 T6460 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 888139 T6460 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 888139 T6460
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 888139 T6460 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 888140 T6460 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 888140 T6460 oash.ReplicationHandler.inform Commits will be
reserved for 10000
[junit4] 2> 888141 T6461 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@51e873c7[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 888141 T6460 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 888142 T6464 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:59232 collection:collection1
shard:shard1
[junit4] 2> 888142 T6375 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 888142 T6375 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 888143 T6464 oasc.ZkController.register We are
https://127.0.0.1:59232/collection1/ and leader is
https://127.0.0.1:43219/collection1/
[junit4] 2> 888143 T6464 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=https://127.0.0.1:59232
[junit4] 2> 888144 T6464 oasc.ZkController.checkRecovery Core needs to
recover:collection1
[junit4] 2> 888144 T6464 oasu.DefaultSolrCoreState.doRecovery Running
recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C1008 name=collection1
org.apache.solr.core.SolrCore@7de5fc0e url=https://127.0.0.1:59232/collection1
node=127.0.0.1:59232_ C1008_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:59232,
node_name=127.0.0.1:59232_, state=down}
[junit4] 2> 888144 T6465 C1008 P59232 oasc.RecoveryStrategy.run Starting
recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 888145 T6465 C1008 P59232 oasc.RecoveryStrategy.doRecovery
###### startupVersions=[]
[junit4] 2> 888146 T6465 C1008 P59232 oasc.RecoveryStrategy.doRecovery
Publishing state of core collection1 as recovering, leader is
https://127.0.0.1:43219/collection1/ and I am
https://127.0.0.1:59232/collection1/
[junit4] 2> 888146 T6465 C1008 P59232 oasc.ZkController.publish publishing
core=collection1 state=recovering collection=collection1
[junit4] 2> 888146 T6465 C1008 P59232 oasc.ZkController.publish numShards
not found on descriptor - reading it from system property
[junit4] 2> 888147 T6406 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 888148 T6407 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:59232";,
[junit4] 2> "node_name":"127.0.0.1:59232_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"recovering",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 888148 T6464 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 5
[junit4] 2> 888148 T6407 oasco.ReplicaMutator.updateState Update state
numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:59232";,
[junit4] 2> "node_name":"127.0.0.1:59232_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"recovering",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 888149 T6407 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 5
[junit4] 2> 888150 T6459 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 888150 T6437 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 888150 T6459 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 6
[junit4] 2> 888151 T6437 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 6
[junit4] 2> 888156 T6375
oasc.AbstractFullDistribZkTestBase.enableAutoSoftCommit Turning on auto soft
commit: 1000
[junit4] 2> C1008_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:59232,
node_name=127.0.0.1:59232_, state=recovering}
[junit4] 2> 888156 T6465 C1008 P59232
oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to
https://127.0.0.1:43219; WaitForState:
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A59232_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
[junit4] 2> ASYNC NEW_CORE C1009 name=collection1
org.apache.solr.core.SolrCore@25a03f08 url=https://127.0.0.1:39766/collection1
node=127.0.0.1:39766_ C1009_STATE=coll:control_collection core:collection1
props:{core=collection1, base_url=https://127.0.0.1:39766,
node_name=127.0.0.1:39766_, state=active, leader=true}
[junit4] 2> 888218 T6396 C1009 P39766 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2}
{deleteByQuery=*:* (-1490089029825200128)} 0 3
[junit4] 2> 888234 T6427 oasha.CoreAdminHandler.handleWaitForStateAction
Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true,
onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 888237 T6427 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 6
[junit4] 2> 888237 T6427 oasha.CoreAdminHandler.handleWaitForSt
[...truncated too long message...]
.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 8103585 T6375 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4] 2> 8103585 T6375 oasc.CachingDirectoryFactory.closeCacheValue
looking to close
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index
[CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index;done=true>>]
[junit4] 2> 8103586 T6375 oasc.CachingDirectoryFactory.close Closing
directory:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index
[junit4] 2> 8103589 T6375 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index
[junit4] 2> 8103609 T6375 oasc.CachingDirectoryFactory.close Closing
StandardDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 8103609 T6375 oasc.CachingDirectoryFactory.closeCacheValue
looking to close
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index
[CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index;done=false>>]
[junit4] 2> 8103609 T6375 oasc.CachingDirectoryFactory.close Closing
directory:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2/index
[junit4] 2> 8103610 T6375 oasc.CachingDirectoryFactory.closeCacheValue
looking to close
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2
[CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2;done=false>>]
[junit4] 2> 8103610 T6375 oasc.CachingDirectoryFactory.close Closing
directory:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001/tempDir-001/jetty2
[junit4] 2> 8104089 T6457 oazc.StaticHostProvider.next WARN Unexpected
exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 8104090 T6375 oejs.AbstractConnector.doStop Stopped
ServerConnector@5ddd5673{SSL-http/1.1}{127.0.0.1:0}
[junit4] 2> 8104091 T6375 oejsh.ContextHandler.doStop Stopped
o.e.j.s.ServletContextHandler@3fbf9092{/,null,UNAVAILABLE}
[junit4] 2> 8104092 T6375 oas.SolrTestCaseJ4.tearDown ###Ending
testDistribSearch
[junit4] 2> 8104092 T6375 oasc.ZkTestServer.send4LetterWord connecting to
127.0.0.1:54337 54337
[junit4] 2> 8104787 T6375 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest
273BBFE4963E4923-001
[junit4] 2> 7223299 T6374 ccr.ThreadLeakControl.tryToInterruptAll All
leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=CheapBastard,
sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_OM,
timezone=America/Havana
[junit4] 2> NOTE: Linux 3.13.0-43-generic amd64/Oracle Corporation
1.8.0_40-ea (64-bit)/cpus=8,threads=1,free=152643504,total=375914496
[junit4] 2> NOTE: All tests run in this JVM: [TestReplicationHandler,
TestDistributedSearch, TestCollectionAPI, RankQueryTest, JSONWriterTest,
SyncSliceTest, BlockCacheTest, SolrIndexConfigTest, TestArbitraryIndexDir,
TestBinaryResponseWriter, TestSort, TestSolrXmlPersistor,
TestDistributedMissingSort, BJQParserTest, PluginInfoTest,
DistributedDebugComponentTest, HighlighterConfigTest, MultiThreadedOCPTest,
TestStressRecovery, NoCacheHeaderTest, OverseerTest, OutputWriterTest,
SoftAutoCommitTest, SharedFSAutoReplicaFailoverUtilsTest,
TestFuzzyAnalyzedSuggestions, TestInitQParser, BasicDistributedZkTest,
SuggesterWFSTTest, TestStressReorder, TestSurroundQueryParser,
TestManagedSchemaDynamicFieldResource, TestUpdate, TestFastWriter,
TestRebalanceLeaders, TestFieldResource, MinimalSchemaTest, ResourceLoaderTest,
TestSolrCoreProperties, TestSolrQueryParserResource, DirectUpdateHandlerTest,
CoreAdminHandlerTest, TestBinaryField, ShowFileRequestHandlerTest,
OverriddenZkACLAndCredentialsProvidersTest, UnloadDistributedZkTest,
TestLeaderElectionZkExpiry, TestCSVLoader, HttpPartitionTest,
HighlighterMaxOffsetTest, EnumFieldTest, CoreContainerCoreInitFailuresTest,
TestReRankQParserPlugin, AnalysisAfterCoreReloadTest, TestClusterStateMutator,
TestOverriddenPrefixQueryForCustomFieldType, SolrCoreCheckLockOnStartupTest,
TestManagedResourceStorage, TestPartialUpdateDeduplication, TestLazyCores,
RequiredFieldsTest, AddSchemaFieldsUpdateProcessorFactoryTest,
TestImplicitCoreProperties, SliceStateTest, TestQuerySenderListener,
SimpleCollectionCreateDeleteTest, TestIBSimilarityFactory, PrimUtilsTest,
CloudMLTQParserTest, TestHighFrequencyDictionaryFactory, FacetPivotSmallTest,
CoreAdminRequestStatusTest, TestManagedSynonymFilterFactory,
TestSolrDynamicMBean, TestCodecSupport, HdfsRecoveryZkTest,
TestDocumentBuilder, PreAnalyzedFieldTest, TestSolrConfigHandler,
TestRequestStatusCollectionAPI, TestStressLucene,
DistributedQueryElevationComponentTest, TestSolrDeletionPolicy2,
PathHierarchyTokenizerFactoryTest, MergeStrategyTest, SuggestComponentTest,
DeleteInactiveReplicaTest, NumericFieldsTest, TestRandomDVFaceting,
DefaultValueUpdateProcessorTest, TestUniqueKeyFieldResource,
RemoteQueryErrorTest, LukeRequestHandlerTest, DistanceFunctionTest,
HdfsSyncSliceTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest,
TestSolrXmlPersistence, TestAnalyzedSuggestions,
TestCursorMarkWithoutUniqueKey, CursorMarkTest, CircularListTest,
TestMaxScoreQueryParser, IndexSchemaRuntimeFieldTest, DOMUtilTest,
RecoveryZkTest, TestDocBasedVersionConstraints, TestCoreContainer,
SolrInfoMBeanTest, TestSweetSpotSimilarityFactory, TestQuerySenderNoQuery,
UUIDFieldTest, TestNoOpRegenerator, TestLRUCache, ZkControllerTest,
TestQueryUtils, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=273BBFE4963E4923
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_OM
-Dtests.timezone=America/Havana -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>=
7200000 msec).
[junit4] > at
__randomizedtesting.SeedInfo.seed([273BBFE4963E4923]:0)
[junit4] Completed on J0 in 7223.33s, 1 test, 2 errors <<< FAILURES!
[...truncated 11 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:519: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:61: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:187: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:510:
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1348:
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:956:
There were test failures: 461 suites, 1870 tests, 1 suite-level error, 1 error,
47 ignored (22 assumptions)
Total time: 160 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b20
-XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
