Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11293/
Java: 64bit/jdk1.8.0_40-ea-b09 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([9C3D118D0592C81B]:0)
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([9C3D118D0592C81B]:0)
Build Log:
[...truncated 12160 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/init-core-data-001
[junit4] 2> 150638 T404 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl
(true) and clientAuth (true)
[junit4] 2> 150638 T404 oas.BaseDistributedSearchTestCase.initHostContext
Setting hostContext system property: /
[junit4] 2> 150642 T404 oas.SolrTestCaseJ4.setUp ###Starting
testDistribSearch
[junit4] 2> 150645 T404 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 150647 T405 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 150746 T404 oasc.ZkTestServer.run start zk server on port:39512
[junit4] 2> 150748 T404
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 150783 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 150813 T411 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@164acec8
name:ZooKeeperConnection Watcher:127.0.0.1:39512 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 150814 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 150814 T404 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 150815 T404 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 150837 T404
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 150838 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 150839 T413 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@de8b75 name:ZooKeeperConnection
Watcher:127.0.0.1:39512/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 150840 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 150840 T404 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 150843 T404 oascc.SolrZkClient.makePath makePath:
/collections/collection1
[junit4] 2> 150847 T404 oascc.SolrZkClient.makePath makePath:
/collections/collection1/shards
[junit4] 2> 150850 T404 oascc.SolrZkClient.makePath makePath:
/collections/control_collection
[junit4] 2> 150852 T404 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/shards
[junit4] 2> 150859 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
to /configs/conf1/solrconfig.xml
[junit4] 2> 150859 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.xml
[junit4] 2> 150863 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml
to /configs/conf1/schema.xml
[junit4] 2> 150864 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/schema.xml
[junit4] 2> 150866 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 150867 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 150873 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
to /configs/conf1/stopwords.txt
[junit4] 2> 150874 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/stopwords.txt
[junit4] 2> 150878 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
to /configs/conf1/protwords.txt
[junit4] 2> 150878 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/protwords.txt
[junit4] 2> 150881 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
to /configs/conf1/currency.xml
[junit4] 2> 150881 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/currency.xml
[junit4] 2> 150884 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
to /configs/conf1/enumsConfig.xml
[junit4] 2> 150885 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/enumsConfig.xml
[junit4] 2> 150889 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
to /configs/conf1/open-exchange-rates.json
[junit4] 2> 150889 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/open-exchange-rates.json
[junit4] 2> 150892 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 150893 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 150897 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
to /configs/conf1/old_synonyms.txt
[junit4] 2> 150897 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/old_synonyms.txt
[junit4] 2> 150899 T404 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
to /configs/conf1/synonyms.txt
[junit4] 2> 150899 T404 oascc.SolrZkClient.makePath makePath:
/configs/conf1/synonyms.txt
[junit4] 2> 150905 T404
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 150906 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 150907 T415 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@3b61d18e
name:ZooKeeperConnection Watcher:127.0.0.1:39512/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 150908 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 150908 T404 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 150910 T406 oazs.NIOServerCnxn.doIO WARN caught end of stream
exception EndOfStreamException: Unable to read additional data from client
sessionid 0x1490e7c71d80002, likely client has closed socket
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 151164 T404 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 151167 T404 oejus.SslContextFactory.doStart Enabled Protocols
[SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1,
TLSv1.1, TLSv1.2]
[junit4] 2> 151169 T404 oejs.AbstractConnector.doStart Started
[email protected]:40473
[junit4] 2> 151170 T404 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()
[junit4] 2> 151171 T404 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 151171 T404 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002
[junit4] 2> 151171 T404 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/'
[junit4] 2> 151181 T404 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/solr.xml
[junit4] 2> 151195 T404 oasc.CoreContainer.<init> New CoreContainer
502061990
[junit4] 2> 151196 T404 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/]
[junit4] 2> 151196 T404 oashc.HttpShardHandlerFactory.getParameter Setting
socketTimeout to: 90000
[junit4] 2> 151196 T404 oashc.HttpShardHandlerFactory.getParameter Setting
urlScheme to:
[junit4] 2> 151196 T404 oashc.HttpShardHandlerFactory.getParameter Setting
connTimeout to: 15000
[junit4] 2> 151197 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxConnectionsPerHost to: 20
[junit4] 2> 151197 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxConnections to: 10000
[junit4] 2> 151197 T404 oashc.HttpShardHandlerFactory.getParameter Setting
corePoolSize to: 0
[junit4] 2> 151197 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maximumPoolSize to: 2147483647
[junit4] 2> 151197 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxThreadIdleTime to: 5
[junit4] 2> 151198 T404 oashc.HttpShardHandlerFactory.getParameter Setting
sizeOfQueue to: -1
[junit4] 2> 151198 T404 oashc.HttpShardHandlerFactory.getParameter Setting
fairnessPolicy to: false
[junit4] 2> 151199 T404 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 151200 T404 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 151200 T404 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 151200 T404 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 151200 T404 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:39512/solr
[junit4] 2> 151208 T404 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 151208 T404
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 151209 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 151211 T426 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@3ff760b5
name:ZooKeeperConnection Watcher:127.0.0.1:39512 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 151212 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 151212 T404 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 151217 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 151218 T428 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@758cc243
name:ZooKeeperConnection Watcher:127.0.0.1:39512/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 151218 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 151224 T404 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 151230 T404 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 151234 T404 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 151237 T404 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 151240 T404 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 151248 T404 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 151249 T404 oasc.ZkController.createEphemeralLiveNode Register
node as live in ZooKeeper:/live_nodes/127.0.0.1:40473_
[junit4] 2> 151251 T404 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:40473_
[junit4] 2> 151254 T404 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 151256 T404 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 151257 T404 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 151263 T404 oasc.OverseerElectionContext.runLeaderProcess I am
going to be the leader 127.0.0.1:40473_
[junit4] 2> 151263 T404 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 151265 T404 oasc.Overseer.start Overseer
(id=92621194986520580-127.0.0.1:40473_-n_0000000000) starting
[junit4] 2> 151273 T404 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 151292 T404 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 151296 T430 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 151296 T404 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 151298 T404 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 151299 T404
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 151308 T429 oasc.Overseer$ClusterStateUpdater.run Starting to
work on the main queue
[junit4] 2> 151311 T432 oasc.ZkController.publish publishing
core=collection1 state=down collection=control_collection
[junit4] 2> 151312 T432 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 151315 T432 oasc.ZkController.waitForCoreNodeName look for our
core node name
[junit4] 2> 151316 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 151323 T429 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40473";,
[junit4] 2> "node_name":"127.0.0.1:40473_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 151323 T429 oasc.Overseer$ClusterStateUpdater.createCollection
Create collection control_collection with shards [shard1]
[junit4] 2> 151329 T429 oasc.Overseer$ClusterStateUpdater.createCollection
state version control_collection 1
[junit4] 2> 151331 T429 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 151338 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 152316 T432 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 152316 T432 oasc.ZkController.createCollectionZkNode Check for
collection zkNode:control_collection
[junit4] 2> 152317 T432 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 152317 T432 oascc.ZkStateReader.readConfigName Load collection
config from:/collections/control_collection
[junit4] 2> 152318 T432 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 152319 T432 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/collection1/'
[junit4] 2> 152319 T432 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/collection1/lib/.svn/'
to classloader
[junit4] 2> 152319 T432 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/collection1/lib/classes/'
to classloader
[junit4] 2> 152320 T432 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/collection1/lib/README'
to classloader
[junit4] 2> 152343 T432 oasc.SolrConfig.<init> Using Lucene MatchVersion:
5.0.0
[junit4] 2> 152352 T432 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 152354 T432 oass.IndexSchema.readSchema Reading Solr Schema
from schema.xml
[junit4] 2> 152357 T432 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 152449 T432 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 152450 T432 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 152454 T432 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 152457 T432 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 152475 T432 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection control_collection
[junit4] 2> 152475 T432 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 152483 T432 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-002/collection1/,
dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/control/data/
[junit4] 2> 152483 T432 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@fb0e367
[junit4] 2> 152490 T432 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/control/data
[junit4] 2> 152491 T432 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/control/data/index/
[junit4] 2> 152491 T432 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/control/data/index'
doesn't exist. Creating new index...
[junit4] 2> 152491 T432 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/control/data/index
[junit4] 2> 152492 T432 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=14, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=76.9111328125,
floorSegmentMB=0.7978515625, forceMergeDeletesPctAllowed=19.695317452037482,
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 152595 T432 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 152595 T432 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 152601 T432 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 152601 T432 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 152601 T432 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 152601 T432 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 152602 T432 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 152602 T432 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 152602 T432 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 152602 T432 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 152602 T432 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 152604 T432 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 152605 T432 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 152605 T432 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 152606 T432 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 152606 T432 oasc.RequestHandlers.initHandlersFromConfig
created /get: solr.RealTimeGetHandler
[junit4] 2> 152606 T432 oasc.RequestHandlers.initHandlersFromConfig adding
lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 152607 T432 oasc.RequestHandlers.initHandlersFromConfig
created /replication: solr.ReplicationHandler
[junit4] 2> 152607 T432 oasc.RequestHandlers.initHandlersFromConfig
created /update: solr.UpdateRequestHandler
[junit4] 2> 152608 T432 oasc.RequestHandlers.initHandlersFromConfig
created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 152608 T432 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 152609 T432 oasc.RequestHandlers.initHandlersFromConfig
created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 152610 T432 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 152611 T432 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 152611 T432 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 152612 T432 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 152613 T432 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 152614 T432 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 152614 T432 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=false, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7946915014752227]
[junit4] 2> 152615 T432 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 152615 T432 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 152615 T432 oass.SolrIndexSearcher.<init> Opening
Searcher@227e6756[collection1] main
[junit4] 2> 152616 T432 oascc.ZkStateReader.readConfigName Load collection
config from:/collections/control_collection
[junit4] 2> 152617 T432 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 152617 T432 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 152619 T432
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 152619 T432 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 152620 T432 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 152620 T432
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 152620 T432 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 152621 T432 oascc.SolrZkClient.makePath makePath:
/configs/conf1/_rest_managed.json
[junit4] 2> 152623 T432
oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new
znode /configs/conf1/_rest_managed.json
[junit4] 2> 152623 T432 oasr.ManagedResourceStorage$JsonStorage.store
Saved JSON object to path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 152623 T432 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 152628 T434 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@227e6756[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 152633 T432 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 152633 T437 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:40473 collection:control_collection
shard:shard1
[junit4] 2> 152634 T404 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 152634 T404 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 152642 T437 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leader_elect/shard1/election
[junit4] 2> 152647 T404
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 152648 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 152649 T437 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard1
[junit4] 2> 152650 T439 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@46ddbddb
name:ZooKeeperConnection Watcher:127.0.0.1:39512/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 152650 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 152651 T404 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 152652 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 152652 T437
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 152652 T404
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 152652 T437 oasc.ShardLeaderElectionContext.runLeaderProcess I
may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C675 name=collection1
org.apache.solr.core.SolrCore@52ead23e url=https://127.0.0.1:40473/collection1
node=127.0.0.1:40473_ C675_STATE=coll:control_collection core:collection1
props:{core=collection1, base_url=https://127.0.0.1:40473,
node_name=127.0.0.1:40473_, state=down}
[junit4] 2> 152653 T437 C675 P40473 oasc.SyncStrategy.sync Sync replicas
to https://127.0.0.1:40473/collection1/
[junit4] 2> 152653 T437 C675 P40473 oasc.SyncStrategy.syncReplicas Sync
Success - now sync replicas to me
[junit4] 2> 152653 T437 C675 P40473 oasc.SyncStrategy.syncToMe
https://127.0.0.1:40473/collection1/ has no replicas
[junit4] 2> 152654 T437 oasc.ShardLeaderElectionContext.runLeaderProcess I
am the new leader: https://127.0.0.1:40473/collection1/ shard1
[junit4] 2> 152655 T437 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leaders/shard1
[junit4] 2> 152657 T404 oasc.ChaosMonkey.monkeyLog monkey: init - expire
sessions:false cause connection loss:false
[junit4] 2> 152659 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 152763 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 152763 T440 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 152811 T437 oasc.ZkController.register We are
https://127.0.0.1:40473/collection1/ and leader is
https://127.0.0.1:40473/collection1/
[junit4] 2> 152811 T437 oasc.ZkController.register No LogReplay needed for
core=collection1 baseURL=https://127.0.0.1:40473
[junit4] 2> 152811 T437 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 152811 T437 oasc.ZkController.publish publishing
core=collection1 state=active collection=control_collection
[junit4] 2> 152811 T437 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 152813 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 152816 T429 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40473";,
[junit4] 2> "node_name":"127.0.0.1:40473_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 152920 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 152920 T440 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 152921 T404 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 1
[junit4] 2> 152921 T404 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 152930 T404 oejus.SslContextFactory.doStart Enabled Protocols
[SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1,
TLSv1.1, TLSv1.2]
[junit4] 2> 152931 T404 oejs.AbstractConnector.doStart Started
[email protected]:60084
[junit4] 2> 152933 T404 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()
[junit4] 2> 152933 T404 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 152933 T404 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003
[junit4] 2> 152934 T404 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/'
[junit4] 2> 152943 T404 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/solr.xml
[junit4] 2> 152951 T404 oasc.CoreContainer.<init> New CoreContainer
208521056
[junit4] 2> 152951 T404 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/]
[junit4] 2> 152952 T404 oashc.HttpShardHandlerFactory.getParameter Setting
socketTimeout to: 90000
[junit4] 2> 152952 T404 oashc.HttpShardHandlerFactory.getParameter Setting
urlScheme to:
[junit4] 2> 152952 T404 oashc.HttpShardHandlerFactory.getParameter Setting
connTimeout to: 15000
[junit4] 2> 152952 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxConnectionsPerHost to: 20
[junit4] 2> 152952 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxConnections to: 10000
[junit4] 2> 152953 T404 oashc.HttpShardHandlerFactory.getParameter Setting
corePoolSize to: 0
[junit4] 2> 152953 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maximumPoolSize to: 2147483647
[junit4] 2> 152953 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxThreadIdleTime to: 5
[junit4] 2> 152953 T404 oashc.HttpShardHandlerFactory.getParameter Setting
sizeOfQueue to: -1
[junit4] 2> 152954 T404 oashc.HttpShardHandlerFactory.getParameter Setting
fairnessPolicy to: false
[junit4] 2> 152955 T404 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 152956 T404 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 152956 T404 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 152956 T404 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 152956 T404 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:39512/solr
[junit4] 2> 152956 T404 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 152957 T404
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 152957 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 152959 T451 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@4b1d8417
name:ZooKeeperConnection Watcher:127.0.0.1:39512 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 152959 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 152959 T404 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 152962 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 152963 T453 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@57a187b8
name:ZooKeeperConnection Watcher:127.0.0.1:39512/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 152963 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 152968 T404
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 153974 T404 oasc.ZkController.createEphemeralLiveNode Register
node as live in ZooKeeper:/live_nodes/127.0.0.1:60084_
[junit4] 2> 153976 T404 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:60084_
[junit4] 2> 153979 T404 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 153984 T455 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 153984 T455 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 153985 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 153985 T455 oasc.ZkController.waitForCoreNodeName look for our
core node name
[junit4] 2> 153988 T429 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:60084";,
[junit4] 2> "node_name":"127.0.0.1:60084_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 153988 T429 oasc.Overseer$ClusterStateUpdater.createCollection
Create collection collection1 with shards [shard1]
[junit4] 2> 153988 T429 oasc.Overseer$ClusterStateUpdater.createCollection
state version collection1 1
[junit4] 2> 153988 T429 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 154092 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 154092 T454 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 154092 T440 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 154986 T455 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 154986 T455 oasc.ZkController.createCollectionZkNode Check for
collection zkNode:collection1
[junit4] 2> 154987 T455 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 154987 T455 oascc.ZkStateReader.readConfigName Load collection
config from:/collections/collection1
[junit4] 2> 154988 T455 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 154988 T455 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/collection1/'
[junit4] 2> 154988 T455 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/collection1/lib/.svn/'
to classloader
[junit4] 2> 154989 T455 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/collection1/lib/classes/'
to classloader
[junit4] 2> 154989 T455 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/collection1/lib/README'
to classloader
[junit4] 2> 155006 T455 oasc.SolrConfig.<init> Using Lucene MatchVersion:
5.0.0
[junit4] 2> 155018 T455 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 155019 T455 oass.IndexSchema.readSchema Reading Solr Schema
from schema.xml
[junit4] 2> 155022 T455 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 155128 T455 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 155129 T455 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 155130 T455 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 155132 T455 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 155148 T455 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 155148 T455 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 155149 T455 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-003/collection1/,
dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty1/
[junit4] 2> 155149 T455 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@fb0e367
[junit4] 2> 155150 T455 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty1
[junit4] 2> 155151 T455 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty1/index/
[junit4] 2> 155151 T455 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty1/index'
doesn't exist. Creating new index...
[junit4] 2> 155151 T455 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty1/index
[junit4] 2> 155153 T455 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=14, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=76.9111328125,
floorSegmentMB=0.7978515625, forceMergeDeletesPctAllowed=19.695317452037482,
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 155162 T455 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 155163 T455 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 155171 T455 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 155172 T455 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 155172 T455 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 155172 T455 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 155172 T455 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 155173 T455 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 155173 T455 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 155173 T455 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 155173 T455 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 155175 T455 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 155175 T455 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 155176 T455 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 155177 T455 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 155177 T455 oasc.RequestHandlers.initHandlersFromConfig
created /get: solr.RealTimeGetHandler
[junit4] 2> 155177 T455 oasc.RequestHandlers.initHandlersFromConfig adding
lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 155178 T455 oasc.RequestHandlers.initHandlersFromConfig
created /replication: solr.ReplicationHandler
[junit4] 2> 155178 T455 oasc.RequestHandlers.initHandlersFromConfig
created /update: solr.UpdateRequestHandler
[junit4] 2> 155179 T455 oasc.RequestHandlers.initHandlersFromConfig
created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 155179 T455 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 155181 T455 oasc.RequestHandlers.initHandlersFromConfig
created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 155183 T455 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 155184 T455 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 155185 T455 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 155185 T455 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 155187 T455 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 155187 T455 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 155188 T455 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=false, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7946915014752227]
[junit4] 2> 155189 T455 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 155189 T455 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 155189 T455 oass.SolrIndexSearcher.<init> Opening
Searcher@58cbe13f[collection1] main
[junit4] 2> 155189 T455 oascc.ZkStateReader.readConfigName Load collection
config from:/collections/collection1
[junit4] 2> 155190 T455 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 155191 T455 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 155191 T455
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 155191 T455 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 155191 T455 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 155192 T455
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes
from znode /configs/conf1/_rest_managed.json
[junit4] 2> 155192 T455 oasr.ManagedResourceStorage.load Loaded
LinkedHashMap at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 155192 T455 oasr.ManagedResource.reloadFromStorage Loaded
initArgs {} for /rest/managed
[junit4] 2> 155193 T455 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 155202 T456 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@58cbe13f[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 155211 T455 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 155212 T459 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:60084 collection:collection1
shard:shard1
[junit4] 2> 155212 T404 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 155213 T404 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 155213 T459 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 155219 T459 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard1
[junit4] 2> 155221 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 155222 T459
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 155223 T459 oasc.ShardLeaderElectionContext.runLeaderProcess I
may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C676 name=collection1
org.apache.solr.core.SolrCore@16f8a6b2 url=https://127.0.0.1:60084/collection1
node=127.0.0.1:60084_ C676_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:60084,
node_name=127.0.0.1:60084_, state=down}
[junit4] 2> 155223 T459 C676 P60084 oasc.SyncStrategy.sync Sync replicas
to https://127.0.0.1:60084/collection1/
[junit4] 2> 155224 T459 C676 P60084 oasc.SyncStrategy.syncReplicas Sync
Success - now sync replicas to me
[junit4] 2> 155224 T459 C676 P60084 oasc.SyncStrategy.syncToMe
https://127.0.0.1:60084/collection1/ has no replicas
[junit4] 2> 155224 T459 oasc.ShardLeaderElectionContext.runLeaderProcess I
am the new leader: https://127.0.0.1:60084/collection1/ shard1
[junit4] 2> 155224 T459 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 155229 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 155333 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 155333 T440 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 155333 T454 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 155383 T459 oasc.ZkController.register We are
https://127.0.0.1:60084/collection1/ and leader is
https://127.0.0.1:60084/collection1/
[junit4] 2> 155383 T459 oasc.ZkController.register No LogReplay needed for
core=collection1 baseURL=https://127.0.0.1:60084
[junit4] 2> 155383 T459 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 155383 T459 oasc.ZkController.publish publishing
core=collection1 state=active collection=collection1
[junit4] 2> 155383 T459 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 155385 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 155387 T429 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:60084";,
[junit4] 2> "node_name":"127.0.0.1:60084_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 155460 T404 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 2
[junit4] 2> 155461 T404 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 155463 T404 oejus.SslContextFactory.doStart Enabled Protocols
[SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1,
TLSv1.1, TLSv1.2]
[junit4] 2> 155464 T404 oejs.AbstractConnector.doStart Started
[email protected]:52702
[junit4] 2> 155466 T404 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()
[junit4] 2> 155467 T404 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 155467 T404 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004
[junit4] 2> 155467 T404 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/'
[junit4] 2> 155477 T404 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/solr.xml
[junit4] 2> 155491 T440 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 155491 T454 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 155491 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 155491 T404 oasc.CoreContainer.<init> New CoreContainer
1862431805
[junit4] 2> 155492 T404 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/]
[junit4] 2> 155492 T404 oashc.HttpShardHandlerFactory.getParameter Setting
socketTimeout to: 90000
[junit4] 2> 155492 T404 oashc.HttpShardHandlerFactory.getParameter Setting
urlScheme to:
[junit4] 2> 155493 T404 oashc.HttpShardHandlerFactory.getParameter Setting
connTimeout to: 15000
[junit4] 2> 155493 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxConnectionsPerHost to: 20
[junit4] 2> 155493 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxConnections to: 10000
[junit4] 2> 155493 T404 oashc.HttpShardHandlerFactory.getParameter Setting
corePoolSize to: 0
[junit4] 2> 155493 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maximumPoolSize to: 2147483647
[junit4] 2> 155494 T404 oashc.HttpShardHandlerFactory.getParameter Setting
maxThreadIdleTime to: 5
[junit4] 2> 155494 T404 oashc.HttpShardHandlerFactory.getParameter Setting
sizeOfQueue to: -1
[junit4] 2> 155494 T404 oashc.HttpShardHandlerFactory.getParameter Setting
fairnessPolicy to: false
[junit4] 2> 155496 T404 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 155497 T404 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 155497 T404 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 155497 T404 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 155498 T404 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:39512/solr
[junit4] 2> 155498 T404 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 155498 T404
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 155499 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 155500 T470 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@78ac3dd2
name:ZooKeeperConnection Watcher:127.0.0.1:39512 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 155501 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 155501 T404 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 155503 T404 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 155510 T472 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@2ad36c37
name:ZooKeeperConnection Watcher:127.0.0.1:39512/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 155510 T404 oascc.ConnectionManager.waitForConnected Client is
connected to ZooKeeper
[junit4] 2> 155518 T404
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 156521 T404 oasc.ZkController.createEphemeralLiveNode Register
node as live in ZooKeeper:/live_nodes/127.0.0.1:52702_
[junit4] 2> 156522 T404 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:52702_
[junit4] 2> 156525 T404 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 156529 T474 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 156529 T474 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 156530 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 156530 T474 oasc.ZkController.waitForCoreNodeName look for our
core node name
[junit4] 2> 156532 T429 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52702";,
[junit4] 2> "node_name":"127.0.0.1:52702_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 156532 T429 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=1
[junit4] 2> 156532 T429 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 156637 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 156637 T473 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 156637 T454 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 156637 T440 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 157530 T474 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 157530 T474 oasc.ZkController.createCollectionZkNode Check for
collection zkNode:collection1
[junit4] 2> 157531 T474 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 157531 T474 oascc.ZkStateReader.readConfigName Load collection
config from:/collections/collection1
[junit4] 2> 157532 T474 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 157532 T474 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/collection1/'
[junit4] 2> 157533 T474 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/collection1/lib/.svn/'
to classloader
[junit4] 2> 157533 T474 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/collection1/lib/classes/'
to classloader
[junit4] 2> 157533 T474 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/collection1/lib/README'
to classloader
[junit4] 2> 157554 T474 oasc.SolrConfig.<init> Using Lucene MatchVersion:
5.0.0
[junit4] 2> 157563 T474 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 157564 T474 oass.IndexSchema.readSchema Reading Solr Schema
from schema.xml
[junit4] 2> 157568 T474 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 157670 T474 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 157671 T474 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 157672 T474 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 157674 T474 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 157688 T474 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 157688 T474 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 157688 T474 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-004/collection1/,
dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty2/
[junit4] 2> 157688 T474 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@fb0e367
[junit4] 2> 157689 T474 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty2
[junit4] 2> 157689 T474 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty2/index/
[junit4] 2> 157689 T474 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty2/index'
doesn't exist. Creating new index...
[junit4] 2> 157690 T474 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty2/index
[junit4] 2> 157690 T474 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=14, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=76.9111328125,
floorSegmentMB=0.7978515625, forceMergeDeletesPctAllowed=19.695317452037482,
segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 157697 T474 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 157697 T474 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 157702 T474 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 157702 T474 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 157703 T474 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 157703 T474 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 157703 T474 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 157703 T474 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 157703 T474 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 157704 T474 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 157704 T474 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 157704 T474 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 157705 T474 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 157705 T474 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 157706 T474 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 157707 T474 oasc.RequestHandlers.initHandlersFromConfig
created /get: solr.RealTimeGetHandler
[junit4] 2> 157707 T474 oasc.RequestHandlers.initHandlersFromConfig adding
lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 157708 T474 oasc.RequestHandlers.initHandlersFromConfig
created /replication: solr.ReplicationHandler
[junit4] 2> 157709 T474 oasc.RequestHandlers.initHandlersFromConfig
created /update: solr.UpdateRequestHandler
[junit4] 2> 157709 T474 oasc.RequestHandlers.initHandlersFromConfig
created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 157710 T474 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 157710 T474 oasc.RequestHandlers.initHandlersFromConfig
created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 157712 T474 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 157712 T474 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 157713 T474 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 157714 T474 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 157715 T474 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 157715 T474 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 157717 T474 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=false, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7946915014752227]
[junit4] 2> 157718 T474 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9C3D118D0592C81B-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 157718 T474 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 157718 T474 oass.SolrIndexSearcher.<init> Opening
Searcher@439b8fd0[collection1] main
[junit4] 2> 157718 T474 oascc.ZkStateReader.readConfigName Load collection
config from:/collections/collection1
[junit4] 2> 157719 T474 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 157719 T474 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 157720 T474
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 157720 T474 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 157720 T474 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 157721 T474
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes
from znode /configs/conf1/_rest_managed.json
[junit4] 2> 157721 T474 oasr.ManagedResourceStorage.load Loaded
LinkedHashMap at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 157721 T474 oasr.ManagedResource.reloadFromStorage Loaded
initArgs {} for /rest/managed
[junit4] 2> 157722 T474 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 157725 T475 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@439b8fd0[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 157736 T474 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 157736 T478 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:52702 collection:collection1
shard:shard1
[junit4] 2> 157737 T404 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 157737 T404 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 157741 T478 oasc.ZkController.register We are
https://127.0.0.1:52702/collection1/ and leader is
https://127.0.0.1:60084/collection1/
[junit4] 2> 157741 T478 oasc.ZkController.register No LogReplay needed for
core=collection1 baseURL=https://127.0.0.1:52702
[junit4] 2> 157741 T478 oasc.ZkController.checkRecovery Core needs to
recover:collection1
[junit4] 2> 157741 T404
oasc.AbstractFullDistribZkTestBase.randomlyEnableAutoSoftCommit Not turning on
auto soft commit
[junit4] 2> 157741 T478 oasu.DefaultSolrCoreState.doRecovery Running
recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C677 name=collection1
org.apache.solr.core.SolrCore@28e224c8 url=https://127.0.0.1:52702/collection1
node=127.0.0.1:52702_ C677_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:52702,
node_name=127.0.0.1:52702_, state=down}
[junit4] 2> 157749 T479 C677 P52702 oasc.RecoveryStrategy.run Starting
recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 157750 T479 C677 P52702 oasc.RecoveryStrategy.doRecovery
###### startupVersions=[]
[junit4] 2> 157750 T479 C677 P52702 oasc.RecoveryStrategy.doRecovery
Publishing state of core collection1 as recovering, leader is
https://127.0.0.1:60084/collection1/ and I am
https://127.0.0.1:52702/collection1/
[junit4] 2> 157751 T479 C677 P52702 oasc.ZkController.publish publishing
core=collection1 state=recovering collection=collection1
[junit4] 2> 157751 T479 C677 P52702 oasc.ZkController.publish numShards
not found on descriptor - reading it from system property
[junit4] 2> 157752 T433 oasc.DistributedQueue$LatchChildWatcher.process
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type
NodeChildrenChanged
[junit4] 2> 157754 T429 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:52702";,
[junit4] 2> "node_name":"127.0.0.1:52702_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"recovering",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 157755 T479 C677 P52702
oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to
https://127.0.0.1:60084; WaitForState:
action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A52702_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
[junit4] 2> ASYNC NEW_CORE C678 name=collection1
org.apache.solr.core.SolrCore@52ead23e url=https://127.0.0.1:40473/collection1
node=127.0.0.1:40473_ C678_STATE=coll:control_collection core:collection1
props:{core=collection1, base_url=https://127.0.0.1:40473,
node_name=127.0.0.1:40473_, state=active, leader=true}
[junit4] 2> 157816 T422 C678 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2}
{deleteByQuery=*:* (-1481939127242850304)} 0 8
[junit4] 2> 157836 T446 oasha.CoreAdminHandler.handleWaitForStateAction
Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true,
onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 157838 T446 oasha.CoreAdminHandler.handleWaitForStateAction
Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have
state: recovering
[junit4] 2> 157839 T446 oasha.CoreAdminHandler.handleWaitForStateAction In
WaitForState(recovering): collection=collection1, shard=shard1,
thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true,
live=true, checkLive=true, currentState=down, localState=active,
nodeName=127.0.0.1:52702_, coreNodeName=core_node2,
onlyIfActiveCheckResult=false, nodeProps:
core_node2:{"core":"collection1","base_url":"https://127.0.0.1:52702","node_name":"127.0.0.1:52702_","state":"down"}
[junit4] 2> 157858 T433 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 157859 T454 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 157860 T473 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 157860 T440 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> C677_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:52702,
node_name=127.0.0.1:52702_, state=recovering}
[junit4] 2> 157962 T466 C677 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&_version_=-1481939127319396352&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{deleteByQuery=*:* (-1481939127319396352)} 0 2
[junit4] 2> ASYNC NEW_CORE C679 name=collection1
org.apache.solr.core.SolrCore@16f8a6b2 url=https://127.0.0.1:60084/collection1
node=127.0.0.1:60084_ C679_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:60084,
node_name=127.0.0.1:60084_, state=active, leader=true}
[junit4] 2> 157963 T448 C679 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2}
{deleteByQuery=*:* (-1481939127319396352)} 0 80
[junit4] 2> 157966 T404 oasc.ChaosMonkey.monkeyLog monkey: starting
[junit4] 2> 157967 T404 oasc.ChaosMonkey.monkeyLog monkey: Jetty will not
commit on close
[junit4] 2> 157979 T423 C678 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[1-0 (1481939127413768192)]} 0 3
[junit4] 2> 157979 T424 C678 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[0-0 (1481939127414816768)]} 0 2
[junit4] 2> 158027 T467 C677 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[1-0 (1481939127445225472)]} 0 10
[junit4] 2> 158027 T465 C677 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[0-0 (1481939127445225473)]} 0 2
[junit4] 2> 158028 T448 C679 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0-0
(1481939127445225473)]} 0 23
[junit4] 2> 158037 T449 C679 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1-0
(1481939127445225472)]} 0 33
[junit4] 2> ASYNC NEW_CORE C680 name=collection1
org.apache.solr.core.SolrCore@52ead23e url=https://127.0.0.1:40473/collection1
node=127.0.0.1:40473_ C680_STATE=coll:control_collection core:collection1
props:{core=collection1, base_url=https://127.0.0.1:40473,
node_name=127.0.0.1:40473_, state=active, leader=true}
[junit4] 2> 158116 T422 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{delete=[1-0 (-1481939127560568833)]} 0 0
[junit4] 2> 158116 T423 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{delete=[0-0 (-1481939127560568832)]} 0 0
[junit4] 2> ASYNC NEW_CORE C681 name=collection1
org.apache.solr.core.SolrCore@28e224c8 url=https://127.0.0.1:52702/collection1
node=127.0.0.1:52702_ C681_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:52702,
node_name=127.0.0.1:52702_, state=recovering}
[junit4] 2> 158128 T465 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{delete=[0-0 (-1481939127567908864)]} 0 1
[junit4] 2> ASYNC NEW_CORE C682 name=collection1
org.apache.solr.core.SolrCore@16f8a6b2 url=https://127.0.0.1:60084/collection1
node=127.0.0.1:60084_ C682_STATE=coll:collection1 core:collection1
props:{core=collection1, base_url=https://127.0.0.1:60084,
node_name=127.0.0.1:60084_, state=active, leader=true}
[junit4] 2> 158129 T449 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[0-0
(-1481939127567908864)]} 0 7
[junit4] 2> 158133 T468 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{delete=[1-0 (-1481939127567908865)]} 0 0
[junit4] 2> 158134 T447 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[1-0
(-1481939127567908865)]} 0 11
[junit4] 2> 158136 T424 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[0-1 (1481939127580491776)]} 0 1
[junit4] 2> 158139 T421 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[1-1 (1481939127583637504)]} 0 1
[junit4] 2> 158153 T468 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[1-1 (1481939127594123264)]} 0 1
[junit4] 2> 158157 T447 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1-1
(1481939127594123264)]} 0 10
[junit4] 2> 158159 T465 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[0-1 (1481939127592026112)]} 0 1
[junit4] 2> 158160 T449 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0-1
(1481939127592026112)]} 0 14
[junit4] 2> 158182 T484 oasc.ChaosMonkey.monkeyLog monkey: only one active
node in shard - monkey cannot kill :(
[junit4] 2> 158216 T424 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[1-2 (1481939127665426432)]} 0 1
[junit4] 2> 158217 T421 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[0-2 (1481939127666475008)]} 0 0
[junit4] 2> 158229 T466 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[1-2 (1481939127671717888)]} 0 1
[junit4] 2> 158229 T468 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[0-2 (1481939127673815040)]} 0 1
[junit4] 2> 158230 T448 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1-2
(1481939127671717888)]} 0 9
[junit4] 2> 158230 T449 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0-2
(1481939127673815040)]} 0 6
[junit4] 2> 158247 T422 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[1-3 (1481939127696883713)]} 0 1
[junit4] 2> 158247 T423 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[0-3 (1481939127696883712)]} 0 1
[junit4] 2> 158260 T466 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[0-3 (1481939127706320896)]} 0 1
[junit4] 2> 158261 T467 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[1-3 (1481939127706320897)]} 0 0
[junit4] 2> 158262 T448 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1-3
(1481939127706320897)]} 0 7
[junit4] 2> 158262 T447 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0-3
(1481939127706320896)]} 0 8
[junit4] 2> 158320 T423 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{delete=[1-3 (-1481939127774478336)]} 0 1
[junit4] 2> 158321 T423 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{delete=[0-3 (-1481939127775526912)]} 0 0
[junit4] 2> 158329 T468 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{delete=[0-3 (-1481939127779721216)]} 0 0
[junit4] 2> 158329 T466 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{delete=[1-3 (-1481939127779721217)]} 0 0
[junit4] 2> 158330 T448 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[0-3
(-1481939127779721216)]} 0 5
[junit4] 2> 158330 T447 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {delete=[1-3
(-1481939127779721217)]} 0 5
[junit4] 2> 158336 T421 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[0-4 (1481939127790206976)]} 0 1
[junit4] 2> 158337 T422 C680 P40473 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2&CONTROL=TRUE}
{add=[1-4 (1481939127792304128)]} 0 0
[junit4] 2> 158347 T468 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[0-4 (1481939127798595584)]} 0 0
[junit4] 2> 158348 T448 C682 P60084 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0-4
(1481939127798595584)]} 0 5
[junit4] 2> 158349 T468 C681 P52702 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60084/collection1/&wt=javabin&version=2}
{add=[1-4 (14819
[...truncated too long message...]
a:101)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> 12) Thread[id=404,
name=TEST-ChaosMonkeySafeLeaderTest.testDistribSearch-seed#[9C3D118D0592C81B],
state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Throwable.fillInStackTrace(Native
Method)
[junit4] 2> at
java.lang.Throwable.fillInStackTrace(Throwable.java:783)
[junit4] 2> at
java.lang.Throwable.<init>(Throwable.java:250)
[junit4] 2> at java.lang.Exception.<init>(Exception.java:54)
[junit4] 2> at
java.lang.InterruptedException.<init>(InterruptedException.java:57)
[junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2067)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1465)
[junit4] 2> at
org.apache.solr.common.util.ExecutorUtil.shutdownAndAwaitTermination(ExecutorUtil.java:54)
[junit4] 2> at
org.apache.solr.update.UpdateShardHandler.close(UpdateShardHandler.java:83)
[junit4] 2> at
org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:381)
[junit4] 2> at
org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:195)
[junit4] 2> at
org.apache.solr.cloud.ChaosMonkey.stopJettySolrRunner(ChaosMonkey.java:197)
[junit4] 2> at
org.apache.solr.cloud.ChaosMonkey.stop(ChaosMonkey.java:550)
[junit4] 2> at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.destroyServers(AbstractFullDistribZkTestBase.java:1577)
[junit4] 2> at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at
java.lang.reflect.Method.invoke(Method.java:497)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50):
{rnd_b=OrdsLucene41(blocksize=128), _version_=FST41, a_t=FST41,
a_i=OrdsLucene41(blocksize=128), id=OrdsLucene41(blocksize=128)}, docValues:{},
sim=DefaultSimilarity, locale=es_MX, timezone=PRC
[junit4] 2> NOTE: Linux 3.13.0-37-generic amd64/Oracle Corporation
1.8.0_40-ea (64-bit)/cpus=8,threads=15,free=46881064,total=163840000
[junit4] 2> NOTE: All tests run in this JVM: [HdfsBasicDistributedZkTest,
IndexSchemaRuntimeFieldTest, SolrCoreTest, TestCSVLoader, TestIndexSearcher,
UUIDUpdateProcessorFallbackTest, TestIndexingPerformance, CopyFieldTest,
TestExtendedDismaxParser, TestManagedStopFilterFactory,
PreAnalyzedUpdateProcessorTest, TestElisionMultitermQuery,
ChangedSchemaMergeTest, SuggesterFSTTest, TestQueryUtils, TestLFUCache,
DebugComponentTest, ResourceLoaderTest, StatsComponentTest, TestRealTimeGet,
DistributedDebugComponentTest, URLClassifyProcessorTest, SoftAutoCommitTest,
TestManagedResource, TestCharFilters, TestDistributedGrouping,
CoreMergeIndexesAdminHandlerTest, TestPHPSerializedResponseWriter,
SynonymTokenizerTest, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=9C3D118D0592C81B
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_MX
-Dtests.timezone=PRC -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>=
7200000 msec).
[junit4] > at
__randomizedtesting.SeedInfo.seed([9C3D118D0592C81B]:0)
[junit4] Completed on J1 in 7223.30s, 1 test, 2 errors <<< FAILURES!
[...truncated 250 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:524: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:472: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:189: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:496: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1360:
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:962:
There were test failures: 437 suites, 1785 tests, 1 suite-level error, 1 error,
318 ignored (21 assumptions)
Total time: 175 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09
-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
