Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.10-Linux/50/
Java: 32bit/jdk1.8.0_31 -client -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([E3E372A485BB9D2C]:0)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([E3E372A485BB9D2C]:0)
Build Log:
[...truncated 12571 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/init-core-data-001
[junit4] 2> 1634839 T3373 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl
(true) and clientAuth (true)
[junit4] 2> 1634839 T3373
oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system
property: /e/h
[junit4] 2> 1634845 T3373 oas.SolrTestCaseJ4.setUp ###Starting
testDistribSearch
[junit4] 2> 1634846 T3373 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1634847 T3374 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 1634946 T3373 oasc.ZkTestServer.run start zk server on
port:51954
[junit4] 2> 1634947 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1634950 T3380 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@13f54cd name:ZooKeeperConnection
Watcher:127.0.0.1:51954 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1634951 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1634951 T3373 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1634955 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1634962 T3382 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@788094 name:ZooKeeperConnection
Watcher:127.0.0.1:51954/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1634963 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1634963 T3373 oascc.SolrZkClient.makePath makePath:
/collections/collection1
[junit4] 2> 1634967 T3373 oascc.SolrZkClient.makePath makePath:
/collections/collection1/shards
[junit4] 2> 1634970 T3373 oascc.SolrZkClient.makePath makePath:
/collections/control_collection
[junit4] 2> 1634972 T3373 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/shards
[junit4] 2> 1634976 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
to /configs/conf1/solrconfig.xml
[junit4] 2> 1634977 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.xml
[junit4] 2> 1634981 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml
to /configs/conf1/schema.xml
[junit4] 2> 1634982 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/schema.xml
[junit4] 2> 1634985 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1634986 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1634988 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
to /configs/conf1/stopwords.txt
[junit4] 2> 1634989 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/stopwords.txt
[junit4] 2> 1634992 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
to /configs/conf1/protwords.txt
[junit4] 2> 1634992 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/protwords.txt
[junit4] 2> 1634995 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
to /configs/conf1/currency.xml
[junit4] 2> 1634996 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/currency.xml
[junit4] 2> 1634998 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
to /configs/conf1/enumsConfig.xml
[junit4] 2> 1634999 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/enumsConfig.xml
[junit4] 2> 1635001 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1635002 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/open-exchange-rates.json
[junit4] 2> 1635004 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1635005 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1635007 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
to /configs/conf1/old_synonyms.txt
[junit4] 2> 1635008 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/old_synonyms.txt
[junit4] 2> 1635010 T3373 oasc.AbstractZkTestCase.putConfig put
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
to /configs/conf1/synonyms.txt
[junit4] 2> 1635011 T3373 oascc.SolrZkClient.makePath makePath:
/configs/conf1/synonyms.txt
[junit4] 2> 1635014 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1635016 T3384 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1fcf19d name:ZooKeeperConnection
Watcher:127.0.0.1:51954/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1635016 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1635334 T3373 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1635343 T3373 oejus.SslContextFactory.doStart Enabled
Protocols [SSLv2Hello, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1,
TLSv1.1, TLSv1.2]
[junit4] 2> 1635350 T3373 oejs.AbstractConnector.doStart Started
[email protected]:47978
[junit4] 2> 1635353 T3373 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()
[junit4] 2> 1635353 T3373 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 1635353 T3373 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002
[junit4] 2> 1635354 T3373 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/'
[junit4] 2> 1635383 T3373 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/solr.xml
[junit4] 2> 1635406 T3373 oasc.CoreContainer.<init> New CoreContainer
844353
[junit4] 2> 1635407 T3373 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/]
[junit4] 2> 1635410 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 1635411 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 1635411 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 1635411 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 1635412 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 1635412 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 1635412 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 1635413 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 1635413 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 1635413 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting useRetries to: false
[junit4] 2> 1635415 T3373 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1635417 T3373 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1635418 T3373 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1635418 T3373 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1635419 T3373 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:51954/solr
[junit4] 2> 1635419 T3373 oasc.ZkController.checkChrootPath zkHost
includes chroot
[junit4] 2> 1635420 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1635423 T3395 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@6b4bec name:ZooKeeperConnection
Watcher:127.0.0.1:51954 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1635423 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1635432 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1635438 T3397 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1955890 name:ZooKeeperConnection
Watcher:127.0.0.1:51954/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1635438 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1635440 T3373 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 1635444 T3373 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 1635448 T3373 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 1635457 T3373 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 1635461 T3373 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 1635466 T3373 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1635468 T3373 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47978_e%2Fh
[junit4] 2> 1635469 T3373 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:47978_e%2Fh
[junit4] 2> 1635472 T3373 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 1635474 T3373 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 1635476 T3373 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1635478 T3373 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:47978_e%2Fh
[junit4] 2> 1635479 T3373 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 1635482 T3373 oasc.Overseer.start Overseer
(id=93411633543249924-127.0.0.1:47978_e%2Fh-n_0000000000) starting
[junit4] 2> 1635484 T3373 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 1635494 T3373 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1635494 T3399 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 1635496 T3373 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 1635498 T3373 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 1635500 T3373
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1635503 T3398 oasc.Overseer$ClusterStateUpdater.run Starting
to work on the main queue
[junit4] 2> 1635505 T3401 oasc.ZkController.publish publishing
core=collection1 state=down collection=control_collection
[junit4] 2> 1635506 T3401 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 1635507 T3401 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 1635507 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1635510 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:47978/e/h";,
[junit4] 2> "node_name":"127.0.0.1:47978_e%2Fh",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1635510 T3398
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
control_collection with shards [shard1]
[junit4] 2> 1635510 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1635514 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1636507 T3401 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 1636508 T3401 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:control_collection
[junit4] 2> 1636509 T3401 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 1636509 T3401 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/control_collection
[junit4] 2> 1636510 T3401 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 1636510 T3401 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/collection1/'
[junit4] 2> 1636511 T3401 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/collection1/lib/.svn/'
to classloader
[junit4] 2> 1636511 T3401 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/collection1/lib/classes/'
to classloader
[junit4] 2> 1636512 T3401 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/collection1/lib/README'
to classloader
[junit4] 2> 1636553 T3401 oasc.SolrConfig.<init> Using Lucene
MatchVersion: 4.10.4
[junit4] 2> 1636571 T3401 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 1636573 T3401 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 1636592 T3401 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 1636731 T3401 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 1636733 T3401 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1636734 T3401 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1636737 T3401 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1636775 T3401 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection control_collection
[junit4] 2> 1636776 T3401 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 1636776 T3401 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-002/collection1/,
dataDir=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/control/data/
[junit4] 2> 1636777 T3401 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@c5fb5f
[junit4] 2> 1636778 T3401 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/control/data
[junit4] 2> 1636778 T3401 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/control/data/index/
[junit4] 2> 1636779 T3401 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/control/data/index'
doesn't exist. Creating new index...
[junit4] 2> 1636779 T3401 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/control/data/index
[junit4] 2> 1636780 T3401 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=false, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.44156486996929667]
[junit4] 2> 1636884 T3401 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 1636884 T3401 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 1636893 T3401 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 1636894 T3401 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 1636895 T3401 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1636895 T3401 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 1636895 T3401 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1636896 T3401 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1636896 T3401 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1636897 T3401 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 1636897 T3401 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1636898 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1636899 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1636899 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1636900 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1636901 T3401 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 1636901 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /get: solr.RealTimeGetHandler
[junit4] 2> 1636902 T3401 oasc.RequestHandlers.initHandlersFromConfig
adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1636902 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /replication: solr.ReplicationHandler
[junit4] 2> 1636903 T3401 oasc.RequestHandlers.initHandlersFromConfig WARN
Multiple requestHandler registered to the same name: /update ignoring:
org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1636903 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /update: solr.UpdateRequestHandler
[junit4] 2> 1636904 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1636905 T3401 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1636907 T3401 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1636909 T3401 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1636911 T3401 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1636912 T3401 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1636914 T3401 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1636916 T3401 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 1636916 T3401 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 1636917 T3401 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=12, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=41.8095703125,
floorSegmentMB=1.3134765625, forceMergeDeletesPctAllowed=25.126880907336027,
segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.8199693950602195
[junit4] 2> 1636919 T3401 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 1636919 T3401 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 1636920 T3401 oass.SolrIndexSearcher.<init> Opening
Searcher@a268a8[collection1] main
[junit4] 2> 1636921 T3401 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/control_collection
[junit4] 2> 1636922 T3401 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 1636922 T3401 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1636923 T3401
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1636923 T3401 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 1636924 T3401 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1636924 T3401
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1636925 T3401 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1636926 T3401 oascc.SolrZkClient.makePath makePath:
/configs/conf1/_rest_managed.json
[junit4] 2> 1636929 T3401
oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new
znode /configs/conf1/_rest_managed.json
[junit4] 2> 1636929 T3401 oasr.ManagedResourceStorage$JsonStorage.store
Saved JSON object to path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1636929 T3401 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 1636932 T3403 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@a268a8[collection1]
main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 1636938 T3401 oasc.CoreContainer.registerCore registering
core: collection1
[junit4] 2> 1636938 T3406 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:47978/e/h
collection:control_collection shard:shard1
[junit4] 2> 1636939 T3373 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1
[junit4] 2> 1636939 T3373 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 1636940 T3406 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1636944 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1636945 T3408 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1656c8 name:ZooKeeperConnection
Watcher:127.0.0.1:51954/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1636957 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1636959 T3406 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard1
[junit4] 2> 1636959 T3373
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1636961 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1636961 T3406
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 1636961 T3406 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C13079 name=collection1
org.apache.solr.core.SolrCore@114615e
url=https://127.0.0.1:47978/e/h/collection1 node=127.0.0.1:47978_e%2Fh
C13079_STATE=coll:control_collection core:collection1 props:{core=collection1,
base_url=https://127.0.0.1:47978/e/h, node_name=127.0.0.1:47978_e%2Fh,
state=down}
[junit4] 2> 1636962 T3406 C13079 P47978 oasc.SyncStrategy.sync Sync
replicas to https://127.0.0.1:47978/e/h/collection1/
[junit4] 2> 1636961 T3373 oasc.ChaosMonkey.monkeyLog monkey: init - expire
sessions:false cause connection loss:false
[junit4] 2> 1636963 T3406 C13079 P47978 oasc.SyncStrategy.syncReplicas
Sync Success - now sync replicas to me
[junit4] 2> 1636964 T3406 C13079 P47978 oasc.SyncStrategy.syncToMe
https://127.0.0.1:47978/e/h/collection1/ has no replicas
[junit4] 2> 1636965 T3406 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: https://127.0.0.1:47978/e/h/collection1/ shard1
[junit4] 2> 1636965 T3406 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leaders/shard1
[junit4] 2> 1636970 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1637075 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1637075 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1637122 T3406 oasc.ZkController.register We are
https://127.0.0.1:47978/e/h/collection1/ and leader is
https://127.0.0.1:47978/e/h/collection1/
[junit4] 2> 1637122 T3406 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=https://127.0.0.1:47978/e/h
[junit4] 2> 1637122 T3406 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 1637123 T3406 oasc.ZkController.publish publishing
core=collection1 state=active collection=control_collection
[junit4] 2> 1637123 T3406 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 1637125 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1637127 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:47978/e/h";,
[junit4] 2> "node_name":"127.0.0.1:47978_e%2Fh",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1637230 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1637231 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1637298 T3373 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 1
[junit4] 2> 1637299 T3373 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1637305 T3373 oejus.SslContextFactory.doStart Enabled
Protocols [SSLv2Hello, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1,
TLSv1.1, TLSv1.2]
[junit4] 2> 1637310 T3373 oejs.AbstractConnector.doStart Started
[email protected]:33495
[junit4] 2> 1637315 T3373 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()
[junit4] 2> 1637315 T3373 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 1637316 T3373 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003
[junit4] 2> 1637316 T3373 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/'
[junit4] 2> 1637339 T3373 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/solr.xml
[junit4] 2> 1637364 T3373 oasc.CoreContainer.<init> New CoreContainer
23537299
[junit4] 2> 1637364 T3373 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/]
[junit4] 2> 1637365 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 1637366 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 1637366 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 1637366 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 1637367 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 1637367 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 1637368 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 1637368 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 1637368 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 1637369 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting useRetries to: false
[junit4] 2> 1637371 T3373 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1637372 T3373 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1637373 T3373 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1637373 T3373 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1637374 T3373 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:51954/solr
[junit4] 2> 1637374 T3373 oasc.ZkController.checkChrootPath zkHost
includes chroot
[junit4] 2> 1637375 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1637377 T3420 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1e7f7cf name:ZooKeeperConnection
Watcher:127.0.0.1:51954 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1637377 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1637382 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1637383 T3422 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@98d987 name:ZooKeeperConnection
Watcher:127.0.0.1:51954/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1637384 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1637392 T3373
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1638397 T3373 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33495_e%2Fh
[junit4] 2> 1638398 T3373 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:33495_e%2Fh
[junit4] 2> 1638402 T3373 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1638407 T3424 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 1638407 T3424 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 1638408 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1638408 T3424 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 1638410 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33495/e/h";,
[junit4] 2> "node_name":"127.0.0.1:33495_e%2Fh",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1638410 T3398
oasc.Overseer$ClusterStateUpdater.createCollection Create collection
collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 1638411 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard2
[junit4] 2> 1638514 T3423 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1638514 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1638514 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1639409 T3424 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 1639409 T3424 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:collection1
[junit4] 2> 1639410 T3424 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 1639410 T3424 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 1639411 T3424 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 1639411 T3424 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/collection1/'
[junit4] 2> 1639413 T3424 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/collection1/lib/.svn/'
to classloader
[junit4] 2> 1639413 T3424 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/collection1/lib/classes/'
to classloader
[junit4] 2> 1639414 T3424 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/collection1/lib/README'
to classloader
[junit4] 2> 1639452 T3424 oasc.SolrConfig.<init> Using Lucene
MatchVersion: 4.10.4
[junit4] 2> 1639469 T3424 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 1639471 T3424 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 1639485 T3424 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 1639623 T3424 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 1639625 T3424 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1639627 T3424 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1639631 T3424 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1639655 T3424 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 1639656 T3424 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 1639656 T3424 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-003/collection1/,
dataDir=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty1/
[junit4] 2> 1639656 T3424 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@c5fb5f
[junit4] 2> 1639661 T3424 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty1
[junit4] 2> 1639662 T3424 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty1/index/
[junit4] 2> 1639662 T3424 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty1/index'
doesn't exist. Creating new index...
[junit4] 2> 1639663 T3424 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty1/index
[junit4] 2> 1639664 T3424 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=false, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.44156486996929667]
[junit4] 2> 1639694 T3424 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 1639694 T3424 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 1639700 T3424 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 1639701 T3424 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 1639701 T3424 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1639702 T3424 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 1639702 T3424 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1639702 T3424 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1639703 T3424 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1639703 T3424 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 1639704 T3424 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1639706 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1639707 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1639709 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1639710 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1639711 T3424 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 1639713 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /get: solr.RealTimeGetHandler
[junit4] 2> 1639713 T3424 oasc.RequestHandlers.initHandlersFromConfig
adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1639714 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /replication: solr.ReplicationHandler
[junit4] 2> 1639716 T3424 oasc.RequestHandlers.initHandlersFromConfig WARN
Multiple requestHandler registered to the same name: /update ignoring:
org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1639716 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /update: solr.UpdateRequestHandler
[junit4] 2> 1639717 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1639718 T3424 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1639722 T3424 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1639723 T3424 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1639725 T3424 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1639726 T3424 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1639728 T3424 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1639729 T3424 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 1639730 T3424 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 1639731 T3424 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=12, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=41.8095703125,
floorSegmentMB=1.3134765625, forceMergeDeletesPctAllowed=25.126880907336027,
segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.8199693950602195
[junit4] 2> 1639732 T3424 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 1639732 T3424 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 1639733 T3424 oass.SolrIndexSearcher.<init> Opening
Searcher@fb9d75[collection1] main
[junit4] 2> 1639733 T3424 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 1639734 T3424 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 1639734 T3424 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1639735 T3424
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1639735 T3424 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 1639735 T3424 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1639736 T3424
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes
from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1639737 T3424 oasr.ManagedResourceStorage.load Loaded
LinkedHashMap at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1639737 T3424 oasr.ManagedResource.reloadFromStorage Loaded
initArgs {} for /rest/managed
[junit4] 2> 1639737 T3424 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 1639743 T3425 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@fb9d75[collection1]
main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 1639747 T3424 oasc.CoreContainer.registerCore registering
core: collection1
[junit4] 2> 1639748 T3428 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:33495/e/h collection:collection1
shard:shard2
[junit4] 2> 1639749 T3373 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1
[junit4] 2> 1639749 T3428 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard2/election
[junit4] 2> 1639749 T3373 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 1639754 T3428 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard2
[junit4] 2> 1639756 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1639756 T3428
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 1639756 T3428 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C13080 name=collection1
org.apache.solr.core.SolrCore@1278c66
url=https://127.0.0.1:33495/e/h/collection1 node=127.0.0.1:33495_e%2Fh
C13080_STATE=coll:collection1 core:collection1 props:{core=collection1,
base_url=https://127.0.0.1:33495/e/h, node_name=127.0.0.1:33495_e%2Fh,
state=down}
[junit4] 2> 1639757 T3428 C13080 P33495 oasc.SyncStrategy.sync Sync
replicas to https://127.0.0.1:33495/e/h/collection1/
[junit4] 2> 1639757 T3428 C13080 P33495 oasc.SyncStrategy.syncReplicas
Sync Success - now sync replicas to me
[junit4] 2> 1639757 T3428 C13080 P33495 oasc.SyncStrategy.syncToMe
https://127.0.0.1:33495/e/h/collection1/ has no replicas
[junit4] 2> 1639758 T3428 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: https://127.0.0.1:33495/e/h/collection1/ shard2
[junit4] 2> 1639758 T3428 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard2
[junit4] 2> 1639762 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1639867 T3423 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1639867 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1639867 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1639878 T3428 oasc.ZkController.register We are
https://127.0.0.1:33495/e/h/collection1/ and leader is
https://127.0.0.1:33495/e/h/collection1/
[junit4] 2> 1639878 T3428 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=https://127.0.0.1:33495/e/h
[junit4] 2> 1639878 T3428 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 1639879 T3428 oasc.ZkController.publish publishing
core=collection1 state=active collection=collection1
[junit4] 2> 1639879 T3428 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 1639881 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1639883 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33495/e/h";,
[junit4] 2> "node_name":"127.0.0.1:33495_e%2Fh",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1639987 T3423 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1639987 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1639987 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1640054 T3373 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 2
[junit4] 2> 1640056 T3373 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1640064 T3373 oejus.SslContextFactory.doStart Enabled
Protocols [SSLv2Hello, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1,
TLSv1.1, TLSv1.2]
[junit4] 2> 1640070 T3373 oejs.AbstractConnector.doStart Started
[email protected]:33180
[junit4] 2> 1640073 T3373 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()
[junit4] 2> 1640073 T3373 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 1640074 T3373 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004
[junit4] 2> 1640074 T3373 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/'
[junit4] 2> 1640098 T3373 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/solr.xml
[junit4] 2> 1640122 T3373 oasc.CoreContainer.<init> New CoreContainer
9012414
[junit4] 2> 1640122 T3373 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/]
[junit4] 2> 1640123 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 1640124 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 1640124 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 1640124 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 1640125 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 1640125 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 1640126 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 1640126 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 1640126 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 1640127 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting useRetries to: false
[junit4] 2> 1640129 T3373 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1640130 T3373 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1640131 T3373 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1640131 T3373 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1640131 T3373 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:51954/solr
[junit4] 2> 1640132 T3373 oasc.ZkController.checkChrootPath zkHost
includes chroot
[junit4] 2> 1640133 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1640139 T3439 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@e52dbf name:ZooKeeperConnection
Watcher:127.0.0.1:51954 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1640139 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1640142 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1640144 T3441 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@2160b4 name:ZooKeeperConnection
Watcher:127.0.0.1:51954/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1640144 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1640150 T3373
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1641154 T3373 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33180_e%2Fh
[junit4] 2> 1641155 T3373 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:33180_e%2Fh
[junit4] 2> 1641158 T3373 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1641163 T3443 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 1641163 T3443 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 1641164 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1641164 T3443 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 1641166 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33180/e/h";,
[junit4] 2> "node_name":"127.0.0.1:33180_e%2Fh",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1641166 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1641167 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard3
[junit4] 2> 1641270 T3442 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1641270 T3423 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1641270 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1641270 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642165 T3443 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 1642165 T3443 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:collection1
[junit4] 2> 1642166 T3443 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 1642166 T3443 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 1642167 T3443 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 1642167 T3443 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/collection1/'
[junit4] 2> 1642168 T3443 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/collection1/lib/.svn/'
to classloader
[junit4] 2> 1642168 T3443 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/collection1/lib/classes/'
to classloader
[junit4] 2> 1642169 T3443 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/collection1/lib/README'
to classloader
[junit4] 2> 1642201 T3443 oasc.SolrConfig.<init> Using Lucene
MatchVersion: 4.10.4
[junit4] 2> 1642222 T3443 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 1642224 T3443 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 1642240 T3443 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 1642399 T3443 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 1642401 T3443 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1642402 T3443 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1642410 T3443 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1642434 T3443 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 1642434 T3443 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 1642434 T3443 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-004/collection1/,
dataDir=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty2/
[junit4] 2> 1642435 T3443 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@c5fb5f
[junit4] 2> 1642436 T3443 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty2
[junit4] 2> 1642436 T3443 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty2/index/
[junit4] 2> 1642437 T3443 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty2/index'
doesn't exist. Creating new index...
[junit4] 2> 1642437 T3443 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty2/index
[junit4] 2> 1642438 T3443 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=false, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.44156486996929667]
[junit4] 2> 1642464 T3443 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 1642464 T3443 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 1642473 T3443 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 1642473 T3443 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 1642474 T3443 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1642474 T3443 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 1642474 T3443 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1642475 T3443 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1642475 T3443 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1642476 T3443 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 1642476 T3443 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1642477 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1642478 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1642479 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1642480 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1642481 T3443 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 1642482 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /get: solr.RealTimeGetHandler
[junit4] 2> 1642482 T3443 oasc.RequestHandlers.initHandlersFromConfig
adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1642483 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /replication: solr.ReplicationHandler
[junit4] 2> 1642484 T3443 oasc.RequestHandlers.initHandlersFromConfig WARN
Multiple requestHandler registered to the same name: /update ignoring:
org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1642484 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /update: solr.UpdateRequestHandler
[junit4] 2> 1642485 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1642486 T3443 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1642489 T3443 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1642490 T3443 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1642491 T3443 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1642493 T3443 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1642494 T3443 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1642496 T3443 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 1642496 T3443 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 1642497 T3443 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=12, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=41.8095703125,
floorSegmentMB=1.3134765625, forceMergeDeletesPctAllowed=25.126880907336027,
segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.8199693950602195
[junit4] 2> 1642499 T3443 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 1642499 T3443 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 1642499 T3443 oass.SolrIndexSearcher.<init> Opening
Searcher@30693a[collection1] main
[junit4] 2> 1642500 T3443 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 1642501 T3443 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 1642501 T3443 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1642501 T3443
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1642502 T3443 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 1642502 T3443 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1642503 T3443
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes
from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1642503 T3443 oasr.ManagedResourceStorage.load Loaded
LinkedHashMap at path _rest_managed.json using
ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1642504 T3443 oasr.ManagedResource.reloadFromStorage Loaded
initArgs {} for /rest/managed
[junit4] 2> 1642504 T3443 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 1642510 T3444 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@30693a[collection1]
main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 1642517 T3443 oasc.CoreContainer.registerCore registering
core: collection1
[junit4] 2> 1642517 T3447 oasc.ZkController.register Register replica -
core:collection1 address:https://127.0.0.1:33180/e/h collection:collection1
shard:shard3
[junit4] 2> 1642518 T3373 oass.SolrDispatchFilter.init
user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1
[junit4] 2> 1642518 T3373 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 1642518 T3447 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard3/election
[junit4] 2> 1642523 T3447 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard3
[junit4] 2> 1642525 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1642526 T3447
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 1642527 T3447 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C13081 name=collection1
org.apache.solr.core.SolrCore@f51058
url=https://127.0.0.1:33180/e/h/collection1 node=127.0.0.1:33180_e%2Fh
C13081_STATE=coll:collection1 core:collection1 props:{core=collection1,
base_url=https://127.0.0.1:33180/e/h, node_name=127.0.0.1:33180_e%2Fh,
state=down}
[junit4] 2> 1642527 T3447 C13081 P33180 oasc.SyncStrategy.sync Sync
replicas to https://127.0.0.1:33180/e/h/collection1/
[junit4] 2> 1642527 T3447 C13081 P33180 oasc.SyncStrategy.syncReplicas
Sync Success - now sync replicas to me
[junit4] 2> 1642528 T3447 C13081 P33180 oasc.SyncStrategy.syncToMe
https://127.0.0.1:33180/e/h/collection1/ has no replicas
[junit4] 2> 1642528 T3447 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: https://127.0.0.1:33180/e/h/collection1/ shard3
[junit4] 2> 1642528 T3447 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard3
[junit4] 2> 1642532 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1642636 T3442 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642636 T3423 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642636 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642637 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642684 T3447 oasc.ZkController.register We are
https://127.0.0.1:33180/e/h/collection1/ and leader is
https://127.0.0.1:33180/e/h/collection1/
[junit4] 2> 1642684 T3447 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=https://127.0.0.1:33180/e/h
[junit4] 2> 1642684 T3447 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 1642685 T3447 oasc.ZkController.publish publishing
core=collection1 state=active collection=collection1
[junit4] 2> 1642685 T3447 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 1642686 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1642689 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33180/e/h";,
[junit4] 2> "node_name":"127.0.0.1:33180_e%2Fh",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1642793 T3442 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642793 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642793 T3423 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642793 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1642876 T3373 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 3
[junit4] 2> 1642877 T3373 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1642885 T3373 oejus.SslContextFactory.doStart Enabled
Protocols [SSLv2Hello, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1,
TLSv1.1, TLSv1.2]
[junit4] 2> 1642897 T3373 oejs.AbstractConnector.doStart Started
[email protected]:45454
[junit4] 2> 1642902 T3373 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()
[junit4] 2> 1642903 T3373 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 1642907 T3373 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005
[junit4] 2> 1642908 T3373 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/'
[junit4] 2> 1642942 T3373 oasc.ConfigSolr.fromFile Loading container
configuration from
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/solr.xml
[junit4] 2> 1642957 T3373 oasc.CoreContainer.<init> New CoreContainer
22868991
[junit4] 2> 1642958 T3373 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/]
[junit4] 2> 1642959 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 1642959 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 1642960 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 1642960 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 1642961 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 1642961 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 1642962 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 1642962 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 1642963 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 1642963 T3373 oashc.HttpShardHandlerFactory.getParameter
Setting useRetries to: false
[junit4] 2> 1642965 T3373 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1642967 T3373 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1642967 T3373 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1642967 T3373 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1642968 T3373 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:51954/solr
[junit4] 2> 1642968 T3373 oasc.ZkController.checkChrootPath zkHost
includes chroot
[junit4] 2> 1642969 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1642970 T3458 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@5286c9 name:ZooKeeperConnection
Watcher:127.0.0.1:51954 got event WatchedEvent state:SyncConnected type:None
path:null path:null type:None
[junit4] 2> 1642971 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1642973 T3373 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 1642983 T3460 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@76a268 name:ZooKeeperConnection
Watcher:127.0.0.1:51954/solr got event WatchedEvent state:SyncConnected
type:None path:null path:null type:None
[junit4] 2> 1642983 T3373 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 1642989 T3373
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 1643991 T3373 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45454_e%2Fh
[junit4] 2> 1643992 T3373 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:45454_e%2Fh
[junit4] 2> 1643996 T3373 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1643999 T3462 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 1643999 T3462 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 1644001 T3462 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 1644001 T3402 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1644004 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Update state numShards=3 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:45454/e/h";,
[junit4] 2> "node_name":"127.0.0.1:45454_e%2Fh",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 1644004 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Collection already exists with numShards=3
[junit4] 2> 1644004 T3398 oasc.Overseer$ClusterStateUpdater.updateState
Assigning new node to shard shard=shard1
[junit4] 2> 1644108 T3442 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1644108 T3461 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1644108 T3409 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1644108 T3402 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1644108 T3423 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 1645001 T3462 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 1645001 T3462 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:collection1
[junit4] 2> 1645002 T3462 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 1645002 T3462 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 1645003 T3462 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 1645003 T3462 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/collection1/'
[junit4] 2> 1645004 T3462 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/collection1/lib/.svn/'
to classloader
[junit4] 2> 1645005 T3462 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/collection1/lib/classes/'
to classloader
[junit4] 2> 1645005 T3462 oasc.SolrResourceLoader.replaceClassLoader
Adding
'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/collection1/lib/README'
to classloader
[junit4] 2> 1645039 T3462 oasc.SolrConfig.<init> Using Lucene
MatchVersion: 4.10.4
[junit4] 2> 1645062 T3462 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 1645064 T3462 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 1645075 T3462 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 1645240 T3462 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 1645242 T3462 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1645243 T3462 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1645246 T3462 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 1645267 T3462 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 1645267 T3462 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 1645267 T3462 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-005/collection1/,
dataDir=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty3/
[junit4] 2> 1645268 T3462 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@c5fb5f
[junit4] 2> 1645268 T3462 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty3
[junit4] 2> 1645269 T3462 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty3/index/
[junit4] 2> 1645269 T3462 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'./solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty3/index'
doesn't exist. Creating new index...
[junit4] 2> 1645270 T3462 oasc.CachingDirectoryFactory.get return new
directory for
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/solr.cloud.ChaosMonkeySafeLeaderTest-E3E372A485BB9D2C-001/tempDir-001/jetty3/index
[junit4] 2> 1645271 T3462 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.LogByteSizeMergePolicy:
[LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=11,
maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807,
calibrateSizeByDeletes=false, maxMergeDocs=2147483647,
maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.44156486996929
[...truncated too long message...]
2> at
org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
[junit4] 2> at
org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610)
[junit4] 2> at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445)
[junit4] 2> at
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
[junit4] 2> at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
[junit4] 2> at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer$Runner.run(ConcurrentUpdateSolrServer.java:233)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 68) Thread[id=3463,
name=searcherExecutor-2048-thread-1, state=WAITING,
group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 69) Thread[id=3490, name=qtp27591507-3490 Selector1,
state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] 2> at
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
[junit4] 2> at
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] 2> at
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] 2> at
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
[junit4] 2> at
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 70) Thread[id=3504,
name=searcherExecutor-2066-thread-1, state=WAITING,
group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 71) Thread[id=3414, name=qtp17600182-3414 Acceptor1
[email protected]:33495, state=BLOCKED,
group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
[junit4] 2> at
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4] 2> at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 8859603 T3440 oaz.ClientCnxn$SendThread.run WARN Session
0x14bdd62c33c0009 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Lucene46,
sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_CR,
timezone=America/Manaus
[junit4] 2> NOTE: Linux 3.13.0-46-generic i386/Oracle Corporation 1.8.0_31
(32-bit)/cpus=8,threads=87,free=62054296,total=518979584
[junit4] 2> NOTE: All tests run in this JVM:
[DistribDocExpirationUpdateProcessorTest, UUIDUpdateProcessorFallbackTest,
SolrXmlInZkTest, TestSchemaResource, TestValueSourceCache, CacheHeaderTest,
SliceStateTest, DistributedTermsComponentTest, TestSolrQueryParser,
TestNonNRTOpen, SuggesterFSTTest, TestConfigSets, SearchHandlerTest,
TestStressUserVersions, UpdateRequestProcessorFactoryTest,
SynonymTokenizerTest, FileBasedSpellCheckerTest, LegacyHTMLStripCharFilterTest,
TestPerFieldSimilarity, TestSystemIdResolver, TestSweetSpotSimilarityFactory,
ZkSolrClientTest, DOMUtilTest, CopyFieldTest, CursorPagingTest,
TestFastLRUCache, LeaderElectionIntegrationTest, AutoCommitTest,
TestExpandComponent, TestCSVResponseWriter, OverseerStatusTest,
RequestHandlersTest, TestNonDefinedSimilarityFactory, SoftAutoCommitTest,
TermVectorComponentTest, UUIDFieldTest, BinaryUpdateRequestHandlerTest,
TestCollapseQParserPlugin, ChaosMonkeyNothingIsSafeTest,
LukeRequestHandlerTest, TestIntervalFaceting, TestPostingsSolrHighlighter,
DeleteLastCustomShardedReplicaTest, TestFiltering, TestClassNameShortening,
TestFieldTypeResource, DirectSolrConnectionTest, CurrencyFieldXmlFileTest,
TestLazyCores, TestSerializedLuceneMatchVersion, PolyFieldTest,
ScriptEngineTest, TestCharFilters, AtomicUpdatesTest, TestMissingGroups,
RemoteQueryErrorTest, DistributedQueueTest, HardAutoCommitTest,
DistributedSuggestComponentTest, TestManagedSchemaFieldResource,
TestShortCircuitedRequests, TestRealTimeGet, SharedFSAutoReplicaFailoverTest,
TestCloudPivotFacet, SolrCoreTest, TestFileDictionaryLookup, TestOmitPositions,
TestMultiCoreConfBootstrap, TestLRUCache, TestCollationFieldDocValues,
DistributedDebugComponentTest, SolrRequestParserTest, AssignTest,
SpatialFilterTest, TestQueryUtils, OpenCloseCoreStressTest,
TestReplicationHandlerBackup, TestUpdate, TestWriterPerf, BasicZkTest,
FieldAnalysisRequestHandlerTest, HdfsLockFactoryTest, TestIBSimilarityFactory,
MultiThreadedOCPTest, QueryParsingTest, TestDynamicFieldResource,
TestFastWriter, TimeZoneUtilsTest, DefaultValueUpdateProcessorTest,
UniqFieldsUpdateProcessorFactoryTest, CoreAdminCreateDiscoverTest,
TestDocumentBuilder, SortByFunctionTest, HdfsDirectoryTest, TestGroupingSearch,
TestDefaultSearchFieldResource, BasicDistributedZk2Test, CursorMarkTest,
SolrInfoMBeanTest, TestRandomFaceting, DebugComponentTest,
ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=E3E372A485BB9D2C
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CR
-Dtests.timezone=America/Manaus -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>=
7200000 msec).
[junit4] > at
__randomizedtesting.SeedInfo.seed([E3E372A485BB9D2C]:0)
[junit4] Completed on J1 in 7224.85s, 1 test, 2 errors <<< FAILURES!
[...truncated 11 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:474: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:454: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:45: The following
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/extra-targets.xml:37: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build.xml:189: The
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/common-build.xml:494:
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:1351:
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:966:
There were test failures: 424 suites, 1762 tests, 1 suite-level error, 1 error,
39 ignored (18 assumptions)
Total time: 206 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_31 -client
-XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
