Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2407/
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([1885E1D8FDE54458]:0)
FAILED:
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([1885E1D8FDE54458]:0)
Build Log:
[...truncated 10380 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/init-core-data-001
[junit4] 2> 652991 T1796 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl
(false) and clientAuth (false)
[junit4] 2> 652991 T1796 oas.BaseDistributedSearchTestCase.initHostContext
Setting hostContext system property: /
[junit4] 2> 652996 T1796 oas.SolrTestCaseJ4.setUp ###Starting
testDistribSearch
[junit4] 2> 652997 T1796 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 652997 T1797 oasc.ZkTestServer$ZKServerMain.runFromConfig
Starting server
[junit4] 2> 653097 T1796 oasc.ZkTestServer.run start zk server on
port:59929
[junit4] 2> 653098 T1796
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 653099 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 653103 T1804 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@69bfe0d6
name:ZooKeeperConnection Watcher:127.0.0.1:59929 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 653103 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 653103 T1796 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 653104 T1796 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 653106 T1796
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 653107 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 653109 T1807 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@3c66dc6f
name:ZooKeeperConnection Watcher:127.0.0.1:59929/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 653109 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 653109 T1796 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 653109 T1796 oascc.SolrZkClient.makePath makePath:
/collections/collection1
[junit4] 2> 653112 T1796 oascc.SolrZkClient.makePath makePath:
/collections/collection1/shards
[junit4] 2> 653113 T1796 oascc.SolrZkClient.makePath makePath:
/collections/control_collection
[junit4] 2> 653114 T1796 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/shards
[junit4] 2> 653116 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
to /configs/conf1/solrconfig.xml
[junit4] 2> 653117 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.xml
[junit4] 2> 653119 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema15.xml
to /configs/conf1/schema.xml
[junit4] 2> 653120 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/schema.xml
[junit4] 2> 653222 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 653222 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 653224 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
to /configs/conf1/stopwords.txt
[junit4] 2> 653225 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/stopwords.txt
[junit4] 2> 653227 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt
to /configs/conf1/protwords.txt
[junit4] 2> 653227 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/protwords.txt
[junit4] 2> 653229 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml
to /configs/conf1/currency.xml
[junit4] 2> 653230 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/currency.xml
[junit4] 2> 653231 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
to /configs/conf1/enumsConfig.xml
[junit4] 2> 653232 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/enumsConfig.xml
[junit4] 2> 653234 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
to /configs/conf1/open-exchange-rates.json
[junit4] 2> 653234 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/open-exchange-rates.json
[junit4] 2> 653236 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 653237 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 653238 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
to /configs/conf1/old_synonyms.txt
[junit4] 2> 653239 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/old_synonyms.txt
[junit4] 2> 653241 T1796 oasc.AbstractZkTestCase.putConfig put
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
to /configs/conf1/synonyms.txt
[junit4] 2> 653241 T1796 oascc.SolrZkClient.makePath makePath:
/configs/conf1/synonyms.txt
[junit4] 2> 654331 T1796 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 654335 T1796 oejs.AbstractConnector.doStart Started
[email protected]:44444
[junit4] 2> 654336 T1796 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@57bd06bf
[junit4] 2> 654336 T1796 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 654336 T1796 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002
[junit4] 2> 654336 T1796 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/'
[junit4] 2> 654366 T1796 oasc.ConfigSolr.fromFile Loading container
configuration from
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/solr.xml
[junit4] 2> 654449 T1796 oasc.CoreContainer.<init> New CoreContainer
320066571
[junit4] 2> 654449 T1796 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/]
[junit4] 2> 654451 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 654451 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 654451 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 654452 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 654452 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnections to: 10000
[junit4] 2> 654452 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 654453 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 654453 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 654453 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 654454 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 654454 T1796 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 654455 T1796 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 654455 T1796 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 654455 T1796 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 654456 T1796 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:59929/solr
[junit4] 2> 654456 T1796 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 654457 T1796
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 654458 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 654459 T1819 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@74ef63da
name:ZooKeeperConnection Watcher:127.0.0.1:59929 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 654460 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 654460 T1796 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 654463 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 654464 T1822 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@5cc2e055
name:ZooKeeperConnection Watcher:127.0.0.1:59929/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 654464 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 654466 T1796 oascc.SolrZkClient.makePath makePath:
/overseer/queue
[junit4] 2> 654467 T1796 oascc.SolrZkClient.makePath makePath:
/overseer/collection-queue-work
[junit4] 2> 654469 T1796 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-running
[junit4] 2> 654471 T1796 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-completed
[junit4] 2> 654473 T1796 oascc.SolrZkClient.makePath makePath:
/overseer/collection-map-failure
[junit4] 2> 654475 T1796 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 654476 T1796 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44444_
[junit4] 2> 654476 T1796 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:44444_
[junit4] 2> 654478 T1796 oascc.SolrZkClient.makePath makePath:
/overseer_elect
[junit4] 2> 654479 T1796 oascc.SolrZkClient.makePath makePath:
/overseer_elect/election
[junit4] 2> 654481 T1796 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 654482 T1796 oasc.OverseerElectionContext.runLeaderProcess I
am going to be the leader 127.0.0.1:44444_
[junit4] 2> 654482 T1796 oascc.SolrZkClient.makePath makePath:
/overseer_elect/leader
[junit4] 2> 654484 T1796 oasc.Overseer.start Overseer
(id=93064854516203523-127.0.0.1:44444_-n_0000000000) starting
[junit4] 2> 654485 T1796 oascc.SolrZkClient.makePath makePath:
/overseer/queue-work
[junit4] 2> 654491 T1796 oasc.OverseerAutoReplicaFailoverThread.<init>
Starting OverseerAutoReplicaFailoverThread
autoReplicaFailoverWorkLoopDelay=10000
autoReplicaFailoverWaitAfterExpiration=30000
autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 654491 T1824 oasc.OverseerCollectionProcessor.run Process
current queue of collection creations
[junit4] 2> 654492 T1796 oascc.SolrZkClient.makePath makePath:
/clusterstate.json
[junit4] 2> 654493 T1796 oascc.SolrZkClient.makePath makePath:
/aliases.json
[junit4] 2> 654494 T1796
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 654496 T1823 oasc.Overseer$ClusterStateUpdater.run Starting to
work on the main queue
[junit4] 2> 654501 T1826 oasc.ZkController.publish publishing
core=collection1 state=down collection=control_collection
[junit4] 2> 654501 T1826 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 654502 T1826 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 654502 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 654504 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 0, message = {
[junit4] 2> "shard":null,
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:44444_",
[junit4] 2> "base_url":"http://127.0.0.1:44444";,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"down"} current state version: 0
[junit4] 2> 654505 T1823 oasco.ReplicaMutator.updateState Update state
numShards=1 message={
[junit4] 2> "shard":null,
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:44444_",
[junit4] 2> "base_url":"http://127.0.0.1:44444";,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"down"}
[junit4] 2> 654505 T1823 oasco.ClusterStateMutator.createCollection
building a new cName: control_collection
[junit4] 2> 654505 T1823 oasco.ReplicaMutator.updateState Assigning new
node to shard shard=shard1
[junit4] 2> 654506 T1822 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 655503 T1826 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 655504 T1826 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:control_collection
[junit4] 2> 655504 T1826 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 655505 T1826 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/control_collection
[junit4] 2> 655505 T1826 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 655505 T1826 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/collection1/'
[junit4] 2> 655507 T1826 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/collection1/lib/README'
to classloader
[junit4] 2> 655508 T1826 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/collection1/lib/.svn/'
to classloader
[junit4] 2> 655508 T1826 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/collection1/lib/classes/'
to classloader
[junit4] 2> 655541 T1826 oasc.ZkController.watchZKConfDir watch zkdir
/configs/conf1
[junit4] 2> 655543 T1826 oasc.Config.<init> loaded config solrconfig.xml
with version 0
[junit4] 2> 655576 T1826 oasc.SolrConfig.<init> Using Lucene MatchVersion:
5.0.0
[junit4] 2> 655623 T1826 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 655724 T1826 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 655734 T1826 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 656074 T1826 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 656077 T1826 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 656079 T1826 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 656083 T1826 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 656103 T1826 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection control_collection
[junit4] 2> 656103 T1826 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 656104 T1826 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-002/collection1/,
dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/control/data/
[junit4] 2> 656104 T1826 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@3f8f6434
[junit4] 2> 656107 T1826 oasc.CachingDirectoryFactory.get return new
directory for
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/control/data
[junit4] 2> 656108 T1826 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/control/data/index/
[junit4] 2> 656109 T1826 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/control/data/index'
doesn't exist. Creating new index...
[junit4] 2> 656109 T1826 oasc.CachingDirectoryFactory.get return new
directory for
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/control/data/index
[junit4] 2> 656110 T1826 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=43, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=37.9697265625,
floorSegmentMB=1.625, forceMergeDeletesPctAllowed=28.278130187488365,
segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.5965526834078169
[junit4] 2> 656134 T1826 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 656134 T1826 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 656140 T1826 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 656140 T1826 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 656140 T1826 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 656141 T1826 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 656141 T1826 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 656141 T1826 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 656142 T1826 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 656142 T1826 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 656142 T1826 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 656143 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 656143 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 656144 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 656144 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 656145 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 656145 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 656146 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 656146 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 656147 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 656148 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 656149 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 656149 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 656150 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 656150 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/properties:
org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 656151 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 656151 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 656152 T1826 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 656152 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 656153 T1826 oasc.RequestHandlers.initHandlersFromConfig
created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 656171 T1826 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 656173 T1826 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 656175 T1826 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 656177 T1826 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 656179 T1826 oasc.SolrCore.initStatsCache Using default
statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 656181 T1826 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 656181 T1826 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 656182 T1826 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=28, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=5.6181640625,
floorSegmentMB=2.068359375, forceMergeDeletesPctAllowed=9.327741889486456,
segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 656183 T1826 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 656183 T1826 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 656184 T1826 oass.SolrIndexSearcher.<init> Opening
Searcher@73d4984f[collection1] main
[junit4] 2> 656184 T1826 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/control_collection
[junit4] 2> 656185 T1826 oascc.ZkStateReader.readConfigName
path=/collections/control_collection configName=conf1 specified config exists
in ZooKeeper
[junit4] 2> 656185 T1826 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 656186 T1826
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 656186 T1826 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 656186 T1826 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 656187 T1826
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 656187 T1826 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 656187 T1826 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 656187 T1826 oash.ReplicationHandler.inform Commits will be
reserved for 10000
[junit4] 2> 656188 T1827 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@73d4984f[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 656189 T1826 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 656190 T1830 oasc.ZkController.register Register replica -
core:collection1 address:http://127.0.0.1:44444 collection:control_collection
shard:shard1
[junit4] 2> 656190 T1796 oass.SolrDispatchFilter.init
user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3
[junit4] 2> 656190 T1796 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 656191 T1830 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leader_elect/shard1/election
[junit4] 2> 656191 T1796
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 656192 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 656194 T1833 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@3d62bf75
name:ZooKeeperConnection Watcher:127.0.0.1:59929/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 656194 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 656194 T1796 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 656196 T1796
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 656196 T1830 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard1
[junit4] 2> 656198 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 656198 T1830
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 656198 T1830 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C163 name=collection1
org.apache.solr.core.SolrCore@3635e6f9 url=http://127.0.0.1:44444/collection1
node=127.0.0.1:44444_ C163_STATE=coll:control_collection core:collection1
props:{node_name=127.0.0.1:44444_, base_url=http://127.0.0.1:44444,
core=collection1, state=down}
[junit4] 2> 656198 T1830 C163 P44444 oasc.SyncStrategy.sync Sync replicas
to http://127.0.0.1:44444/collection1/
[junit4] 2> 656199 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state
version: 1
[junit4] 2> 656199 T1830 C163 P44444 oasc.SyncStrategy.syncReplicas Sync
Success - now sync replicas to me
[junit4] 2> 656199 T1796 oasc.ChaosMonkey.monkeyLog monkey: init - expire
sessions:false cause connection loss:false
[junit4] 2> 656199 T1830 C163 P44444 oasc.SyncStrategy.syncToMe
http://127.0.0.1:44444/collection1/ has no replicas
[junit4] 2> 656200 T1796 oasc.AbstractFullDistribZkTestBase.createJettys
Creating collection1 with stateFormat=2
[junit4] 2> 656200 T1830 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: http://127.0.0.1:44444/collection1/ shard1
[junit4] 2> 656200 T1833 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 656200 T1796
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 656200 T1822 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 656200 T1830 oascc.SolrZkClient.makePath makePath:
/collections/control_collection/leaders/shard1
[junit4] 2> 656202 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 656203 T1836 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@2bff9812
name:ZooKeeperConnection Watcher:127.0.0.1:59929/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 656204 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 656204 T1796 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 656206 T1837 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 656207 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "stateFormat":2} current state version: 2
[junit4] 2> 656207 T1823 oasco.ClusterStateMutator.createCollection
building a new cName: collection1
[junit4] 2> 656210 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:44444";,
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 2
[junit4] 2> 656210 T1823 oasco.ZkStateWriter.writePendingUpdates going to
create_collection /collections/collection1/state.json
[junit4] 2> 656211 T1833 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 656214 T1837 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 656214 T1833 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 656257 T1830 oasc.ZkController.register We are
http://127.0.0.1:44444/collection1/ and leader is
http://127.0.0.1:44444/collection1/
[junit4] 2> 656258 T1830 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=http://127.0.0.1:44444
[junit4] 2> 656258 T1830 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 656258 T1830 oasc.ZkController.publish publishing
core=collection1 state=active collection=control_collection
[junit4] 2> 656258 T1830 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 656259 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 656261 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "shard":"shard1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:44444_",
[junit4] 2> "base_url":"http://127.0.0.1:44444";,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active"} current state version: 4
[junit4] 2> 656261 T1823 oasco.ReplicaMutator.updateState Update state
numShards=3 message={
[junit4] 2> "shard":"shard1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:44444_",
[junit4] 2> "base_url":"http://127.0.0.1:44444";,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active"}
[junit4] 2> 656364 T1833 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 656364 T1822 oascc.ZkStateReader$2.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 657327 T1796 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 1
[junit4] 2> 657328 T1796 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 657332 T1796 oejs.AbstractConnector.doStart Started
[email protected]:63069
[junit4] 2> 657332 T1796 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@57bd06bf
[junit4] 2> 657333 T1796 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 657333 T1796 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003
[junit4] 2> 657333 T1796 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/'
[junit4] 2> 657363 T1796 oasc.ConfigSolr.fromFile Loading container
configuration from
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/solr.xml
[junit4] 2> 657416 T1796 oasc.CoreContainer.<init> New CoreContainer
726103743
[junit4] 2> 657416 T1796 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/]
[junit4] 2> 657417 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 657417 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 657417 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 657418 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 657418 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnections to: 10000
[junit4] 2> 657418 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 657418 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 657419 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 657419 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 657419 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 657420 T1796 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 657420 T1796 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 657420 T1796 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 657420 T1796 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 657421 T1796 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:59929/solr
[junit4] 2> 657421 T1796 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 657421 T1796
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 657422 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 657424 T1849 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@74f2e8bd
name:ZooKeeperConnection Watcher:127.0.0.1:59929 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 657425 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 657425 T1796 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 657427 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 657428 T1852 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@2cb97209
name:ZooKeeperConnection Watcher:127.0.0.1:59929/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 657429 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 657433 T1796
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 658438 T1796 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63069_
[junit4] 2> 658439 T1796 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:63069_
[junit4] 2> 658442 T1796 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 658451 T1853 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 658451 T1853 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 658452 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 658453 T1853 oasc.ZkController.preRegister Registering watch
for external collection collection1
[junit4] 2> 658453 T1853 oascc.ZkStateReader.addZkWatch addZkWatch
collection1
[junit4] 2> 658453 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "shard":null,
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:63069_",
[junit4] 2> "base_url":"http://127.0.0.1:63069";,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"down"} current state version: 5
[junit4] 2> 658453 T1853 oascc.ZkStateReader.addZkWatch Updating
collection state at /collections/collection1/state.json from ZooKeeper...
[junit4] 2> 658454 T1823 oasco.ReplicaMutator.updateState Update state
numShards=3 message={
[junit4] 2> "shard":null,
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:63069_",
[junit4] 2> "base_url":"http://127.0.0.1:63069";,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"down"}
[junit4] 2> 658454 T1823 oasco.ReplicaMutator.updateState Collection
already exists with numShards=3
[junit4] 2> 658454 T1823 oasco.ReplicaMutator.updateState Assigning new
node to shard shard=shard3
[junit4] 2> 658454 T1853 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 0
[junit4] 2> 658455 T1853 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 658456 T1823 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 0
[junit4] 2> 658456 T1852 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 658457 T1852 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 1
[junit4] 2> 659455 T1853 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 659456 T1853 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:collection1
[junit4] 2> 659456 T1853 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 659457 T1853 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 659457 T1853 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 659457 T1853 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/collection1/'
[junit4] 2> 659459 T1853 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/collection1/lib/README'
to classloader
[junit4] 2> 659460 T1853 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/collection1/lib/classes/'
to classloader
[junit4] 2> 659461 T1853 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/collection1/lib/.svn/'
to classloader
[junit4] 2> 659502 T1853 oasc.ZkController.watchZKConfDir watch zkdir
/configs/conf1
[junit4] 2> 659504 T1853 oasc.Config.<init> loaded config solrconfig.xml
with version 0
[junit4] 2> 659542 T1853 oasc.SolrConfig.<init> Using Lucene MatchVersion:
5.0.0
[junit4] 2> 659595 T1853 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 659697 T1853 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 659718 T1853 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 659958 T1853 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 659960 T1853 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 659962 T1853 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 659965 T1853 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 659981 T1853 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 659981 T1853 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 659981 T1853 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-003/collection1/,
dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty1/
[junit4] 2> 659982 T1853 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@3f8f6434
[junit4] 2> 659983 T1853 oasc.CachingDirectoryFactory.get return new
directory for
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty1
[junit4] 2> 659984 T1853 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty1/index/
[junit4] 2> 659984 T1853 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty1/index'
doesn't exist. Creating new index...
[junit4] 2> 659986 T1853 oasc.CachingDirectoryFactory.get return new
directory for
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty1/index
[junit4] 2> 659986 T1853 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=43, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=37.9697265625,
floorSegmentMB=1.625, forceMergeDeletesPctAllowed=28.278130187488365,
segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.5965526834078169
[junit4] 2> 660007 T1853 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 660040 T1853 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 660046 T1853 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 660046 T1853 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 660046 T1853 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 660046 T1853 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 660046 T1853 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 660047 T1853 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 660047 T1853 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 660047 T1853 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 660047 T1853 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 660048 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 660048 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 660049 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 660049 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 660050 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 660050 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 660052 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 660053 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 660053 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 660054 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 660055 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 660055 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 660055 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 660056 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/properties:
org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 660056 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 660057 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 660057 T1853 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 660058 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 660058 T1853 oasc.RequestHandlers.initHandlersFromConfig
created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 660078 T1853 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 660080 T1853 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 660082 T1853 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 660084 T1853 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 660085 T1853 oasc.SolrCore.initStatsCache Using default
statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 660087 T1853 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 660087 T1853 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 660088 T1853 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=28, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=5.6181640625,
floorSegmentMB=2.068359375, forceMergeDeletesPctAllowed=9.327741889486456,
segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 660090 T1853 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 660090 T1853 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 660090 T1853 oass.SolrIndexSearcher.<init> Opening
Searcher@49b0ee10[collection1] main
[junit4] 2> 660091 T1853 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 660091 T1853 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 660092 T1853 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 660092 T1853
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 660092 T1853 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 660092 T1853 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 660093 T1853
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 660093 T1853 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 660093 T1853 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 660093 T1853 oash.ReplicationHandler.inform Commits will be
reserved for 10000
[junit4] 2> 660094 T1854 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@49b0ee10[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 660094 T1853 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 660095 T1857 oasc.ZkController.register Register replica -
core:collection1 address:http://127.0.0.1:63069 collection:collection1
shard:shard3
[junit4] 2> 660095 T1796 oass.SolrDispatchFilter.init
user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3
[junit4] 2> 660096 T1857 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard3/election
[junit4] 2> 660096 T1796 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 660101 T1857 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard3
[junit4] 2> 660102 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 660103 T1857
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 660103 T1857 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C164 name=collection1
org.apache.solr.core.SolrCore@7d2b51fa url=http://127.0.0.1:63069/collection1
node=127.0.0.1:63069_ C164_STATE=coll:collection1 core:collection1
props:{node_name=127.0.0.1:63069_, base_url=http://127.0.0.1:63069,
core=collection1, state=down}
[junit4] 2> 660103 T1857 C164 P63069 oasc.SyncStrategy.sync Sync replicas
to http://127.0.0.1:63069/collection1/
[junit4] 2> 660103 T1857 C164 P63069 oasc.SyncStrategy.syncReplicas Sync
Success - now sync replicas to me
[junit4] 2> 660103 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1"} current state version: 5
[junit4] 2> 660103 T1857 C164 P63069 oasc.SyncStrategy.syncToMe
http://127.0.0.1:63069/collection1/ has no replicas
[junit4] 2> 660104 T1857 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: http://127.0.0.1:63069/collection1/ shard3
[junit4] 2> 660104 T1857 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard3
[junit4] 2> 660105 T1823 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 1
[junit4] 2> 660105 T1852 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 660106 T1852 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 2
[junit4] 2> 660108 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:63069";,
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 5
[junit4] 2> 660110 T1823 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 2
[junit4] 2> 660110 T1852 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 660111 T1852 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 3
[junit4] 2> 660112 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 660159 T1857 oasc.ZkController.register We are
http://127.0.0.1:63069/collection1/ and leader is
http://127.0.0.1:63069/collection1/
[junit4] 2> 660160 T1857 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=http://127.0.0.1:63069
[junit4] 2> 660160 T1857 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 660160 T1857 oasc.ZkController.publish publishing
core=collection1 state=active collection=collection1
[junit4] 2> 660160 T1857 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 660162 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 660163 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "shard":"shard3",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:63069_",
[junit4] 2> "base_url":"http://127.0.0.1:63069";,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active"} current state version: 5
[junit4] 2> 660163 T1823 oasco.ReplicaMutator.updateState Update state
numShards=3 message={
[junit4] 2> "shard":"shard3",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:63069_",
[junit4] 2> "base_url":"http://127.0.0.1:63069";,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active"}
[junit4] 2> 660165 T1823 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 3
[junit4] 2> 660165 T1857 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 3
[junit4] 2> 660165 T1852 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 660166 T1852 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 4
[junit4] 2> 662168 T1796 oasc.AbstractFullDistribZkTestBase.createJettys
create jetty 2
[junit4] 2> 662169 T1796 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 662172 T1796 oejs.AbstractConnector.doStart Started
[email protected]:21024
[junit4] 2> 662173 T1796 oass.SolrDispatchFilter.init
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@57bd06bf
[junit4] 2> 662173 T1796 oasc.SolrResourceLoader.locateSolrHome JNDI not
configured for solr (NoInitialContextEx)
[junit4] 2> 662174 T1796 oasc.SolrResourceLoader.locateSolrHome using
system property solr.solr.home:
../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004
[junit4] 2> 662174 T1796 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/'
[junit4] 2> 662236 T1796 oasc.ConfigSolr.fromFile Loading container
configuration from
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/solr.xml
[junit4] 2> 662341 T1796 oasc.CoreContainer.<init> New CoreContainer
419728569
[junit4] 2> 662342 T1796 oasc.CoreContainer.load Loading cores into
CoreContainer
[instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/]
[junit4] 2> 662343 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting socketTimeout to: 90000
[junit4] 2> 662343 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting urlScheme to:
[junit4] 2> 662344 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting connTimeout to: 15000
[junit4] 2> 662344 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnectionsPerHost to: 20
[junit4] 2> 662344 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxConnections to: 10000
[junit4] 2> 662345 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting corePoolSize to: 0
[junit4] 2> 662345 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maximumPoolSize to: 2147483647
[junit4] 2> 662345 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting maxThreadIdleTime to: 5
[junit4] 2> 662346 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting sizeOfQueue to: -1
[junit4] 2> 662346 T1796 oashc.HttpShardHandlerFactory.getParameter
Setting fairnessPolicy to: false
[junit4] 2> 662347 T1796 oasu.UpdateShardHandler.<init> Creating
UpdateShardHandler HTTP client with params:
socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 662347 T1796 oasl.LogWatcher.createWatcher SLF4J impl is
org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 662347 T1796 oasl.LogWatcher.newRegisteredLogWatcher
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 662348 T1796 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 662348 T1796 oasc.ZkContainer.initZooKeeper Zookeeper
client=127.0.0.1:59929/solr
[junit4] 2> 662349 T1796 oasc.ZkController.checkChrootPath zkHost includes
chroot
[junit4] 2> 662349 T1796
oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
ZkCredentialsProvider
[junit4] 2> 662350 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 662352 T1869 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@18bf8e52
name:ZooKeeperConnection Watcher:127.0.0.1:59929 got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 662353 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 662353 T1796 oascc.SolrZkClient.createZkACLProvider Using
default ZkACLProvider
[junit4] 2> 662356 T1796 oascc.ConnectionManager.waitForConnected Waiting
for client to connect to ZooKeeper
[junit4] 2> 662357 T1872 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@24d2261c
name:ZooKeeperConnection Watcher:127.0.0.1:59929/solr got event WatchedEvent
state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 662357 T1796 oascc.ConnectionManager.waitForConnected Client
is connected to ZooKeeper
[junit4] 2> 662362 T1796
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state
from ZooKeeper...
[junit4] 2> 663367 T1796 oasc.ZkController.createEphemeralLiveNode
Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21024_
[junit4] 2> 663368 T1796 oascc.SolrZkClient.makePath makePath:
/live_nodes/127.0.0.1:21024_
[junit4] 2> 663372 T1796 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 663380 T1873 oasc.ZkController.publish publishing
core=collection1 state=down collection=collection1
[junit4] 2> 663380 T1873 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 663382 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 663382 T1873 oasc.ZkController.preRegister Registering watch
for external collection collection1
[junit4] 2> 663383 T1873 oascc.ZkStateReader.addZkWatch addZkWatch
collection1
[junit4] 2> 663383 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "shard":null,
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:21024_",
[junit4] 2> "base_url":"http://127.0.0.1:21024";,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"down"} current state version: 5
[junit4] 2> 663384 T1873 oascc.ZkStateReader.addZkWatch Updating
collection state at /collections/collection1/state.json from ZooKeeper...
[junit4] 2> 663384 T1823 oasco.ReplicaMutator.updateState Update state
numShards=3 message={
[junit4] 2> "shard":null,
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:21024_",
[junit4] 2> "base_url":"http://127.0.0.1:21024";,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "roles":null,
[junit4] 2> "state":"down"}
[junit4] 2> 663384 T1823 oasco.ReplicaMutator.updateState Collection
already exists with numShards=3
[junit4] 2> 663385 T1823 oasco.ReplicaMutator.updateState Assigning new
node to shard shard=shard1
[junit4] 2> 663385 T1873 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 4
[junit4] 2> 663385 T1873 oasc.ZkController.waitForCoreNodeName look for
our core node name
[junit4] 2> 663386 T1823 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 4
[junit4] 2> 663386 T1872 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 663386 T1852 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 663387 T1872 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 5
[junit4] 2> 663387 T1852 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 5
[junit4] 2> 664385 T1873 oasc.ZkController.waitForShardId waiting to find
shard id in clusterstate for collection1
[junit4] 2> 664386 T1873 oasc.ZkController.createCollectionZkNode Check
for collection zkNode:collection1
[junit4] 2> 664386 T1873 oasc.ZkController.createCollectionZkNode
Collection zkNode exists
[junit4] 2> 664387 T1873 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 664387 T1873 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 664388 T1873 oasc.SolrResourceLoader.<init> new
SolrResourceLoader for directory:
'../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/collection1/'
[junit4] 2> 664390 T1873 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/collection1/lib/README'
to classloader
[junit4] 2> 664391 T1873 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/collection1/lib/classes/'
to classloader
[junit4] 2> 664392 T1873 oasc.SolrResourceLoader.replaceClassLoader Adding
'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/collection1/lib/.svn/'
to classloader
[junit4] 2> 664429 T1873 oasc.ZkController.watchZKConfDir watch zkdir
/configs/conf1
[junit4] 2> 664431 T1873 oasc.Config.<init> loaded config solrconfig.xml
with version 0
[junit4] 2> 664465 T1873 oasc.SolrConfig.<init> Using Lucene MatchVersion:
5.0.0
[junit4] 2> 664516 T1873 oasc.SolrConfig.<init> Loaded SolrConfig:
solrconfig.xml
[junit4] 2> 664617 T1873 oass.IndexSchema.readSchema Reading Solr Schema
from /configs/conf1/schema.xml
[junit4] 2> 664627 T1873 oass.IndexSchema.readSchema [collection1] Schema
name=test
[junit4] 2> 664979 T1873 oass.IndexSchema.readSchema default search field
in schema is text
[junit4] 2> 664982 T1873 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 664984 T1873 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 664989 T1873 oass.FileExchangeRateProvider.reload Reloading
exchange rates from file currency.xml
[junit4] 2> 665008 T1873 oasc.CoreContainer.create Creating SolrCore
'collection1' using configuration from collection collection1
[junit4] 2> 665008 T1873 oasc.SolrCore.initDirectoryFactory
solr.StandardDirectoryFactory
[junit4] 2> 665009 T1873 oasc.SolrCore.<init> [collection1] Opening new
SolrCore at
../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-004/collection1/,
dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty2/
[junit4] 2> 665009 T1873 oasc.JmxMonitoredMap.<init> JMX monitoring is
enabled. Adding Solr mbeans to JMX Server:
com.sun.jmx.mbeanserver.JmxMBeanServer@3f8f6434
[junit4] 2> 665011 T1873 oasc.CachingDirectoryFactory.get return new
directory for
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty2
[junit4] 2> 665012 T1873 oasc.SolrCore.getNewIndexDir New index directory
detected: old=null
new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty2/index/
[junit4] 2> 665012 T1873 oasc.SolrCore.initIndex WARN [collection1] Solr
index directory
'/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty2/index'
doesn't exist. Creating new index...
[junit4] 2> 665014 T1873 oasc.CachingDirectoryFactory.get return new
directory for
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty2/index
[junit4] 2> 665015 T1873 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=43, maxMergeAtOnceExplicit=24, maxMergedSegmentMB=37.9697265625,
floorSegmentMB=1.625, forceMergeDeletesPctAllowed=28.278130187488365,
segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12,
noCFSRatio=0.5965526834078169
[junit4] 2> 665020 T1873 oasc.SolrDeletionPolicy.onCommit
SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2>
commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 665021 T1873 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 665027 T1873 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "nodistrib"
[junit4] 2> 665027 T1873 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "dedupe"
[junit4] 2> 665028 T1873 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 665028 T1873 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "stored_sig"
[junit4] 2> 665028 T1873 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 665028 T1873 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 665029 T1873 oasup.UpdateRequestProcessorChain.init creating
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 665029 T1873 oasup.UpdateRequestProcessorChain.init inserting
DistributedUpdateProcessorFactory into updateRequestProcessorChain
"distrib-dup-test-chain-implicit"
[junit4] 2> 665029 T1873 oasc.SolrCore.loadUpdateProcessorChains no
updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 665030 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 665031 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 665031 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 665032 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 665033 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 665033 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 665035 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 665036 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 665037 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 665038 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 665038 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 665039 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 665040 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 665040 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/properties:
org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 665041 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 665041 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 665042 T1873 oasc.RequestHandlers.initHandlersFromConfig
created standard: solr.StandardRequestHandler
[junit4] 2> 665043 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 665043 T1873 oasc.RequestHandlers.initHandlersFromConfig
created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 665064 T1873 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 665066 T1873 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 665068 T1873 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 665070 T1873 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 665072 T1873 oasc.SolrCore.initStatsCache Using default
statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 665074 T1873 oasu.CommitTracker.<init> Hard AutoCommit:
disabled
[junit4] 2> 665074 T1873 oasu.CommitTracker.<init> Soft AutoCommit:
disabled
[junit4] 2> 665075 T1873 oasu.RandomMergePolicy.<init> RandomMergePolicy
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy:
maxMergeAtOnce=28, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=5.6181640625,
floorSegmentMB=2.068359375, forceMergeDeletesPctAllowed=9.327741889486456,
segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 665077 T1873 oasc.SolrDeletionPolicy.onInit
SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2>
commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3/temp/solr.cloud.ChaosMonkeySafeLeaderTest-1885E1D8FDE54458-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 665077 T1873 oasc.SolrDeletionPolicy.updateCommits newest
commit generation = 1
[junit4] 2> 665077 T1873 oass.SolrIndexSearcher.<init> Opening
Searcher@2c1101d4[collection1] main
[junit4] 2> 665078 T1873 oascc.ZkStateReader.readConfigName Load
collection config from:/collections/collection1
[junit4] 2> 665079 T1873 oascc.ZkStateReader.readConfigName
path=/collections/collection1 configName=conf1 specified config exists in
ZooKeeper
[junit4] 2> 665079 T1873 oasr.ManagedResourceStorage.newStorageIO Setting
up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 665079 T1873
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 665079 T1873 oasr.RestManager.init Initializing RestManager
with initArgs: {}
[junit4] 2> 665080 T1873 oasr.ManagedResourceStorage.load Reading
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 665080 T1873
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found
for znode /configs/conf1/_rest_managed.json
[junit4] 2> 665081 T1873 oasr.ManagedResourceStorage.load Loaded null at
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 665081 T1873 oasr.RestManager.init Initializing 0 registered
ManagedResources
[junit4] 2> 665081 T1873 oash.ReplicationHandler.inform Commits will be
reserved for 10000
[junit4] 2> 665082 T1874 oasc.SolrCore.registerSearcher [collection1]
Registered new searcher Searcher@2c1101d4[collection1]
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 665083 T1873 oasc.CoreContainer.registerCore registering core:
collection1
[junit4] 2> 665083 T1877 oasc.ZkController.register Register replica -
core:collection1 address:http://127.0.0.1:21024 collection:collection1
shard:shard1
[junit4] 2> 665084 T1796 oass.SolrDispatchFilter.init
user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J3
[junit4] 2> 665084 T1796 oass.SolrDispatchFilter.init
SolrDispatchFilter.init() done
[junit4] 2> 665084 T1877 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leader_elect/shard1/election
[junit4] 2> 665087 T1877 oasc.ShardLeaderElectionContext.runLeaderProcess
Running the leader process for shard shard1
[junit4] 2> 665089 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 665089 T1877
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found
to continue.
[junit4] 2> 665089 T1877 oasc.ShardLeaderElectionContext.runLeaderProcess
I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C165 name=collection1
org.apache.solr.core.SolrCore@4e7725df url=http://127.0.0.1:21024/collection1
node=127.0.0.1:21024_ C165_STATE=coll:collection1 core:collection1
props:{node_name=127.0.0.1:21024_, base_url=http://127.0.0.1:21024,
core=collection1, state=down}
[junit4] 2> 665089 T1877 C165 P21024 oasc.SyncStrategy.sync Sync replicas
to http://127.0.0.1:21024/collection1/
[junit4] 2> 665090 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1"} current state version: 5
[junit4] 2> 665090 T1877 C165 P21024 oasc.SyncStrategy.syncReplicas Sync
Success - now sync replicas to me
[junit4] 2> 665090 T1877 C165 P21024 oasc.SyncStrategy.syncToMe
http://127.0.0.1:21024/collection1/ has no replicas
[junit4] 2> 665090 T1877 oasc.ShardLeaderElectionContext.runLeaderProcess
I am the new leader: http://127.0.0.1:21024/collection1/ shard1
[junit4] 2> 665091 T1877 oascc.SolrZkClient.makePath makePath:
/collections/collection1/leaders/shard1
[junit4] 2> 665091 T1823 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 5
[junit4] 2> 665092 T1852 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 665092 T1872 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 665092 T1852 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 6
[junit4] 2> 665092 T1872 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 6
[junit4] 2> 665094 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:21024";,
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 5
[junit4] 2> 665095 T1823 oasco.ZkStateWriter.writePendingUpdates going to
update_collection /collections/collection1/state.json version: 6
[junit4] 2> 665096 T1872 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 665096 T1852 oascc.ZkStateReader$7.process A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/collections/collection1/state.json, has occurred - updating...
[junit4] 2> 665097 T1872 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 7
[junit4] 2> 665097 T1852 oascc.ZkStateReader.updateWatchedCollection
Updating data for collection1 to ver 7
[junit4] 2> 665098 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 665144 T1877 oasc.ZkController.register We are
http://127.0.0.1:21024/collection1/ and leader is
http://127.0.0.1:21024/collection1/
[junit4] 2> 665145 T1877 oasc.ZkController.register No LogReplay needed
for core=collection1 baseURL=http://127.0.0.1:21024
[junit4] 2> 665145 T1877 oasc.ZkController.checkRecovery I am the leader,
no recovery necessary
[junit4] 2> 665145 T1877 oasc.ZkController.publish publishing
core=collection1 state=active collection=collection1
[junit4] 2> 665145 T1877 oasc.ZkController.publish numShards not found on
descriptor - reading it from system property
[junit4] 2> 665146 T1822 oasc.DistributedQueue$LatchWatcher.process
NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 665147 T1823 oasc.Overseer$ClusterStateUpdater.run
processMessage: queueSize: 1, message = {
[junit4] 2> "shard":"shard1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:21024_",
[junit4] 2> "base_url":"http://127.0.0.1:21024";,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "r
[...truncated too long message...]
at
org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:377)
[junit4] 2> at
org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:361)
[junit4] 2> at
org.apache.solr.cloud.ZkController.publish(ZkController.java:1167)
[junit4] 2> at
org.apache.solr.cloud.ZkController.publish(ZkController.java:1085)
[junit4] 2> at
org.apache.solr.cloud.ZkController.publish(ZkController.java:1081)
[junit4] 2> at
org.apache.solr.core.ZkContainer.publishCoresAsDown(ZkContainer.java:245)
[junit4] 2> at
org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:340)
[junit4] 2> at
org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:194)
[junit4] 2> at
org.apache.solr.cloud.ChaosMonkey.stopJettySolrRunner(ChaosMonkey.java:197)
[junit4] 2> at
org.apache.solr.cloud.ChaosMonkey.stop(ChaosMonkey.java:550)
[junit4] 2> at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.destroyServers(AbstractFullDistribZkTestBase.java:1577)
[junit4] 2> at
org.apache.solr.BaseDistributedSearchTestCase.tearDown(BaseDistributedSearchTestCase.java:283)
[junit4] 2> at
org.apache.solr.cloud.AbstractDistribZkTestBase.tearDown(AbstractDistribZkTestBase.java:231)
[junit4] 2> at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.tearDown(AbstractFullDistribZkTestBase.java:1551)
[junit4] 2> at
org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.tearDown(ChaosMonkeySafeLeaderTest.java:86)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4] 2> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at
java.lang.reflect.Method.invoke(Method.java:606)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are:
codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION,
chunkSize=2756, maxDocsPerChunk=746),
termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION,
chunkSize=2756)), sim=RandomSimilarityProvider(queryNorm=true,coord=no): {},
locale=ca, timezone=CNT
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65
(64-bit)/cpus=16,threads=19,free=83034040,total=287834112
[junit4] 2> NOTE: All tests run in this JVM: [TestNRTOpen,
StandardRequestHandlerTest, BinaryUpdateRequestHandlerTest,
ConvertedLegacyTest, TestBulkSchemaConcurrent, ClusterStateUpdateTest,
TestNoOpRegenerator, TestCopyFieldCollectionResource, TestExpandComponent,
ResourceLoaderTest, TestSimpleQParserPlugin, TestCodecSupport, TestQueryUtils,
VMParamsZkACLAndCredentialsProvidersTest, ZkSolrClientTest,
TestReversedWildcardFilterFactory, PreAnalyzedFieldTest, TestMissingGroups,
TestReRankQParserPlugin, TestLazyCores, DistributedSpellCheckComponentTest,
TestDefaultSimilarityFactory, TestSolr4Spatial, TestReplicaProperties,
AnalyticsMergeStrategyTest, FastVectorHighlighterTest,
CloudExitableDirectoryReaderTest, QueryEqualityTest,
TestOverriddenPrefixQueryForCustomFieldType, LegacyHTMLStripCharFilterTest,
TestSolrXmlPersistence, TestDefaultSearchFieldResource,
ExternalFileFieldSortTest, DeleteInactiveReplicaTest, SyncSliceTest,
ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=1885E1D8FDE54458
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ca -Dtests.timezone=CNT
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J3 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>=
7200000 msec).
[junit4] > at
__randomizedtesting.SeedInfo.seed([1885E1D8FDE54458]:0)
[junit4] Completed on J3 in 7223.89s, 1 test, 2 errors <<< FAILURES!
[...truncated 78 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:529:
The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:477:
The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/build.xml:61:
The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/extra-targets.xml:39:
The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build.xml:190:
The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/common-build.xml:508:
The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:1363:
The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/lucene/common-build.xml:970:
There were test failures: 458 suites, 1859 tests, 1 suite-level error, 1
error, 140 ignored (23 assumptions)
Total time: 151 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-5.x-Java7 #2405
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 200991880 bytes
Compression is 0.0%
Took 35 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
