Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/599/

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
ERROR: SolrIndexSearcher opens=16 closes=15

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=16 closes=15
        at __randomizedtesting.SeedInfo.seed([A07445A293ED20EE]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:436)
        at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:183)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
2 threads leaked from SUITE scope at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:     1) Thread[id=197, 
name=qtp1182665876-197, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] 
        at java.lang.Object.wait(Native Method)         at 
java.lang.Object.wait(Object.java:503)         at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer.blockUntilFinished(ConcurrentUpdateSolrServer.java:374)
         at 
org.apache.solr.update.StreamingSolrServers.blockUntilFinished(StreamingSolrServers.java:103)
         at 
org.apache.solr.update.SolrCmdDistributor.blockAndDoRetries(SolrCmdDistributor.java:228)
         at 
org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:89)    
     at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:766)
         at 
org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1662)
         at 
org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:179)
         at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)
         at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
         at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)         
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780) 
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
         at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
         at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
         at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
         at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
         at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)      
   at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
         at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)   
      at 
org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)       
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
         at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)       
  at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
         at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
         at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)   
      at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) 
        at org.eclipse.jetty.server.Server.handle(Server.java:368)         at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
         at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
         at 
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
         at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
         at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)    
     at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)   
      at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
         at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    2) Thread[id=205, 
name=searcherExecutor-87-thread-1, state=WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 
   1) Thread[id=197, name=qtp1182665876-197, state=WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:503)
        at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrServer.blockUntilFinished(ConcurrentUpdateSolrServer.java:374)
        at 
org.apache.solr.update.StreamingSolrServers.blockUntilFinished(StreamingSolrServers.java:103)
        at 
org.apache.solr.update.SolrCmdDistributor.blockAndDoRetries(SolrCmdDistributor.java:228)
        at 
org.apache.solr.update.SolrCmdDistributor.finish(SolrCmdDistributor.java:89)
        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:766)
        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1662)
        at 
org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:179)
        at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:83)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1984)
        at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
        at 
org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
        at org.eclipse.jetty.server.Server.handle(Server.java:368)
        at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
        at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
        at 
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
        at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:953)
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
        at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
        at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=205, name=searcherExecutor-87-thread-1, state=WAITING, 
group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([A07445A293ED20EE]:0)


FAILED:  org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch

Error Message:
We have a failed SPLITSHARD task

Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
        at 
__randomizedtesting.SeedInfo.seed([A07445A293ED20EE:2192CBBAE4B240D2]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
        at 
org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
        at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
        at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13326 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> Creating dataDir: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/init-core-data-001
   [junit4]   2> 64454 T154 oas.BaseDistributedSearchTestCase.initHostContext 
Setting hostContext system property: /pb_/
   [junit4]   2> 64489 T154 oas.SolrTestCaseJ4.setUp ###Starting 
testDistribSearch
   [junit4]   2> 64490 T154 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 64491 T155 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 64590 T154 oasc.ZkTestServer.run start zk server on port:40090
   [junit4]   2> 64591 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 64595 T161 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@517f37f5 
name:ZooKeeperConnection Watcher:127.0.0.1:40090 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 64595 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 64596 T154 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 64600 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 64601 T163 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@289f65df 
name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 64602 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 64602 T154 oascc.SolrZkClient.makePath makePath: 
/collections/collection1
   [junit4]   2> 64606 T154 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/shards
   [junit4]   2> 64608 T154 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection
   [junit4]   2> 64611 T154 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/shards
   [junit4]   2> 64614 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 64615 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/solrconfig.xml
   [junit4]   2> 64619 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 64620 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/schema.xml
   [junit4]   2> 64624 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 64624 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 64627 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 64628 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/stopwords.txt
   [junit4]   2> 64631 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 64632 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/protwords.txt
   [junit4]   2> 64635 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 64636 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/currency.xml
   [junit4]   2> 64639 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 64640 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/enumsConfig.xml
   [junit4]   2> 64643 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 64643 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/open-exchange-rates.json
   [junit4]   2> 64646 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 64647 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 64650 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 64651 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/old_synonyms.txt
   [junit4]   2> 64654 T154 oasc.AbstractZkTestCase.putConfig put 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 64654 T154 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/synonyms.txt
   [junit4]   2> 67611 T154 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 67616 T154 oejs.AbstractConnector.doStart Started 
[email protected]:53547
   [junit4]   2> 67617 T154 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()
   [junit4]   2> 67617 T154 oasc.SolrResourceLoader.locateSolrHome JNDI not 
configured for solr (NoInitialContextEx)
   [junit4]   2> 67617 T154 oasc.SolrResourceLoader.locateSolrHome using system 
property solr.solr.home: 
../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002
   [junit4]   2> 67618 T154 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/'
   [junit4]   2> 67678 T154 oasc.ConfigSolr.fromFile Loading container 
configuration from 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/solr.xml
   [junit4]   2> 67792 T154 oasc.CoreContainer.<init> New CoreContainer 
1286591841
   [junit4]   2> 67792 T154 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/]
   [junit4]   2> 67794 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
socketTimeout to: 90000
   [junit4]   2> 67795 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
urlScheme to: 
   [junit4]   2> 67795 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
connTimeout to: 15000
   [junit4]   2> 67795 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maxConnectionsPerHost to: 20
   [junit4]   2> 67796 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
corePoolSize to: 0
   [junit4]   2> 67796 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maximumPoolSize to: 2147483647
   [junit4]   2> 67796 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maxThreadIdleTime to: 5
   [junit4]   2> 67797 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
sizeOfQueue to: -1
   [junit4]   2> 67797 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
fairnessPolicy to: false
   [junit4]   2> 67798 T154 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 67798 T154 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 67799 T154 oasl.LogWatcher.newRegisteredLogWatcher Registering 
Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 67799 T154 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 67800 T154 oasc.ZkContainer.initZooKeeper Zookeeper 
client=127.0.0.1:40090/solr
   [junit4]   2> 67800 T154 oasc.ZkController.checkChrootPath zkHost includes 
chroot
   [junit4]   2> 67801 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 67803 T174 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@10c2e46 name:ZooKeeperConnection 
Watcher:127.0.0.1:40090 got event WatchedEvent state:SyncConnected type:None 
path:null path:null type:None
   [junit4]   2> 67803 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 67806 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 67808 T176 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@52aea950 
name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 67808 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 67809 T154 oascc.SolrZkClient.makePath makePath: 
/overseer/queue
   [junit4]   2> 67812 T154 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-queue-work
   [junit4]   2> 67815 T154 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-map-running
   [junit4]   2> 67818 T154 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-map-completed
   [junit4]   2> 67820 T154 oascc.SolrZkClient.makePath makePath: 
/overseer/collection-map-failure
   [junit4]   2> 67823 T154 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 67825 T154 oasc.ZkController.createEphemeralLiveNode Register 
node as live in ZooKeeper:/live_nodes/127.0.0.1:53547_pb_
   [junit4]   2> 67826 T154 oascc.SolrZkClient.makePath makePath: 
/live_nodes/127.0.0.1:53547_pb_
   [junit4]   2> 67828 T154 oascc.SolrZkClient.makePath makePath: 
/overseer_elect
   [junit4]   2> 67830 T154 oascc.SolrZkClient.makePath makePath: 
/overseer_elect/election
   [junit4]   2> 67834 T154 oasc.OverseerElectionContext.runLeaderProcess I am 
going to be the leader 127.0.0.1:53547_pb_
   [junit4]   2> 67834 T154 oascc.SolrZkClient.makePath makePath: 
/overseer_elect/leader
   [junit4]   2> 67836 T154 oasc.Overseer.start Overseer 
(id=92237202507694083-127.0.0.1:53547_pb_-n_0000000000) starting
   [junit4]   2> 67839 T154 oascc.SolrZkClient.makePath makePath: 
/overseer/queue-work
   [junit4]   2> 67847 T178 oasc.OverseerCollectionProcessor.run Process 
current queue of collection creations
   [junit4]   2> 67847 T154 oascc.SolrZkClient.makePath makePath: 
/clusterstate.json
   [junit4]   2> 67849 T154 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 67851 T154 
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 67853 T177 oasc.Overseer$ClusterStateUpdater.run Starting to 
work on the main queue
   [junit4]   2> 67875 T179 oasc.ZkController.publish publishing 
core=collection1 state=down collection=control_collection
   [junit4]   2> 67875 T179 oasc.ZkController.publish numShards not found on 
descriptor - reading it from system property
   [junit4]   2> 67877 T179 oasc.ZkController.waitForCoreNodeName look for our 
core node name
   [junit4]   2> 67877 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 67883 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 67883 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Update state numShards=1 message={
   [junit4]   2>          "operation":"state",
   [junit4]   2>          "state":"down",
   [junit4]   2>          "base_url":"http://127.0.0.1:53547/pb_";,
   [junit4]   2>          "core":"collection1",
   [junit4]   2>          "roles":null,
   [junit4]   2>          "node_name":"127.0.0.1:53547_pb_",
   [junit4]   2>          "shard":null,
   [junit4]   2>          "collection":"control_collection",
   [junit4]   2>          "numShards":"1",
   [junit4]   2>          "core_node_name":null}
   [junit4]   2> 67884 T177 oasc.Overseer$ClusterStateUpdater.createCollection 
Create collection control_collection with shards [shard1]
   [junit4]   2> 67914 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Assigning new node to shard shard=shard1
   [junit4]   2> 67917 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 67918 T180 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 68878 T179 oasc.ZkController.waitForShardId waiting to find 
shard id in clusterstate for collection1
   [junit4]   2> 68878 T179 oasc.ZkController.createCollectionZkNode Check for 
collection zkNode:control_collection
   [junit4]   2> 68879 T179 oasc.ZkController.createCollectionZkNode Collection 
zkNode exists
   [junit4]   2> 68879 T179 oascc.ZkStateReader.readConfigName Load collection 
config from:/collections/control_collection
   [junit4]   2> 68880 T179 oascc.ZkStateReader.readConfigName 
path=/collections/control_collection configName=conf1 specified config exists 
in ZooKeeper
   [junit4]   2> 68881 T179 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/'
   [junit4]   2> 68882 T179 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/lib/classes/'
 to classloader
   [junit4]   2> 68883 T179 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/lib/README'
 to classloader
   [junit4]   2> 68883 T179 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/lib/.svn/'
 to classloader
   [junit4]   2> 68937 T179 oasc.SolrConfig.<init> Using Lucene MatchVersion: 
LUCENE_5_0
   [junit4]   2> 68980 T179 oasc.SolrConfig.<init> Loaded SolrConfig: 
solrconfig.xml
   [junit4]   2> 68981 T179 oass.IndexSchema.readSchema Reading Solr Schema 
from schema.xml
   [junit4]   2> 69001 T179 oass.IndexSchema.readSchema [collection1] Schema 
name=test
   [junit4]   2> 69414 T179 oass.IndexSchema.readSchema default search field in 
schema is text
   [junit4]   2> 69417 T179 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 69420 T179 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 69426 T179 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 69471 T179 oasc.CoreContainer.create Creating SolrCore 
'collection1' using configuration from collection control_collection
   [junit4]   2> 69471 T179 oasc.SolrCore.initDirectoryFactory 
solr.StandardDirectoryFactory
   [junit4]   2> 69542 T179 oasc.SolrCore.<init> [collection1] Opening new 
SolrCore at 
../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-002/collection1/,
 
dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/
   [junit4]   2> 69543 T179 oasc.JmxMonitoredMap.<init> JMX monitoring is 
enabled. Adding Solr mbeans to JMX Server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@54b83713
   [junit4]   2> 69545 T179 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data
   [junit4]   2> 69547 T179 oasc.SolrCore.getNewIndexDir New index directory 
detected: old=null 
new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index/
   [junit4]   2> 69548 T179 oasc.SolrCore.initIndex WARN [collection1] Solr 
index directory 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index'
 doesn't exist. Creating new index...
   [junit4]   2> 69549 T179 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index
   [junit4]   2> 69550 T179 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=50, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=0.4306640625, 
floorSegmentMB=1.0390625, forceMergeDeletesPctAllowed=3.007953945489976, 
segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 69582 T179 oasc.SolrDeletionPolicy.onCommit 
SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>                
commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 69583 T179 oasc.SolrDeletionPolicy.updateCommits newest commit 
generation = 1
   [junit4]   2> 69590 T179 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 69591 T179 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 69591 T179 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 69591 T179 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 69592 T179 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 69592 T179 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 69592 T179 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 69593 T179 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 69593 T179 oasc.SolrCore.loadUpdateProcessorChains no 
updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 69594 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69594 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69595 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69596 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69596 T179 oasc.RequestHandlers.initHandlersFromConfig created 
standard: solr.StandardRequestHandler
   [junit4]   2> 69597 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/get: solr.RealTimeGetHandler
   [junit4]   2> 69597 T179 oasc.RequestHandlers.initHandlersFromConfig adding 
lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 69597 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/replication: solr.ReplicationHandler
   [junit4]   2> 69598 T179 oasc.RequestHandlers.initHandlersFromConfig WARN 
Multiple requestHandler registered to the same name: /update ignoring: 
org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 69598 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/update: solr.UpdateRequestHandler
   [junit4]   2> 69599 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 69599 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 69600 T179 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 69619 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69622 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69624 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69626 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69628 T179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 69637 T179 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 69638 T179 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 69639 T179 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=3, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=56.107421875, 
floorSegmentMB=0.4912109375, forceMergeDeletesPctAllowed=26.23316267623028, 
segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.28139084101318934
   [junit4]   2> 69641 T179 oasc.SolrDeletionPolicy.onInit 
SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>                
commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 69642 T179 oasc.SolrDeletionPolicy.updateCommits newest commit 
generation = 1
   [junit4]   2> 69642 T179 oass.SolrIndexSearcher.<init> Opening 
Searcher@36e6512c[collection1] main
   [junit4]   2> 69643 T179 oascc.ZkStateReader.readConfigName Load collection 
config from:/collections/control_collection
   [junit4]   2> 69644 T179 oascc.ZkStateReader.readConfigName 
path=/collections/control_collection configName=conf1 specified config exists 
in ZooKeeper
   [junit4]   2> 69644 T179 oasr.ManagedResourceStorage.newStorageIO Setting up 
ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 69656 T179 
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured 
ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 69657 T179 oasr.RestManager.init Initializing RestManager with 
initArgs: {}
   [junit4]   2> 69657 T179 oasr.ManagedResourceStorage.load Reading 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 69658 T179 
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found 
for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 69659 T179 oasr.ManagedResourceStorage.load Loaded null at 
path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 69660 T179 oascc.SolrZkClient.makePath makePath: 
/configs/conf1/_rest_managed.json
   [junit4]   2> 69663 T179 
oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new 
znode /configs/conf1/_rest_managed.json
   [junit4]   2> 69664 T179 oasr.ManagedResourceStorage$JsonStorage.store Saved 
JSON object to path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 69664 T179 oasr.RestManager.init Initializing 0 registered 
ManagedResources
   [junit4]   2> 69668 T181 oasc.SolrCore.registerSearcher [collection1] 
Registered new searcher Searcher@36e6512c[collection1] 
main{UninvertingDirectoryReader()}
   [junit4]   2> 69674 T179 oasc.CoreContainer.registerCore registering core: 
collection1
   [junit4]   2> 69675 T184 oasc.ZkController.register Register replica - 
core:collection1 address:http://127.0.0.1:53547/pb_ 
collection:control_collection shard:shard1
   [junit4]   2> 69676 T154 oass.SolrDispatchFilter.init 
user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 69676 T154 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init() done
   [junit4]   2> 69714 T184 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 69721 T184 oasc.ShardLeaderElectionContext.runLeaderProcess 
Running the leader process for shard shard1
   [junit4]   2> 69723 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 69724 T184 
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found 
to continue.
   [junit4]   2> 69724 T184 oasc.ShardLeaderElectionContext.runLeaderProcess I 
may be the new leader - try and sync
   [junit4]   2> 69725 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C49 name=collection1 
org.apache.solr.core.SolrCore@3856a77c 
url=http://127.0.0.1:53547/pb_/collection1 node=127.0.0.1:53547_pb_ 
C49_STATE=coll:control_collection core:collection1 props:{state=down, 
base_url=http://127.0.0.1:53547/pb_, core=collection1, 
node_name=127.0.0.1:53547_pb_}
   [junit4]   2> 69732 T184 C49 P53547 oasc.SyncStrategy.sync Sync replicas to 
http://127.0.0.1:53547/pb_/collection1/
   [junit4]   2> 69732 T184 C49 P53547 oasc.SyncStrategy.syncReplicas Sync 
Success - now sync replicas to me
   [junit4]   2> 69733 T184 C49 P53547 oasc.SyncStrategy.syncToMe 
http://127.0.0.1:53547/pb_/collection1/ has no replicas
   [junit4]   2> 69733 T184 oasc.ShardLeaderElectionContext.runLeaderProcess I 
am the new leader: http://127.0.0.1:53547/pb_/collection1/ shard1
   [junit4]   2> 69734 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 69735 T180 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 69747 T184 oascc.SolrZkClient.makePath makePath: 
/collections/control_collection/leaders/shard1
   [junit4]   2> 69753 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 69755 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 69758 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 69760 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 69761 T186 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@7ae41e92 
name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 69762 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 69763 T154 
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 69769 T154 oasc.ChaosMonkey.monkeyLog monkey: init - expire 
sessions:false cause connection loss:false
   [junit4]   2> 69861 T180 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 69861 T187 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 69906 T184 oasc.ZkController.register We are 
http://127.0.0.1:53547/pb_/collection1/ and leader is 
http://127.0.0.1:53547/pb_/collection1/
   [junit4]   2> 69907 T184 oasc.ZkController.register No LogReplay needed for 
core=collection1 baseURL=http://127.0.0.1:53547/pb_
   [junit4]   2> 69907 T184 oasc.ZkController.checkRecovery I am the leader, no 
recovery necessary
   [junit4]   2> 69907 T184 oasc.ZkController.publish publishing 
core=collection1 state=active collection=control_collection
   [junit4]   2> 69907 T184 oasc.ZkController.publish numShards not found on 
descriptor - reading it from system property
   [junit4]   2> 69910 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 69910 T188 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 69910 T189 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 69910 T184 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 69912 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 69913 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Update state numShards=2 message={
   [junit4]   2>          "operation":"state",
   [junit4]   2>          "state":"active",
   [junit4]   2>          "base_url":"http://127.0.0.1:53547/pb_";,
   [junit4]   2>          "core":"collection1",
   [junit4]   2>          "roles":null,
   [junit4]   2>          "node_name":"127.0.0.1:53547_pb_",
   [junit4]   2>          "shard":"shard1",
   [junit4]   2>          "collection":"control_collection",
   [junit4]   2>          "numShards":"2",
   [junit4]   2>          "core_node_name":"core_node1"}
   [junit4]   2> 69916 T189 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 70019 T189 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 70019 T187 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 71199 T154 oasc.AbstractFullDistribZkTestBase.createJettys 
create jetty 1
   [junit4]   2> 71200 T154 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 71202 T154 oejs.AbstractConnector.doStart Started 
[email protected]:45461
   [junit4]   2> 71203 T154 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()
   [junit4]   2> 71203 T154 oasc.SolrResourceLoader.locateSolrHome JNDI not 
configured for solr (NoInitialContextEx)
   [junit4]   2> 71203 T154 oasc.SolrResourceLoader.locateSolrHome using system 
property solr.solr.home: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003
   [junit4]   2> 71204 T154 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/'
   [junit4]   2> 71236 T154 oasc.ConfigSolr.fromFile Loading container 
configuration from 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/solr.xml
   [junit4]   2> 71299 T154 oasc.CoreContainer.<init> New CoreContainer 95952472
   [junit4]   2> 71300 T154 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/]
   [junit4]   2> 71301 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
socketTimeout to: 90000
   [junit4]   2> 71301 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
urlScheme to: 
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
connTimeout to: 15000
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maxConnectionsPerHost to: 20
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
corePoolSize to: 0
   [junit4]   2> 71302 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maximumPoolSize to: 2147483647
   [junit4]   2> 71303 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maxThreadIdleTime to: 5
   [junit4]   2> 71303 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
sizeOfQueue to: -1
   [junit4]   2> 71303 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
fairnessPolicy to: false
   [junit4]   2> 71304 T154 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 71304 T154 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 71304 T154 oasl.LogWatcher.newRegisteredLogWatcher Registering 
Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 71304 T154 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 71305 T154 oasc.ZkContainer.initZooKeeper Zookeeper 
client=127.0.0.1:40090/solr
   [junit4]   2> 71305 T154 oasc.ZkController.checkChrootPath zkHost includes 
chroot
   [junit4]   2> 71306 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 71308 T200 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@7ad02656 
name:ZooKeeperConnection Watcher:127.0.0.1:40090 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 71308 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 71310 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 71312 T202 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@d8b69db name:ZooKeeperConnection 
Watcher:127.0.0.1:40090/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 71312 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 71319 T154 
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 72323 T154 oasc.ZkController.createEphemeralLiveNode Register 
node as live in ZooKeeper:/live_nodes/127.0.0.1:45461_pb_
   [junit4]   2> 72324 T154 oascc.SolrZkClient.makePath makePath: 
/live_nodes/127.0.0.1:45461_pb_
   [junit4]   2> 72342 T204 oasc.ZkController.publish publishing 
core=collection1 state=down collection=collection1
   [junit4]   2> 72342 T204 oasc.ZkController.publish numShards not found on 
descriptor - reading it from system property
   [junit4]   2> 72343 T189 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 72343 T188 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 72343 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 72343 T204 oasc.ZkController.waitForCoreNodeName look for our 
core node name
   [junit4]   2> 72345 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 72346 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Update state numShards=2 message={
   [junit4]   2>          "operation":"state",
   [junit4]   2>          "state":"down",
   [junit4]   2>          "base_url":"http://127.0.0.1:45461/pb_";,
   [junit4]   2>          "core":"collection1",
   [junit4]   2>          "roles":null,
   [junit4]   2>          "node_name":"127.0.0.1:45461_pb_",
   [junit4]   2>          "shard":null,
   [junit4]   2>          "collection":"collection1",
   [junit4]   2>          "numShards":"2",
   [junit4]   2>          "core_node_name":null}
   [junit4]   2> 72346 T177 oasc.Overseer$ClusterStateUpdater.createCollection 
Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 72346 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Assigning new node to shard shard=shard2
   [junit4]   2> 72349 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 72349 T203 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 72350 T187 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 72349 T180 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 73344 T204 oasc.ZkController.waitForShardId waiting to find 
shard id in clusterstate for collection1
   [junit4]   2> 73345 T204 oasc.ZkController.createCollectionZkNode Check for 
collection zkNode:collection1
   [junit4]   2> 73346 T204 oasc.ZkController.createCollectionZkNode Collection 
zkNode exists
   [junit4]   2> 73346 T204 oascc.ZkStateReader.readConfigName Load collection 
config from:/collections/collection1
   [junit4]   2> 73347 T204 oascc.ZkStateReader.readConfigName 
path=/collections/collection1 configName=conf1 specified config exists in 
ZooKeeper
   [junit4]   2> 73348 T204 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/'
   [junit4]   2> 73349 T204 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/lib/.svn/'
 to classloader
   [junit4]   2> 73350 T204 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/lib/classes/'
 to classloader
   [junit4]   2> 73351 T204 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/lib/README'
 to classloader
   [junit4]   2> 73425 T204 oasc.SolrConfig.<init> Using Lucene MatchVersion: 
LUCENE_5_0
   [junit4]   2> 73479 T204 oasc.SolrConfig.<init> Loaded SolrConfig: 
solrconfig.xml
   [junit4]   2> 73481 T204 oass.IndexSchema.readSchema Reading Solr Schema 
from schema.xml
   [junit4]   2> 73511 T204 oass.IndexSchema.readSchema [collection1] Schema 
name=test
   [junit4]   2> 73935 T204 oass.IndexSchema.readSchema default search field in 
schema is text
   [junit4]   2> 73939 T204 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 73942 T204 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 73948 T204 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 73994 T204 oasc.CoreContainer.create Creating SolrCore 
'collection1' using configuration from collection collection1
   [junit4]   2> 73995 T204 oasc.SolrCore.initDirectoryFactory 
solr.StandardDirectoryFactory
   [junit4]   2> 73995 T204 oasc.SolrCore.<init> [collection1] Opening new 
SolrCore at 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-003/collection1/,
 
dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/
   [junit4]   2> 73995 T204 oasc.JmxMonitoredMap.<init> JMX monitoring is 
enabled. Adding Solr mbeans to JMX Server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@54b83713
   [junit4]   2> 73997 T204 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1
   [junit4]   2> 73998 T204 oasc.SolrCore.getNewIndexDir New index directory 
detected: old=null 
new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index/
   [junit4]   2> 73999 T204 oasc.SolrCore.initIndex WARN [collection1] Solr 
index directory 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index'
 doesn't exist. Creating new index...
   [junit4]   2> 74001 T204 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index
   [junit4]   2> 74002 T204 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=50, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=0.4306640625, 
floorSegmentMB=1.0390625, forceMergeDeletesPctAllowed=3.007953945489976, 
segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 74037 T204 oasc.SolrDeletionPolicy.onCommit 
SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>                
commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 74038 T204 oasc.SolrDeletionPolicy.updateCommits newest commit 
generation = 1
   [junit4]   2> 74048 T204 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 74048 T204 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 74049 T204 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 74049 T204 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 74049 T204 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 74050 T204 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 74050 T204 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 74051 T204 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 74051 T204 oasc.SolrCore.loadUpdateProcessorChains no 
updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 74052 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74052 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74053 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74054 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74055 T204 oasc.RequestHandlers.initHandlersFromConfig created 
standard: solr.StandardRequestHandler
   [junit4]   2> 74056 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/get: solr.RealTimeGetHandler
   [junit4]   2> 74056 T204 oasc.RequestHandlers.initHandlersFromConfig adding 
lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 74056 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/replication: solr.ReplicationHandler
   [junit4]   2> 74057 T204 oasc.RequestHandlers.initHandlersFromConfig WARN 
Multiple requestHandler registered to the same name: /update ignoring: 
org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 74057 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/update: solr.UpdateRequestHandler
   [junit4]   2> 74058 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 74059 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 74059 T204 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 74081 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74084 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74086 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74088 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74090 T204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 74095 T204 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 74095 T204 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 74096 T204 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=3, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=56.107421875, 
floorSegmentMB=0.4912109375, forceMergeDeletesPctAllowed=26.23316267623028, 
segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.28139084101318934
   [junit4]   2> 74099 T204 oasc.SolrDeletionPolicy.onInit 
SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>                
commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 74099 T204 oasc.SolrDeletionPolicy.updateCommits newest commit 
generation = 1
   [junit4]   2> 74100 T204 oass.SolrIndexSearcher.<init> Opening 
Searcher@2fe00fed[collection1] main
   [junit4]   2> 74100 T204 oascc.ZkStateReader.readConfigName Load collection 
config from:/collections/collection1
   [junit4]   2> 74101 T204 oascc.ZkStateReader.readConfigName 
path=/collections/collection1 configName=conf1 specified config exists in 
ZooKeeper
   [junit4]   2> 74102 T204 oasr.ManagedResourceStorage.newStorageIO Setting up 
ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 74102 T204 
oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured 
ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 74103 T204 oasr.RestManager.init Initializing RestManager with 
initArgs: {}
   [junit4]   2> 74103 T204 oasr.ManagedResourceStorage.load Reading 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 74104 T204 
oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes 
from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 74105 T204 oasr.ManagedResourceStorage.load Loaded 
LinkedHashMap at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 74105 T204 oasr.ManagedResource.reloadFromStorage Loaded 
initArgs {} for /rest/managed
   [junit4]   2> 74105 T204 oasr.RestManager.init Initializing 0 registered 
ManagedResources
   [junit4]   2> 74111 T205 oasc.SolrCore.registerSearcher [collection1] 
Registered new searcher Searcher@2fe00fed[collection1] 
main{UninvertingDirectoryReader()}
   [junit4]   2> 74118 T204 oasc.CoreContainer.registerCore registering core: 
collection1
   [junit4]   2> 74119 T208 oasc.ZkController.register Register replica - 
core:collection1 address:http://127.0.0.1:45461/pb_ collection:collection1 
shard:shard2
   [junit4]   2> 74120 T154 oass.SolrDispatchFilter.init 
user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 74120 T154 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init() done
   [junit4]   2> 74120 T208 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leader_elect/shard2/election
   [junit4]   2> 74127 T208 oasc.ShardLeaderElectionContext.runLeaderProcess 
Running the leader process for shard shard2
   [junit4]   2> 74130 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 74130 T208 
oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found 
to continue.
   [junit4]   2> 74131 T208 oasc.ShardLeaderElectionContext.runLeaderProcess I 
may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C50 name=collection1 
org.apache.solr.core.SolrCore@47c1facb 
url=http://127.0.0.1:45461/pb_/collection1 node=127.0.0.1:45461_pb_ 
C50_STATE=coll:collection1 core:collection1 props:{state=down, 
base_url=http://127.0.0.1:45461/pb_, core=collection1, 
node_name=127.0.0.1:45461_pb_}
   [junit4]   2> 74131 T208 C50 P45461 oasc.SyncStrategy.sync Sync replicas to 
http://127.0.0.1:45461/pb_/collection1/
   [junit4]   2> 74131 T208 C50 P45461 oasc.SyncStrategy.syncReplicas Sync 
Success - now sync replicas to me
   [junit4]   2> 74132 T208 C50 P45461 oasc.SyncStrategy.syncToMe 
http://127.0.0.1:45461/pb_/collection1/ has no replicas
   [junit4]   2> 74132 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 74132 T208 oasc.ShardLeaderElectionContext.runLeaderProcess I 
am the new leader: http://127.0.0.1:45461/pb_/collection1/ shard2
   [junit4]   2> 74132 T208 oascc.SolrZkClient.makePath makePath: 
/collections/collection1/leaders/shard2
   [junit4]   2> 74135 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 74136 T180 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74137 T187 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74136 T203 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74142 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 74145 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 74248 T203 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74248 T187 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74248 T180 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74293 T208 oasc.ZkController.register We are 
http://127.0.0.1:45461/pb_/collection1/ and leader is 
http://127.0.0.1:45461/pb_/collection1/
   [junit4]   2> 74294 T208 oasc.ZkController.register No LogReplay needed for 
core=collection1 baseURL=http://127.0.0.1:45461/pb_
   [junit4]   2> 74294 T208 oasc.ZkController.checkRecovery I am the leader, no 
recovery necessary
   [junit4]   2> 74294 T208 oasc.ZkController.publish publishing 
core=collection1 state=active collection=collection1
   [junit4]   2> 74294 T208 oasc.ZkController.publish numShards not found on 
descriptor - reading it from system property
   [junit4]   2> 74297 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 74297 T189 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 74297 T208 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 74297 T188 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 74299 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 74300 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Update state numShards=2 message={
   [junit4]   2>          "operation":"state",
   [junit4]   2>          "state":"active",
   [junit4]   2>          "base_url":"http://127.0.0.1:45461/pb_";,
   [junit4]   2>          "core":"collection1",
   [junit4]   2>          "roles":null,
   [junit4]   2>          "node_name":"127.0.0.1:45461_pb_",
   [junit4]   2>          "shard":"shard2",
   [junit4]   2>          "collection":"collection1",
   [junit4]   2>          "numShards":"2",
   [junit4]   2>          "core_node_name":"core_node1"}
   [junit4]   2> 74303 T188 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 74406 T188 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74406 T203 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 74406 T187 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 75674 T154 oasc.AbstractFullDistribZkTestBase.createJettys 
create jetty 2
   [junit4]   2> 75675 T154 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 75677 T154 oejs.AbstractConnector.doStart Started 
[email protected]:41438
   [junit4]   2> 75678 T154 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()
   [junit4]   2> 75678 T154 oasc.SolrResourceLoader.locateSolrHome JNDI not 
configured for solr (NoInitialContextEx)
   [junit4]   2> 75678 T154 oasc.SolrResourceLoader.locateSolrHome using system 
property solr.solr.home: 
../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004
   [junit4]   2> 75678 T154 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/'
   [junit4]   2> 75711 T154 oasc.ConfigSolr.fromFile Loading container 
configuration from 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/solr.xml
   [junit4]   2> 75774 T154 oasc.CoreContainer.<init> New CoreContainer 
1040609761
   [junit4]   2> 75774 T154 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/]
   [junit4]   2> 75775 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
socketTimeout to: 90000
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
urlScheme to: 
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
connTimeout to: 15000
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maxConnectionsPerHost to: 20
   [junit4]   2> 75776 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
corePoolSize to: 0
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maximumPoolSize to: 2147483647
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
maxThreadIdleTime to: 5
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
sizeOfQueue to: -1
   [junit4]   2> 75777 T154 oashc.HttpShardHandlerFactory.getParameter Setting 
fairnessPolicy to: false
   [junit4]   2> 75778 T154 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 75778 T154 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 75778 T154 oasl.LogWatcher.newRegisteredLogWatcher Registering 
Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 75779 T154 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 75779 T154 oasc.ZkContainer.initZooKeeper Zookeeper 
client=127.0.0.1:40090/solr
   [junit4]   2> 75779 T154 oasc.ZkController.checkChrootPath zkHost includes 
chroot
   [junit4]   2> 75780 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 75782 T219 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@191e8a97 
name:ZooKeeperConnection Watcher:127.0.0.1:40090 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75782 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 75785 T154 oascc.ConnectionManager.waitForConnected Waiting 
for client to connect to ZooKeeper
   [junit4]   2> 75795 T221 oascc.ConnectionManager.process Watcher 
org.apache.solr.common.cloud.ConnectionManager@562cf72d 
name:ZooKeeperConnection Watcher:127.0.0.1:40090/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75795 T154 oascc.ConnectionManager.waitForConnected Client is 
connected to ZooKeeper
   [junit4]   2> 75802 T154 
oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 76806 T154 oasc.ZkController.createEphemeralLiveNode Register 
node as live in ZooKeeper:/live_nodes/127.0.0.1:41438_pb_
   [junit4]   2> 76807 T154 oascc.SolrZkClient.makePath makePath: 
/live_nodes/127.0.0.1:41438_pb_
   [junit4]   2> 76819 T223 oasc.ZkController.publish publishing 
core=collection1 state=down collection=collection1
   [junit4]   2> 76819 T223 oasc.ZkController.publish numShards not found on 
descriptor - reading it from system property
   [junit4]   2> 76820 T188 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 76820 T180 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 76820 T189 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 76820 T223 oasc.ZkController.waitForCoreNodeName look for our 
core node name
   [junit4]   2> 76822 T177 oascc.ZkStateReader.updateClusterState Updating 
cloud state from ZooKeeper... 
   [junit4]   2> 76823 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Update state numShards=2 message={
   [junit4]   2>          "operation":"state",
   [junit4]   2>          "state":"down",
   [junit4]   2>          "base_url":"http://127.0.0.1:41438/pb_";,
   [junit4]   2>          "core":"collection1",
   [junit4]   2>          "roles":null,
   [junit4]   2>          "node_name":"127.0.0.1:41438_pb_",
   [junit4]   2>          "shard":null,
   [junit4]   2>          "collection":"collection1",
   [junit4]   2>          "numShards":"2",
   [junit4]   2>          "core_node_name":null}
   [junit4]   2> 76823 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Collection already exists with numShards=2
   [junit4]   2> 76823 T177 oasc.Overseer$ClusterStateUpdater.updateState 
Assigning new node to shard shard=shard1
   [junit4]   2> 76826 T189 oasc.DistributedQueue$LatchChildWatcher.process 
LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type 
NodeChildrenChanged
   [junit4]   2> 76827 T189 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 76827 T203 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 76827 T187 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 76827 T222 oascc.ZkStateReader$2.process A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 77821 T223 oasc.ZkController.waitForShardId waiting to find 
shard id in clusterstate for collection1
   [junit4]   2> 77822 T223 oasc.ZkController.createCollectionZkNode Check for 
collection zkNode:collection1
   [junit4]   2> 77823 T223 oasc.ZkController.createCollectionZkNode Collection 
zkNode exists
   [junit4]   2> 77823 T223 oascc.ZkStateReader.readConfigName Load collection 
config from:/collections/collection1
   [junit4]   2> 77824 T223 oascc.ZkStateReader.readConfigName 
path=/collections/collection1 configName=conf1 specified config exists in 
ZooKeeper
   [junit4]   2> 77824 T223 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/'
   [junit4]   2> 77827 T223 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/lib/README'
 to classloader
   [junit4]   2> 77828 T223 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/lib/classes/'
 to classloader
   [junit4]   2> 77828 T223 oasc.SolrResourceLoader.replaceClassLoader Adding 
'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/lib/.svn/'
 to classloader
   [junit4]   2> 77902 T223 oasc.SolrConfig.<init> Using Lucene MatchVersion: 
LUCENE_5_0
   [junit4]   2> 77956 T223 oasc.SolrConfig.<init> Loaded SolrConfig: 
solrconfig.xml
   [junit4]   2> 77958 T223 oass.IndexSchema.readSchema Reading Solr Schema 
from schema.xml
   [junit4]   2> 77983 T223 oass.IndexSchema.readSchema [collection1] Schema 
name=test
   [junit4]   2> 78440 T223 oass.IndexSchema.readSchema default search field in 
schema is text
   [junit4]   2> 78444 T223 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 78446 T223 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 78452 T223 oass.FileExchangeRateProvider.reload Reloading 
exchange rates from file currency.xml
   [junit4]   2> 78496 T223 oasc.CoreContainer.create Creating SolrCore 
'collection1' using configuration from collection collection1
   [junit4]   2> 78496 T223 oasc.SolrCore.initDirectoryFactory 
solr.StandardDirectoryFactory
   [junit4]   2> 78497 T223 oasc.SolrCore.<init> [collection1] Opening new 
SolrCore at 
../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-004/collection1/,
 
dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/
   [junit4]   2> 78497 T223 oasc.JmxMonitoredMap.<init> JMX monitoring is 
enabled. Adding Solr mbeans to JMX Server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@54b83713
   [junit4]   2> 78499 T223 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2
   [junit4]   2> 78500 T223 oasc.SolrCore.getNewIndexDir New index directory 
detected: old=null 
new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index/
   [junit4]   2> 78501 T223 oasc.SolrCore.initIndex WARN [collection1] Solr 
index directory 
'/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index'
 doesn't exist. Creating new index...
   [junit4]   2> 78502 T223 oasc.CachingDirectoryFactory.get return new 
directory for 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index
   [junit4]   2> 78503 T223 oasu.RandomMergePolicy.<init> RandomMergePolicy 
wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=50, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=0.4306640625, 
floorSegmentMB=1.0390625, forceMergeDeletesPctAllowed=3.007953945489976, 
segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 78515 T223 oasc.SolrDeletionPolicy.onCommit 
SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>                
commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest-A07445A293ED20EE-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 78515 T223 oasc.SolrDeletionPolicy.updateCommits newest commit 
generation = 1
   [junit4]   2> 78525 T223 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 78525 T223 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 78526 T223 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 78527 T223 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 78527 T223 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 78527 T223 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 78528 T223 oasup.UpdateRequestProcessorChain.init creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 78528 T223 oasup.UpdateRequestProcessorChain.init inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 78529 T223 oasc.SolrCore.loadUpdateProcessorChains no 
updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 78530 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78530 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78531 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78532 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78533 T223 oasc.RequestHandlers.initHandlersFromConfig created 
standard: solr.StandardRequestHandler
   [junit4]   2> 78533 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/get: solr.RealTimeGetHandler
   [junit4]   2> 78534 T223 oasc.RequestHandlers.initHandlersFromConfig adding 
lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 78534 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/replication: solr.ReplicationHandler
   [junit4]   2> 78535 T223 oasc.RequestHandlers.initHandlersFromConfig WARN 
Multiple requestHandler registered to the same name: /update ignoring: 
org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 78535 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/update: solr.UpdateRequestHandler
   [junit4]   2> 78535 T223 oasc.RequestHandlers.initHandlersFromConfig created 
/admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 78536

[...truncated too long message...]

it4]   2>       at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2>        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
   [junit4]   2> 
   [junit4]   2> 4118055 T16920 oascc.ZkStateReader$3.process WARN ZooKeeper 
watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 4118057 T15349 oejsh.ContextHandler.doStop stopped 
o.e.j.s.ServletContextHandler{/m_gef/be,null}
   [junit4]   2> 4120433 T15349 oas.SolrTestCaseJ4.tearDown ###Ending 
testDistribSearch
   [junit4]   2> 4120435 T15349 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:39587 39587
   [junit4]   2> 4120617 T15350 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:39587 39587
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch 
-Dtests.seed=A07445A293ED20EE -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=zh_TW -Dtests.timezone=Europe/Stockholm 
-Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 57.0s J0 | MultiThreadedOCPTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: We have a failed 
SPLITSHARD task
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([A07445A293ED20EE:2192CBBAE4B240D2]:0)
   [junit4]    >        at 
org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
   [junit4]    >        at 
org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4120628 T15349 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-A07445A293ED20EE-001
   [junit4]   2> 56998 T15348 ccr.ThreadLeakControl.checkThreadLeaks WARNING 
Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=SimpleText, 
sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=zh_TW, 
timezone=Europe/Stockholm
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 
(64-bit)/cpus=16,threads=1,free=210295832,total=467140608
   [junit4]   2> NOTE: All tests run in this JVM: [DirectSolrSpellCheckerTest, 
HdfsCollectionsAPIDistributedZkTest, TermsComponentTest, EchoParamsTest, 
TestDefaultSearchFieldResource, ZkControllerTest, TestAnalyzedSuggestions, 
TestStressVersions, TestFieldTypeCollectionResource, 
ChaosMonkeyNothingIsSafeTest, SuggesterWFSTTest, TestCollationFieldDocValues, 
RAMDirectoryFactoryTest, CoreAdminRequestStatusTest, 
SolrCoreCheckLockOnStartupTest, OverseerTest, PrimUtilsTest, 
DeleteInactiveReplicaTest, TestHighlightDedupGrouping, PrimitiveFieldTypeTest, 
HdfsSyncSliceTest, ChangedSchemaMergeTest, CursorMarkTest, 
TestManagedSynonymFilterFactory, ClusterStateUpdateTest, 
XsltUpdateRequestHandlerTest, TestDefaultSimilarityFactory, 
AnalyticsMergeStrategyTest, TestBinaryResponseWriter, PeerSyncTest, 
TestComponentsName, UnloadDistributedZkTest, TestFoldingMultitermQuery, 
PluginInfoTest, TestCodecSupport, TestStressRecovery, TestInfoStreamLogging, 
PingRequestHandlerTest, XmlUpdateRequestHandlerTest, TestDFRSimilarityFactory, 
DirectUpdateHandlerTest, SpellCheckCollatorTest, SolrCmdDistributorTest, 
TestFastOutputStream, DocumentBuilderTest, ReplicationFactorTest, 
MBeansHandlerTest, TestSolrDeletionPolicy1, TestUtils, TestSolrIndexConfig, 
ZkNodePropsTest, TestSolrDeletionPolicy2, AssignTest, 
AbstractAnalyticsStatsTest, TestExceedMaxTermLength, TestSolr4Spatial, 
TestFastLRUCache, SuggestComponentTest, BadIndexSchemaTest, 
TestSolrXmlPersistence, SimpleFacetsTest, OverseerCollectionProcessorTest, 
TestMissingGroups, ReturnFieldsTest, MoreLikeThisHandlerTest, 
FastVectorHighlighterTest, NoCacheHeaderTest, TestStressReorder, 
TestPartialUpdateDeduplication, JsonLoaderTest, ShardRoutingTest, 
TestFastWriter, HdfsBasicDistributedZk2Test, TestSchemaVersionResource, 
DocExpirationUpdateProcessorFactoryTest, TestFuzzyAnalyzedSuggestions, 
TestFieldResource, TestSchemaResource, SolrIndexSplitterTest, SuggesterTSTTest, 
TestAnalyzeInfixSuggestions, TimeZoneUtilsTest, TestLuceneMatchVersion, 
ExternalFileFieldSortTest, DisMaxRequestHandlerTest, 
AddSchemaFieldsUpdateProcessorFactoryTest, TestComplexPhraseQParserPlugin, 
TestManagedResource, HighlighterMaxOffsetTest, TestReplicationHandlerBackup, 
TestLeaderElectionZkExpiry, TestJmxIntegration, 
TestCopyFieldCollectionResource, SolrXmlInZkTest, ShowFileRequestHandlerTest, 
HighlighterConfigTest, HdfsWriteToMultipleCollectionsTest, UpdateParamsTest, 
RegexBoostProcessorTest, TestSolrXml, TestAddFieldRealTimeGet, 
OpenExchangeRatesOrgProviderTest, BasicDistributedZkTest, 
ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, SyncSliceTest, 
TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, ZkCLITest, 
ShardRoutingCustomTest, TestDistributedSearch, TestDistributedGrouping, 
TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, 
TestRealTimeGet, TestReload, DistributedTermsComponentTest, TestRangeQuery, 
SpellCheckComponentTest, TestGroupingSearch, QueryElevationComponentTest, 
TestSort, BasicFunctionalityTest, HighlighterTest, 
CurrencyFieldOpenExchangeTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, 
SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, 
CoreAdminHandlerTest, SuggesterTest, TestTrie, PolyFieldTest, 
WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, 
FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, 
QueryEqualityTest, DefaultValueUpdateProcessorTest, 
StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, 
DocValuesMultiTest, DistanceFunctionTest, SolrInfoMBeanTest, 
DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, 
TestSurroundQueryParser, LukeRequestHandlerTest, TestQueryUtils, 
StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, 
DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, 
FileBasedSpellCheckerTest, TestValueSourceCache, 
PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, 
FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, 
IndexSchemaRuntimeFieldTest, LoggingHandlerTest, IndexSchemaTest, 
UpdateRequestProcessorFactoryTest, JSONWriterTest, TestCSVResponseWriter, 
UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, 
CoreContainerCoreInitFailuresTest, TestLFUCache, SOLR749Test, 
TestQuerySenderListener, TestQuerySenderNoQuery, CopyFieldTest, 
ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, SampleTest, 
TestDocSet, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestConfig, 
TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, 
TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, 
SynonymTokenizerTest, TestXIncludeConfig, TestSweetSpotSimilarityFactory, 
TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, 
TestBM25SimilarityFactory, TestJmxMonitoredMap, ResourceLoaderTest, 
ScriptEngineTest, URLClassifyProcessorTest, PreAnalyzedFieldTest, 
DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, ClusterStateTest, 
TestSolrJ, TestDocumentBuilder, SliceStateTest, SystemInfoHandlerTest, 
FileUtilsTest, TestRTGBase, CursorPagingTest, TestCursorMarkWithoutUniqueKey, 
TestDistributedMissingSort, TestTolerantSearch, NoFacetTest, ExpressionTest, 
AbstractAnalyticsFacetTest, FieldFacetExtrasTest, FieldFacetTest, 
RangeFacetTest, FunctionTest, CustomCollectionTest, 
DistribDocExpirationUpdateProcessorTest, MultiThreadedOCPTest]
   [junit4] Completed on J0 in 57.28s, 1 test, 1 failure <<< FAILURES!

[...truncated 234 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:481:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:454:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:45:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:496:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1298:
 The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:922:
 There were test failures: 422 suites, 1761 tests, 2 suite-level errors, 1 
failure, 51 ignored (9 assumptions)

Total time: 200 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #592
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 719889489 bytes
Compression is 0.0%
Took 2 min 5 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to