[
https://issues.apache.org/jira/browse/SOLR-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675354#comment-16675354
]
Kevin Risden commented on SOLR-12932:
-------------------------------------
I had stopped the runs for the weekend but this morning 5/5 failed with the
same test failure.
Commit:34500a1262f2a7a6602e43904facc7da958e0cd1
Test:AutoscalingHistoryHandlerTest
{code:java}
reproduce with: ant test -Dtestcase=AutoscalingHistoryHandlerTest
-Dtests.seed=8F97179111B556A4 -Dtests.slow=true -Dtests.badapples=true
-Dtests.locale=sr-BA -Dtests.timezone=Cuba -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | AutoscalingHistoryHandlerTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ObjectTracker found 2
object(s) that were not released!!! [ZkStateReader, SolrZkClient]
[junit4] >
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException:
org.apache.solr.common.cloud.ZkStateReader
[junit4] > at
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at
org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:328)
[junit4] > at
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
[junit4] > at
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
[junit4] > at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
[junit4] > at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at
org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:936)
[junit4] > at
org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:899)
[junit4] > at
org.apache.solr.metrics.rrd.SolrRrdBackendFactory.remove(SolrRrdBackendFactory.java:320)
[junit4] > at
org.apache.solr.handler.admin.MetricsHistoryHandler.removeHistory(MetricsHistoryHandler.java:302)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$1(CoreAdminOperation.java:121)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
[junit4] > at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] > at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] >
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException:
org.apache.solr.common.cloud.SolrZkClient
[junit4] > at
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:203)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:126)
[junit4] > at
org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:116)
[junit4] > at
org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:306)
[junit4] > at
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:160)
[junit4] > at
org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:399)
[junit4] > at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
[junit4] > at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
[junit4] > at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at
org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:936)
[junit4] > at
org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:899)
[junit4] > at
org.apache.solr.metrics.rrd.SolrRrdBackendFactory.remove(SolrRrdBackendFactory.java:320)
[junit4] > at
org.apache.solr.handler.admin.MetricsHistoryHandler.removeHistory(MetricsHistoryHandler.java:302)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$1(CoreAdminOperation.java:121)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
[junit4] > at
org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
[junit4] > at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] > at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at
__randomizedtesting.SeedInfo.seed([8F97179111B556A4]:0)
[junit4] > at
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:317)
[junit4] > at java.lang.Thread.run(Thread.java:748)Throwable #2:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE
scope at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest:
[junit4] > 1) Thread[id=23113, name=zkCallback-5763-thread-1,
state=TIMED_WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=23071,
name=parallelCoreAdminExecutor-6574-thread-1-processing-n:127.0.0.1:46027_solr
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1
AutoscalingHistoryHandlerTest_node_added_trigger//354f56a1a5c62T97h2je8841fj16tp15d7ws8zi//0937839259663825
UNLOAD-EventThread, state=WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4] > 3) Thread[id=23070,
name=parallelCoreAdminExecutor-6574-thread-1-processing-n:127.0.0.1:46027_solr
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1
AutoscalingHistoryHandlerTest_node_added_trigger//354f56a1a5c62T97h2je8841fj16tp15d7ws8zi//0937839259663825
UNLOAD-SendThread(127.0.0.1:40291), state=TIMED_WAITING,
group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > 4) Thread[id=23174, name=zkCallback-5763-thread-2,
state=TIMED_WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=23175, name=zkCallback-5763-thread-3,
state=TIMED_WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 6) Thread[id=23072,
name=zkConnectionManagerCallback-5764-thread-1, state=WAITING,
group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at
__randomizedtesting.SeedInfo.seed([8F97179111B556A4]:0)Throwable #3:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
[junit4] > 1) Thread[id=23113, name=zkCallback-5763-thread-1,
state=TIMED_WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=23070,
name=parallelCoreAdminExecutor-6574-thread-1-processing-n:127.0.0.1:46027_solr
x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1
AutoscalingHistoryHandlerTest_node_added_trigger//354f56a1a5c62T97h2je8841fj16tp15d7ws8zi//0937839259663825
UNLOAD-SendThread(127.0.0.1:40291), state=TIMED_WAITING,
group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > 3) Thread[id=23174, name=zkCallback-5763-thread-2,
state=TIMED_WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 4) Thread[id=23175, name=zkCallback-5763-thread-3,
state=TIMED_WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=23072,
name=zkConnectionManagerCallback-5764-thread-1, state=WAITING,
group=TGRP-AutoscalingHistoryHandlerTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at
__randomizedtesting.SeedInfo.seed([8F97179111B556A4]:0)
{code}
Reference: risdenk/nuc#182-186
> ant test (without badapples=false) should pass easily for developers.
> ---------------------------------------------------------------------
>
> Key: SOLR-12932
> URL: https://issues.apache.org/jira/browse/SOLR-12932
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
> Reporter: Mark Miller
> Assignee: Mark Miller
> Priority: Major
>
> If we fix the tests we will end up here anyway, but we can shortcut this.
> Once I get my first patch in, anyone who mentions a test that fails locally
> for them at any time (not jenkins), I will fix it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
