[JENKINS] Lucene-Solr-8.5-MacOSX (64bit/jdk1.8.0) - Build # 9 - Still Unstable!

Policeman Jenkins Server Mon, 09 Mar 2020 23:17:24 -0700

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.5-MacOSX/9/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC


2 tests failed.
FAILED:  org.apache.solr.cloud.SystemCollectionCompatTest.testBackCompat

Error Message:
re-indexing warning not found

Stack Trace:
java.lang.AssertionError: re-indexing warning not found
        at 
__randomizedtesting.SeedInfo.seed([5C993D44F92559DC:2C6C9EED99EDF0AA]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at 
org.apache.solr.cloud.SystemCollectionCompatTest.testBackCompat(SystemCollectionCompatTest.java:215)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore

Error Message:
ObjectTracker found 7 object(s) that were not released!!! [InternalHttpClient, 
MMapDirectory, MMapDirectory, InternalHttpClient, SolrCore, MMapDirectory, 
MMapDirectory] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
  at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)  
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)  at 
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1241) 
 at 
org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
  at 
org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1320)
  at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:694)
  at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)  
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)  
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) 
 at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424) 
 at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1210)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)  at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
  at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:388)  at 
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:765)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
  at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)  
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)  at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420) 
 at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:251) 
 at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:663)
  at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)  
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.core.SolrCore  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1078)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at 
org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:536)  
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:981)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)  at 
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:797)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)   expected null, but 
was:<ObjectTracker found 7 object(s) that were not released!!! 
[InternalHttpClient, MMapDirectory, MMapDirectory, InternalHttpClient, 
SolrCore, MMapDirectory, MMapDirectory] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
  at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)  
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)  at 
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1241) 
 at 
org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
  at 
org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1320)
  at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:694)
  at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)  
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)  
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) 
 at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424) 
 at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1210)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)  at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
  at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:388)  at 
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:765)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
  at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)  
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)  at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420) 
 at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:251) 
 at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:663)
  at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)  
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.core.SolrCore  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1078)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at 
org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:536)  
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:981)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)  at 
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:797)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)  at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) 
 at org.eclipse.jetty.server.Server.handle(Server.java:500)  at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at 
org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at 
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  at java.lang.Thread.run(Thread.java:748)  >

Stack Trace:
java.lang.AssertionError: ObjectTracker found 7 object(s) that were not 
released!!! [InternalHttpClient, MMapDirectory, MMapDirectory, 
InternalHttpClient, SolrCore, MMapDirectory, MMapDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
        at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
        at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
        at 
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1241)
        at 
org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
        at 
org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1320)
        at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:694)
        at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)
        at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)
        at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)
        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
        at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1210)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:388)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:765)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
        at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
        at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420)
        at 
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:251)
        at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:663)
        at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)
        at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)
        at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.core.SolrCore
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1078)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at 
org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:536)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:981)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at 
org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:797)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

 expected null, but was:<ObjectTracker found 7 object(s) that were not 
released!!! [InternalHttpClient, MMapDirectory, MMapDirectory, 
InternalHttpClient, SolrCore, MMapDirectory, MMapDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
        at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
        at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
        at 
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1241)
        at 
org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
        at 
org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1320)
        at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:694)
        at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)
        at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)
        at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)
        at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
        at 
org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1210)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:388)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:765)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
        at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
        at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
        at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
        at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420)
        at 
org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:251)
        at 
org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:663)
        at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:337)
        at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:318)
        at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.core.SolrCore
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1078)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at 
org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:536)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:981)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.MMapDirectory
        at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
        at 
org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
        at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:797)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:989)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
        at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1306)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)
        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:808)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:420)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:352)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1607)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1297)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1577)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1212)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:500)
        at 
org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.lang.Thread.run(Thread.java:748)

>
        at __randomizedtesting.SeedInfo.seed([5C993D44F92559DC]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotNull(Assert.java:755)
        at org.junit.Assert.assertNull(Assert.java:737)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:334)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:901)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13388 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
   [junit4]   2> 71957 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to 
test-framework derived value of 
'/Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 71958 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.SolrTestCaseJ4 Created dataDir: 
/Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_5C993D44F92559DC-001/data-dir-5-001
   [junit4]   2> 71958 WARN  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=32 numCloses=32
   [junit4]   2> 71959 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 71966 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) 
w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 71966 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 80033 WARN  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 80171 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 
363d5f2df3a8a28de40604320230664b9c793c16; jvm 1.8.0_201-b09
   [junit4]   2> 80178 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 80178 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 80178 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 80185 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@41cd223d{static,/static,jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
   [junit4]   2> 80619 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.w.WebAppContext@6f27edb3{hdfs,/,file:///Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX/solr/build/solr-core/test/J2/temp/jetty-localhost-61480-hadoop-hdfs-3_2_0-tests_jar-_-any-4334739255663030498.dir/webapp/,AVAILABLE}{jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/hdfs}
   [junit4]   2> 80621 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@49f7dac7{HTTP/1.1,[http/1.1]}{localhost:61480}
   [junit4]   2> 80621 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.Server Started @80717ms
   [junit4]   2> 84798 WARN  (StorageLocationChecker thread 0) [     ] 
o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your 
platform... using builtin-java classes where applicable
   [junit4]   2> 85424 WARN  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 85466 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 
363d5f2df3a8a28de40604320230664b9c793c16; jvm 1.8.0_201-b09
   [junit4]   2> 85470 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 85471 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 85471 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 85503 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1fff7d59{static,/static,jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
   [junit4]   2> 85729 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.w.WebAppContext@5ba52931{datanode,/,file:///Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX/solr/build/solr-core/test/J2/temp/jetty-localhost-61591-hadoop-hdfs-3_2_0-tests_jar-_-any-2353919956224314969.dir/webapp/,AVAILABLE}{jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/datanode}
   [junit4]   2> 85730 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@24a6630a{HTTP/1.1,[http/1.1]}{localhost:61591}
   [junit4]   2> 85730 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.Server Started @85826ms
   [junit4]   2> 88396 WARN  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 88412 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 
363d5f2df3a8a28de40604320230664b9c793c16; jvm 1.8.0_201-b09
   [junit4]   2> 88427 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 88428 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 88428 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 88430 INFO  
(SUITE-TestHdfsCloudBackupRestore-seed#[5C993D44F92559DC]-worker) [     ] 
o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@fb

[...truncated too long message...]

ccurred - updating... (live nodes size: [1])
   [junit4]   2> 234322 INFO  (qtp1643566546-53) [n:127.0.0.1:58671_solr     ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={name=.system&action=DELETE&wt=javabin&version=2} status=0 QTime=996
   [junit4]   2> 234453 INFO  
(TEST-SystemCollectionCompatTest.testBackCompat-seed#[5C993D44F92559DC]) [     
] o.a.s.SolrTestCaseJ4 ###Ending testBackCompat
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=SystemCollectionCompatTest -Dtests.method=testBackCompat 
-Dtests.seed=5C993D44F92559DC -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=th-TH-u-nu-thai-x-lvariant-TH -Dtests.timezone=America/Detroit 
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE  213s J0 | SystemCollectionCompatTest.testBackCompat <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: re-indexing warning 
not found
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([5C993D44F92559DC:2C6C9EED99EDF0AA]:0)
   [junit4]    >        at 
org.apache.solr.cloud.SystemCollectionCompatTest.testBackCompat(SystemCollectionCompatTest.java:215)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 234590 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.c.CoreContainer Shutting down CoreContainer instance=1615327768
   [junit4]   2> 234590 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.c.ZkController Remove node as live in 
ZooKeeper:/live_nodes/127.0.0.1:58671_solr
   [junit4]   2> 234593 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 234593 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.c.ZkController Publish node=127.0.0.1:58671_solr as DOWN
   [junit4]   2> 234778 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, 
tag=null
   [junit4]   2> 234780 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@30bb8355: rootName = 
solr_58671, domain = solr.node, service url = null, agent id = null] for 
registry solr.node / com.codahale.metrics.MetricRegistry@90f6e4f
   [junit4]   2> 234786 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, 
tag=null
   [junit4]   2> 234787 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@47d885e3: rootName = 
solr_58671, domain = solr.jvm, service url = null, agent id = null] for 
registry solr.jvm / com.codahale.metrics.MetricRegistry@7cd75048
   [junit4]   2> 234789 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, 
tag=null
   [junit4]   2> 234789 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.m.r.SolrJmxReporter Closing reporter 
[org.apache.solr.metrics.reporters.SolrJmxReporter@5dcf7174: rootName = 
solr_58671, domain = solr.jetty, service url = null, agent id = null] for 
registry solr.jetty / com.codahale.metrics.MetricRegistry@34866a18
   [junit4]   2> 234789 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, 
tag=null
   [junit4]   2> 234792 INFO  (closeThreadPool-44258-thread-2) [     ] 
o.a.s.c.Overseer Overseer 
(id=72110353700290568-127.0.0.1:58671_solr-n_0000000001) closing
   [junit4]   2> 234806 INFO  
(OverseerStateUpdate-72110353700290568-127.0.0.1:58671_solr-n_0000000001) 
[n:127.0.0.1:58671_solr     ] o.a.s.c.Overseer Overseer Loop exiting : 
127.0.0.1:58671_solr
   [junit4]   2> 234810 INFO  
(OverseerAutoScalingTriggerThread-72110353700290568-127.0.0.1:58671_solr-n_0000000001)
 [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we 
are closed, exiting.
   [junit4]   2> 234883 INFO  (closeThreadPool-44258-thread-1) [     ] 
o.a.s.c.Overseer Overseer 
(id=72110353700290568-127.0.0.1:58671_solr-n_0000000001) closing
   [junit4]   2> 234996 INFO  (jetty-closer-44254-thread-1) [     ] 
o.a.s.c.Overseer Overseer 
(id=72110353700290568-127.0.0.1:58671_solr-n_0000000001) closing
   [junit4]   2> 235078 INFO  (jetty-closer-44254-thread-1) [     ] 
o.e.j.s.AbstractConnector Stopped ServerConnector@3b2f032{HTTP/1.1,[http/1.1, 
h2c]}{127.0.0.1:?}
   [junit4]   2> 235098 INFO  (jetty-closer-44254-thread-1) [     ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@1139f00c{/solr,null,UNAVAILABLE}
   [junit4]   2> 235098 INFO  (jetty-closer-44254-thread-1) [     ] 
o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 235108 INFO  
(SUITE-SystemCollectionCompatTest-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 235361 WARN  (ZkTestServer Run Thread) [     ] 
o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        9       /solr/configs/.system/managed-schema
   [junit4]   2>        9       /solr/collections/.system/terms/shard1
   [junit4]   2>        8       /solr/clusterprops.json
   [junit4]   2>        8       /solr/aliases.json
   [junit4]   2>        6       /solr/configs/.system
   [junit4]   2>        5       /solr/collections/.system/collectionprops.json
   [junit4]   2>        3       /solr/collections/.system/state.json
   [junit4]   2>        2       /solr/packages.json
   [junit4]   2>        2       /solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        25      /solr/collections/.system/state.json
   [junit4]   2>        8       /solr/clusterstate.json
   [junit4]   2>        5       /solr/aliases.json
   [junit4]   2>        3       /solr/configs/.system/managed-schema
   [junit4]   2>        2       /solr/autoscaling.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        27      /solr/overseer/queue
   [junit4]   2>        19      /solr/overseer/collection-queue-work
   [junit4]   2>        16      /solr/collections
   [junit4]   2>        15      /solr/live_nodes
   [junit4]   2>        2       /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2>        2       /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2>        2       /solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> 235372 INFO  
(SUITE-SystemCollectionCompatTest-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:58645
   [junit4]   2> 235372 INFO  
(SUITE-SystemCollectionCompatTest-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:58645
   [junit4]   2> 235372 INFO  
(SUITE-SystemCollectionCompatTest-seed#[5C993D44F92559DC]-worker) [     ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1 58645
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.SystemCollectionCompatTest_5C993D44F92559DC-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene84): 
{id=PostingsFormat(name=MockRandom), type=BlockTreeOrds(blocksize=128)}, 
docValues:{timestamp_l=DocValuesFormat(name=Asserting), 
_version_=DocValuesFormat(name=Lucene80), 
time_l=DocValuesFormat(name=Lucene80), timestamp=DocValuesFormat(name=Direct)}, 
maxPointsInLeafNode=628, maxMBSortInHeap=5.106123100828586, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@299cea06),
 locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=America/Detroit
   [junit4]   2> NOTE: Mac OS X 10.14.6 x86_64/Oracle Corporation 1.8.0_201 
(64-bit)/cpus=6,threads=2,free=59096208,total=201326592
   [junit4]   2> NOTE: All tests run in this JVM: [SystemCollectionCompatTest]
   [junit4] Completed [10/10 (3!)] on J0 in 234.70s, 1 test, 1 failure <<< 
FAILURES!

[...truncated 16 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX/lucene/common-build.xml:1594: 
The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX/lucene/common-build.xml:1121: 
There were test failures: 10 suites, 15 tests, 3 failures, 10 ignored [seed: 
5C993D44F92559DC]

Total time: 5 minutes 4 seconds

[repro] Setting last failure code to 256

[repro] Failures w/original seeds:
[repro]   0/5 failed: 
org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
[repro]   3/5 failed: org.apache.solr.cloud.SystemCollectionCompatTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
No credentials specified
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all 
warnings.
<Git Blamer> GIT_COMMIT=0ae8db0aa0d21508adce12c9926269a69c089520, 
workspace=/Users/jenkins/workspace/Lucene-Solr-8.5-MacOSX
[WARNINGS] Computing warning deltas based on reference build #8
Recording test results
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting 
ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

[JENKINS] Lucene-Solr-8.5-MacOSX (64bit/jdk1.8.0) - Build # 9 - Still Unstable!

Reply via email to