[ 
https://issues.apache.org/jira/browse/SOLR-8280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-8280:
---------------------------
    Attachment: SOLR-8280.patch


With the attached patch, it's trivial to trigger a suite failure in 
TestCloudSchemaless, although lots of other cloud related tests (using 
schema.xml via AbstractFullDistribZkTestBase) function just fine.

(NOTE: there are in fact a handful of other test failures with this patch, but 
they exist due to specific Sim related assumptions in those tests vs the 
general change of schema.xml to using SchemaSimilarityFactory ... i included 
the change to schema.xml in this patch purely as an easy way to demonstrate 
that most cloud related tests -- even ones using schemaless and/or involving 
core reloads -- don't ffreak out the same way as TestCloudSchemaless)

----

Quick walk through of the TestCloudSchemaless failure...

{noformat}
ant test  -Dtestcase=TestCloudSchemaless -Dtests.seed=8F1B744FBE000231 
-Dtests.slow=true
{noformat}

The test itself runs fine, and reaches the {{SolrTestCaseJ4 ###Ending test" 
stage}} at which point shutdown begins -- and eventually stalls in the dir 
factory waiting for directories to close...

{noformat}
   [junit4]   2> 52621 INFO  
(TEST-TestCloudSchemaless.test-seed#[E3CE46BC04E096EF]) [n:127.0.0.1:35554_ 
c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Closing main searcher on request.
   [junit4]   2> 52621 INFO  
(TEST-TestCloudSchemaless.test-seed#[E3CE46BC04E096EF]) [n:127.0.0.1:35554_ 
c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory Closing NRTCachingDirectoryFactory - 2 
directories currently being tracked
   [junit4]   2> 64733 ERROR 
(TEST-TestCloudSchemaless.test-seed#[E3CE46BC04E096EF]) [n:127.0.0.1:35554_ 
c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory Timeout waiting for all directory ref counts to 
be released - gave up waiting on 
CachedDir<<refCount=1;path=/home/hossman/lucene/dev/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_E3CE46BC04E096EF-001/shard-1-001/cores/collection1/data/index;done=false>>
   [junit4]   2> 64733 ERROR 
(TEST-TestCloudSchemaless.test-seed#[E3CE46BC04E096EF]) [n:127.0.0.1:35554_ 
c:collection1 s:shard2 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory Error closing 
directory:org.apache.solr.common.SolrException: Timeout waiting for all 
directory ref counts to be released - gave up waiting on 
CachedDir<<refCount=1;path=/home/hossman/lucene/dev/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_E3CE46BC04E096EF-001/shard-1-001/cores/collection1/data/index;done=false>>
   [junit4]   2>        at 
org.apache.solr.core.CachingDirectoryFactory.close(CachingDirectoryFactory.java:187)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.close(SolrCore.java:1268)
   [junit4]   2>        at 
org.apache.solr.core.SolrCores.close(SolrCores.java:124)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:570)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:172)
   [junit4]   2>        at 
org.apache.solr.cloud.ChaosMonkey.stopJettySolrRunner(ChaosMonkey.java:186)
   [junit4]   2>        at 
org.apache.solr.cloud.ChaosMonkey.stop(ChaosMonkey.java:518)
   [junit4]   2>        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.destroyServers(AbstractFullDistribZkTestBase.java:1514)
   [junit4]   2>        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
   [junit4]   2>        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 64733 ERROR 
(TEST-TestCloudSchemaless.test-seed#[E3CE46BC04E096EF]) [n:127.0.0.1:35554_ 
c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore 
java.lang.AssertionError: 1
   [junit4]   2>        at 
org.apache.solr.core.CachingDirectoryFactory.close(CachingDirectoryFactory.java:201)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.close(SolrCore.java:1268)
   [junit4]   2>        at 
org.apache.solr.core.SolrCores.close(SolrCores.java:124)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:570)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:172)
   [junit4]   2>        at 
org.apache.solr.cloud.ChaosMonkey.stopJettySolrRunner(ChaosMonkey.java:186)
   [junit4]   2>        at 
org.apache.solr.cloud.ChaosMonkey.stop(ChaosMonkey.java:518)
   [junit4]   2>        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.destroyServers(AbstractFullDistribZkTestBase.java:1514)
   [junit4]   2>        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:964)
   [junit4]   2>        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
{noformat}


...at that point, things go from bad to worse, as RecoveryThreads kick in (why 
is the collection attempting to recover it it was deliberaly being shutdown???) 
but of course that doesn't turn out well because the CoreContainer's are all in 
shutdown mode...

{noformat}
   [junit4]   2> 69727 INFO  (qtp1007998027-153) [n:127.0.0.1:44195_    ] 
o.a.s.h.a.CoreAdminHandler Going to wait for coreNodeName: core_node8, state: 
recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 69727 INFO  
(zkCallback-44-thread-2-processing-n:127.0.0.1:42678_) [n:127.0.0.1:42678_    ] 
o.a.s.c.c.ZkStateReader Updating data for collection1 from 29 to 30 
   [junit4]   2> 69728 INFO  
(zkCallback-39-thread-2-processing-n:127.0.0.1:52349_) [n:127.0.0.1:52349_    ] 
o.a.s.c.c.ZkStateReader Updating data for collection1 from 29 to 30 
   [junit4]   2> 69728 INFO  (qtp1007998027-153) [n:127.0.0.1:44195_    ] 
o.a.s.h.a.CoreAdminHandler Will wait a max of 183 seconds to see collection1 
(shard1 of collection1) have state: recovering
   [junit4]   2> 69728 INFO  (qtp1007998027-153) [n:127.0.0.1:44195_    ] 
o.a.s.h.a.CoreAdminHandler In WaitForState(recovering): collection=collection1, 
shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? 
true, live=true, checkLive=true, currentState=recovering, localState=active, 
nodeName=127.0.0.1:42678_, coreNodeName=core_node8, 
onlyIfActiveCheckResult=false, nodeProps: 
core_node8:{"core":"collection1","base_url":"http://127.0.0.1:42678","node_name":"127.0.0.1:42678_","state":"recovering"}
   [junit4]   2> 69729 INFO  (qtp1007998027-153) [n:127.0.0.1:44195_    ] 
o.a.s.h.a.CoreAdminHandler Waited coreNodeName: core_node8, state: recovering, 
checkLive: true, onlyIfLeader: true for: 0 seconds.
   [junit4]   2> 69729 INFO  (qtp1007998027-153) [n:127.0.0.1:44195_    ] 
o.a.s.s.SolrDispatchFilter [admin] webapp=null path=/admin/cores 
params={nodeName=127.0.0.1:42678_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node8&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
 status=0 QTime=1 
   [junit4] HEARTBEAT J0 PID(17711@tray): 2015-11-11T16:22:31, stalled for 
70.2s at: TestCloudSchemaless.test
   [junit4]   2> 72390 INFO  (RecoveryThread-collection1) [n:127.0.0.1:44717_ 
c:collection1 s:shard3 r:core_node6 x:collection1] o.a.s.c.RecoveryStrategy 
Starting Replication Recovery.
   [junit4]   2> 72390 INFO  (RecoveryThread-collection1) [n:127.0.0.1:44717_ 
c:collection1 s:shard3 r:core_node6 x:collection1] o.a.s.c.RecoveryStrategy 
Attempting to replicate from http://127.0.0.1:54475/collection1/.
   [junit4]   2> 72391 ERROR (qtp1661135858-94) [n:127.0.0.1:54475_    ] 
o.a.s.s.SolrDispatchFilter Error processing the request. CoreContainer is 
either not initialized or shutting down.
   [junit4]   2> 72392 WARN  (qtp1661135858-94) [n:127.0.0.1:54475_    ] 
o.e.j.s.ServletHandler /collection1/update
   [junit4]   2> org.apache.solr.common.SolrException: Error processing the 
request. CoreContainer is either not initialized or shutting down.
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:187)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]   2>        at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
   [junit4]   2>        at 
org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:499)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
{noformat}

...and then things *really* get weird, as the RecoveryThread starts trying to 
do replication recovery, and then nodes start complaining that they can't open 
a new searcher because SolrCoreAware.inform() was never called on the 
SimilarityFactory (an assert that exists inside 
SchemaSimilarityFactory.getSimilarity()) ...

{noformat}
   [junit4]   2> 74329 INFO  (RecoveryThread-collection1) [n:127.0.0.1:52349_ 
c:collection1 s:shard4 r:core_node7 x:collection1] o.a.s.c.RecoveryStrategy 
Attempting to replicate from http://127.0.0.1:54695/collection1/.
   [junit4]   2> 74331 INFO  (qtp1972553455-125) [n:127.0.0.1:54695_ 
c:collection1 s:shard4 r:core_node3 x:collection1] o.a.s.u.UpdateHandler start 
commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 74359 INFO  (qtp1972553455-125) [n:127.0.0.1:54695_ 
c:collection1 s:shard4 r:core_node3 x:collection1] o.a.s.c.SolrCore 
SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2>        
commit{dir=NRTCachingDirectory(MMapDirectory@/home/hossman/lucene/dev/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_E3CE46BC04E096EF-001/shard-3-001/cores/collection1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5720ab9c; 
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
   [junit4]   2>        
commit{dir=NRTCachingDirectory(MMapDirectory@/home/hossman/lucene/dev/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_E3CE46BC04E096EF-001/shard-3-001/cores/collection1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5720ab9c; 
maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
   [junit4]   2> 74359 INFO  (qtp1972553455-125) [n:127.0.0.1:54695_ 
c:collection1 s:shard4 r:core_node3 x:collection1] o.a.s.c.SolrCore newest 
commit generation = 5
   [junit4]   2> 74361 INFO  (qtp1972553455-125) [n:127.0.0.1:54695_ 
c:collection1 s:shard4 r:core_node3 x:collection1] o.a.s.s.SolrIndexSearcher 
Opening Searcher@5428727f[collection1] realtime
   [junit4]   2> 74361 INFO  (qtp1972553455-125) [n:127.0.0.1:54695_ 
c:collection1 s:shard4 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessor 
[collection1] webapp= path=/update 
params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}
 {} 0 30
   [junit4]   2> 74362 ERROR (qtp1972553455-125) [n:127.0.0.1:54695_ 
c:collection1 s:shard4 r:core_node3 x:collection1] o.a.s.s.SolrDispatchFilter 
null:java.lang.RuntimeException: java.lang.AssertionError: inform must be 
called first
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:616)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:475)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]   2>        at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
   [junit4]   2>        at 
org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:499)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.AssertionError: inform must be called 
first
   [junit4]   2>        at 
org.apache.solr.search.similarities.SchemaSimilarityFactory.getSimilarity(SchemaSimilarityFactory.java:86)
   [junit4]   2>        at 
org.apache.solr.schema.IndexSchema.getSimilarity(IndexSchema.java:269)
   [junit4]   2>        at 
org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:255)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1628)
   [junit4]   2>        at 
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:635)
   [junit4]   2>        at 
org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2>        at 
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2>        at 
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2>        at 
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2>        at 
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2>        at 
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2>        at 
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2>        at 
org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2>        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1616)
   [junit4]   2>        at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1593)
   [junit4]   2>        at 
org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:161)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
   [junit4]   2>        at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:667)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
   [junit4]   2>        ... 23 more
{noformat}

...and then eventually the suite times out.

----

There are lots of questions here, notably:

# what makes this tests special that it fails so spectaculaly like this?
#* in particular why does this SimilarityFactory change cause the directory ref 
counting to fail?
# why is recovery happening after shutdown?
# regardless of how/why the other things in the test go so bizare in the first 
place, what code path is leading to the SchemaSimilarityFactory that hasn't had 
{{.inform(SolrCore)}} called on it?


> TestCloudSchemaless fails weirdly if you try to use SolrCoreAware sim 
> factory: SchemaSimilarityFactory 
> -------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-8280
>                 URL: https://issues.apache.org/jira/browse/SOLR-8280
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: Trunk
>            Reporter: Hoss Man
>         Attachments: SOLR-8280.patch
>
>
> Something about the code path(s) involved in TestCloudSchemaless doesn't play 
> nicely with a SimilarityFactory that is SolrCoreAware -- notably: 
> SchemaSimilarityFactory.
> I discovered this while trying to implement SOLR-8271, but it can be 
> reproduced trivially by modifying the 
> schema-add-schema-fields-update-processor.xml file used by 
> TestCloudSchemaless to refer to SchemaSimilarityFactory explicitly.  Other 
> cloud tests (such as CollectionReloadTest) or cloud+schemaless (ex: 
> TestCloudManagedSchema) tests don't seem to demonstrate the same problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to