[ https://issues.apache.org/jira/browse/SOLR-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joachim Sauer updated SOLR-13291: --------------------------------- Description: We have a weird workload that at some times involves deletion and re-creation of collections with the same name in a short period of time (don't ask why). When running in a SolrCloud cluster this will occasionally leave a random core lying around and locked even though the Collection deletion was reported to have finished successfully. This results in an error the next time a collection of that given name should be created. The attached shell script is consistently able to reproduce the error states within a small number of iterations against the 7.7.1 binary distribution running the default cloud example (`solr start -e cloud`, accept all default values). Log entries that seemed relevant to me are: At the time when the collection is deleted: {code} 2019-03-04 16:56:44.037 WARN (Thread-24) [c:myCollection s:shard2 r:core_node4 x:myCollection_shard2_replica_n2] o.a.s.c.ZkController listener throws error org.apache.solr.common.SolrException: Unable to reload core [myCollection_shard2_replica_n2] at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1463) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3041) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$21(ZkController.java:2803) [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at java.lang.Thread.run(Thread.java:834) [?:?] Caused by: org.apache.solr.common.SolrException at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1048) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1439) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] ... 3 more Caused by: java.lang.NullPointerException at org.apache.solr.metrics.SolrMetricManager.loadShardReporters(SolrMetricManager.java:1160) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.metrics.SolrCoreMetricManager.loadReporters(SolrCoreMetricManager.java:92) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.core.SolrCore.<init>(SolrCore.java:920) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1439) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] {code} Later, when trying to re-create the collection: {code} 2019-03-04 16:56:51.982 ERROR (OverseerThreadFactory-9-thread-5-processing-n:127.0.1.1:8983_solr) [ ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error from shard: http://127.0.1.1:8983/solr org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.1.1:8983/solr: Error CREATEing SolrCore 'myCollection_shard2_replica_n2': Unable to create core [myCollection_shard2_replica_n2 ] Caused by: Lock held by this virtual machine: /home/joachim/workspaces/devtools/solr-7.7.1/example/cloud/node1/solr/myCollection_shard2_replica_n2/data/index/write.lock at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09] at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09] at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09] at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1260) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09] at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:173) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6] at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] {code} was: We have a weird workload that at some times involves deletion and re-creation of collections with the same name in a short period of time (don't ask why). When running in a SolrCloud cluster this will occasionally leave a random core lying around and locked even though the Collection deletion was reported to have finished successfully. This results in an error the next time a collection of that given name should be created. The attached shell script is consistently able to reproduce the error states within a small number of iterations against the 7.7.1 binary distribution running the default cloud example (`solr start -e cloud`, accept all default values). Log entries that seemed relevant to me are: At the time when the collection is deleted: {{2019-03-04 16:56:44.037 ERROR (Thread-24) [c:myCollection s:shard2 r:core_node4 x:myCollection_shard2_replica_n2] o.a.s.c.SolrCore Error while closing}} {{java.lang.NullPointerException: null}} {{ at org.apache.solr.core.SolrCore.close(SolrCore.java:1639) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1040) [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1439) [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3041) [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$21(ZkController.java:2803) [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at java.lang.Thread.run(Thread.java:834) [?:?]}} {{2019-03-04 16:56:44.037 WARN (Thread-24) [c:myCollection s:shard2 r:core_node4 x:myCollection_shard2_replica_n2] o.a.s.c.ZkController listener throws error}} {{org.apache.solr.common.SolrException: Unable to reload core [myCollection_shard2_replica_n2]}} {{ at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1463) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3041) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$21(ZkController.java:2803) [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at java.lang.Thread.run(Thread.java:834) [?:?]}} {{Caused by: org.apache.solr.common.SolrException}} {{ at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1048) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1439) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ ... 3 more}} {{Caused by: java.lang.NullPointerException}} {{ at org.apache.solr.metrics.SolrMetricManager.loadShardReporters(SolrMetricManager.java:1160) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.metrics.SolrCoreMetricManager.loadReporters(SolrCoreMetricManager.java:92) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.SolrCore.<init>(SolrCore.java:920) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1439) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ ... 3 more}} Later, when trying to re-create the collection: {{2019-03-04 16:56:51.982 ERROR (OverseerThreadFactory-9-thread-5-processing-n:127.0.1.1:8983_solr) [ ] o.a.s.c.a.c.OverseerCollectionMessageHandler Error from shard: http://127.0.1.1:8983/solr}} {{org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.1.1:8983/solr: Error CREATEing SolrCore 'myCollection_shard2_replica_n2': Unable to create core [myCollection_shard2_replica_n2] Caused by: Lock held by this virtual machine: /.../solr-7.7.1/example/cloud/node1/solr/myCollection_shard2_replica_n2/data/index/write.lock}} {{ at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09]}} {{ at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09]}} {{ at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09]}} {{ at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1260) ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09]}} {{ at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:173) ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:07]}} {{ at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]}} {{ at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]}} {{ at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]}} {{ at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) ~[metrics-core-3.2.6.jar:3.2.6]}} {{ at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) [solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan - 2019-02-23 02:39:09]}} {{ at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]}} {{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]}} {{ at java.lang.Thread.run(Thread.java:834) [?:?]}} Summary: Failed to create collection due to lock held by this virtual machine (was: Failed to create collection due to ) > Failed to create collection due to lock held by this virtual machine > -------------------------------------------------------------------- > > Key: SOLR-13291 > URL: https://issues.apache.org/jira/browse/SOLR-13291 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud > Affects Versions: 7.5, 7.7 > Environment: * Solr 7.7.1 (also reproduced on 7.5) > * running on Ubuntu 18.04 (also reproduced on AWS instances using Amazon > Linux) > * Using OpenJDK 11.0.1 as distributedby AdoptOpenJDK > * setting up a solr example cloud using `solr start -e cloud` and accepting > all default values (2 cluster nodes) > Reporter: Joachim Sauer > Priority: Major > Attachments: tortureSolr.sh > > > We have a weird workload that at some times involves deletion and re-creation > of collections with the same name in a short period of time (don't ask why). > > When running in a SolrCloud cluster this will occasionally leave a random > core lying around and locked even though the Collection deletion was reported > to have finished successfully. > > This results in an error the next time a collection of that given name should > be created. > > The attached shell script is consistently able to reproduce the error states > within a small number of iterations against the 7.7.1 binary distribution > running the default cloud example (`solr start -e cloud`, accept all default > values). > > Log entries that seemed relevant to me are: > At the time when the collection is deleted: > {code} > 2019-03-04 16:56:44.037 WARN (Thread-24) [c:myCollection s:shard2 > r:core_node4 x:myCollection_shard2_replica_n2] o.a.s.c.ZkController listener > throws error > org.apache.solr.common.SolrException: Unable to reload core > [myCollection_shard2_replica_n2] > at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1463) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at > org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3041) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at > org.apache.solr.cloud.ZkController.lambda$fireEventListeners$21(ZkController.java:2803) > [solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at java.lang.Thread.run(Thread.java:834) [?:?] > Caused by: org.apache.solr.common.SolrException > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1048) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1439) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > ... 3 more > Caused by: java.lang.NullPointerException > at > org.apache.solr.metrics.SolrMetricManager.loadShardReporters(SolrMetricManager.java:1160) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at > org.apache.solr.metrics.SolrCoreMetricManager.loadReporters(SolrCoreMetricManager.java:92) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:920) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at org.apache.solr.core.SolrCore.reload(SolrCore.java:666) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1439) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > {code} > > Later, when trying to re-create the collection: > > {code} > 2019-03-04 16:56:51.982 ERROR > (OverseerThreadFactory-9-thread-5-processing-n:127.0.1.1:8983_solr) [ ] > o.a.s.c.a.c.OverseerCollectionMessageHandler Error from shard: > http://127.0.1.1:8983/solr > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at http://127.0.1.1:8983/solr: Error CREATEing SolrCore > 'myCollection_shard2_replica_n2': Unable to create core > [myCollection_shard2_replica_n2 > ] Caused by: Lock held by this virtual machine: > /home/joachim/workspaces/devtools/solr-7.7.1/example/cloud/node1/solr/myCollection_shard2_replica_n2/data/index/write.lock > at > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643) > ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - > ishan - 2019-02-23 02:39:09] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) > ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - > ishan - 2019-02-23 02:39:09] > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) > ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - > ishan - 2019-02-23 02:39:09] > at > org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1260) > ~[solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:09] > at > org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:173) > ~[solr-core-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:07] > at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] > at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] > at > com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) > ~[metrics-core-3.2.6.jar:3.2.6] > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) > [solr-solrj-7.7.1.jar:7.7.1 5bf96d32f88eb8a2f5e775339885cd6ba84a3b58 - ishan > - 2019-02-23 02:39:09] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > [?:?] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > [?:?] > at java.lang.Thread.run(Thread.java:834) [?:?] > {code} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org