[jira] [Updated] (LUCENE-7976) Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of very large segments

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson updated LUCENE-7976:
---
Attachment: LUCENE-7976.patch

> Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of 
> very large segments
> -
>
> Key: LUCENE-7976
> URL: https://issues.apache.org/jira/browse/LUCENE-7976
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, 
> LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch
>
>
> We're seeing situations "in the wild" where there are very large indexes (on 
> disk) handled quite easily in a single Lucene index. This is particularly 
> true as features like docValues move data into MMapDirectory space. The 
> current TMP algorithm allows on the order of 50% deleted documents as per a 
> dev list conversation with Mike McCandless (and his blog here:  
> https://www.elastic.co/blog/lucenes-handling-of-deleted-documents).
> Especially in the current era of very large indexes in aggregate, (think many 
> TB) solutions like "you need to distribute your collection over more shards" 
> become very costly. Additionally, the tempting "optimize" button exacerbates 
> the issue since once you form, say, a 100G segment (by 
> optimizing/forceMerging) it is not eligible for merging until 97.5G of the 
> docs in it are deleted (current default 5G max segment size).
> The proposal here would be to add a new parameter to TMP, something like 
>  (no, that's not serious name, suggestions 
> welcome) which would default to 100 (or the same behavior we have now).
> So if I set this parameter to, say, 20%, and the max segment size stays at 
> 5G, the following would happen when segments were selected for merging:
> > any segment with > 20% deleted documents would be merged or rewritten NO 
> > MATTER HOW LARGE. There are two cases,
> >> the segment has < 5G "live" docs. In that case it would be merged with 
> >> smaller segments to bring the resulting segment up to 5G. If no smaller 
> >> segments exist, it would just be rewritten
> >> The segment has > 5G "live" docs (the result of a forceMerge or optimize). 
> >> It would be rewritten into a single segment removing all deleted docs no 
> >> matter how big it is to start. The 100G example above would be rewritten 
> >> to an 80G segment for instance.
> Of course this would lead to potentially much more I/O which is why the 
> default would be the same behavior we see now. As it stands now, though, 
> there's no way to recover from an optimize/forceMerge except to re-index from 
> scratch. We routinely see 200G-300G Lucene indexes at this point "in the 
> wild" with 10s of  shards replicated 3 or more times. And that doesn't even 
> include having these over HDFS.
> Alternatives welcome! Something like the above seems minimally invasive. A 
> new merge policy is certainly an alternative.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7976) Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of very large segments

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449265#comment-16449265
 ] 

Erick Erickson commented on LUCENE-7976:


Fix for the failing test

> Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of 
> very large segments
> -
>
> Key: LUCENE-7976
> URL: https://issues.apache.org/jira/browse/LUCENE-7976
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Attachments: LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, 
> LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch
>
>
> We're seeing situations "in the wild" where there are very large indexes (on 
> disk) handled quite easily in a single Lucene index. This is particularly 
> true as features like docValues move data into MMapDirectory space. The 
> current TMP algorithm allows on the order of 50% deleted documents as per a 
> dev list conversation with Mike McCandless (and his blog here:  
> https://www.elastic.co/blog/lucenes-handling-of-deleted-documents).
> Especially in the current era of very large indexes in aggregate, (think many 
> TB) solutions like "you need to distribute your collection over more shards" 
> become very costly. Additionally, the tempting "optimize" button exacerbates 
> the issue since once you form, say, a 100G segment (by 
> optimizing/forceMerging) it is not eligible for merging until 97.5G of the 
> docs in it are deleted (current default 5G max segment size).
> The proposal here would be to add a new parameter to TMP, something like 
>  (no, that's not serious name, suggestions 
> welcome) which would default to 100 (or the same behavior we have now).
> So if I set this parameter to, say, 20%, and the max segment size stays at 
> 5G, the following would happen when segments were selected for merging:
> > any segment with > 20% deleted documents would be merged or rewritten NO 
> > MATTER HOW LARGE. There are two cases,
> >> the segment has < 5G "live" docs. In that case it would be merged with 
> >> smaller segments to bring the resulting segment up to 5G. If no smaller 
> >> segments exist, it would just be rewritten
> >> The segment has > 5G "live" docs (the result of a forceMerge or optimize). 
> >> It would be rewritten into a single segment removing all deleted docs no 
> >> matter how big it is to start. The 100G example above would be rewritten 
> >> to an 80G segment for instance.
> Of course this would lead to potentially much more I/O which is why the 
> default would be the same behavior we see now. As it stands now, though, 
> there's no way to recover from an optimize/forceMerge except to re-index from 
> scratch. We routinely see 200G-300G Lucene indexes at this point "in the 
> wild" with 10s of  shards replicated 3 or more times. And that doesn't even 
> include having these over HDFS.
> Alternatives welcome! Something like the above seems minimally invasive. A 
> new merge policy is certainly an alternative.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-BadApples-master-Linux (64bit/jdk1.8.0_162) - Build # 27 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/27/
Java: 64bit/jdk1.8.0_162 -XX:-UseCompressedOops -XX:+UseG1GC

7 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:35839/_ya/c/collection1: Async exception 
during distributed update: Error from server at 
http://127.0.0.1:46611/_ya/c/collection1_shard2_replica_n41: Server Error
request: 
http://127.0.0.1:46611/_ya/c/collection1_shard2_replica_n41/update?update.chain=distrib-dup-test-chain-explicit=TOLEADER=http%3A%2F%2F127.0.0.1%3A35839%2F_ya%2Fc%2Fcollection1_shard1_replica_n47%2F=javabin=2
 Remote error message: ERROR adding document SolrInputDocument(fields: [id=45, 
regex_dup_A_s=x_x, regex_dup_B_s=x_x, _version_=1598600016229826560])

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:35839/_ya/c/collection1: Async exception during 
distributed update: Error from server at 
http://127.0.0.1:46611/_ya/c/collection1_shard2_replica_n41: Server Error



request: 
http://127.0.0.1:46611/_ya/c/collection1_shard2_replica_n41/update?update.chain=distrib-dup-test-chain-explicit=TOLEADER=http%3A%2F%2F127.0.0.1%3A35839%2F_ya%2Fc%2Fcollection1_shard1_replica_n47%2F=javabin=2
Remote error message: ERROR adding document SolrInputDocument(fields: [id=45, 
regex_dup_A_s=x_x, regex_dup_B_s=x_x, _version_=1598600016229826560])
at 
__randomizedtesting.SeedInfo.seed([AC56303341F3D845:24020FE9EF0FB5BD]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.BaseDistributedSearchTestCase.add(BaseDistributedSearchTestCase.java:507)
at 
org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:684)
at 
org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:378)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4582 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4582/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations

Error Message:
Expected 2x2 for collection: collection2 null Live Nodes: 
[127.0.0.1:62700_solr, 127.0.0.1:62701_solr, 127.0.0.1:62702_solr] Last 
available state: null

Stack Trace:
java.lang.AssertionError: Expected 2x2 for collection: collection2
null
Live Nodes: [127.0.0.1:62700_solr, 127.0.0.1:62701_solr, 127.0.0.1:62702_solr]
Last available state: null
at 
__randomizedtesting.SeedInfo.seed([B6DB978A921C7E08:7149E5CC6F50C5DE]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:278)
at 
org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations(TestSkipOverseerOperations.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 

[jira] [Commented] (LUCENE-8264) Allow an option to rewrite all segments

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449219#comment-16449219
 ] 

Shawn Heisey commented on LUCENE-8264:
--

I'm with Erick.  I had always understood that you could use IndexUpgrader to 
migrate one major version at a time and use the index with a much newer 
version, as long as all the field type classes writing data into the index are 
still around in the newer version.

Now I'm hearing differently ... that any user who has successfully done this 
has just gotten lucky, and that there's no guarantee for the future.

I am not greatly impacted by this, because I always prefer to build indexes 
from scratch whenever I upgrade, and I recommend to anyone who will listen that 
they do the same.  We do have users with extremely large indexes, who have a 
very difficult time reindexing from scratch.

> Allow an option to rewrite all segments
> ---
>
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> For the background, see SOLR-12259.
> There are several use-cases that would be much easier, especially during 
> upgrades, if we could specify that all segments get rewritten. 
> One example: Upgrading 5x->6x->7x. When segments are merged, they're 
> rewritten into the current format. However, there's no guarantee that a 
> particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily 
> be successful.
> How many merge policies support this is an open question. I propose to start 
> with TMP and raise other JIRAs as necessary for other merge policies.
> So far the usual response has been "re-index from scratch", but that's 
> increasingly difficult as systems get larger.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk1.8.0_144) - Build # 561 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/561/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseSerialGC

27 tests failed.
FAILED:  org.apache.solr.cloud.CreateRoutedAliasTest.testTimezoneAbsoluteDate

Error Message:
Error from server at http://127.0.0.1:60500/solr: Can't create collection alias 
for collections='testTimezoneAbsoluteDate_2018-01-15', 
'testTimezoneAbsoluteDate_2018-01-15' is not an existing collection or alias

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:60500/solr: Can't create collection alias for 
collections='testTimezoneAbsoluteDate_2018-01-15', 
'testTimezoneAbsoluteDate_2018-01-15' is not an existing collection or alias
at 
__randomizedtesting.SeedInfo.seed([BAF8AEB345245EC1:1F18D3DED16307BF]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at 
org.apache.solr.cloud.CreateRoutedAliasTest.testTimezoneAbsoluteDate(CreateRoutedAliasTest.java:240)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)

[JENKINS] Lucene-Solr-repro - Build # 548 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/548/

[...truncated 28 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/47/consoleText

[repro] Revision: c2d01c2bc75f76bf7fd3b41f93d3edb01974e5ad

[repro] Repro line:  ant test  -Dtestcase=AutoscalingHistoryHandlerTest 
-Dtests.method=testHistory -Dtests.seed=3276C176959E7A6F -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ar-AE 
-Dtests.timezone=Asia/Srednekolymsk -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[repro] Repro line:  ant test  -Dtestcase=IndexSizeTriggerTest 
-Dtests.method=testTrigger -Dtests.seed=3276C176959E7A6F -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sr-Latn 
-Dtests.timezone=America/Ojinaga -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
1409ab8f84ab0949b1da095f03dc94d3b74db5cf
[repro] git fetch

[...truncated 2 lines...]
[repro] git checkout c2d01c2bc75f76bf7fd3b41f93d3edb01974e5ad

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   IndexSizeTriggerTest
[repro]   AutoscalingHistoryHandlerTest
[repro] ant compile-test

[...truncated 3316 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 
-Dtests.class="*.IndexSizeTriggerTest|*.AutoscalingHistoryHandlerTest" 
-Dtests.showOutput=onerror  -Dtests.seed=3276C176959E7A6F -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sr-Latn 
-Dtests.timezone=America/Ojinaga -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[...truncated 6289 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: 
org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[repro]   4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[repro] git checkout 1409ab8f84ab0949b1da095f03dc94d3b74db5cf

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 5 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9304) SOLR_SSL_CHECK_PEER_NAME=false (aka -Dsolr.ssl.checkPeerName=false) has no effect

[Carlton Findley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449112#comment-16449112
 ] 

Carlton Findley commented on SOLR-9304:
---

Thanks for getting this in.

> SOLR_SSL_CHECK_PEER_NAME=false (aka -Dsolr.ssl.checkPeerName=false) has no 
> effect
> -
>
> Key: SOLR-9304
> URL: https://issues.apache.org/jira/browse/SOLR-9304
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 7.0
>Reporter: Hoss Man
>Assignee: Hoss Man
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-9304-uses-deprecated.patch, SOLR-9304.patch, 
> SOLR-9304.patch, SOLR-9304.patch, SOLR-9304.patch, SOLR-9304.patch, 
> SOLR-9304.patch, SOLR-9304.patch
>
>
> {{-Dsolr.ssl.checkPeerName=false}} is completely ignored on master...
> {noformat}
> hossman@tray:~/lucene/dev/solr [master] $ find -name \*.java | xargs grep 
> checkPeerName
> ./solrj/src/java/org/apache/solr/client/solrj/impl/HttpClientUtil.java:  
> public static final String SYS_PROP_CHECK_PEER_NAME = 
> "solr.ssl.checkPeerName";
> hossman@tray:~/lucene/dev/solr [master] $ find -name \*.java | xargs grep 
> SYS_PROP_CHECK_PEER_NAME
> ./test-framework/src/java/org/apache/solr/util/SSLTestConfig.java:  
> boolean sslCheckPeerName = 
> toBooleanDefaultIfNull(toBooleanObject(System.getProperty(HttpClientUtil.SYS_PROP_CHECK_PEER_NAME)),
>  true);
> ./solrj/src/java/org/apache/solr/client/solrj/impl/HttpClientUtil.java:  
> public static final String SYS_PROP_CHECK_PEER_NAME = 
> "solr.ssl.checkPeerName";
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 547 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/547/

[...truncated 49 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-Tests-master/2503/consoleText

[repro] Revision: 4e0e8e979b66abdf0778fc0ea86ae5ef5d8f2f91

[repro] Repro line:  ant test  -Dtestcase=IndexSizeTriggerTest 
-Dtests.method=testTrigger -Dtests.seed=C5660E6BBF13 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=be-BY -Dtests.timezone=Europe/Copenhagen 
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] Repro line:  ant test  -Dtestcase=TimeRoutedAliasUpdateProcessorTest 
-Dtests.method=test -Dtests.seed=C5660E6BBF13 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=es-UY -Dtests.timezone=Australia/Queensland 
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
1409ab8f84ab0949b1da095f03dc94d3b74db5cf
[repro] git fetch

[...truncated 2 lines...]
[repro] git checkout 4e0e8e979b66abdf0778fc0ea86ae5ef5d8f2f91

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   IndexSizeTriggerTest
[repro]   TimeRoutedAliasUpdateProcessorTest
[repro] ant compile-test

[...truncated 3298 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 
-Dtests.class="*.IndexSizeTriggerTest|*.TimeRoutedAliasUpdateProcessorTest" 
-Dtests.showOutput=onerror  -Dtests.seed=C5660E6BBF13 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=be-BY -Dtests.timezone=Europe/Copenhagen 
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[...truncated 5362 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: 
org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest
[repro]   4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[repro] git checkout 1409ab8f84ab0949b1da095f03dc94d3b74db5cf

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 5 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_144) - Build # 7284 - Still Unstable!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8264) Allow an option to rewrite all segments

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448895#comment-16448895
 ] 

Erick Erickson commented on LUCENE-8264:


So let me see if I have this straight. The guarantee is that Lucene N will read 
indexes produced with N-1 if (and only if) the index was originally created 
entirely by version N-1? IOW if Lucene N-2 was used at any point in history no 
guarantees are made.

And any upgrade process that would overcome that issue would be on a 
case-by-case, one-off basis that would stand no chance of making it into the 
Lucene code base.

> Allow an option to rewrite all segments
> ---
>
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> For the background, see SOLR-12259.
> There are several use-cases that would be much easier, especially during 
> upgrades, if we could specify that all segments get rewritten. 
> One example: Upgrading 5x->6x->7x. When segments are merged, they're 
> rewritten into the current format. However, there's no guarantee that a 
> particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily 
> be successful.
> How many merge policies support this is an open question. I propose to start 
> with TMP and raise other JIRAs as necessary for other merge policies.
> So far the usual response has been "re-index from scratch", but that's 
> increasingly difficult as systems get larger.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

BeastIt! Report for Apache Solr Master

[BeastIt BeastIt]

BeastIt Unit Test Beasting Summary Report for Apache Solr Master

BeastIt gives unit tests a chance to duke it out in a fair but difficult
environment. Each test is beasted and then judged. See a link to the full
reports below.

Number of Tests: 1122
Number Passed: 1090
% Passed: 97.15%

Ran 30 iterations, 5 at a time

Markers (Marked tests have one of these markers)
 @AwaitsFix: 1
 @BadApple: 12
 @Ignore: 3

 *** Worst Test - Not Marked (It's your Moby Dick!)

 - TestTriggerIntegration 53% screwy ''

  If you catch that whale, next try
   - IndexSizeTriggerTest 23% unreliable ''

 *** New Test(s) Failing?! Sound the alarm, we may have a cowboy here.

 - IndexSizeTriggerTest 23% unreliable
 - MathExpressionTest 6% flakey

 *** New Failures in Test(s)?! Everything looked so good!

 - RequestLoggingTest 3% flakey was 0,0,0,0
 - TestSegmentSorting 3% flakey was 0,0,0,0
 - TestLeaderElectionZkExpiry 3% flakey was 0,0,0,0
 - TestExecutePlanAction 3% flakey was 0,0,0,0
 - SolrZkClientTest 3% flakey was 0,0,0,0

 *** Slowest Test

 - CdcrReplicationDistributedZkTest 1560.65 mission-failed '@BadApple
@Nightly  ' Can you speed me up?

 *** Worst Test - Marked (How long must it suffer the mark?)

 - CdcrReplicationDistributedZkTest 112% mission-failed '@BadApple
@Nightly  '

 *** Non Running Tests - Coverage Holes?!

 - ChaosMonkeyShardSplitTest '@Ignore '
 - TestRankQueryPlugin '@Ignore '
 - TestMailEntityProcessor '@Ignore '


Full Reports http://apache-solr.bitballoon.com

(Report maintained by Mark Miller)

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 572 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/572/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

8 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
at 
__randomizedtesting.SeedInfo.seed([824EF3FC2E7A9091:BBC04ABC0185596F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:298)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
at 

[jira] [Updated] (SOLR-12261) Deleting collections should sync aliases before prematurely failing when alias is deleted

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-12261:

Attachment: SOLR-12261.patch

> Deleting collections should sync aliases before prematurely failing when 
> alias is deleted
> -
>
> Key: SOLR-12261
> URL: https://issues.apache.org/jira/browse/SOLR-12261
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Attachments: SOLR-12261.patch
>
>
> In SOLR-11218 [~erickerickson] ensured that we can't delete a collection that 
> is referenced by an alias. However It may be that the alias is deleted but 
> the node servicing the request doesn't know about this yet. It should call 
> AliasesManager.update() first (which now sync()'s with ZK).
> I believe this is the cause of some sporadic failures to 
> org.apache.solr.cloud.AliasIntegrationTest#tearDown which deletes the alias 
> then all collections.
> It's debatable if this is an improvement or a bug. Sadly most of SolrCloud 
> simply seems to operate this way despite it being eventually consistent. Thus 
> users using SolrCloud may have to add sleep()s after calls to Solr admin 
> calls :-/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12261) Deleting collections should sync aliases before prematurely failing when alias is deleted

[David Smiley (JIRA)]

David Smiley created SOLR-12261:
---

 Summary: Deleting collections should sync aliases before 
prematurely failing when alias is deleted
 Key: SOLR-12261
 URL: https://issues.apache.org/jira/browse/SOLR-12261
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrCloud
Reporter: David Smiley
Assignee: David Smiley


In SOLR-11218 [~erickerickson] ensured that we can't delete a collection that 
is referenced by an alias. However It may be that the alias is deleted but the 
node servicing the request doesn't know about this yet. It should call 
AliasesManager.update() first (which now sync()'s with ZK).

I believe this is the cause of some sporadic failures to 
org.apache.solr.cloud.AliasIntegrationTest#tearDown which deletes the alias 
then all collections.

It's debatable if this is an improvement or a bug. Sadly most of SolrCloud 
simply seems to operate this way despite it being eventually consistent. Thus 
users using SolrCloud may have to add sleep()s after calls to Solr admin calls 
:-/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12260) edismax: Include phrase clauses as terms in pf/pf2/pf3

[Elizabeth Haubert (JIRA)]

Elizabeth Haubert created SOLR-12260:


 Summary: edismax: Include phrase clauses as terms in pf/pf2/pf3 
 Key: SOLR-12260
 URL: https://issues.apache.org/jira/browse/SOLR-12260
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
  Components: query parsers
Reporter: Elizabeth Haubert


Phrase queries are currently built only on bareword clauses, which causes 
unexpected behavior for queries with mixed quoted and bareword terms:

q:cat "allergic reaction" dog  

will flag "allergic reaction" as a phrase, and so will include it in none of 
pf/pf2/pf3

pf or pf2 will be generated as "cat dog".

At a minimum, it would be nice if phrases would be applied as stand-alone 
entities to pf2/pf3, if they contain the appropriate number of terms.  But I 
think the work that has been done to accommodate graph queries should also be 
able to handle these phrase terms following the pattern of:

spanNear[text:cat, spanNear(text:allergic, text:reaction, 0, true), text:dog]

      

 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12258) V2 API should "retry" for unresolved collections/aliases (like V1 does)

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448816#comment-16448816
 ] 

David Smiley commented on SOLR-12258:
-

To be extra clear, here's a small excerpt from 
org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest#test (line 
97-111) that is minimally sufficient to be showing the problem. _The payload 
part is irrelevant.; what matters is that it's simply a V2 request_:
{code:java}
CollectionAdminRequest.createCollection(configName, configName, 1, 
1).process(solrClient);

// manipulate the config...
checkNoError(solrClient.request(new V2Request.Builder("/collections/" + 
configName + "/config")
.withMethod(SolrRequest.METHOD.POST)
.withPayload("{" +
"  'set-user-property' : {'update.autoCreateFields':false}," + // 
no data driven
"  'add-updateprocessor' : {" +
"'name':'tolerant', 
'class':'solr.TolerantUpdateProcessorFactory'" +
"  }," +
"  'add-updateprocessor' : {" + // for testing
"'name':'inc', 'class':'" + IncrementURPFactory.class.getName() 
+ "'," +
"'fieldName':'" + intField + "'" +
"  }," +
"}").build()));
{code}
The second call, where we manipulate the config, sometimes/rarely fails because 
V2HttpCall can't resolve the collection (line 119). It's ZK state simply isn't 
up to date (I surmise). In principle, a V1 call could fail as well but in 
practice maybe it's more rare because the "retry" aspect of V1 buys it 
sufficient extra time. Adding a SolrCloudTestCase.waitForState in-between the 
calls here _may_ help but again there's no guarantee since waitForState waits 
for _the state of the client's state reader_ (not for state readers of Solr 
nodes).

For aliases, we can call ZooKeeper.sync("/aliases.json",...) -- and in fact I 
made sure ZkStateReader now does this in update(). For cases where our code 
expects to operate on a collection (thus it had better exist or we have an 
error) we could try and do a similar thing for collections? In fact we have 
ZkStateReader.forceUpdateCollection(collection) added by [~shalinmangar] in 
SOLR-8745 though it doesn't call ZooKeeper.sync() but shouldn't it?

> V2 API should "retry" for unresolved collections/aliases (like V1 does)
> ---
>
> Key: SOLR-12258
> URL: https://issues.apache.org/jira/browse/SOLR-12258
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud, v2 API
>Reporter: David Smiley
>Priority: Major
>
> When using V1, if the request refers to a possible collection/alias that 
> fails to resolve, HttpSolrCall will invoke AliasesManager.update() then retry 
> the request as if anew (in collaboration with SolrDispatchFilter).  If it 
> fails to resolve again we stop there and return an error; it doesn't go on 
> forever.
> V2 (V2HttpCall specifically) doesn't have this retry mechanism.  It'll return 
> "no such collection or alias".
> The retry will not only work for an alias but the retrying is a delay that 
> will at least help the odds of a newly made collection from being known to 
> this Solr node.  It'd be nice if this was more explicit – i.e. if there was a 
> mechanism similar to AliasesManager.update() but for a collection.  I'm not 
> sure how to do that?
> BTW I discovered this while debugging a Jenkins failure of 
> TimeRoutedAliasUpdateProcessorTest.test where it early on simply goes to 
> issue a V2 based request to change the configuration of a collection that was 
> created immediately before it.  It's pretty mysterious.  I am aware of 
> SolrCloudTestCase.waitForState which is maybe something that needs to be 
> called?  But if that were true then *every* SolrCloud test would need to use 
> it; it just seems wrong to me that we ought to use this method commonly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448788#comment-16448788
 ] 

Robert Muir commented on LUCENE-8267:
-

{quote}
Thanks for the suggestion to use MMapDirectory.preload, I didn't know about it, 
but that appears to only help warmup, not sustained performance; right?
{quote}

loading stuff into heap memory gives no higher guarantee than doing it this way 
under pressure, it still depends on vm parameters.

{quote}
I get the maintenance aspect but we need community input on such decisions to 
ascertain real-world use.
{quote}

That is not how it works: this is open source. These memory/direct formats 
cause excessive maintenance hassle with the tests. I saw Alan and Dawid 
fighting with them and it seemed clear to me its not worth the trouble. We 
should remove them: the cost is too high.

Someone can always pull in the source code themselves for their esoteric 
use-case: but unless we have *maintainers* coming up then they need to go: this 
doesn't come down to a vote by users.

If you want to make it hard for us to clean up tech debt like this, by -1s and 
so on, thats your choice. But it is also my choice to make it hard to add 
things. 

Trust me, I will make it equally hard to add code as it is to remove code. It 
is the only way to make things sustainable.


> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-7.x - Build # 207 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/207/

No tests ran.

Build Log:
[...truncated 24220 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: 
invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid 
part, must have at least one section (e.g., chapter, appendix, etc.)
 [java] Processed 2190 links (1747 relative) to 2947 anchors in 228 files
 [echo] Validated Links & Anchors via: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr-ref-guide/bare-bones-html/

-dist-changes:
 [copy] Copying 4 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/changes

-dist-keys:
  [get] Getting: http://home.apache.org/keys/group/lucene.asc
  [get] To: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/KEYS

package:

-unpack-solr-tgz:

-ensure-solr-tgz-exists:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr.tgz.unpacked
[untar] Expanding: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/solr-7.4.0.tgz
 into 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr.tgz.unpacked

generate-maven-artifacts:

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448771#comment-16448771
 ] 

Dawid Weiss commented on LUCENE-8267:
-

bq. I get the maintenance aspect but we need community input on such decisions 
to ascertain real-world use.

I think filing a JIRA issue is kind of soliciting feedback, don't you think? I 
agree with Simon and Robert that there are classes that, while useful, are not 
at the forefront of what a broad "Lucene API" is... We should have the liberty 
to adjust or remove such things. I scanned the code of both Lucene and Solr and 
there were no references (other than in tests) to those classes, so it's not 
just "Lucene land".

Also, given the size and diversity of the Lucene/Solr user community I'm fairly 
confident there will always be somebody who finds something very useful, no 
matter what you'd like to change or remove. Hell, I use a lot of internal 
Lucene infrastructure in my own projects and sometimes I miss things that go 
away myself... (and frequently I just grab the latest source of something and 
copy it over to maintain in my own source tree, that's part of the beauty of 
open source).

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448748#comment-16448748
 ] 

David Smiley commented on LUCENE-8267:
--

Ah; I incorrectly assumed the proposal included the FST postings formats but 
apparently not. It's too bad FSTPulsingFormat is long gone though since in the 
text-tagging use-case it'd effectively be a substitute for 
MemoryPostingsFormat. The FSTTermsReader accepts a PostingsReaderBase; maybe 
it's possible to write an in-memory version of PostingsReaderBase, at least for 
the "pulsed" (single posting) case. Nonetheless lets see how the text tagger 
performs with these codec options.

Thanks for the suggestion to use MMapDirectory.preload, I didn't know about it, 
but that appears to only help warmup, not sustained performance; right? And I 
believe even with FileSwitchDirectory, on shutdown files with certain 
extensions would vanish; right?
{quote}So I perceive your veto as an aggressive step. To me it's a last resort 
after we can't find a solution that is good for all of us. The conversation 
already has a tone that is not appropriate and could have been prevented by 
formulating objections as questions. like I am using this postings format in X 
and it's serving well, what are the alternatives. - I am sure you would have 
got an awesome answer.
{quote}
The "sorry" word immediately after my veto was intended to prevent 
misperceptions about tone; I don't mean to be aggressive – sorry!  I agree I 
could have asked for alternatives up-front; I'll try and remember that next 
time. I was thinking my early vote could prevent work that someone does in vein 
to remove these pieces. In retrospect I didn't need to vote yet to accomplish 
that (e.g. convey disagreement with others).  In this way I was trying to offer 
improved communication where from other's I've seen no veto but a confusing 
cloud of doubt as to wether there would be a veto or not (which in my mind is 
worse).  I respect you may feel differently though; just please understand my 
intended tone is not aggressive.
{quote}if you can't remove stuff without others jumping in vetoing the reaction 
will be to prevent additions in the same way due to _fear_  created by the 
veto. This is a terrible place to be in, we have seen this in the past we 
should prevent it.
{quote}
Do you mean if we add some new thingamajig, we might feel that we *have* to 
support it indefinitely?  (I wouldn't use the word "fear" for this; maybe I've 
got your intent wrong still)  Hmmm; I think it's very situationally dependent.  
For example with queryNorm & coords, LUCENE-7347, I had concerns but ultimately 
understood that maintaining these things were making things awkward for us.  
But the PostingsFormats seem different to me.  They conform to our APIs; they 
don't get in the way or tie our hands.  Yes there is maintenance though.  I 
think what I objected to most in the description of this issue was the notion 
that, because Lucene-core doesn't use something and because there is 
maintenance to that something, then we should delete that something.  I get the 
maintenance aspect but we need community input on such decisions to ascertain 
real-world use.

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12242) UnifiedHighlighter does not work with Surround query parser (SurroundQParser)

[Andy Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448731#comment-16448731
 ] 

Andy Liu edited comment on SOLR-12242 at 4/23/18 7:14 PM:
--

Yeah, I noticed that.  I did it in an super hacky way – in 
highlightFieldsAsObjects() I noticed that the output of 
UnifiedHighlighter.extractTerms() is passed into getFieldHighlighter(), which 
is an instance method.  So I subclassed UnifiedHighlighter and with an 
overridden implementation of  getFieldHighlighter() which does the term 
extraction with the live searcher, and passes the result to  
super.getFieldHighlighter() .

Super brittle but works for now.


was (Author: andyliu1227):
Yeah, I noticed that.  I did it in an super hacky way – in 
highlightFieldsAsObjects() I noticed that the output of 
UnifiedHighlighter.extractTerms() is passed into getFieldHighlighter(), which 
is an instance method.  So I subclassed UnifiedHighlighter and with an 
overridden implementation of  getFieldHighlighter() which does the term 
extraction with the live searcher.

Super brittle but works for now.

> UnifiedHighlighter does not work with Surround query parser (SurroundQParser)
> -
>
> Key: SOLR-12242
> URL: https://issues.apache.org/jira/browse/SOLR-12242
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Affects Versions: 7.2.1
>Reporter: Andy Liu
>Priority: Major
> Attachments: TestUnifiedHighlighterSurround.java
>
>
> I'm attempting to use the UnifiedHighlighter in conjunction with queries 
> parsed by Solr's SurroundQParserPlugin. When doing so, the response yields 
> empty arrays for documents that should contain highlighted snippets.
> I've attached a test for UnifiedHighlighter that uses the surround's 
> QueryParser and preprocesses the query in a similar fashion as 
> SurroundQParser, which results in test failure.  When creating a SpanQuery 
> directly (rather via surround's QueryParser), the test passes.
> The problem can be isolated to the code path initiated by 
> UnifiedHighlighter.extractTerms(), which uses EMPTY_INDEXSEARCHER to extract 
> terms from the query. After a series of method calls, we end up at 
> DistanceQuery.getSpanNearQuery(), where 
> {{((DistanceSubQuery)sqi.next()).addSpanQueries(sncf)}} fails silently and 
> doesn't add any span queries.  
> Another data point: If I hack UnifiedHighlighter and pass in a live 
> IndexSearcher to extractTerms(), highlighting works. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12242) UnifiedHighlighter does not work with Surround query parser (SurroundQParser)

[Andy Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448731#comment-16448731
 ] 

Andy Liu commented on SOLR-12242:
-

Yeah, I noticed that.  I did it in an super hacky way – in 
highlightFieldsAsObjects() I noticed that the output of 
UnifiedHighlighter.extractTerms() is passed into getFieldHighlighter(), which 
is an instance method.  So I subclassed UnifiedHighlighter and with an 
overridden implementation of  getFieldHighlighter() which does the term 
extraction with the live searcher.

Super brittle but works for now.

> UnifiedHighlighter does not work with Surround query parser (SurroundQParser)
> -
>
> Key: SOLR-12242
> URL: https://issues.apache.org/jira/browse/SOLR-12242
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Affects Versions: 7.2.1
>Reporter: Andy Liu
>Priority: Major
> Attachments: TestUnifiedHighlighterSurround.java
>
>
> I'm attempting to use the UnifiedHighlighter in conjunction with queries 
> parsed by Solr's SurroundQParserPlugin. When doing so, the response yields 
> empty arrays for documents that should contain highlighted snippets.
> I've attached a test for UnifiedHighlighter that uses the surround's 
> QueryParser and preprocesses the query in a similar fashion as 
> SurroundQParser, which results in test failure.  When creating a SpanQuery 
> directly (rather via surround's QueryParser), the test passes.
> The problem can be isolated to the code path initiated by 
> UnifiedHighlighter.extractTerms(), which uses EMPTY_INDEXSEARCHER to extract 
> terms from the query. After a series of method calls, we end up at 
> DistanceQuery.getSpanNearQuery(), where 
> {{((DistanceSubQuery)sqi.next()).addSpanQueries(sncf)}} fails silently and 
> doesn't add any span queries.  
> Another data point: If I hack UnifiedHighlighter and pass in a live 
> IndexSearcher to extractTerms(), highlighting works. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_162) - Build # 1784 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1784/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore

Error Message:
ObjectTracker found 4 object(s) that were not released!!! [InternalHttpClient, 
NIOFSDirectory, SolrCore, InternalHttpClient] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.http.impl.client.InternalHttpClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319)
  at 
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:328)
  at 
org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:248)  
at org.apache.solr.handler.IndexFetcher.(IndexFetcher.java:290)  at 
org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1190) 
 at 
org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:106)
  at 
org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1178)
  at 
org.apache.solr.cloud.RecoveryStrategy.doReplicateOnlyRecovery(RecoveryStrategy.java:381)
  at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:312)  
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:294)  at 
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
 at java.lang.Thread.run(Thread.java:748)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.lucene.store.NIOFSDirectory  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95)  at 
org.apache.solr.core.SolrCore.initIndex(SolrCore.java:762)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:955)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:864)  at 
org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:957)  at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:90)
  at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:356)
  at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
  at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)  
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)  
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:393)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:339)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) 
 at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:455)  
at 

[jira] [Updated] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms

[Elizabeth Haubert (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elizabeth Haubert updated SOLR-12243:
-
Attachment: SOLR-12243.patch

> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Priority: Major
> Attachments: SOLR-12243.patch
>
>
> synonyms.txt:
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> request handler:
> 
>  
> 
>  edismax
>   0.4
>  title^100
>  title~20^5000
>  title~11
>  title~22^1000
>  text
>  
>  3-1 6-3 930%
>  *:*
>  25
> 
>  
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin"  against the 
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not 
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction 
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin 
> dose" or pf3:"aspirin dose ?"
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk1.8.0) - Build # 591 - Still Failing!

[Policeman Jenkins Server]

Error processing tokens: Error while parsing action 
'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input 
position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11914) Remove/move questionable SolrParams methods

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448525#comment-16448525
 ] 

David Smiley commented on SOLR-11914:
-

I'm unsure of what to do RE master branch.  I could commit their removal here & 
now.  If so, I could add an upgrade note to 8.0, or simply not bother; the loss 
of the methods is no big deal.  Or do nothing at all now, waiting till some 
future 8.0 upgrade to remove all deprecated stuff en-masse.

> Remove/move questionable SolrParams methods
> ---
>
> Key: SOLR-11914
> URL: https://issues.apache.org/jira/browse/SOLR-11914
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
>  Labels: newdev
> Attachments: SOLR-11914.patch, SOLR-11914.patch
>
>
> {{Map getAll(Map sink, Collection 
> params)}} 
> Is only used by the CollectionsHandler, and has particular rules about how it 
> handles multi-valued data that make it not very generic, and thus I think 
> doesn't belong here.  Furthermore the existence of this method is confusing 
> in that it gives the user another choice against it use versus toMap (there 
> are two overloaded variants).
> {{SolrParams toFilteredSolrParams(List names)}}
> Is only called in one place, and something about it bothers me, perhaps just 
> the name or that it ought to be a view maybe.
> {{static Map toMap(NamedList params)}}
> Isn't used and I don't like it; it doesn't even involve a SolrParams!  Legacy 
> of 2006.
> {{static Map toMultiMap(NamedList params)}}
> It doesn't even involve a SolrParams! Legacy of 2006 with some updates since. 
> Used in some places. Perhaps should be moved to NamedList as an instance 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11914) Remove/move questionable SolrParams methods

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448505#comment-16448505
 ] 

ASF subversion and git services commented on SOLR-11914:


Commit 9e36c51e6575a8d750ddc62683c528df197f2833 in lucene-solr's branch 
refs/heads/branch_7x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9e36c51 ]

SOLR-11914: Deprecated some SolrParams methods.
* toSolrParams(nl) moved to a NamedList method, which is more natural.

(cherry picked from commit 1409ab8)


> Remove/move questionable SolrParams methods
> ---
>
> Key: SOLR-11914
> URL: https://issues.apache.org/jira/browse/SOLR-11914
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
>  Labels: newdev
> Attachments: SOLR-11914.patch, SOLR-11914.patch
>
>
> {{Map getAll(Map sink, Collection 
> params)}} 
> Is only used by the CollectionsHandler, and has particular rules about how it 
> handles multi-valued data that make it not very generic, and thus I think 
> doesn't belong here.  Furthermore the existence of this method is confusing 
> in that it gives the user another choice against it use versus toMap (there 
> are two overloaded variants).
> {{SolrParams toFilteredSolrParams(List names)}}
> Is only called in one place, and something about it bothers me, perhaps just 
> the name or that it ought to be a view maybe.
> {{static Map toMap(NamedList params)}}
> Isn't used and I don't like it; it doesn't even involve a SolrParams!  Legacy 
> of 2006.
> {{static Map toMultiMap(NamedList params)}}
> It doesn't even involve a SolrParams! Legacy of 2006 with some updates since. 
> Used in some places. Perhaps should be moved to NamedList as an instance 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11914) Remove/move questionable SolrParams methods

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448504#comment-16448504
 ] 

ASF subversion and git services commented on SOLR-11914:


Commit 1409ab8f84ab0949b1da095f03dc94d3b74db5cf in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1409ab8 ]

SOLR-11914: Deprecated some SolrParams methods.
* toSolrParams(nl) moved to a NamedList method, which is more natural.


> Remove/move questionable SolrParams methods
> ---
>
> Key: SOLR-11914
> URL: https://issues.apache.org/jira/browse/SOLR-11914
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
>  Labels: newdev
> Attachments: SOLR-11914.patch, SOLR-11914.patch
>
>
> {{Map getAll(Map sink, Collection 
> params)}} 
> Is only used by the CollectionsHandler, and has particular rules about how it 
> handles multi-valued data that make it not very generic, and thus I think 
> doesn't belong here.  Furthermore the existence of this method is confusing 
> in that it gives the user another choice against it use versus toMap (there 
> are two overloaded variants).
> {{SolrParams toFilteredSolrParams(List names)}}
> Is only called in one place, and something about it bothers me, perhaps just 
> the name or that it ought to be a view maybe.
> {{static Map toMap(NamedList params)}}
> Isn't used and I don't like it; it doesn't even involve a SolrParams!  Legacy 
> of 2006.
> {{static Map toMultiMap(NamedList params)}}
> It doesn't even involve a SolrParams! Legacy of 2006 with some updates since. 
> Used in some places. Perhaps should be moved to NamedList as an instance 
> method.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_162) - Build # 21892 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21892/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
at 
__randomizedtesting.SeedInfo.seed([9B38859C53FDCD9D:F8F3B31ECA32BEB0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:187)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14403 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> Creating 

[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 208 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/208/

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at 
__randomizedtesting.SeedInfo.seed([87F9482F6E12289D:BE77F16F41EDE163]:0)
at 
org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:109)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:299)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14952 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> 3803268 INFO  
(SUITE-IndexSizeTriggerTest-seed#[87F9482F6E12289D]-worker) [] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: 

[jira] [Commented] (SOLR-12242) UnifiedHighlighter does not work with Surround query parser (SurroundQParser)

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448420#comment-16448420
 ] 

David Smiley commented on SOLR-12242:
-

sadly UnifiedHighlighter.extractTerms is static so you can't actually override 
it.  That's a shame; this should be changed.  How did you go about this in your 
code?  A replaced variation of highlightWithoutSearcher() perhaps?

> UnifiedHighlighter does not work with Surround query parser (SurroundQParser)
> -
>
> Key: SOLR-12242
> URL: https://issues.apache.org/jira/browse/SOLR-12242
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Affects Versions: 7.2.1
>Reporter: Andy Liu
>Priority: Major
> Attachments: TestUnifiedHighlighterSurround.java
>
>
> I'm attempting to use the UnifiedHighlighter in conjunction with queries 
> parsed by Solr's SurroundQParserPlugin. When doing so, the response yields 
> empty arrays for documents that should contain highlighted snippets.
> I've attached a test for UnifiedHighlighter that uses the surround's 
> QueryParser and preprocesses the query in a similar fashion as 
> SurroundQParser, which results in test failure.  When creating a SpanQuery 
> directly (rather via surround's QueryParser), the test passes.
> The problem can be isolated to the code path initiated by 
> UnifiedHighlighter.extractTerms(), which uses EMPTY_INDEXSEARCHER to extract 
> terms from the query. After a series of method calls, we end up at 
> DistanceQuery.getSpanNearQuery(), where 
> {{((DistanceSubQuery)sqi.next()).addSpanQueries(sncf)}} fails silently and 
> doesn't add any span queries.  
> Another data point: If I hack UnifiedHighlighter and pass in a live 
> IndexSearcher to extractTerms(), highlighting works. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8196) Add IntervalQuery and IntervalsSource to expose minimum interval semantics across term fields

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448387#comment-16448387
 ] 

Alan Woodward commented on LUCENE-8196:
---

bq. How would we prevent matching at the same interval?

The original paper doesn't look like it addresses this.  I'll try and work out 
the best way of dealing with things, I guess we'll need to keep track of the 
positions of internal intervals in the priority queue, and when we advance make 
sure that they don't collide.

> Add IntervalQuery and IntervalsSource to expose minimum interval semantics 
> across term fields
> -
>
> Key: LUCENE-8196
> URL: https://issues.apache.org/jira/browse/LUCENE-8196
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.4
>
> Attachments: LUCENE-8196.patch, LUCENE-8196.patch, LUCENE-8196.patch, 
> LUCENE-8196.patch, LUCENE-8196.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This ticket proposes an alternative implementation of the SpanQuery family 
> that uses minimum-interval semantics from 
> [http://vigna.di.unimi.it/ftp/papers/EfficientAlgorithmsMinimalIntervalSemantics.pdf]
>  to implement positional queries across term-based fields.  Rather than using 
> TermQueries to construct the interval operators, as in LUCENE-2878 or the 
> current Spans implementation, we instead use a new IntervalsSource object, 
> which will produce IntervalIterators over a particular segment and field.  
> These are constructed using various static helper methods, and can then be 
> passed to a new IntervalQuery which will return documents that contain one or 
> more intervals so defined.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12250) NegativeArraySizeException on TransactionLog if previous document more than 1.9GB

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448356#comment-16448356
 ] 

Hoss Man commented on SOLR-12250:
-

Dat: even after simon's addition of {{assumeWorkingMockito}} this test is still 
causing jenkins failures...

{noformat}
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1781/
Java: 64bit/jdk-11-ea+5 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.update.TransactionLogTest.testBigLastAddSize

Error Message:
 Mockito cannot mock this class: class org.apache.solr.update.AddUpdateCommand. 
 Mockito can only mock non-private &
non-final classes. If you're not sure why you're getting this error, please 
report to the mailing list.   Java
: 11 JVM vendor name: "Oracle Corporation" JVM vendor version : 11-ea+5 JVM 
name   : OpenJDK 64-Bit
Server VM JVM version: 11-ea+5 JVM info   : mixed mode OS name  
  : Linux OS version
: 4.13.0-37-generic   Underlying exception : 
java.lang.UnsupportedOperationException: Cannot define class using
reflection

Stack Trace:
org.mockito.exceptions.base.MockitoException: 
Mockito cannot mock this class: class org.apache.solr.update.AddUpdateCommand.

Mockito can only mock non-private & non-final classes.
If you're not sure why you're getting this error, please report to the mailing 
list.

{noformat}

> NegativeArraySizeException on TransactionLog if previous document more than 
> 1.9GB
> -
>
> Key: SOLR-12250
> URL: https://issues.apache.org/jira/browse/SOLR-12250
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-12250.patch, SOLR-12250.patch
>
>
> In TransactionLog, we have
> {code:java}
> bufSize = Math.min(1024*1024, lastAddSize+(lastAddSize>>3)+256);
> MemOutputStream out = new MemOutputStream(new byte[bufSize]);
> {code}
> Note that bufSize will be a negative number if lastAddSize > 1908874127 ( 
> which is around 1.9GB).
> Although this seems to relate to user's error because of sending such a big 
> document. But the exception is thrown for the update after the big one. 
> Therefore it is better to fix the problem and solving how we can prohibit 
> users from sending very big documents in other issues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12242) UnifiedHighlighter does not work with Surround query parser (SurroundQParser)

[Andy Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448343#comment-16448343
 ] 

Andy Liu commented on SOLR-12242:
-

Gotcha, thanks for the tip.  Seems like the fix is rather involved.

For my purposes (non-production) I will override UnifiedHighlighter to use a 
live searcher and take note of when or if performance / memory utilization goes 
haywire if wildcard queries are used.

> UnifiedHighlighter does not work with Surround query parser (SurroundQParser)
> -
>
> Key: SOLR-12242
> URL: https://issues.apache.org/jira/browse/SOLR-12242
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Affects Versions: 7.2.1
>Reporter: Andy Liu
>Priority: Major
> Attachments: TestUnifiedHighlighterSurround.java
>
>
> I'm attempting to use the UnifiedHighlighter in conjunction with queries 
> parsed by Solr's SurroundQParserPlugin. When doing so, the response yields 
> empty arrays for documents that should contain highlighted snippets.
> I've attached a test for UnifiedHighlighter that uses the surround's 
> QueryParser and preprocesses the query in a similar fashion as 
> SurroundQParser, which results in test failure.  When creating a SpanQuery 
> directly (rather via surround's QueryParser), the test passes.
> The problem can be isolated to the code path initiated by 
> UnifiedHighlighter.extractTerms(), which uses EMPTY_INDEXSEARCHER to extract 
> terms from the query. After a series of method calls, we end up at 
> DistanceQuery.getSpanNearQuery(), where 
> {{((DistanceSubQuery)sqi.next()).addSpanQueries(sncf)}} fails silently and 
> doesn't add any span queries.  
> Another data point: If I hack UnifiedHighlighter and pass in a live 
> IndexSearcher to extractTerms(), highlighting works. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8270) Remove MatchesIterator.term()

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448341#comment-16448341
 ] 

ASF subversion and git services commented on LUCENE-8270:
-

Commit e167e9124757b3f3597db8149c49b7f388c48627 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e167e91 ]

LUCENE-8270: Remove MatchesIterator.term()


> Remove MatchesIterator.term()
> -
>
> Key: LUCENE-8270
> URL: https://issues.apache.org/jira/browse/LUCENE-8270
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8270.patch
>
>
> As discussed on LUCENE-8268, we don't have a clear use-case for this yet, and 
> it's complicating adding Matches to phrase queries, so let's just remove it 
> for now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8265) WordDelimiterFilter should pass through terms marked as keywords

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448339#comment-16448339
 ] 

David Smiley commented on LUCENE-8265:
--

bq.  code up a TokenFilter that wraps another TokenFilter, and bypasses the 
wrapped filter if a certain condition is met?

Yes; this has been my very long Lucene/Solr idea TODO list.   Or perhaps 
alternatively, some TokenFilters could extend a new TokenFilter subclass that 
checks a condition.   By default it could be a Predicate that simply returns 
true.  This would address Mike Sokolov's concern on propagating the lifecycle 
calls... I've had to delegate a tokenizer/filter before and it was a bit 
annoying to get right.

> WordDelimiterFilter should pass through terms marked as keywords
> 
>
> Key: LUCENE-8265
> URL: https://issues.apache.org/jira/browse/LUCENE-8265
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mike Sokolov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This will help in cases where some terms containing separator characters 
> should be split, but others should not.  For example, this will enable a 
> filter that identifies things that look like fractions and identifies them as 
> keywords so that 1/2 does not become 12, while doing splitting and joining on 
> terms that look like part numbers containing slashes, eg something like 
> "sn-999123/1" might sometimes be written "sn-999123-1".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8270) Remove MatchesIterator.term()

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448340#comment-16448340
 ] 

ASF subversion and git services commented on LUCENE-8270:
-

Commit 9a3a43771ee7122d291a96c7543e099c010005cf in lucene-solr's branch 
refs/heads/branch_7x from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9a3a437 ]

LUCENE-8270: Remove MatchesIterator.term()


> Remove MatchesIterator.term()
> -
>
> Key: LUCENE-8270
> URL: https://issues.apache.org/jira/browse/LUCENE-8270
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8270.patch
>
>
> As discussed on LUCENE-8268, we don't have a clear use-case for this yet, and 
> it's complicating adding Matches to phrase queries, so let's just remove it 
> for now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8269) Detach downstream classes from IndexWriter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448303#comment-16448303
 ] 

ASF subversion and git services commented on LUCENE-8269:
-

Commit 422955eb0d0297ed1f13269eb24e37c4a5c86129 in lucene-solr's branch 
refs/heads/branch_7x from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=422955e ]

LUCENE-8269: Detach downstream classes from IndexWriter

IndexWriter today is shared with many classes like BufferedUpdateStream,
DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
on the writer instance or assert that the current thread doesn't hold a lock.
This makes it very difficult to have a manageable threading model.

This change separates out the IndexWriter from those classes and makes them all
independent of IW. IW now implements a new interface for DocumentsWriter to 
communicate
on failed or successful flushes and tragic events. This allows IW to make it's 
critical
methods private and execute all lock critical actions on it's private queue 
that ensures
that the IW lock is not held. Follow-up changes will try to detach more code 
like
publishing flushed segments to ensure we never call back into IW in an 
uncontrolled way.

> Detach downstream classes from IndexWriter
> --
>
> Key: LUCENE-8269
> URL: https://issues.apache.org/jira/browse/LUCENE-8269
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8269.patch
>
>
> IndexWriter today is shared with many classes like BufferedUpdateStream,
> DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
> on the writer instance or assert that the current thread doesn't hold a lock.
> This makes it very difficult to have a manageable threading model.
> 
> This change separates out the IndexWriter from those classes and makes 
> them all
> independent of IW. IW now implements a new interface for DocumentsWriter 
> to communicate
> on failed or successful flushes and tragic events. This allows IW to make 
> it's critical
> methods private and execute all lock critical actions on it's private 
> queue that ensures
> that the IW lock is not held. Follow-up changes will try to detach more 
> code like
> publishing flushed segments to ensure we never call back into IW in an 
> uncontrolled way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-8269) Detach downstream classes from IndexWriter

[Simon Willnauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer resolved LUCENE-8269.
-
Resolution: Fixed

> Detach downstream classes from IndexWriter
> --
>
> Key: LUCENE-8269
> URL: https://issues.apache.org/jira/browse/LUCENE-8269
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8269.patch
>
>
> IndexWriter today is shared with many classes like BufferedUpdateStream,
> DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
> on the writer instance or assert that the current thread doesn't hold a lock.
> This makes it very difficult to have a manageable threading model.
> 
> This change separates out the IndexWriter from those classes and makes 
> them all
> independent of IW. IW now implements a new interface for DocumentsWriter 
> to communicate
> on failed or successful flushes and tragic events. This allows IW to make 
> it's critical
> methods private and execute all lock critical actions on it's private 
> queue that ensures
> that the IW lock is not held. Follow-up changes will try to detach more 
> code like
> publishing flushed segments to ensure we never call back into IW in an 
> uncontrolled way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8269) Detach downstream classes from IndexWriter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448300#comment-16448300
 ] 

ASF subversion and git services commented on LUCENE-8269:
-

Commit 6f0a884582a3d58342f98dc1df2c06418defb317 in lucene-solr's branch 
refs/heads/master from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6f0a884 ]

LUCENE-8269: Detach downstream classes from IndexWriter

IndexWriter today is shared with many classes like BufferedUpdateStream,
DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
on the writer instance or assert that the current thread doesn't hold a lock.
This makes it very difficult to have a manageable threading model.

This change separates out the IndexWriter from those classes and makes them all
independent of IW. IW now implements a new interface for DocumentsWriter to 
communicate
on failed or successful flushes and tragic events. This allows IW to make it's 
critical
methods private and execute all lock critical actions on it's private queue 
that ensures
that the IW lock is not held. Follow-up changes will try to detach more code 
like
publishing flushed segments to ensure we never call back into IW in an 
uncontrolled way.

> Detach downstream classes from IndexWriter
> --
>
> Key: LUCENE-8269
> URL: https://issues.apache.org/jira/browse/LUCENE-8269
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8269.patch
>
>
> IndexWriter today is shared with many classes like BufferedUpdateStream,
> DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
> on the writer instance or assert that the current thread doesn't hold a lock.
> This makes it very difficult to have a manageable threading model.
> 
> This change separates out the IndexWriter from those classes and makes 
> them all
> independent of IW. IW now implements a new interface for DocumentsWriter 
> to communicate
> on failed or successful flushes and tragic events. This allows IW to make 
> it's critical
> methods private and execute all lock critical actions on it's private 
> queue that ensures
> that the IW lock is not held. Follow-up changes will try to detach more 
> code like
> publishing flushed segments to ensure we never call back into IW in an 
> uncontrolled way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448287#comment-16448287
 ] 

Simon Willnauer commented on LUCENE-8267:
-

+1 to what [~rcmuir] said so many more efficient options

{quote}Do you mean to say I should have said all I said without voting first? 
Lets have a conversation! (we _are_ having a conversation){quote}

So I perceive your veto as an aggressive step. To me it's a last resort after 
we can't find a solution that is good for all of us. The conversation already 
has a tone that is not appropriate and could have been prevented by formulating 
objections as questions. like _I am using this postings format in X and it's 
serving well, what are the alternatives._ - I am sure you would have got an 
awesome answer.

{quote}I don't understand this point of view; can you please elaborate? Fear of 
what?{quote}

if you can't remove stuff without others jumping in vetoing the reaction will 
be to prevent additions in the same way due to _fear_  created by the veto. 
This is a terrible place to be in, we have seen this in the past we should 
prevent it.

 

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_162) - Build # 1783 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1783/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.testRestoreState

Error Message:
Did not expect the processor to fire on first run! event={   
"id":"726561cccb05eT5zvtwban4xm68dzgh8qfdjfi5",   
"source":"node_added_trigger",   "eventTime":2012476129194078,   
"eventType":"NODEADDED",   "properties":{ "eventTimes":[   
2012476129194078,   2012476129195658], "nodeNames":[   
"127.0.0.1:44783_solr",   "127.0.0.1:44341_solr"]}}

Stack Trace:
java.lang.AssertionError: Did not expect the processor to fire on first run! 
event={
  "id":"726561cccb05eT5zvtwban4xm68dzgh8qfdjfi5",
  "source":"node_added_trigger",
  "eventTime":2012476129194078,
  "eventType":"NODEADDED",
  "properties":{
"eventTimes":[
  2012476129194078,
  2012476129195658],
"nodeNames":[
  "127.0.0.1:44783_solr",
  "127.0.0.1:44341_solr"]}}
at 
__randomizedtesting.SeedInfo.seed([55E23CD92FEAE9A3:9B4C984AD7D391B5]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.lambda$new$0(NodeAddedTriggerTest.java:49)
at 
org.apache.solr.cloud.autoscaling.NodeAddedTrigger.run(NodeAddedTrigger.java:161)
at 
org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.testRestoreState(NodeAddedTriggerTest.java:257)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

[jira] [Comment Edited] (LUCENE-8265) WordDelimiterFilter should pass through terms marked as keywords

[Mike Sokolov (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448257#comment-16448257
 ] 

Mike Sokolov edited comment on LUCENE-8265 at 4/23/18 3:02 PM:
---

Good point [~khitrin]. While I am having difficulty coming up with an example 
of something that would be marked as keyword in order to protect it from 
stemming, but that *should* be split up by WDF (or conversely something that 
should be stemmed, but not split), I agree that in principle these are 
different decisions, and adding an option is low-complexity. I'll post a change 
later today I hope.

[~romseygeek] that idea is really cool, but I think it is more complexity than 
I want to take on for this issue? I have tried making a wrapping filter before, 
and it is pretty tricky. In my experience you have to be very careful about (1) 
 how reset() calls propagate, and (2) the signal to switch behavior.  EG you 
probably want to be able to call incrementToken() on either of two different 
upstream filters that both share the same input, based on the value of some 
attribute. However, by the time you see the attribute, it is already too late 
to change! So you have to introduce a delay token that only carries this 
"switching" info. 


was (Author: sokolov):
Good point [~khitrin]. I agree, we should add an option. I'll post a change 
later today I hope.

[~romseygeek] that idea is really cool, but I think it is more complexity than 
I want to take on for this issue? I have tried making a wrapping filter before, 
and it is pretty tricky. In my experience you have to be very careful about (1) 
 how reset() calls propagate, and (2) the signal to switch behavior.  EG you 
probably want to be able to call incrementToken() on either of two different 
upstream filters that both share the same input, based on the value of some 
attribute. However, by the time you see the attribute, it is already too late 
to change! So you have to introduce a delay token that only carries this 
"switching" info. 

> WordDelimiterFilter should pass through terms marked as keywords
> 
>
> Key: LUCENE-8265
> URL: https://issues.apache.org/jira/browse/LUCENE-8265
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mike Sokolov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This will help in cases where some terms containing separator characters 
> should be split, but others should not.  For example, this will enable a 
> filter that identifies things that look like fractions and identifies them as 
> keywords so that 1/2 does not become 12, while doing splitting and joining on 
> terms that look like part numbers containing slashes, eg something like 
> "sn-999123/1" might sometimes be written "sn-999123-1".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8269) Detach downstream classes from IndexWriter

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448275#comment-16448275
 ] 

Michael McCandless commented on LUCENE-8269:


+1, great cleanup.

> Detach downstream classes from IndexWriter
> --
>
> Key: LUCENE-8269
> URL: https://issues.apache.org/jira/browse/LUCENE-8269
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8269.patch
>
>
> IndexWriter today is shared with many classes like BufferedUpdateStream,
> DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
> on the writer instance or assert that the current thread doesn't hold a lock.
> This makes it very difficult to have a manageable threading model.
> 
> This change separates out the IndexWriter from those classes and makes 
> them all
> independent of IW. IW now implements a new interface for DocumentsWriter 
> to communicate
> on failed or successful flushes and tragic events. This allows IW to make 
> it's critical
> methods private and execute all lock critical actions on it's private 
> queue that ensures
> that the IW lock is not held. Follow-up changes will try to detach more 
> code like
> publishing flushed segments to ensure we never call back into IW in an 
> uncontrolled way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448272#comment-16448272
 ] 

Robert Muir commented on LUCENE-8267:
-

There are a lot of other alternatives to putting data in heap memory directly 
in the postings format.

The best (IMO) is for the user to use MMapDirectory.preload with the standard 
index format. This way it doesn't impact their java heap and they use supported 
index format. Users can also use RAMDirectory/FileSwitchDirectory to load 
specified files into heap. 

Finally, users can use FSTPostingsFormat which will load *term dictionary only* 
into heap fst. This is way different than Memory/Direct which load not only 
terms but also postings lists and positions and stuff all into heap RAM.

So i don't really see any technical merit for your objection: there are many 
other ways to have a ram-resident terms dictionary, many of them better than 
the inefficient Memory/Direct formats.

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8265) WordDelimiterFilter should pass through terms marked as keywords

[Mike Sokolov (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448257#comment-16448257
 ] 

Mike Sokolov commented on LUCENE-8265:
--

Good point [~khitrin]. I agree, we should add an option. I'll post a change 
later today I hope.

[~romseygeek] that idea is really cool, but I think it is more complexity than 
I want to take on for this issue? I have tried making a wrapping filter before, 
and it is pretty tricky. In my experience you have to be very careful about (1) 
 how reset() calls propagate, and (2) the signal to switch behavior.  EG you 
probably want to be able to call incrementToken() on either of two different 
upstream filters that both share the same input, based on the value of some 
attribute. However, by the time you see the attribute, it is already too late 
to change! So you have to introduce a delay token that only carries this 
"switching" info. 

> WordDelimiterFilter should pass through terms marked as keywords
> 
>
> Key: LUCENE-8265
> URL: https://issues.apache.org/jira/browse/LUCENE-8265
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mike Sokolov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This will help in cases where some terms containing separator characters 
> should be split, but others should not.  For example, this will enable a 
> filter that identifies things that look like fractions and identifies them as 
> keywords so that 1/2 does not become 12, while doing splitting and joining on 
> terms that look like part numbers containing slashes, eg something like 
> "sn-999123/1" might sometimes be written "sn-999123-1".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448249#comment-16448249
 ] 

David Smiley commented on LUCENE-8267:
--

bq. given that you know that you are using your veto here we are already in a 
terrible position to have any conversation

Do you mean to say I should have said all I said without voting first?  Lets 
have a conversation!  (we _are_ having a conversation)

bq.  we will have a super hard time adding stuff. It creates fear driven 
decisions.

I don't understand this point of view; can you please elaborate?  Fear of what?

bq.  Can you quantify the "it's nice"?

Yes, I shall do that.  My preferred route to do this is find an existing user 
of the "Solr Text Tagger" who can experiment with the postingsFormat setting to 
try a comparison with the default format.  Failing that, I'll create a 
benchmark using that project.

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8207) Modernise cloud tab on Admin UI

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-8207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448243#comment-16448243
 ] 

Jan Høydahl commented on SOLR-8207:
---

Ok, I got a bit further, see [GitHub Pull Request 
#360|https://github.com/apache/lucene-solr/pull/360] for code. The screenshot 
below is a *live* view of data captured from 6 nodes running on my Mac:
 * Proxying SystemInofHandler call and MetricsHandler call through localhost to 
all other Solr nodes
 * Pulling various data and metrics from the APIs and plotting in the table
 ** Under node name is server-level data such as OS, version, JVM, physical 
RAM, CPUs
 ** Uptime is JVM uptime for this Solr node
 ** Load, CPU & heap as reported by system handler (per-process)
 ** Disk is based on {{CONTAINER.fs.totalSpace}} - not per core. Not sure if we 
get {{solr.data.dir}} space here as well as {{solr.solr.home}} space??
 ** GC is number of major collections 
(solr.jvm.{{gc.ConcurrentMarkSweep.countper}}) 5min since JVM start and the 
same for minor collections ({{solr.jvm.gc.ParNew.count}})
 ** Requests is from 
{{solr.jetty.org.eclipse.jetty.server.handler.DefaultHandler.get-requests}}, 
i.e. not per core but for jetty overall, and we show requests per minute 1min 
rate and 95 percentile latency
 ** Collections and cores simply list what runs on the node. Cores link to the 
Admin of that node.
 * Some mouse-overs (title attr) for more details on many metrics. Plan to also 
create a CSS popup with the raw JSON for each node

!nodes-tab-real.png|width=1100!

TODO:
 * Limit number of collections/cores displayed, now all are listed
 * Add a refresh button
 * Add possibility to sort by other columns
 * Add some graphical gauges in place of the percentages??
 * Should the collection names link somewhere?
 * Should we show #docs somewhere? Per core or per node? As well as avg 
size/doc?

Feel free to take it for a spin by cloning the Github fork or checking out [the 
patch 
file|https://patch-diff.githubusercontent.com/raw/apache/lucene-solr/pull/360.patch]

> Modernise cloud tab on Admin UI
> ---
>
> Key: SOLR-8207
> URL: https://issues.apache.org/jira/browse/SOLR-8207
> Project: Solr
>  Issue Type: Improvement
>  Components: Admin UI
>Affects Versions: 5.3
>Reporter: Upayavira
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: nodes-tab-real.png, nodes-tab.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The various sub-tabs of the "Cloud tab" were designed before anyone was 
> making real use of SolrCloud, and when we didn't really know the use-cases we 
> would need to support. I would argue that, whilst they are pretty (and 
> clever) they aren't really fit for purpose (with the exception of tree view).
> Issues:
> * Radial view doesn't scale beyond a small number of nodes/collections
> * Paging on the graph view is based on collections - so a collection with 
> many replicas won't be subject to pagination
> * The Dump feature is kinda redundant and should be removed
> * There is now a major overlap in functionality with the new Collections tab
> What I'd propose is that we:
>  * promote the tree tab to top level
>  * remove the graph views and the dump tab
>  * add a new Nodes tab
> This nodes tab would complement the collections tab - showing nodes, and 
> their associated replicas/collections. From this view, it would be possible 
> to add/remove replicas and to see the status of nodes. It would also be 
> possible to filter nodes by status: "show me only up nodes", "show me nodes 
> that are in trouble", "show me nodes that have leaders on them", etc.
> Presumably, if we have APIs to support it, we might have a "decommission 
> node" option, that would ensure that no replicas on this node are leaders, 
> and then remove all replicas from the node, ready for it to be removed from 
> the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8270) Remove MatchesIterator.term()

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448219#comment-16448219
 ] 

Adrien Grand commented on LUCENE-8270:
--

+1

> Remove MatchesIterator.term()
> -
>
> Key: LUCENE-8270
> URL: https://issues.apache.org/jira/browse/LUCENE-8270
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8270.patch
>
>
> As discussed on LUCENE-8268, we don't have a clear use-case for this yet, and 
> it's complicating adding Matches to phrase queries, so let's just remove it 
> for now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448218#comment-16448218
 ] 

Adrien Grand commented on LUCENE-8268:
--

What I meant is that if you run "(a OR b) NEAR c" with an IntervalQuery, you 
would only have one query, the top-level IntervalQuery since inner nodes are 
IntervalsSource instances. So you won't be able to distinguish matches against 
"a" from matches against "b".

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch, LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8270) Remove MatchesIterator.term()

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8270:
--
Attachment: LUCENE-8270.patch

> Remove MatchesIterator.term()
> -
>
> Key: LUCENE-8270
> URL: https://issues.apache.org/jira/browse/LUCENE-8270
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8270.patch
>
>
> As discussed on LUCENE-8268, we don't have a clear use-case for this yet, and 
> it's complicating adding Matches to phrase queries, so let's just remove it 
> for now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk1.8.0_144) - Build # 560 - Still unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/560/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

15 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 8

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for 
intGSF: 8
at 
__randomizedtesting.SeedInfo.seed([AEFAE33CF336BE3:9154C06B826B59BD]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:379)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
at 

[jira] [Created] (LUCENE-8270) Remove MatchesIterator.term()

[Alan Woodward (JIRA)]

Alan Woodward created LUCENE-8270:
-

 Summary: Remove MatchesIterator.term()
 Key: LUCENE-8270
 URL: https://issues.apache.org/jira/browse/LUCENE-8270
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Alan Woodward
Assignee: Alan Woodward


As discussed on LUCENE-8268, we don't have a clear use-case for this yet, and 
it's complicating adding Matches to phrase queries, so let's just remove it for 
now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #360: SOLR-8207 Admin UI Nodes tab

[janhoy]

GitHub user janhoy opened a pull request:

https://github.com/apache/lucene-solr/pull/360

SOLR-8207 Admin UI Nodes tab

New nodes tab

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cominvent/lucene-solr solr8207-ui-nodes-tab

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/360.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #360


commit 88765050e4234cc907885afc19fa3964334a284c
Author: Jan Høydahl 
Date:   2018-04-13T13:54:33Z

SOLR-8207: First mock version of the new nodes tab

commit a2aabdf64e68f54f2ddcfd80a53160025740a2d1
Author: Jan Høydahl 
Date:   2018-04-18T08:40:00Z

Proxy request to node WIP

commit 05ec26e8d1abfcc6c1690c92326ab846bdd6669b
Author: Jan Høydahl 
Date:   2018-04-20T14:50:22Z

Merge branch 'master' into solr8207-ui-nodes-tab

commit 256ae42c14d5128248a63e71112b7825c7ba92ad
Author: Jan Høydahl 
Date:   2018-04-20T23:55:54Z

Proxying system info call to all live nodes and fill some data in table

commit c6e168d121acfc48b1199a3377863509f882f186
Author: Jan Høydahl 
Date:   2018-04-20T23:57:04Z

Merge remote-tracking branch 'origin/master' into solr8207-ui-nodes-tab

commit 779af18d82e8279e72363039d9c6c0ce13833608
Author: Jan Høydahl 
Date:   2018-04-23T10:32:53Z

Get back graphs

commit b39fe7e2554f6ddd7f373f20053416804edcf743
Author: Jan Høydahl 
Date:   2018-04-23T10:33:11Z

More mouseover info in table

commit 792d4b7d06fa8fde5eb35ea797fd1a72f1c1c138
Author: Jan Høydahl 
Date:   2018-04-23T10:39:39Z

Icon for the nodes entry

commit ba9cf1054471b9b56aaa81a76ca73eceb2acdecc
Author: Jan Høydahl 
Date:   2018-04-23T11:29:05Z

Refactored proxying to external nodes into separate class

commit 26b8e5b4d06d837cf90e9ac5127dbb43ebe71dd0
Author: Jan Høydahl 
Date:   2018-04-23T11:40:02Z

Added "nodes" support also to MetricsHandler

commit 59965c2b5a29985fc0ac355c19c76fe3bc37448b
Author: Jan Høydahl 
Date:   2018-04-23T13:23:18Z

Adding disk space and request rate metrics to table
Moved calculations from HTML to cloud.js

commit 33f4cab0e1c2fd8c6f35ac2a35073ea0ce27183d
Author: Jan Høydahl 
Date:   2018-04-23T14:25:50Z

Added GC to the table




---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448211#comment-16448211
 ] 

Alan Woodward commented on LUCENE-8268:
---

bq. since inner nodes are not queries? 

Sorry, I'm not following here - inner nodes are always generated by a Weight, 
which in turn has a parent query.

bq. Maybe we should just remove this method for now

I think that may be the most sensible, I'll close this as Won't Fix and open a 
new issue to just remove it entirely.

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch, LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward resolved LUCENE-8268.
---
Resolution: Won't Fix

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch, LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8207) Modernise cloud tab on Admin UI

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-8207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-8207:
--
Attachment: nodes-tab-real.png

> Modernise cloud tab on Admin UI
> ---
>
> Key: SOLR-8207
> URL: https://issues.apache.org/jira/browse/SOLR-8207
> Project: Solr
>  Issue Type: Improvement
>  Components: Admin UI
>Affects Versions: 5.3
>Reporter: Upayavira
>Assignee: Jan Høydahl
>Priority: Major
> Attachments: nodes-tab-real.png, nodes-tab.png
>
>
> The various sub-tabs of the "Cloud tab" were designed before anyone was 
> making real use of SolrCloud, and when we didn't really know the use-cases we 
> would need to support. I would argue that, whilst they are pretty (and 
> clever) they aren't really fit for purpose (with the exception of tree view).
> Issues:
> * Radial view doesn't scale beyond a small number of nodes/collections
> * Paging on the graph view is based on collections - so a collection with 
> many replicas won't be subject to pagination
> * The Dump feature is kinda redundant and should be removed
> * There is now a major overlap in functionality with the new Collections tab
> What I'd propose is that we:
>  * promote the tree tab to top level
>  * remove the graph views and the dump tab
>  * add a new Nodes tab
> This nodes tab would complement the collections tab - showing nodes, and 
> their associated replicas/collections. From this view, it would be possible 
> to add/remove replicas and to see the status of nodes. It would also be 
> possible to filter nodes by status: "show me only up nodes", "show me nodes 
> that are in trouble", "show me nodes that have leaders on them", etc.
> Presumably, if we have APIs to support it, we might have a "decommission 
> node" option, that would ensure that no replicas on this node are leaders, 
> and then remove all replicas from the node, ready for it to be removed from 
> the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8267) Remove memory codecs from the codebase

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448208#comment-16448208
 ] 

Simon Willnauer edited comment on LUCENE-8267 at 4/23/18 2:19 PM:
--

{quote}If we are going to make it harder to remove stuff, I have no problem 
being the one to make it equally harder to add stuff.
{quote}

I agree this is one of these issues that we have to face. if we put the bar 
very high to remove stuff that is not mainstream then we will have a super hard 
time adding stuff. It creates fear driven decisions. It sucks I agree with 
[~rcmuir] 100% here.
  
{quote}
-1 sorry. I've used the MemoryPostingsFormat for a text-tagging use-case where 
there are intense lookups against the terms dictionary. It's highly beneficial 
to have the terms dictionary be entirely memory resident, albeit in a compact 
FST. The issue description mentions "We don't use those memory codecs anywhere 
outside of tests" – this should be no surprise as it's not the default codec. 
I'm sure it may be hard to gauge the level of use of something outside of 
core-Lucene. When we ponder removing something that Lucene doesn't even _need_, 
I propose we raise the issue more openly to the community. Perhaps the question 
could be proposed in CHANGES.txt and/or release announcements to solicit 
community input?
{quote}
 
 given that you know that you are using your veto here we are already in a 
terrible position to have any conversation. Can you quantify the "it's nice"? 
since there are alternatives that (standard codec) can you go and provide some 
numbers. We should not use vetos based on non-quantifiable arguments IMO. We 
can go and ask the community but I don't expect much useful outcome, most of 
the folks don't know what they are using here and there. Nevertheless, I am 
happy to send a mail to dev to get this information. 


was (Author: simonw):
{quote}
If we are going to make it harder to remove stuff, I have no problem being the 
one to make it equally harder to add stuff.
 \{quote}
 
I agree this is one of these issues that we have to face. if we put the bar 
very high to remove stuff that is not mainstream then we will have a super hard 
time adding stuff. It creates fear driven decisions. It sucks I agree with 
[~rcmuir] 100% here.
 
{quote}
-1 sorry. I've used the MemoryPostingsFormat for a text-tagging use-case where 
there are intense lookups against the terms dictionary. It's highly beneficial 
to have the terms dictionary be entirely memory resident, albeit in a compact 
FST. The issue description mentions "We don't use those memory codecs anywhere 
outside of tests" – this should be no surprise as it's not the default codec. 
I'm sure it may be hard to gauge the level of use of something outside of 
core-Lucene. When we ponder removing something that Lucene doesn't even _need_, 
I propose we raise the issue more openly to the community. Perhaps the question 
could be proposed in CHANGES.txt and/or release announcements to solicit 
community input?
{quote}
 
given that you know that you are using your veto here we are already in a 
terrible position to have any conversation. Can you quantify the "it's nice"? 
since there are alternatives that (standard codec) can you go and provide some 
numbers. We should not use vetos based on non-quantifiable arguments IMO. We 
can go and ask the community but I don't expect much useful outcome, most of 
the folks don't know what they are using here and there. Nevertheless, I am 
happy to send a mail to dev to get this information. 

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448208#comment-16448208
 ] 

Simon Willnauer commented on LUCENE-8267:
-

{quote}
If we are going to make it harder to remove stuff, I have no problem being the 
one to make it equally harder to add stuff.
 \{quote}
 
I agree this is one of these issues that we have to face. if we put the bar 
very high to remove stuff that is not mainstream then we will have a super hard 
time adding stuff. It creates fear driven decisions. It sucks I agree with 
[~rcmuir] 100% here.
 
{quote}
-1 sorry. I've used the MemoryPostingsFormat for a text-tagging use-case where 
there are intense lookups against the terms dictionary. It's highly beneficial 
to have the terms dictionary be entirely memory resident, albeit in a compact 
FST. The issue description mentions "We don't use those memory codecs anywhere 
outside of tests" – this should be no surprise as it's not the default codec. 
I'm sure it may be hard to gauge the level of use of something outside of 
core-Lucene. When we ponder removing something that Lucene doesn't even _need_, 
I propose we raise the issue more openly to the community. Perhaps the question 
could be proposed in CHANGES.txt and/or release announcements to solicit 
community input?
{quote}
 
given that you know that you are using your veto here we are already in a 
terrible position to have any conversation. Can you quantify the "it's nice"? 
since there are alternatives that (standard codec) can you go and provide some 
numbers. We should not use vetos based on non-quantifiable arguments IMO. We 
can go and ask the community but I don't expect much useful outcome, most of 
the folks don't know what they are using here and there. Nevertheless, I am 
happy to send a mail to dev to get this information. 

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448190#comment-16448190
 ] 

Adrien Grand commented on LUCENE-8268:
--

Returning a query means that intervals won't be able to say what matched, since 
inner nodes are not queries? Maybe we should just remove this method for now 
and add something back when use-cases for it are clearer?

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch, LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448159#comment-16448159
 ] 

Alan Woodward commented on LUCENE-8268:
---

Here's a patch removing term() and adding getLeafQuery()

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch, LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8268:
--
Attachment: LUCENE-8268.patch

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch, LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8265) WordDelimiterFilter should pass through terms marked as keywords

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448145#comment-16448145
 ] 

Alan Woodward commented on LUCENE-8265:
---

I wonder if there's a better way of handling this than using the 
KeywordAttribute, which as Nikolay says is heavily overloaded.  Would it be 
possible to somehow code up a TokenFilter that wraps another TokenFilter, and 
bypasses the wrapped filter if a certain condition is met?

> WordDelimiterFilter should pass through terms marked as keywords
> 
>
> Key: LUCENE-8265
> URL: https://issues.apache.org/jira/browse/LUCENE-8265
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mike Sokolov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This will help in cases where some terms containing separator characters 
> should be split, but others should not.  For example, this will enable a 
> filter that identifies things that look like fractions and identifies them as 
> keywords so that 1/2 does not become 12, while doing splitting and joining on 
> terms that look like part numbers containing slashes, eg something like 
> "sn-999123/1" might sometimes be written "sn-999123-1".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448143#comment-16448143
 ] 

Robert Muir commented on LUCENE-8267:
-

If we are going to make it harder to remove stuff, I have no problem being the 
one to make it equally harder to add stuff.

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8265) WordDelimiterFilter should pass through terms marked as keywords

[Nikolay Khitrin (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448135#comment-16448135
 ] 

Nikolay Khitrin commented on LUCENE-8265:
-

This is the breaking change.

For example keyword attribute can be used for bypass stemming (as mentioned in 
KeywordAttribute javadoc) _after_ WordDelimiterFilter.

Should be at least marked as breaking in changelog. Might be better solution is 
to provide this as an option for delimiter filter.

> WordDelimiterFilter should pass through terms marked as keywords
> 
>
> Key: LUCENE-8265
> URL: https://issues.apache.org/jira/browse/LUCENE-8265
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mike Sokolov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This will help in cases where some terms containing separator characters 
> should be split, but others should not.  For example, this will enable a 
> filter that identifies things that look like fractions and identifies them as 
> keywords so that 1/2 does not become 12, while doing splitting and joining on 
> terms that look like part numbers containing slashes, eg something like 
> "sn-999123/1" might sometimes be written "sn-999123-1".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448123#comment-16448123
 ] 

David Smiley commented on LUCENE-8267:
--

-1 sorry. I've used the MemoryPostingsFormat for a text-tagging use-case where 
there are intense lookups against the terms dictionary.  It's highly beneficial 
to have the terms dictionary be entirely memory resident, albeit in a compact 
FST.  The issue description mentions "We don't use those memory codecs anywhere 
outside of tests" -- this should be no surprise as it's not the default codec.  
I'm sure it may be hard to gauge the level of use of something outside of 
core-Lucene.  When we ponder removing something that Lucene doesn't even 
_need_, I propose we raise the issue more openly to the community.  Perhaps the 
question could be proposed in CHANGES.txt and/or release announcements to 
solicit community input?

Perhaps BaseRangeFieldQueryTestCase.verify should ascertain if the postings 
format is a known "memory" postings format (of which there are several, to 
include "Direct"), and if so then use JUnit's Assume to bail out?  If this is 
hard to do, we ought to add a convenience method to make it easier.

Speaking of memory postings formats, I'm in favor of the Direct postings format 
going away since it ought to be re-imagined as some sort of read-time 
FilterCodecReader that does not require an index format.  Credit to Alan for 
that idea years ago.  Though that's more of a re-orientation of something that 
exists rather than saying it should go away entirely.

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms

[Elizabeth Haubert (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448067#comment-16448067
 ] 

Elizabeth Haubert commented on SOLR-12243:
--

OK. Patch pending  to handle that else if.

Duplicated the tests from TestEdismaxQParser.testPsPf with a multiterm synonym 
"bar, tropical cyclone", which is mostly passing except for those where the 
queryphrase is re-ordered with slop.  Default on those SpanNear queries is 
inorder=true, which is very different semantics from existing phrase queries 
with slop.

Looking into that.

 

 

 

> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Priority: Major
>
> synonyms.txt:
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> request handler:
> 
>  
> 
>  edismax
>   0.4
>  title^100
>  title~20^5000
>  title~11
>  title~22^1000
>  text
>  
>  3-1 6-3 930%
>  *:*
>  25
> 
>  
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin"  against the 
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not 
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction 
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin 
> dose" or pf3:"aspirin dose ?"
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Windows (64bit/jdk-11-ea+5) - Build # 7283 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7283/
Java: 64bit/jdk-11-ea+5 -XX:-UseCompressedOops -XX:+UseSerialGC

14 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart

Error Message:
null

Stack Trace:
java.lang.NumberFormatException: null
at 
__randomizedtesting.SeedInfo.seed([CC90A3587E509C80:146767BCD58B5EDC]:0)
at java.base/java.lang.Integer.parseInt(Integer.java:614)
at java.base/java.lang.Integer.parseInt(Integer.java:770)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:687)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:841)


FAILED:  org.apache.solr.handler.TestSQLHandler.doTest

Error Message:
--> http://127.0.0.1:53914/collection1_shard2_replica_n41:Failed to execute 
sqlQuery 'select id, field_i, str_s from collection1 where (text='()' OR 
text='') AND text='' order by field_i desc' against JDBC 

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448060#comment-16448060
 ] 

Robert Muir commented on LUCENE-8267:
-

+1

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8269) Detach downstream classes from IndexWriter

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448026#comment-16448026
 ] 

Simon Willnauer commented on LUCENE-8269:
-

[https://github.com/s1monw/lucene-solr/pull/13/] /cc [~mikemccand]

> Detach downstream classes from IndexWriter
> --
>
> Key: LUCENE-8269
> URL: https://issues.apache.org/jira/browse/LUCENE-8269
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8269.patch
>
>
> IndexWriter today is shared with many classes like BufferedUpdateStream,
> DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
> on the writer instance or assert that the current thread doesn't hold a lock.
> This makes it very difficult to have a manageable threading model.
> 
> This change separates out the IndexWriter from those classes and makes 
> them all
> independent of IW. IW now implements a new interface for DocumentsWriter 
> to communicate
> on failed or successful flushes and tragic events. This allows IW to make 
> it's critical
> methods private and execute all lock critical actions on it's private 
> queue that ensures
> that the IW lock is not held. Follow-up changes will try to detach more 
> code like
> publishing flushed segments to ensure we never call back into IW in an 
> uncontrolled way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8269) Detach downstream classes from IndexWriter

[Simon Willnauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-8269:

Attachment: LUCENE-8269.patch

> Detach downstream classes from IndexWriter
> --
>
> Key: LUCENE-8269
> URL: https://issues.apache.org/jira/browse/LUCENE-8269
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8269.patch
>
>
> IndexWriter today is shared with many classes like BufferedUpdateStream,
> DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
> on the writer instance or assert that the current thread doesn't hold a lock.
> This makes it very difficult to have a manageable threading model.
> 
> This change separates out the IndexWriter from those classes and makes 
> them all
> independent of IW. IW now implements a new interface for DocumentsWriter 
> to communicate
> on failed or successful flushes and tragic events. This allows IW to make 
> it's critical
> methods private and execute all lock critical actions on it's private 
> queue that ensures
> that the IW lock is not held. Follow-up changes will try to detach more 
> code like
> publishing flushed segments to ensure we never call back into IW in an 
> uncontrolled way.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8269) Detach downstream classes from IndexWriter

[Simon Willnauer (JIRA)]

Simon Willnauer created LUCENE-8269:
---

 Summary: Detach downstream classes from IndexWriter
 Key: LUCENE-8269
 URL: https://issues.apache.org/jira/browse/LUCENE-8269
 Project: Lucene - Core
  Issue Type: Improvement
Affects Versions: 7.4, master (8.0)
Reporter: Simon Willnauer
 Fix For: 7.4, master (8.0)


IndexWriter today is shared with many classes like BufferedUpdateStream,
DocumentsWriter and DocumentsWriterPerThread. Some of them even acquire locks
on the writer instance or assert that the current thread doesn't hold a lock.
This makes it very difficult to have a manageable threading model.

This change separates out the IndexWriter from those classes and makes them 
all
independent of IW. IW now implements a new interface for DocumentsWriter to 
communicate
on failed or successful flushes and tragic events. This allows IW to make 
it's critical
methods private and execute all lock critical actions on it's private queue 
that ensures
that the IW lock is not held. Follow-up changes will try to detach more 
code like
publishing flushed segments to ensure we never call back into IW in an 
uncontrolled way.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 546 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/546/

[...truncated 28 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/206/consoleText

[repro] Revision: b16b380b2cec1614597df6a045599307628988c2

[repro] Ant options: -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9
[repro] Repro line:  ant test  -Dtestcase=IndexSizeTriggerTest 
-Dtests.method=testSplitIntegration -Dtests.seed=7A0A6677597C2E2B 
-Dtests.multiplier=2 -Dtests.locale=ca-ES-VALENCIA -Dtests.timezone=Egypt 
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
e8c36f489e25af7194c5b9b3ac8355db5a6132cc
[repro] git fetch

[...truncated 2 lines...]
[repro] git checkout b16b380b2cec1614597df6a045599307628988c2

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   IndexSizeTriggerTest
[repro] ant compile-test

[...truncated 3316 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.IndexSizeTriggerTest" -Dtests.showOutput=onerror 
-DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 
-Dtests.seed=7A0A6677597C2E2B -Dtests.multiplier=2 
-Dtests.locale=ca-ES-VALENCIA -Dtests.timezone=Egypt -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII

[...truncated 4669 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[repro] git checkout e8c36f489e25af7194c5b9b3ac8355db5a6132cc

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 5 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8998) JSON Facet API child roll-ups

[Lucene/Solr QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448006#comment-16448006
 ] 

Lucene/Solr QA commented on SOLR-8998:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  1m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  1m 33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  1m 33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 55m 25s{color} 
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 10s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.search.QueryEqualityTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-8998 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12920177/SOLR-8998.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene1-us-west 3.13.0-88-generic #135-Ubuntu SMP Wed Jun 8 
21:10:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / e8c36f4 |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on April 8 2014 |
| Default Java | 1.8.0_152 |
| unit | 
https://builds.apache.org/job/PreCommit-SOLR-Build/65/artifact/out/patch-unit-solr_core.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-SOLR-Build/65/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output | 
https://builds.apache.org/job/PreCommit-SOLR-Build/65/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> JSON Facet API child roll-ups
> -
>
> Key: SOLR-8998
> URL: https://issues.apache.org/jira/browse/SOLR-8998
> Project: Solr
>  Issue Type: New Feature
>  Components: Facet Module
>Reporter: Yonik Seeley
>Priority: Major
> Attachments: SOLR-8998.patch, SOLR_8998.patch, SOLR_8998.patch, 
> SOLR_8998.patch
>
>
> The JSON Facet API currently has the ability to map between parents and 
> children ( see http://yonik.com/solr-nested-objects/ )
> This issue is about adding a true rollup ability where parents would take on 
> derived values from their children.  The most important part (and the most 
> difficult part) will be the external API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8265) WordDelimiterFilter should pass through terms marked as keywords

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447980#comment-16447980
 ] 

Michael McCandless commented on LUCENE-8265:


Thanks [~sokolov]; the PR looks great; I'll wait a day or so and then push!

> WordDelimiterFilter should pass through terms marked as keywords
> 
>
> Key: LUCENE-8265
> URL: https://issues.apache.org/jira/browse/LUCENE-8265
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Mike Sokolov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This will help in cases where some terms containing separator characters 
> should be split, but others should not.  For example, this will enable a 
> filter that identifies things that look like fractions and identifies them as 
> keywords so that 1/2 does not become 12, while doing splitting and joining on 
> terms that look like part numbers containing slashes, eg something like 
> "sn-999123/1" might sometimes be written "sn-999123-1".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447948#comment-16447948
 ] 

Simon Willnauer commented on LUCENE-8268:
-

{quote}
So at the moment there isn't anything that actually uses this. My reason for 
adding it was to make it possible to identify the leaf query that returned each 
position, but maybe it would be a better idea to remove terms() entirely, and 
add a getLeafQuery() method instead?
{quote}

hard to tell since I don't know the API well enough. But if this is the 
purpose, I agree.

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 571 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/571/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeLost

Error Message:
org.apache.solr.common.SolrException: 

Stack Trace:
org.apache.solr.common.SolrException: org.apache.solr.common.SolrException: 
at 
__randomizedtesting.SeedInfo.seed([369A1C2DDF9782A6:898FD2D35C7DE720]:0)
at 
org.apache.solr.common.cloud.rule.ImplicitSnitch.getTags(ImplicitSnitch.java:78)
at 
org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getNodeValues(SolrClientNodeStateProvider.java:111)
at 
org.apache.solr.cloud.autoscaling.ComputePlanActionTest.printState(ComputePlanActionTest.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: 
at 
org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$AutoScalingSnitch.getRemoteInfo(SolrClientNodeStateProvider.java:250)
at 
org.apache.solr.common.cloud.rule.ImplicitSnitch.getTags(ImplicitSnitch.java:76)
... 39 more
Caused by: 

[jira] [Resolved] (LUCENE-8266) Standard polygon is created with bogus tile

[Ignacio Vera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ignacio Vera resolved LUCENE-8266.
--
   Resolution: Fixed
Fix Version/s: master (8.0)
   7.4
   6.7

> Standard polygon is created with bogus tile
> ---
>
> Key: LUCENE-8266
> URL: https://issues.apache.org/jira/browse/LUCENE-8266
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Ignacio Vera
>Priority: Major
> Fix For: 6.7, 7.4, master (8.0)
>
> Attachments: LUCENE-8266-test.patch, LUCENE-8266.patch
>
>
> Beasting polygon tests produces the time to time errors due to standard 
> polygons with a bogus tile. We need to improve the checks for those 
> situations and throw a {{TileException}} in those cases so we fall back to 
> complex polygons.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8266) Standard polygon is created with bogus tile

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447874#comment-16447874
 ] 

ASF subversion and git services commented on LUCENE-8266:
-

Commit 7026095f61a4b36daf83b5d468baf99d226ac858 in lucene-solr's branch 
refs/heads/branch_6x from [~ivera]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7026095 ]

LUCENE-8266: Detect bogus tiles when creating a standard polygon and throw a 
TileException


> Standard polygon is created with bogus tile
> ---
>
> Key: LUCENE-8266
> URL: https://issues.apache.org/jira/browse/LUCENE-8266
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Ignacio Vera
>Priority: Major
> Attachments: LUCENE-8266-test.patch, LUCENE-8266.patch
>
>
> Beasting polygon tests produces the time to time errors due to standard 
> polygons with a bogus tile. We need to improve the checks for those 
> situations and throw a {{TileException}} in those cases so we fall back to 
> complex polygons.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8266) Standard polygon is created with bogus tile

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447870#comment-16447870
 ] 

ASF subversion and git services commented on LUCENE-8266:
-

Commit c97d81d94f3091b9475e203a1747bbea8618b9c4 in lucene-solr's branch 
refs/heads/branch_7x from [~ivera]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c97d81d ]

LUCENE-8266: Detect bogus tiles when creating a standard polygon and throw a 
TileException


> Standard polygon is created with bogus tile
> ---
>
> Key: LUCENE-8266
> URL: https://issues.apache.org/jira/browse/LUCENE-8266
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Ignacio Vera
>Priority: Major
> Attachments: LUCENE-8266-test.patch, LUCENE-8266.patch
>
>
> Beasting polygon tests produces the time to time errors due to standard 
> polygons with a bogus tile. We need to improve the checks for those 
> situations and throw a {{TileException}} in those cases so we fall back to 
> complex polygons.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8266) Standard polygon is created with bogus tile

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447869#comment-16447869
 ] 

ASF subversion and git services commented on LUCENE-8266:
-

Commit e8c36f489e25af7194c5b9b3ac8355db5a6132cc in lucene-solr's branch 
refs/heads/master from [~ivera]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e8c36f4 ]

LUCENE-8266: Detect bogus tiles when creating a standard polygon and throw a 
TileException


> Standard polygon is created with bogus tile
> ---
>
> Key: LUCENE-8266
> URL: https://issues.apache.org/jira/browse/LUCENE-8266
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Ignacio Vera
>Priority: Major
> Attachments: LUCENE-8266-test.patch, LUCENE-8266.patch
>
>
> Beasting polygon tests produces the time to time errors due to standard 
> polygons with a bogus tile. We need to improve the checks for those 
> situations and throw a {{TileException}} in those cases so we fall back to 
> complex polygons.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (LUCENE-8254) LRUQueryCache can leak locks

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward reopened LUCENE-8254:
---

> LRUQueryCache can leak locks
> 
>
> Key: LUCENE-8254
> URL: https://issues.apache.org/jira/browse/LUCENE-8254
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.3.1
>
> Attachments: LUCENE-8254.patch, LUCENE-8254.patch
>
>
> If a QueryCache is shared between two searchers, one of which has an 
> IndexReader with no CacheHelper, then CachingWrapperWeight can leak locks in 
> scorerSupplier() and bulkScorer().  This can cause the IndexReader that does 
> have a CacheHelper to hang on close.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8254) LRUQueryCache can leak locks

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8254:
--
Fix Version/s: (was: 7.4)
   7.3.1

> LRUQueryCache can leak locks
> 
>
> Key: LUCENE-8254
> URL: https://issues.apache.org/jira/browse/LUCENE-8254
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.3.1
>
> Attachments: LUCENE-8254.patch, LUCENE-8254.patch
>
>
> If a QueryCache is shared between two searchers, one of which has an 
> IndexReader with no CacheHelper, then CachingWrapperWeight can leak locks in 
> scorerSupplier() and bulkScorer().  This can cause the IndexReader that does 
> have a CacheHelper to hang on close.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-8254) LRUQueryCache can leak locks

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward resolved LUCENE-8254.
---
Resolution: Fixed

Re-opened to change the fix version to 7.3.1

> LRUQueryCache can leak locks
> 
>
> Key: LUCENE-8254
> URL: https://issues.apache.org/jira/browse/LUCENE-8254
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Fix For: 7.3.1
>
> Attachments: LUCENE-8254.patch, LUCENE-8254.patch
>
>
> If a QueryCache is shared between two searchers, one of which has an 
> IndexReader with no CacheHelper, then CachingWrapperWeight can leak locks in 
> scorerSupplier() and bulkScorer().  This can cause the IndexReader that does 
> have a CacheHelper to hang on close.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8266) Standard polygon is created with bogus tile

[Karl Wright (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447838#comment-16447838
 ] 

Karl Wright commented on LUCENE-8266:
-

This seems reasonable -- please go ahead and commit.


> Standard polygon is created with bogus tile
> ---
>
> Key: LUCENE-8266
> URL: https://issues.apache.org/jira/browse/LUCENE-8266
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Ignacio Vera
>Priority: Major
> Attachments: LUCENE-8266-test.patch, LUCENE-8266.patch
>
>
> Beasting polygon tests produces the time to time errors due to standard 
> polygons with a bogus tile. We need to improve the checks for those 
> situations and throw a {{TileException}} in those cases so we fall back to 
> complex polygons.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-8266) Standard polygon is created with bogus tile

[Karl Wright (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karl Wright reassigned LUCENE-8266:
---

Assignee: Ignacio Vera

> Standard polygon is created with bogus tile
> ---
>
> Key: LUCENE-8266
> URL: https://issues.apache.org/jira/browse/LUCENE-8266
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ignacio Vera
>Assignee: Ignacio Vera
>Priority: Major
> Attachments: LUCENE-8266-test.patch, LUCENE-8266.patch
>
>
> Beasting polygon tests produces the time to time errors due to standard 
> polygons with a bogus tile. We need to improve the checks for those 
> situations and throw a {{TileException}} in those cases so we fall back to 
> complex polygons.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8264) Allow an option to rewrite all segments

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447804#comment-16447804
 ] 

Uwe Schindler edited comment on LUCENE-8264 at 4/23/18 9:16 AM:


Ah, I was always using a FilterLeafReader with a SlowCodecReaderWrapper 
(because those people needed to add DocValues from UninvertingReader - the one 
from solr). So yes, in my case, the FilterLeafReader war returning {{new 
LeafMetadata(7, Version.LUCENE_7_0_0, null)}} in {{getMetadata()}}. Very easy 
and worked. Broken offsets were not an issue there, if they would have been we 
may have removed them by changing the field metadata from the filter reader.

Alternatively, you can first upgrade to 6.x withe the same construction of 
filter readers, leaving out metadata (I think I did this, because they were 
coming from Lucene 3.6). As all segments are written completely new, you can 
then migrate to 7 without hassle.

I just wanted to say: It's good that we still allow to do the migration, but I 
agree with you: This should not be provided by default in Solr or Lucene. 
People need to know how about the side-effects.


was (Author: thetaphi):
Ah, I was always using a FilterLeafReader with a SlowCodecReaderWrapper 
(because those people needed to add DocValues from UninvertingReader - the one 
from solr). So yes, in my case, the FilterLeafReader war returning {{new 
LeafMetadata(Version.LUCENE_7_0_0, Version.LUCENE_7_0_0, null)}} in 
{{getMetadata()}}. Very easy and worked. Broken offsets were not an issue 
there, if they would have been we may have removed them by changing the field 
metadata from the filter reader.

Alternatively, you can first upgrade to 6.x withe the same construction of 
filter readers, leaving out metadata (I think I did this, because they were 
coming from Lucene 3.6). As all segments are written completely new, you can 
then migrate to 7 without hassle.

I just wanted to say: It's good that we still allow to do the migration, but I 
agree with you: This should not be provided by default in Solr or Lucene. 
People need to know how about the side-effects.

> Allow an option to rewrite all segments
> ---
>
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> For the background, see SOLR-12259.
> There are several use-cases that would be much easier, especially during 
> upgrades, if we could specify that all segments get rewritten. 
> One example: Upgrading 5x->6x->7x. When segments are merged, they're 
> rewritten into the current format. However, there's no guarantee that a 
> particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily 
> be successful.
> How many merge policies support this is an open question. I propose to start 
> with TMP and raise other JIRAs as necessary for other merge policies.
> So far the usual response has been "re-index from scratch", but that's 
> increasingly difficult as systems get larger.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8264) Allow an option to rewrite all segments

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447804#comment-16447804
 ] 

Uwe Schindler edited comment on LUCENE-8264 at 4/23/18 9:13 AM:


Ah, I was always using a FilterLeafReader with a SlowCodecReaderWrapper 
(because those people needed to add DocValues from UninvertingReader - the one 
from solr). So yes, in my case, the FilterLeafReader war returning {{new 
LeafMetadata(Version.LUCENE_7_0_0, Version.LUCENE_7_0_0, null)}} in 
{{getMetadata()}}. Very easy and worked. Broken offsets were not an issue 
there, if they would have been we may have removed them by changing the field 
metadata from the filter reader.

Alternatively, you can first upgrade to 6.x withe the same construction of 
filter readers, leaving out metadata (I think I did this, because they were 
coming from Lucene 3.6). As all segments are written completely new, you can 
then migrate to 7 without hassle.

I just wanted to say: It's good that we still allow to do the migration, but I 
agree with you: This should not be provided by default in Solr or Lucene. 
People need to know how about the side-effects.


was (Author: thetaphi):
Ah, I was always using a FilterLeafReader with a SlowCodecReaderWrapper 
(because those people needed to add DocValues from UninvertingReader - the one 
from solr). So yes, in my case, the FilterLeafReader war returning {{new 
LeafMetadata(Version.LUCENE_7_0_0, Version.LUCENE_7_0_0, null)}} in 
{getMetadata()}. Very easy and worked. Broken offsets were not an issue there, 
if they would have been we may have removed them by changing the field metadata 
from the filter reader.

I just wanted to say: It's good that we still allow to do the migration, but I 
agree with you: This should not be provided by default in Solr or Lucene. 
People need to know how about the side-effects.

> Allow an option to rewrite all segments
> ---
>
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> For the background, see SOLR-12259.
> There are several use-cases that would be much easier, especially during 
> upgrades, if we could specify that all segments get rewritten. 
> One example: Upgrading 5x->6x->7x. When segments are merged, they're 
> rewritten into the current format. However, there's no guarantee that a 
> particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily 
> be successful.
> How many merge policies support this is an open question. I propose to start 
> with TMP and raise other JIRAs as necessary for other merge policies.
> So far the usual response has been "re-index from scratch", but that's 
> increasingly difficult as systems get larger.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8264) Allow an option to rewrite all segments

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447804#comment-16447804
 ] 

Uwe Schindler commented on LUCENE-8264:
---

Ah, I was always using a FilterLeafReader with a SlowCodecReaderWrapper 
(because those people needed to add DocValues from UninvertingReader - the one 
from solr). So yes, in my case, the FilterLeafReader war returning {{new 
LeafMetadata(Version.LUCENE_7_0_0, Version.LUCENE_7_0_0, null)}} in 
{getMetadata()}. Very easy and worked. Broken offsets were not an issue there, 
if they would have been we may have removed them by changing the field metadata 
from the filter reader.

I just wanted to say: It's good that we still allow to do the migration, but I 
agree with you: This should not be provided by default in Solr or Lucene. 
People need to know how about the side-effects.

> Allow an option to rewrite all segments
> ---
>
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> For the background, see SOLR-12259.
> There are several use-cases that would be much easier, especially during 
> upgrades, if we could specify that all segments get rewritten. 
> One example: Upgrading 5x->6x->7x. When segments are merged, they're 
> rewritten into the current format. However, there's no guarantee that a 
> particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily 
> be successful.
> How many merge policies support this is an open question. I propose to start 
> with TMP and raise other JIRAs as necessary for other merge policies.
> So far the usual response has been "re-index from scratch", but that's 
> increasingly difficult as systems get larger.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447802#comment-16447802
 ] 

Alan Woodward commented on LUCENE-8268:
---

bq. Can we add a users of multiple terms?

So at the moment there isn't anything that actually uses this.  My reason for 
adding it was to make it possible to identify the leaf query that returned each 
position, but maybe it would be a better idea to remove terms() entirely, and 
add a getLeafQuery() method instead?


> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8267) Remove memory codecs from the codebase

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447787#comment-16447787
 ] 

Simon Willnauer commented on LUCENE-8267:
-

+1

> Remove memory codecs from the codebase
> --
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Dawid Weiss
>Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random 
> selection of codecs for tests and cause occasional OOMs when a test with huge 
> data is selected. We don't use those memory codecs anywhere outside of tests, 
> it has been suggested to just remove them to avoid maintenance costs and OOMs 
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1814 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1814/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
at 
__randomizedtesting.SeedInfo.seed([974B28D58D535AD1:C4F26A656F42CF2B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:404)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at 
__randomizedtesting.SeedInfo.seed([974B28D58D535AD1:AEC59195A2AC932F]:0)
at 

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447779#comment-16447779
 ] 

Simon Willnauer commented on LUCENE-8268:
-

a couple of questions:

* in _compareBytesRefArrays_ how can you tell that comparing each individual 
term is correct? 
* is _BytesRefIterator_ an option as a return value and would it make sense. 
It's hart do tell without a single user of this. 
* In the current context there is no gain changing this interface. Can we add a 
users of multiple terms?

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447768#comment-16447768
 ] 

Alan Woodward commented on LUCENE-8268:
---

Patch attached.  It changes the signature from {code}BytesRef term(){code} to 
{code}BytesRef[] terms(){code}, and adds a test to ensure that matches at the 
same position are iterated over in term order.

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8264) Allow an option to rewrite all segments

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447767#comment-16447767
 ] 

Simon Willnauer commented on LUCENE-8264:
-

> It worked at least until 7.x. As I said, you can remove offsets if needed. 
> And of course a FilterLeafReader together with SlowCodecReaderWrapper is 
> definitly needed.

I am not so sure about this, at least 
[this|https://github.com/apache/lucene-solr/blob/branch_7x/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java#L2756]
 will fail and it's in there since 7.0



> Allow an option to rewrite all segments
> ---
>
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> For the background, see SOLR-12259.
> There are several use-cases that would be much easier, especially during 
> upgrades, if we could specify that all segments get rewritten. 
> One example: Upgrading 5x->6x->7x. When segments are merged, they're 
> rewritten into the current format. However, there's no guarantee that a 
> particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily 
> be successful.
> How many merge policies support this is an open question. I propose to start 
> with TMP and raise other JIRAs as necessary for other merge policies.
> So far the usual response has been "re-index from scratch", but that's 
> increasingly difficult as systems get larger.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-11-ea+5) - Build # 1781 - Still Failing!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1781/
Java: 64bit/jdk-11-ea+5 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.update.TransactionLogTest.testBigLastAddSize

Error Message:
 Mockito cannot mock this class: class org.apache.solr.update.AddUpdateCommand. 
 Mockito can only mock non-private & non-final classes. If you're not sure why 
you're getting this error, please report to the mailing list.   Java
   : 11 JVM vendor name: "Oracle Corporation" JVM vendor version : 11-ea+5 
JVM name   : OpenJDK 64-Bit Server VM JVM version: 11-ea+5 JVM 
info   : mixed mode OS name: Linux OS version : 
4.13.0-37-generic   Underlying exception : 
java.lang.UnsupportedOperationException: Cannot define class using reflection

Stack Trace:
org.mockito.exceptions.base.MockitoException: 
Mockito cannot mock this class: class org.apache.solr.update.AddUpdateCommand.

Mockito can only mock non-private & non-final classes.
If you're not sure why you're getting this error, please report to the mailing 
list.


Java   : 11
JVM vendor name: "Oracle Corporation"
JVM vendor version : 11-ea+5
JVM name   : OpenJDK 64-Bit Server VM
JVM version: 11-ea+5
JVM info   : mixed mode
OS name: Linux
OS version : 4.13.0-37-generic


Underlying exception : java.lang.UnsupportedOperationException: Cannot define 
class using reflection
at 
__randomizedtesting.SeedInfo.seed([9F1443AAE163476:110BEA914758C877]:0)
at 
org.apache.solr.update.TransactionLogTest.testBigLastAddSize(TransactionLogTest.java:38)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[jira] [Created] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

Alan Woodward created LUCENE-8268:
-

 Summary: MatchesIterator.term() should return an array
 Key: LUCENE-8268
 URL: https://issues.apache.org/jira/browse/LUCENE-8268
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Alan Woodward
Assignee: Alan Woodward
 Attachments: LUCENE-8268.patch

At the moment, we return a single BytesRef from MatchesIterator.term(), which 
works well for the queries that currently implement this.  This won't be enough 
for queries that operate on more than one term, however, such as phrase or Span 
queries.

In preparation for LUCENE-8249, this issue will change the method to return an 
array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8268) MatchesIterator.term() should return an array

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8268:
--
Attachment: LUCENE-8268.patch

> MatchesIterator.term() should return an array
> -
>
> Key: LUCENE-8268
> URL: https://issues.apache.org/jira/browse/LUCENE-8268
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8268.patch
>
>
> At the moment, we return a single BytesRef from MatchesIterator.term(), which 
> works well for the queries that currently implement this.  This won't be 
> enough for queries that operate on more than one term, however, such as 
> phrase or Span queries.
> In preparation for LUCENE-8249, this issue will change the method to return 
> an array of BytesRef



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8264) Allow an option to rewrite all segments

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447759#comment-16447759
 ] 

Uwe Schindler commented on LUCENE-8264:
---

It worked at least until 7.x. As I said, you can remove offsets if needed. And 
of course a FilterLeafReader together with SlowCodecReaderWrapper is definitly 
needed.

> Allow an option to rewrite all segments
> ---
>
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> For the background, see SOLR-12259.
> There are several use-cases that would be much easier, especially during 
> upgrades, if we could specify that all segments get rewritten. 
> One example: Upgrading 5x->6x->7x. When segments are merged, they're 
> rewritten into the current format. However, there's no guarantee that a 
> particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily 
> be successful.
> How many merge policies support this is an open question. I propose to start 
> with TMP and raise other JIRAs as necessary for other merge policies.
> So far the usual response has been "re-index from scratch", but that's 
> increasingly difficult as systems get larger.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org