[jira] [Commented] (LUCENE-7976) Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of very large segments
[ https://issues.apache.org/jira/browse/LUCENE-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497581#comment-16497581 ] Erick Erickson commented on LUCENE-7976: The failure was a bad test assumption, only a test change. > Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of > very large segments > - > > Key: LUCENE-7976 > URL: https://issues.apache.org/jira/browse/LUCENE-7976 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, > LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, > LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch > > > We're seeing situations "in the wild" where there are very large indexes (on > disk) handled quite easily in a single Lucene index. This is particularly > true as features like docValues move data into MMapDirectory space. The > current TMP algorithm allows on the order of 50% deleted documents as per a > dev list conversation with Mike McCandless (and his blog here: > https://www.elastic.co/blog/lucenes-handling-of-deleted-documents). > Especially in the current era of very large indexes in aggregate, (think many > TB) solutions like "you need to distribute your collection over more shards" > become very costly. Additionally, the tempting "optimize" button exacerbates > the issue since once you form, say, a 100G segment (by > optimizing/forceMerging) it is not eligible for merging until 97.5G of the > docs in it are deleted (current default 5G max segment size). > The proposal here would be to add a new parameter to TMP, something like > (no, that's not serious name, suggestions > welcome) which would default to 100 (or the same behavior we have now). > So if I set this parameter to, say, 20%, and the max segment size stays at > 5G, the following would happen when segments were selected for merging: > > any segment with > 20% deleted documents would be merged or rewritten NO > > MATTER HOW LARGE. There are two cases, > >> the segment has < 5G "live" docs. In that case it would be merged with > >> smaller segments to bring the resulting segment up to 5G. If no smaller > >> segments exist, it would just be rewritten > >> The segment has > 5G "live" docs (the result of a forceMerge or optimize). > >> It would be rewritten into a single segment removing all deleted docs no > >> matter how big it is to start. The 100G example above would be rewritten > >> to an 80G segment for instance. > Of course this would lead to potentially much more I/O which is why the > default would be the same behavior we see now. As it stands now, though, > there's no way to recover from an optimize/forceMerge except to re-index from > scratch. We routinely see 200G-300G Lucene indexes at this point "in the > wild" with 10s of shards replicated 3 or more times. And that doesn't even > include having these over HDFS. > Alternatives welcome! Something like the above seems minimally invasive. A > new merge policy is certainly an alternative. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 656 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/656/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
9 tests failed.
FAILED:
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/38)={
"replicationFactor":"2", "pullReplicas":"0",
"router":{"name":"compositeId"}, "maxShardsPerNode":"2",
"autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0",
"autoCreated":"true", "shards":{ "shard2":{ "replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true", "SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":11}, "core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":11}}, "range":"0-7fff",
"state":"active"}, "shard1":{ "stateTimestamp":"1527828643018728300",
"replicas":{ "core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true", "SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":14}, "core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":14}}, "range":"8000-",
"state":"inactive"}, "shard1_1":{ "parent":"shard1",
"stateTimestamp":"1527828643035506100", "range":"c000-",
"state":"active", "replicas":{ "core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{
"parent":"shard1", "stateTimestamp":"1527828643035233050",
"range":"8000-bfff", "state":"active", "replicas":{
"core_node7":{ "leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7}
Stack Trace:
java.util.concurrent.TimeoutException: last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/38)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
[JENKINS] Lucene-Solr-repro - Build # 734 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/734/ [...truncated 28 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1555/consoleText [repro] Revision: d243f35a5480163fb02e1d36541bf115cec35172 [repro] Ant options: -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testMergeIntegration -Dtests.seed=E9995B841F2A19F5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=sr -Dtests.timezone=Brazil/DeNoronha -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=TestStressLiveNodes -Dtests.method=testStress -Dtests.seed=E9995B841F2A19F5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=nl-BE -Dtests.timezone=Africa/Malabo -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=TestComputePlanAction -Dtests.method=testNodeAdded -Dtests.seed=E9995B841F2A19F5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=zh -Dtests.timezone=Asia/Hebron -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 1d33130fcba48c7a59ef0518c741b647d5826873 [repro] git fetch [...truncated 2 lines...] [repro] git checkout d243f35a5480163fb02e1d36541bf115cec35172 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] IndexSizeTriggerTest [repro] TestStressLiveNodes [repro] TestComputePlanAction [repro] ant compile-test [...truncated 3299 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 -Dtests.class="*.IndexSizeTriggerTest|*.TestStressLiveNodes|*.TestComputePlanAction" -Dtests.showOutput=onerror -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.seed=E9995B841F2A19F5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=sr -Dtests.timezone=Brazil/DeNoronha -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [...truncated 7258 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 0/5 failed: org.apache.solr.cloud.TestStressLiveNodes [repro] 1/5 failed: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction [repro] 4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest [repro] git checkout 1d33130fcba48c7a59ef0518c741b647d5826873 [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 5 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12435) Fix bin/solr help and ref guide text to describe ZK_HOST in solr.in.sh/solr.in.cmd as an alternative to -z cmdline param
Steve Rowe created SOLR-12435:
-
Summary: Fix bin/solr help and ref guide text to describe ZK_HOST
in solr.in.sh/solr.in.cmd as an alternative to -z cmdline param
Key: SOLR-12435
URL: https://issues.apache.org/jira/browse/SOLR-12435
Project: Solr
Issue Type: Task
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Steve Rowe
E.g. {{bin/solr start}}'s help text says about the {{-z}} param:
bq. To launch an embedded Zookeeper instance, don't pass this parameter.
which is wrong if {{ZK_HOST}} is specified in {{solr.in.sh}}/{{solr.in.cmd}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12427) Status 500 on Incorrect value for start and rows
[
https://issues.apache.org/jira/browse/SOLR-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497525#comment-16497525
]
Munendra S N commented on SOLR-12427:
-
[~gerlowskija]
Thanks for looking into this.
* I didn't find a good place to add the test. So, I have added it in
_TestGroupingSearch_ (As grouping also goes through the same code block).
Please move it to a better place.
* While working on this, I have observed that
In *params.getInt(String param)*, when an exception is thrown the error message
doesn't include for which param it has failed. (This is true for all number
conversions in SolrParams)
This can be changed to something like this. This helps user as it is more
verbose.
{code:java}
public Integer getInt(String param) {
String val = get(param);
try {
return val==null ? null : Integer.valueOf(val);
}
catch( Exception ex ) {
throw new SolrException( SolrException.ErrorCode.BAD_REQUEST, "for param"
+ param + "error" + ex.getMessage(), ex );
}
}
{code}
I haven't made the change in this patch because the change is not related to
this issue. Probably it should be covered separately.
> Status 500 on Incorrect value for start and rows
>
>
> Key: SOLR-12427
> URL: https://issues.apache.org/jira/browse/SOLR-12427
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Munendra S N
>Assignee: Jason Gerlowski
>Priority: Trivial
> Attachments: SOLR-12427.patch, SOLR-12427.patch
>
>
> With SOLR-7254,
> Cases, when start and rows are negatives, was handled but the case when an
> invalid value is passed is not handled.
> Hence, Solr returns 500. It is better to return 400, as it is the client error
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12434) bin/solr config ignores ZK_HOST in solr.in.{sh,cmd}
Steve Rowe created SOLR-12434:
-
Summary: bin/solr config ignores ZK_HOST in solr.in.{sh,cmd}
Key: SOLR-12434
URL: https://issues.apache.org/jira/browse/SOLR-12434
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Steve Rowe
Assignee: Steve Rowe
{{bin/solr config}} should be usable without specifying {{-z}} or {{-solrUrl}}
params, if {{ZK_HOST}} is specified in {{solr.in.sh}}/{{solr.in.cmd}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9.0.4) - Build # 22151 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22151/
Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction.testExecute
Error Message:
last state: DocCollection(testExecute//clusterstate.json/18)={
"replicationFactor":"2", "pullReplicas":"0",
"router":{"name":"compositeId"}, "maxShardsPerNode":"1",
"autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0",
"autoCreated":"true", "shards":{"shard1":{ "replicas":{
"core_node1":{ "core":"testExecute_shard1_replica_n1",
"leader":"true", "SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:1_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":0}, "core_node2":{
"core":"testExecute_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":0, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10002_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":0}, "core_node4":{
"node_name":"127.0.0.1:1_solr",
"core":"testExecute_shard1_replica_n3", "state":"active",
"INDEX.sizeInBytes":1, "type":"NRT"}},
"range":"8000-7fff", "state":"active"}}}
Stack Trace:
java.util.concurrent.TimeoutException: last state:
DocCollection(testExecute//clusterstate.json/18)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{"shard1":{
"replicas":{
"core_node1":{
"core":"testExecute_shard1_replica_n1",
"leader":"true",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:1_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node2":{
"core":"testExecute_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":0,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10002_solr",
"state":"active",
"type":"NRT",
"SEARCHER.searcher.numDocs":0},
"core_node4":{
"node_name":"127.0.0.1:1_solr",
"core":"testExecute_shard1_replica_n3",
"state":"active",
"INDEX.sizeInBytes":1,
"type":"NRT"}},
"range":"8000-7fff",
"state":"active"}}}
at
__randomizedtesting.SeedInfo.seed([7C379D404E1AB110:4D878F4D5CED4E97]:0)
at
org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
at
org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction.testExecute(TestExecutePlanAction.java:152)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl
[JENKINS] Lucene-Solr-repro - Build # 733 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/733/ [...truncated 28 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/231/consoleText [repro] Revision: 528b96540e4d65ff69bc4f2d6e0f78615c5e317e [repro] Ant options: -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt [repro] Repro line: ant test -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.method=test -Dtests.seed=F6BF429EE676516E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ga -Dtests.timezone=Asia/Bishkek -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=TestPKIAuthenticationPlugin -Dtests.method=test -Dtests.seed=F6BF429EE676516E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=en-US -Dtests.timezone=Africa/Bamako -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testSplitIntegration -Dtests.seed=F6BF429EE676516E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=es -Dtests.timezone=Asia/Jayapura -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=preferLocalShardsTest -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Atlantic/Bermuda -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=testWrongZkChrootTest -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Atlantic/Bermuda -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=testRouting -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Atlantic/Bermuda -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=stateVersionParamTest -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Atlantic/Bermuda -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=customHttpClientTest -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Atlantic/Bermuda -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=testNonRetryableRequests -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Atlantic/Bermuda -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=testInitializationWithSolrUrls -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Atlantic/Bermuda -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=CloudSolrClientTest -Dtests.method=testOverwriteOption -Dtests.seed=BC65572CF977B1B7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=
[jira] [Commented] (SOLR-12279) Validate Boolean "bin/solr auth" Inputs
[
https://issues.apache.org/jira/browse/SOLR-12279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497515#comment-16497515
]
Steve Rowe commented on SOLR-12279:
---
Jason, can this issue be resolved?
> Validate Boolean "bin/solr auth" Inputs
> ---
>
> Key: SOLR-12279
> URL: https://issues.apache.org/jira/browse/SOLR-12279
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Affects Versions: master (8.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Trivial
> Attachments: SOLR-12279.patch, repro.sh
>
>
> The "auth" command in the {{bin/solr}} scripts has a handful of different
> parameters which take in boolean arguments. However, {{bin/solr}} blithely
> accepts invalid values without warning administrators in any way of the
> mistake.
> In most cases, the results are innocuous. But in some cases, silently
> handling invalid input causes real issues. Consider:
> {code}
> $ bin/solr auth enable -type basicAuth -credentials anyUser:anyPass
> -blockUnknown ture
> Successfully enabled basic auth with username [anyUser] and password
> [anyPass].
> $ bin/solr auth enable -type basicAuth -credentials anyUser:anyPass
> -blockUnknown ture
> Security is already enabled. You can disable it with 'bin/solr auth disable'.
> Existing security.json:
> {
> "authentication":{
>"blockUnknown": false,
>"class":"solr.BasicAuthPlugin",
>"credentials":{"mount":"3FLVxpOGLt4dlqlyqxgsiFDbGX+i+dc81L6qEhuBdcI=
> lrH1W1pFGyGoAdTJ/Isuclh042fvz66ggG7YZ4e7YwA="}
> },
> ...
> }
> {code}
> If an administrator accidentally mistypes or fatfingers "true" when enabling
> authentication, their Solr instance will remain unprotected without any
> warning!
> The {{bin/solr auth}} tool should refuse to process invalid boolean
> arguments, or at the least spit out a warning in such cases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10282) bin/solr support for enabling Kerberos security
[ https://issues.apache.org/jira/browse/SOLR-10282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497507#comment-16497507 ] Steve Rowe commented on SOLR-10282: --- Ping [~ichattopadhyaya]: AFAICT this was never documented in the ref guide. > bin/solr support for enabling Kerberos security > --- > > Key: SOLR-10282 > URL: https://issues.apache.org/jira/browse/SOLR-10282 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCLI >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: 7.0 > > Attachments: SOLR-10282.patch, SOLR-10282.patch > > > This is in the same spirit as SOLR-8440. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8165) ban Arrays.copyOfRange with forbidden APIs
[ https://issues.apache.org/jira/browse/LUCENE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497498#comment-16497498 ] Robert Muir commented on LUCENE-8165: - This looks good, thanks! I had forgotten about this issue, great to have more progress. > ban Arrays.copyOfRange with forbidden APIs > -- > > Key: LUCENE-8165 > URL: https://issues.apache.org/jira/browse/LUCENE-8165 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Attachments: LUCENE-8165_copy_of_range.patch, > LUCENE-8165_start.patch, LUCENE-8165_start.patch > > > This method is no good, because instead of throwing AIOOBE for bad bounds, it > will silently fill with zeros (essentially silent corruption). Unfortunately > it is used in quite a few places so replacing it with e.g. arrayCopy may > uncover some interesting surprises. > See LUCENE-8164 for motivation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1900 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1900/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC 4 tests failed. FAILED: org.apache.solr.cloud.CreateRoutedAliasTest.testV1 Error Message: Unexpected status: HTTP/1.1 400 Bad Request Stack Trace: java.lang.AssertionError: Unexpected status: HTTP/1.1 400 Bad Request at __randomizedtesting.SeedInfo.seed([4D9A3AEB28EDA238:E254FD9B69000EA8]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.CreateRoutedAliasTest.assertSuccess(CreateRoutedAliasTest.java:360) at org.apache.solr.cloud.CreateRoutedAliasTest.testV1(CreateRoutedAliasTest.java:187) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.solr.cloud.CreateRoutedAliasTest.testV2 Error Message: Unexpected status: HTTP/1.1 400 Bad Request Stack Trace: java.lang.AssertionError: Unexpected status: HTTP/1.1 400 Bad Request at __randomizedtesting.S
[JENKINS] Lucene-Solr-SmokeRelease-7.x - Build # 231 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/231/ No tests ran. Build Log: [...truncated 24220 lines...] [asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [java] Processed 2201 links (1756 relative) to 2974 anchors in 229 files [echo] Validated Links & Anchors via: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr-ref-guide/bare-bones-html/ -dist-changes: [copy] Copying 4 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/changes -dist-keys: [get] Getting: http://home.apache.org/keys/group/lucene.asc [get] To: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/KEYS package: -unpack-solr-tgz: -ensure-solr-tgz-exists: [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr.tgz.unpacked [untar] Expanding: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/solr-7.4.0.tgz into /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr.tgz.unpacked generate-maven-artifacts: resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml r
[jira] [Commented] (SOLR-9922) Write buffering updates to another tlog
[ https://issues.apache.org/jira/browse/SOLR-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497491#comment-16497491 ] Cao Manh Dat commented on SOLR-9922: Attached a patch for this ticket which can be applied on the master branch. This issue is required for SOLR-12305. > Write buffering updates to another tlog > --- > > Key: SOLR-9922 > URL: https://issues.apache.org/jira/browse/SOLR-9922 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Cao Manh Dat >Assignee: Cao Manh Dat >Priority: Major > Attachments: SOLR-9922.patch, SOLR-9922.patch, SOLR-9922.patch, > SOLR-9922.patch, SOLR-9922.patch > > > Currently, we write buffering logs to current tlog and not apply that updates > to index. Then we rely on replay log to apply that updates to index. But at > the same time there are some updates also write to current tlog and applied > to the index. > For example, during peersync, if new updates come to replica we will end up > with this tlog > tlog : old1, new1, new2, old2, new3, old3 > old updates belong to peersync, and these updates are applied to the index. > new updates belong to buffering updates, and these updates are not applied to > the index. > But writing all the updates to same current tlog make code base very complex. > We should write buffering updates to another tlog file. > By doing this, it will help our code base simpler. It also makes replica > recovery for SOLR-9835 more easier. Because after peersync success we can > copy new updates from temporary file to current tlog, for example > tlog : old1, old2, old3 > temporary tlog : new1, new2, new3 > --> > tlog : old1, old2, old3, new1, new2, new3 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9922) Write buffering updates to another tlog
[ https://issues.apache.org/jira/browse/SOLR-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao Manh Dat updated SOLR-9922: --- Attachment: SOLR-9922.patch > Write buffering updates to another tlog > --- > > Key: SOLR-9922 > URL: https://issues.apache.org/jira/browse/SOLR-9922 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Cao Manh Dat >Priority: Major > Attachments: SOLR-9922.patch, SOLR-9922.patch, SOLR-9922.patch, > SOLR-9922.patch, SOLR-9922.patch > > > Currently, we write buffering logs to current tlog and not apply that updates > to index. Then we rely on replay log to apply that updates to index. But at > the same time there are some updates also write to current tlog and applied > to the index. > For example, during peersync, if new updates come to replica we will end up > with this tlog > tlog : old1, new1, new2, old2, new3, old3 > old updates belong to peersync, and these updates are applied to the index. > new updates belong to buffering updates, and these updates are not applied to > the index. > But writing all the updates to same current tlog make code base very complex. > We should write buffering updates to another tlog file. > By doing this, it will help our code base simpler. It also makes replica > recovery for SOLR-9835 more easier. Because after peersync success we can > copy new updates from temporary file to current tlog, for example > tlog : old1, old2, old3 > temporary tlog : new1, new2, new3 > --> > tlog : old1, old2, old3, new1, new2, new3 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-9922) Write buffering updates to another tlog
[ https://issues.apache.org/jira/browse/SOLR-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao Manh Dat reassigned SOLR-9922: -- Assignee: Cao Manh Dat > Write buffering updates to another tlog > --- > > Key: SOLR-9922 > URL: https://issues.apache.org/jira/browse/SOLR-9922 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Cao Manh Dat >Assignee: Cao Manh Dat >Priority: Major > Attachments: SOLR-9922.patch, SOLR-9922.patch, SOLR-9922.patch, > SOLR-9922.patch, SOLR-9922.patch > > > Currently, we write buffering logs to current tlog and not apply that updates > to index. Then we rely on replay log to apply that updates to index. But at > the same time there are some updates also write to current tlog and applied > to the index. > For example, during peersync, if new updates come to replica we will end up > with this tlog > tlog : old1, new1, new2, old2, new3, old3 > old updates belong to peersync, and these updates are applied to the index. > new updates belong to buffering updates, and these updates are not applied to > the index. > But writing all the updates to same current tlog make code base very complex. > We should write buffering updates to another tlog file. > By doing this, it will help our code base simpler. It also makes replica > recovery for SOLR-9835 more easier. Because after peersync success we can > copy new updates from temporary file to current tlog, for example > tlog : old1, old2, old3 > temporary tlog : new1, new2, new3 > --> > tlog : old1, old2, old3, new1, new2, new3 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-12433) Recovering flag of a replica is set equals to leader even it failed to receive update on recovering
[
https://issues.apache.org/jira/browse/SOLR-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao Manh Dat resolved SOLR-12433.
-
Resolution: Fixed
Fix Version/s: 7.4
> Recovering flag of a replica is set equals to leader even it failed to
> receive update on recovering
> ---
>
> Key: SOLR-12433
> URL: https://issues.apache.org/jira/browse/SOLR-12433
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Fix For: 7.4
>
>
> When digging into RestartWhileUpdatingTest failure, I see that term of
> replicas is kinda mess up.
> {quote}
> [junit4] 1> /collections/collection1/terms/shard1 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core_node24_recovering":4,
> [junit4] 1> "core_node22":4,
> [junit4] 1> "core_node26_recovering":4,
> [junit4] 1> "core_node24":2,
> [junit4] 1> "core_node26":3}
> {quote}
> By design, the {{core_node24_recovering}} and {{core_node24}} should be
> always equals to each other. The reason here is
> {{ZkShardTerms.ensureTermsIsHigher}} also increase the
> {{core_node24_recovering}} is a higher number. This will lead to a case when
> a replica finished recovering but it won't be able to become active.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12433) Recovering flag of a replica is set equals to leader even it failed to receive update on recovering
[
https://issues.apache.org/jira/browse/SOLR-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497478#comment-16497478
]
ASF subversion and git services commented on SOLR-12433:
Commit 1d33130fcba48c7a59ef0518c741b647d5826873 in lucene-solr's branch
refs/heads/master from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1d33130 ]
SOLR-12433: Recovering flag of a replica is set equals to leader even it failed
to receive update on recovering
> Recovering flag of a replica is set equals to leader even it failed to
> receive update on recovering
> ---
>
> Key: SOLR-12433
> URL: https://issues.apache.org/jira/browse/SOLR-12433
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> When digging into RestartWhileUpdatingTest failure, I see that term of
> replicas is kinda mess up.
> {quote}
> [junit4] 1> /collections/collection1/terms/shard1 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core_node24_recovering":4,
> [junit4] 1> "core_node22":4,
> [junit4] 1> "core_node26_recovering":4,
> [junit4] 1> "core_node24":2,
> [junit4] 1> "core_node26":3}
> {quote}
> By design, the {{core_node24_recovering}} and {{core_node24}} should be
> always equals to each other. The reason here is
> {{ZkShardTerms.ensureTermsIsHigher}} also increase the
> {{core_node24_recovering}} is a higher number. This will lead to a case when
> a replica finished recovering but it won't be able to become active.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12433) Recovering flag of a replica is set equals to leader even it failed to receive update on recovering
[
https://issues.apache.org/jira/browse/SOLR-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497479#comment-16497479
]
ASF subversion and git services commented on SOLR-12433:
Commit b58cd6415f09cfd3c3f47630f11667cd1850a202 in lucene-solr's branch
refs/heads/branch_7x from [~caomanhdat]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b58cd64 ]
SOLR-12433: Recovering flag of a replica is set equals to leader even it failed
to receive update on recovering
> Recovering flag of a replica is set equals to leader even it failed to
> receive update on recovering
> ---
>
> Key: SOLR-12433
> URL: https://issues.apache.org/jira/browse/SOLR-12433
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> When digging into RestartWhileUpdatingTest failure, I see that term of
> replicas is kinda mess up.
> {quote}
> [junit4] 1> /collections/collection1/terms/shard1 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core_node24_recovering":4,
> [junit4] 1> "core_node22":4,
> [junit4] 1> "core_node26_recovering":4,
> [junit4] 1> "core_node24":2,
> [junit4] 1> "core_node26":3}
> {quote}
> By design, the {{core_node24_recovering}} and {{core_node24}} should be
> always equals to each other. The reason here is
> {{ZkShardTerms.ensureTermsIsHigher}} also increase the
> {{core_node24_recovering}} is a higher number. This will lead to a case when
> a replica finished recovering but it won't be able to become active.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12433) Recovering flag of a replica is set equals to leader even it failed to receive update on recovering
[
https://issues.apache.org/jira/browse/SOLR-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao Manh Dat updated SOLR-12433:
Summary: Recovering flag of a replica is set equals to leader even it
failed to receive update on recovering (was: Raft term of a replica is set
equals to leader even it failed to receive update on recovering)
> Recovering flag of a replica is set equals to leader even it failed to
> receive update on recovering
> ---
>
> Key: SOLR-12433
> URL: https://issues.apache.org/jira/browse/SOLR-12433
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> When digging into RestartWhileUpdatingTest failure, I see that term of
> replicas is kinda mess up.
> {quote}
> [junit4] 1> /collections/collection1/terms/shard1 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core_node24_recovering":4,
> [junit4] 1> "core_node22":4,
> [junit4] 1> "core_node26_recovering":4,
> [junit4] 1> "core_node24":2,
> [junit4] 1> "core_node26":3}
> {quote}
> By design, the {{core_node24_recovering}} and {{core_node24}} should be
> always equals to each other. The reason here is
> {{ZkShardTerms.ensureTermsIsHigher}} also increase the
> {{core_node24_recovering}} is a higher number. This will lead to a case when
> a replica finished recovering but it won't be able to become active.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8165) ban Arrays.copyOfRange with forbidden APIs
[ https://issues.apache.org/jira/browse/LUCENE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497476#comment-16497476 ] Nhat Nguyen commented on LUCENE-8165: - [~rcmuir] I continued your initial patch and completed the first round which removes all Arrays#copyOfRange usages. I will do another round for Arrays#copyOf. Could you please have a look? Thank you! > ban Arrays.copyOfRange with forbidden APIs > -- > > Key: LUCENE-8165 > URL: https://issues.apache.org/jira/browse/LUCENE-8165 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Attachments: LUCENE-8165_copy_of_range.patch, > LUCENE-8165_start.patch, LUCENE-8165_start.patch > > > This method is no good, because instead of throwing AIOOBE for bad bounds, it > will silently fill with zeros (essentially silent corruption). Unfortunately > it is used in quite a few places so replacing it with e.g. arrayCopy may > uncover some interesting surprises. > See LUCENE-8164 for motivation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8165) ban Arrays.copyOfRange with forbidden APIs
[ https://issues.apache.org/jira/browse/LUCENE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8165: Attachment: LUCENE-8165_copy_of_range.patch > ban Arrays.copyOfRange with forbidden APIs > -- > > Key: LUCENE-8165 > URL: https://issues.apache.org/jira/browse/LUCENE-8165 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Attachments: LUCENE-8165_copy_of_range.patch, > LUCENE-8165_start.patch, LUCENE-8165_start.patch > > > This method is no good, because instead of throwing AIOOBE for bad bounds, it > will silently fill with zeros (essentially silent corruption). Unfortunately > it is used in quite a few places so replacing it with e.g. arrayCopy may > uncover some interesting surprises. > See LUCENE-8164 for motivation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9272) Auto resolve zkHost for bin/solr zk for running Solr
[
https://issues.apache.org/jira/browse/SOLR-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497468#comment-16497468
]
Steve Rowe commented on SOLR-9272:
--
{quote}
Whether the current cluster will be SSL configured (HTTPS) or conventional
HTTP, is RANDOMIZED. Either https or http will be supported, not both in any
case.
Now, in our code, specifically say, we look for zk host corresponding to ::
http://localhost:[solr_port]/solr. What should be done?
[...]
I have fixed the SSL issue by smartly retrying whenever HTTP schema fails and
SSL is configured. Attached updated patch. Requesting feedback and comments.
{quote}
{{RunExampleTool}} passes the {{-urlScheme}} param ({{bin/solr}} &
{{bin\solr.cmd}} pass {{SOLR_URL_SCHEME}} env. var.'s value); couldn't the
{{zk}} sub-commands do the same? Then testing could add a param for this, and
you wouldn't have to do the "smart retry" thing.
> Auto resolve zkHost for bin/solr zk for running Solr
>
>
> Key: SOLR-9272
> URL: https://issues.apache.org/jira/browse/SOLR-9272
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Affects Versions: 6.2
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>Priority: Major
> Labels: newdev
> Attachments: SOLR-9272.patch, SOLR-9272.patch, SOLR-9272.patch,
> SOLR-9272.patch, SOLR-9272.patch, SOLR-9272.patch, SOLR-9272.patch,
> SOLR-9272.patch, SOLR-9272.patch
>
>
> Spinoff from SOLR-9194:
> We can skip requiring {{-z}} for {{bin/solr zk}} for a Solr that is already
> running. We can optionally accept the {{-p}} parameter instead, and with that
> use StatusTool to fetch the {{cloud/ZooKeeper}} property from there. It's
> easier to remember solr port than zk string.
> Example:
> {noformat}
> bin/solr start -c -p 9090
> bin/solr zk ls / -p 9090
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12433) Raft term of a replica is set equals to leader even it failed to receive update on recovering
[
https://issues.apache.org/jira/browse/SOLR-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao Manh Dat updated SOLR-12433:
Environment: (was: When digging into RestartWhileUpdatingTest failure,
I see that term of replicas is kinda mess up.
{quote}
[junit4] 1> /collections/collection1/terms/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core_node24_recovering":4,
[junit4] 1> "core_node22":4,
[junit4] 1> "core_node26_recovering":4,
[junit4] 1> "core_node24":2,
[junit4] 1> "core_node26":3}
{quote}
By design, the {{core_node24_recovering}} and {{core_node24}} should be always
equals to each other. The reason here is {{ZkShardTerms.ensureTermsIsHigher}}
also increase the {{core_node24_recovering}} is a higher number. This will lead
to a case when a replica finished recovering but it won't be able to become
active. )
> Raft term of a replica is set equals to leader even it failed to receive
> update on recovering
> -
>
> Key: SOLR-12433
> URL: https://issues.apache.org/jira/browse/SOLR-12433
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12433) Raft term of a replica is set equals to leader even it failed to receive update on recovering
Cao Manh Dat created SOLR-12433:
---
Summary: Raft term of a replica is set equals to leader even it
failed to receive update on recovering
Key: SOLR-12433
URL: https://issues.apache.org/jira/browse/SOLR-12433
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Environment: When digging into RestartWhileUpdatingTest failure, I see
that term of replicas is kinda mess up.
{quote}
[junit4] 1> /collections/collection1/terms/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core_node24_recovering":4,
[junit4] 1> "core_node22":4,
[junit4] 1> "core_node26_recovering":4,
[junit4] 1> "core_node24":2,
[junit4] 1> "core_node26":3}
{quote}
By design, the {{core_node24_recovering}} and {{core_node24}} should be always
equals to each other. The reason here is {{ZkShardTerms.ensureTermsIsHigher}}
also increase the {{core_node24_recovering}} is a higher number. This will lead
to a case when a replica finished recovering but it won't be able to become
active.
Reporter: Cao Manh Dat
Assignee: Cao Manh Dat
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12433) Raft term of a replica is set equals to leader even it failed to receive update on recovering
[
https://issues.apache.org/jira/browse/SOLR-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao Manh Dat updated SOLR-12433:
Description:
When digging into RestartWhileUpdatingTest failure, I see that term of replicas
is kinda mess up.
{quote}
[junit4] 1> /collections/collection1/terms/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core_node24_recovering":4,
[junit4] 1> "core_node22":4,
[junit4] 1> "core_node26_recovering":4,
[junit4] 1> "core_node24":2,
[junit4] 1> "core_node26":3}
{quote}
By design, the {{core_node24_recovering}} and {{core_node24}} should be always
equals to each other. The reason here is {{ZkShardTerms.ensureTermsIsHigher}}
also increase the {{core_node24_recovering}} is a higher number. This will lead
to a case when a replica finished recovering but it won't be able to become
active.
> Raft term of a replica is set equals to leader even it failed to receive
> update on recovering
> -
>
> Key: SOLR-12433
> URL: https://issues.apache.org/jira/browse/SOLR-12433
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> When digging into RestartWhileUpdatingTest failure, I see that term of
> replicas is kinda mess up.
> {quote}
> [junit4] 1> /collections/collection1/terms/shard1 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core_node24_recovering":4,
> [junit4] 1> "core_node22":4,
> [junit4] 1> "core_node26_recovering":4,
> [junit4] 1> "core_node24":2,
> [junit4] 1> "core_node26":3}
> {quote}
> By design, the {{core_node24_recovering}} and {{core_node24}} should be
> always equals to each other. The reason here is
> {{ZkShardTerms.ensureTermsIsHigher}} also increase the
> {{core_node24_recovering}} is a higher number. This will lead to a case when
> a replica finished recovering but it won't be able to become active.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12433) Raft term of a replica is set equals to leader even it failed to receive update on recovering
[
https://issues.apache.org/jira/browse/SOLR-12433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cao Manh Dat updated SOLR-12433:
Issue Type: Bug (was: Improvement)
> Raft term of a replica is set equals to leader even it failed to receive
> update on recovering
> -
>
> Key: SOLR-12433
> URL: https://issues.apache.org/jira/browse/SOLR-12433
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
>
> When digging into RestartWhileUpdatingTest failure, I see that term of
> replicas is kinda mess up.
> {quote}
> [junit4] 1> /collections/collection1/terms/shard1 (0)
> [junit4] 1> DATA:
> [junit4] 1> {
> [junit4] 1> "core_node24_recovering":4,
> [junit4] 1> "core_node22":4,
> [junit4] 1> "core_node26_recovering":4,
> [junit4] 1> "core_node24":2,
> [junit4] 1> "core_node26":3}
> {quote}
> By design, the {{core_node24_recovering}} and {{core_node24}} should be
> always equals to each other. The reason here is
> {{ZkShardTerms.ensureTermsIsHigher}} also increase the
> {{core_node24_recovering}} is a higher number. This will lead to a case when
> a replica finished recovering but it won't be able to become active.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12432) Replace our modified Apache Blur HDFS cache with Apache Ignite.
Mark Miller created SOLR-12432: -- Summary: Replace our modified Apache Blur HDFS cache with Apache Ignite. Key: SOLR-12432 URL: https://issues.apache.org/jira/browse/SOLR-12432 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: Mark Miller -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12427) Status 500 on Incorrect value for start and rows
[
https://issues.apache.org/jira/browse/SOLR-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497327#comment-16497327
]
Jason Gerlowski commented on SOLR-12427:
Thanks for adding in tests for this too. I might move them out of
{{TestGroupingSearch}} unless you had a good reason for putting them there?
Nothing about the change seems specific to grouping.
But otherwise this looks good to me. Hope to have this in this weekend.
> Status 500 on Incorrect value for start and rows
>
>
> Key: SOLR-12427
> URL: https://issues.apache.org/jira/browse/SOLR-12427
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Munendra S N
>Assignee: Jason Gerlowski
>Priority: Trivial
> Attachments: SOLR-12427.patch, SOLR-12427.patch
>
>
> With SOLR-7254,
> Cases, when start and rows are negatives, was handled but the case when an
> invalid value is passed is not handled.
> Hence, Solr returns 500. It is better to return 400, as it is the client error
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-10) - Build # 2026 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2026/
Java: 64bit/jdk-10 -XX:-UseCompressedOops -XX:+UseParallelGC
2 tests failed.
FAILED:
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/95)={
"replicationFactor":"2", "pullReplicas":"0",
"router":{"name":"compositeId"}, "maxShardsPerNode":"2",
"autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0",
"autoCreated":"true", "shards":{ "shard2":{ "replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true", "SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:1_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":11}, "core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":11}}, "range":"0-7fff",
"state":"active"}, "shard1":{ "stateTimestamp":"1527808486270123250",
"replicas":{ "core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true", "SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:1_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":14}, "core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":14}}, "range":"8000-",
"state":"inactive"}, "shard1_1":{ "parent":"shard1",
"stateTimestamp":"1527808486281884600", "range":"c000-",
"state":"active", "replicas":{ "core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{
"parent":"shard1", "stateTimestamp":"1527808486281697350",
"range":"8000-bfff", "state":"active", "replicas":{
"core_node7":{ "leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7}
Stack Trace:
java.util.concurrent.TimeoutException: last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/95)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:1_solr",
"st
[jira] [Commented] (LUCENE-8342) Can we enforce more properties on fields like uniquness
[ https://issues.apache.org/jira/browse/LUCENE-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497263#comment-16497263 ] Robert Muir commented on LUCENE-8342: - +1: it would be a big improvement to validate this in IW, merge, checkindex, etc. > Can we enforce more properties on fields like uniquness > > > Key: LUCENE-8342 > URL: https://issues.apache.org/jira/browse/LUCENE-8342 > Project: Lucene - Core > Issue Type: New Feature >Affects Versions: master (8.0) >Reporter: Simon Willnauer >Priority: Major > > This is a spin-off from LUCENE-8335 where we discuss adding a boolean to > FieldInfo to check if the field is used for soft deletes. This has been a a > very delicate line we drew in the past but if we take a step back and think > about how we'd design a feature like IW#updateDocument(Term, Document) today > would we allow passing different fields to this API? It's just one example > ie. storing floats and ints in the same DV field is a different one. I > personally think it would be a good idea to be more strict on that end and I > wonder what others think. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field
[ https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497261#comment-16497261 ] Robert Muir commented on LUCENE-8335: - as a followup, I think lets support checkindex validation (e.g. LUCENE-8341). I am happy to see LUCENE-8342 opened up, thats probably an easier win than int vs float anyway, but it addresses the kind of concerns i had here, I think its important to also enforce stuff for typical usecases as well. > Do not allow changing soft-deletes field > > > Key: LUCENE-8335 > URL: https://issues.apache.org/jira/browse/LUCENE-8335 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Nhat Nguyen >Assignee: Simon Willnauer >Priority: Minor > Attachments: LUCENE-8335.patch > > > Today we do not enforce an index to use a single soft-deletes field. A user > can create an index with one soft-deletes field then open an IW with another > field or add an index with a different soft-deletes field. This should not be > allowed and reported the error to users as soon as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8342) Can we enforce more properties on fields like uniquness
Simon Willnauer created LUCENE-8342: --- Summary: Can we enforce more properties on fields like uniquness Key: LUCENE-8342 URL: https://issues.apache.org/jira/browse/LUCENE-8342 Project: Lucene - Core Issue Type: New Feature Affects Versions: master (8.0) Reporter: Simon Willnauer This is a spin-off from LUCENE-8335 where we discuss adding a boolean to FieldInfo to check if the field is used for soft deletes. This has been a a very delicate line we drew in the past but if we take a step back and think about how we'd design a feature like IW#updateDocument(Term, Document) today would we allow passing different fields to this API? It's just one example ie. storing floats and ints in the same DV field is a different one. I personally think it would be a good idea to be more strict on that end and I wonder what others think. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-EA] Lucene-Solr-7.x-Windows (64bit/jdk-11-ea+14) - Build # 622 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/622/ Java: 64bit/jdk-11-ea+14 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC 18 tests failed. FAILED: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded Error Message: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@32ab20ea, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@4e0d1a67] expected:<1> but was:<2> Stack Trace: java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@32ab20ea, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@4e0d1a67] expected:<1> but was:<2> at __randomizedtesting.SeedInfo.seed([DDF02F7B84B866FD:B833790C261BCEFE]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded(TestComputePlanAction.java:313) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIg
[jira] [Commented] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
[
https://issues.apache.org/jira/browse/LUCENE-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497240#comment-16497240
]
Simon Willnauer commented on LUCENE-8341:
-
{quote}
Yes I agree it is a shame CheckIndex cannot enforce the check yet. But there
are many other such possible checks (ones i would argue are more
mainstream/bigger wins) that it also can't do for similar reasons. That was the
reason why I made the point on the LUCENE-8335 issue, lets just figure it out
there.
{quote}
I totally agree there are many more things we could do here if we knew a field
is an ID like enforcing uniqueness.
> Record soft deletes in SegmentCommitInfo
> -
>
> Key: LUCENE-8341
> URL: https://issues.apache.org/jira/browse/LUCENE-8341
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8341.patch, LUCENE-8341.patch, LUCENE-8341.patch
>
>
> This change add the number of documents that are soft deletes but
> not hard deleted to the segment commit info. This is the last step
> towards making soft deletes as powerful as hard deltes since now the
> number of document can be read from commit points without opening a
> full blown reader. This also allows merge posliies to make decisions
> without requiring an NRT reader to get the relevant statistics. This
> change doesn't enforce any field to be used as soft deletes and the
> statistic
> is maintained per segment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
[ https://issues.apache.org/jira/browse/LUCENE-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497237#comment-16497237 ] Robert Muir commented on LUCENE-8341: - Yes I agree it is a shame CheckIndex cannot enforce the check yet. But there are many other such possible checks (ones i would argue are more mainstream/bigger wins) that it also can't do for similar reasons. That was the reason why I made the point on the LUCENE-8335 issue, lets just figure it out there. > Record soft deletes in SegmentCommitInfo > - > > Key: LUCENE-8341 > URL: https://issues.apache.org/jira/browse/LUCENE-8341 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8341.patch, LUCENE-8341.patch, LUCENE-8341.patch > > > This change add the number of documents that are soft deletes but > not hard deleted to the segment commit info. This is the last step > towards making soft deletes as powerful as hard deltes since now the > number of document can be read from commit points without opening a > full blown reader. This also allows merge posliies to make decisions > without requiring an NRT reader to get the relevant statistics. This > change doesn't enforce any field to be used as soft deletes and the > statistic > is maintained per segment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr issue #384: LUCENE-8332 move CompletionTokenStream to Concatenat...
Github user jimczi commented on the issue: https://github.com/apache/lucene-solr/pull/384 > If we really want to insist that every component be cranky when mistreated, then perhaps we could have toAutomaton not close the inputTokenStream so that we can delegate that from CGF's close? I think it's the right thing to do and the change is minimal. Good catch ! --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
[ https://issues.apache.org/jira/browse/LUCENE-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497200#comment-16497200 ] Simon Willnauer commented on LUCENE-8341: - [~mikemccand] I added an option to checkindex that is for now manual. can you take you another look > Record soft deletes in SegmentCommitInfo > - > > Key: LUCENE-8341 > URL: https://issues.apache.org/jira/browse/LUCENE-8341 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8341.patch, LUCENE-8341.patch, LUCENE-8341.patch > > > This change add the number of documents that are soft deletes but > not hard deleted to the segment commit info. This is the last step > towards making soft deletes as powerful as hard deltes since now the > number of document can be read from commit points without opening a > full blown reader. This also allows merge posliies to make decisions > without requiring an NRT reader to get the relevant statistics. This > change doesn't enforce any field to be used as soft deletes and the > statistic > is maintained per segment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
[ https://issues.apache.org/jira/browse/LUCENE-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-8341: Attachment: LUCENE-8341.patch > Record soft deletes in SegmentCommitInfo > - > > Key: LUCENE-8341 > URL: https://issues.apache.org/jira/browse/LUCENE-8341 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8341.patch, LUCENE-8341.patch, LUCENE-8341.patch > > > This change add the number of documents that are soft deletes but > not hard deleted to the segment commit info. This is the last step > towards making soft deletes as powerful as hard deltes since now the > number of document can be read from commit points without opening a > full blown reader. This also allows merge posliies to make decisions > without requiring an NRT reader to get the relevant statistics. This > change doesn't enforce any field to be used as soft deletes and the > statistic > is maintained per segment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr issue #384: LUCENE-8332 move CompletionTokenStream to Concatenat...
Github user dsmiley commented on the issue: https://github.com/apache/lucene-solr/pull/384 There's a test failure in TestFactories.test which calls BaseTokenStreamTestCase.checkResetException. `ant test -Dtestcase=TestFactories -Dtests.seed=543C55AE2375085C` ``` [junit4]> Throwable #1: java.lang.AssertionError: didn't get expected exception when close() not called [junit4]>at __randomizedtesting.SeedInfo.seed([543C55AE2375085C:DC686A748D8965A4]:0) [junit4]>at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkResetException(BaseTokenStreamTestCase.java:453) [junit4]>at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:546) [junit4]>at org.apache.lucene.analysis.core.TestFactories.doTestTokenFilter(TestFactories.java:118) [junit4]>at org.apache.lucene.analysis.core.TestFactories.test(TestFactories.java:68) [junit4]>at java.lang.Thread.run(Thread.java:748) ``` The issue is that if some faulty code forgets to call close(), an exception would be ideal to be thrown but for this filter it won't be because the inputTokenStream is closed properly internally correctly by toAutomaton. Checks like this are kinda... I dunno, I don't like it. IMO it's asking too much of a TokenFilter to insist that it's cranky when it's mistreated; maybe some components (like this one) are more resilient? Maybe BaseTokenStreamTestCase (or somewhere else) should keep track of components that are resilient in this way so that it doesn't exercise these sort of tests? If we really want to insist that every component be cranky when mistreated, then perhaps we could have toAutomaton _not_ close the inputTokenStream so that we can delegate that from CGF's close? --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
REMINDER: Apache EU Roadshow 2018 in Berlin is less than 2 weeks away!
Hello Apache Supporters and Enthusiasts This is a reminder that our Apache EU Roadshow in Berlin is less than two weeks away and we need your help to spread the word. Please let your work colleagues, friends and anyone interested in any attending know about our Apache EU Roadshow event. We have a great schedule including tracks on Apache Tomcat, Apache Http Server, Microservices, Internet of Things (IoT) and Cloud Technologies. You can find more details at the link below: https://s.apache.org/0hnG Ticket prices will be going up on 8^th June 2018, so please make sure that you register soon if you want to beat the price increase. https://foss-backstage.de/tickets Remember that registering for the Apache EU Roadshow also gives you access to FOSS Backstage so you can attend any talks and workshops from both conferences. And don’t forget that our Apache Lounge will be open throughout the whole conference as a place to meet up, hack and relax. We look forward to seeing you in Berlin! Thanks Sharan Foga, VP Apache Community Development http://apachecon.com/ @apachecon PLEASE NOTE: You are receiving this message because you are subscribed to a user@ or dev@ list of one or more Apache Software Foundation projects.
[JENKINS] Lucene-Solr-repro - Build # 732 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/732/ [...truncated 28 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/230/consoleText [repro] Revision: 528b96540e4d65ff69bc4f2d6e0f78615c5e317e [repro] Ant options: -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testSplitIntegration -Dtests.seed=A85429CF407BCA25 -Dtests.multiplier=2 -Dtests.locale=ar-DZ -Dtests.timezone=Pacific/Tongatapu -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testMergeIntegration -Dtests.seed=A85429CF407BCA25 -Dtests.multiplier=2 -Dtests.locale=ar-DZ -Dtests.timezone=Pacific/Tongatapu -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 252a8145d9374978776f3fc10bfa12e14bf8433a [repro] git fetch [...truncated 2 lines...] [repro] git checkout 528b96540e4d65ff69bc4f2d6e0f78615c5e317e [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] IndexSizeTriggerTest [repro] ant compile-test [...truncated 3317 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.IndexSizeTriggerTest" -Dtests.showOutput=onerror -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 -Dtests.seed=A85429CF407BCA25 -Dtests.multiplier=2 -Dtests.locale=ar-DZ -Dtests.timezone=Pacific/Tongatapu -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 7855 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest [repro] Re-testing 100% failures at the tip of branch_7x [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] IndexSizeTriggerTest [repro] ant compile-test [...truncated 3317 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.IndexSizeTriggerTest" -Dtests.showOutput=onerror -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 -Dtests.seed=A85429CF407BCA25 -Dtests.multiplier=2 -Dtests.locale=ar-DZ -Dtests.timezone=Pacific/Tongatapu -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 10805 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of branch_7x: [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest [repro] Re-testing 100% failures at the tip of branch_7x without a seed [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] IndexSizeTriggerTest [repro] ant compile-test [...truncated 3317 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.IndexSizeTriggerTest" -Dtests.showOutput=onerror -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 -Dtests.multiplier=2 -Dtests.locale=ar-DZ -Dtests.timezone=Pacific/Tongatapu -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 8536 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of branch_7x without a seed: [repro] 4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest [repro] git checkout 252a8145d9374978776f3fc10bfa12e14bf8433a [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 5 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_172) - Build # 7347 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7347/ Java: 32bit/jdk1.8.0_172 -server -XX:+UseConcMarkSweepGC 8 tests failed. FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration Error Message: did not finish processing in time Stack Trace: java.lang.AssertionError: did not finish processing in time at __randomizedtesting.SeedInfo.seed([84F7061C0172E259:D74E44ACE36377A3]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:425) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration Error Message: did not finish processing in time Stack Trace: java.lang.AssertionError: did not finish processing in time at __randomizedtesting.SeedInfo.seed([8
[jira] [Commented] (SOLR-12388) Enable a strict ZooKeeper-connected search request mode, in which search requests will fail when the coordinating node can't communicate with ZooKeeper
[
https://issues.apache.org/jira/browse/SOLR-12388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497096#comment-16497096
]
ASF subversion and git services commented on SOLR-12388:
Commit 61a65d5928f4563f5c8284b1100528257e5c6834 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=61a65d5 ]
SOLR-12388: Add an expected exception message to
SearchHandlerTest.testRequireZkConnectedDistrib()
> Enable a strict ZooKeeper-connected search request mode, in which search
> requests will fail when the coordinating node can't communicate with ZooKeeper
> ---
>
> Key: SOLR-12388
> URL: https://issues.apache.org/jira/browse/SOLR-12388
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search, SolrCloud
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12388.patch
>
>
> Right now, a Solr node will return the results of a search request even if it
> cannot communicate with ZooKeeper at the time it receives the request. This
> may result in stale or incorrect results if there have been major changes to
> the collection structure that the node has not been informed of via
> ZooKeeper. When this happens, as long as all known shards respond, the
> response will succeed, and a {{zkConnected}} header set to {{false}} is
> included in the search response.
> There should be an option to instead fail requests under these conditions, to
> prevent stale or incorrect results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12388) Enable a strict ZooKeeper-connected search request mode, in which search requests will fail when the coordinating node can't communicate with ZooKeeper
[
https://issues.apache.org/jira/browse/SOLR-12388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497097#comment-16497097
]
ASF subversion and git services commented on SOLR-12388:
Commit 252a8145d9374978776f3fc10bfa12e14bf8433a in lucene-solr's branch
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=252a814 ]
SOLR-12388: Add an expected exception message to
SearchHandlerTest.testRequireZkConnectedDistrib()
> Enable a strict ZooKeeper-connected search request mode, in which search
> requests will fail when the coordinating node can't communicate with ZooKeeper
> ---
>
> Key: SOLR-12388
> URL: https://issues.apache.org/jira/browse/SOLR-12388
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search, SolrCloud
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12388.patch
>
>
> Right now, a Solr node will return the results of a search request even if it
> cannot communicate with ZooKeeper at the time it receives the request. This
> may result in stale or incorrect results if there have been major changes to
> the collection structure that the node has not been informed of via
> ZooKeeper. When this happens, as long as all known shards respond, the
> response will succeed, and a {{zkConnected}} header set to {{false}} is
> included in the search response.
> There should be an option to instead fail requests under these conditions, to
> prevent stale or incorrect results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12388) Enable a strict ZooKeeper-connected search request mode, in which search requests will fail when the coordinating node can't communicate with ZooKeeper
[
https://issues.apache.org/jira/browse/SOLR-12388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497095#comment-16497095
]
ASF subversion and git services commented on SOLR-12388:
Commit 114461cbeb2c7a2c9f610a46d4e01ca2ee9cf171 in lucene-solr's branch
refs/heads/branch_7x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=114461c ]
SOLR-12388: print out exception when assert fails
> Enable a strict ZooKeeper-connected search request mode, in which search
> requests will fail when the coordinating node can't communicate with ZooKeeper
> ---
>
> Key: SOLR-12388
> URL: https://issues.apache.org/jira/browse/SOLR-12388
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: search, SolrCloud
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Minor
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12388.patch
>
>
> Right now, a Solr node will return the results of a search request even if it
> cannot communicate with ZooKeeper at the time it receives the request. This
> may result in stale or incorrect results if there have been major changes to
> the collection structure that the node has not been informed of via
> ZooKeeper. When this happens, as long as all known shards respond, the
> response will succeed, and a {{zkConnected}} header set to {{false}} is
> included in the search response.
> There should be an option to instead fail requests under these conditions, to
> prevent stale or incorrect results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-9.0.4) - Build # 2025 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2025/ Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseG1GC 1 tests failed. FAILED: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded Error Message: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@7a02e1e2, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@bf29883] expected:<1> but was:<2> Stack Trace: java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@7a02e1e2, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@bf29883] expected:<1> but was:<2> at __randomizedtesting.SeedInfo.seed([B4921AA6A77AB73B:D1514CD105D91F38]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded(TestComputePlanAction.java:313) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.jav
[jira] [Commented] (SOLR-12297) Create a good SolrClient for SolrCloud paving the way for async requests, HTTP2, multiplexing, and the latest & greatest Jetty features.
[ https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497002#comment-16497002 ] Mark Miller commented on SOLR-12297: It's a unified client, not sure what that references. Jetty HttpClient was originally just HTTP/1.1 and still can be, that is how I first configured it and I left in the code to do it - it existed before there was any Jetty client side support for HTTP/2. For HTTP/2 there is actually a new lower level HTTP/2 client, but conveniently and unusual in the world of HTTP Java clients, the high level original Jetty HttpClient can do Async, HTTP/1.1, and HTTP/2 with the same API. Most other HTTP clients require using different client API's depending on which of those you want. So using the Jetty HttpClient without HTTP/2 is as easy as flipping the code path in Http2SolrClient to 1.1 support. We also can use the ALPN support to negotiate protocols on a single connector - which means we will be able to serve HTTP/1.1 and HTTP/2 on the same port depending on the clients capabilities. > Create a good SolrClient for SolrCloud paving the way for async requests, > HTTP2, multiplexing, and the latest & greatest Jetty features. > > > Key: SOLR-12297 > URL: https://issues.apache.org/jira/browse/SOLR-12297 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Mark Miller >Priority: Major > > Blocking or async support as well as HTTP2 compatible with multiplexing. > Once it supports enough and is stable, replace internal usage, allowing > async, and eventually move to HTTP2 connector and allow multiplexing. Could > support HTTP1.1 and HTTP2 on different ports depending on state of the world > then. > The goal of the client itself is to work against HTTP1.1 or HTTP2 with > minimal or no code path differences and the same for async requests (should > initially work for both 1.1 and 2 and share majority of code). > The client should also be able to replace HttpSolrClient and plug into the > other clients the same way. > I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually > though. > I evaluated some clients and while there are a few options, I went with > Jetty's HttpClient. It's more mature than Apache HttpClient's support (in 5 > beta) and we would have to update to a new API for Apache HttpClient anyway. > Meanwhile, the Jetty guys have been very supportive of helping Solr with any > issues and I like having the client and server from the same project. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 655 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/655/ Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC 5 tests failed. FAILED: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings Error Message: last stage: inconsistent endOffset at pos=18: 53 vs 61; token=pyjavey⦽yjaveywyjavey￢ Stack Trace: java.lang.IllegalStateException: last stage: inconsistent endOffset at pos=18: 53 vs 61; token=pyjavey⦽yjaveywyjavey￢ at __randomizedtesting.SeedInfo.seed([A1EFDDA8D1065D61:CBB462B988487D92]:0) at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:122) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:748) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:659) at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:561) at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:882) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings Error Message: last stage: inconsistent endOffset at pos=18:
[jira] [Commented] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
[
https://issues.apache.org/jira/browse/LUCENE-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496958#comment-16496958
]
Simon Willnauer commented on LUCENE-8341:
-
{quote}Why did you need the SoftDeletesFilterCodecReader?{quote}
I added this to make it easier to pass these readers to IW#addIndices
{quote}Can we also fix CheckIndex to verify the soft delete count? Hmm I guess
we cannot until/unless we record the soft deletes field in the index
(LUCENE-8335).{quote}
correct.
{quote}Do we have a test case that verifies the softDelCount after new soft
deletes (by applying DV updates)?{quote}
There are several tests that verify that in TestIndexWriter
> Record soft deletes in SegmentCommitInfo
> -
>
> Key: LUCENE-8341
> URL: https://issues.apache.org/jira/browse/LUCENE-8341
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8341.patch, LUCENE-8341.patch
>
>
> This change add the number of documents that are soft deletes but
> not hard deleted to the segment commit info. This is the last step
> towards making soft deletes as powerful as hard deltes since now the
> number of document can be read from commit points without opening a
> full blown reader. This also allows merge posliies to make decisions
> without requiring an NRT reader to get the relevant statistics. This
> change doesn't enforce any field to be used as soft deletes and the
> statistic
> is maintained per segment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-repro - Build # 731 - Still unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/731/ [...truncated 28 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/68/consoleText [repro] Revision: d27a2e8996199c395482d06284f5582eeaa8c181 [repro] Repro line: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeLostTriggerRestoreState -Dtests.seed=7D9D96624973904E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=nl-NL -Dtests.timezone=GMT0 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=TestGenericDistributedQueue -Dtests.method=testDistributedQueue -Dtests.seed=7D9D96624973904E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sr-Latn -Dtests.timezone=America/Rainy_River -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=TestGenericDistributedQueue -Dtests.seed=7D9D96624973904E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sr-Latn -Dtests.timezone=America/Rainy_River -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: ce8735556d994f365e9c95c61243c352a7d50e99 [repro] git fetch [repro] git checkout d27a2e8996199c395482d06284f5582eeaa8c181 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestGenericDistributedQueue [repro] TestTriggerIntegration [repro] ant compile-test [...truncated 3299 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.TestGenericDistributedQueue|*.TestTriggerIntegration" -Dtests.showOutput=onerror -Dtests.seed=7D9D96624973904E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sr-Latn -Dtests.timezone=America/Rainy_River -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 8942 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration [repro] Re-testing 100% failures at the tip of master [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestTriggerIntegration [repro] ant compile-test [...truncated 3299 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestTriggerIntegration" -Dtests.showOutput=onerror -Dtests.seed=7D9D96624973904E -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=nl-NL -Dtests.timezone=GMT0 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 6850 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of master: [repro] 4/5 failed: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration [repro] git checkout ce8735556d994f365e9c95c61243c352a7d50e99 [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 5 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12427) Status 500 on Incorrect value for start and rows
[ https://issues.apache.org/jira/browse/SOLR-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496940#comment-16496940 ] Jason Gerlowski commented on SOLR-12427: Thanks for the patch Munendra. I'll try and take this forward. Will take a look at the patch soon and get some feedback back to you. > Status 500 on Incorrect value for start and rows > > > Key: SOLR-12427 > URL: https://issues.apache.org/jira/browse/SOLR-12427 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Munendra S N >Assignee: Jason Gerlowski >Priority: Trivial > Attachments: SOLR-12427.patch, SOLR-12427.patch > > > With SOLR-7254, > Cases, when start and rows are negatives, was handled but the case when an > invalid value is passed is not handled. > Hence, Solr returns 500. It is better to return 400, as it is the client error -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-12427) Status 500 on Incorrect value for start and rows
[ https://issues.apache.org/jira/browse/SOLR-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gerlowski reassigned SOLR-12427: -- Assignee: Jason Gerlowski > Status 500 on Incorrect value for start and rows > > > Key: SOLR-12427 > URL: https://issues.apache.org/jira/browse/SOLR-12427 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Munendra S N >Assignee: Jason Gerlowski >Priority: Trivial > Attachments: SOLR-12427.patch, SOLR-12427.patch > > > With SOLR-7254, > Cases, when start and rows are negatives, was handled but the case when an > invalid value is passed is not handled. > Hence, Solr returns 500. It is better to return 400, as it is the client error -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12427) Status 500 on Incorrect value for start and rows
[
https://issues.apache.org/jira/browse/SOLR-12427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496936#comment-16496936
]
Lucene/Solr QA commented on SOLR-12427:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 4m 57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 4m 57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 4m 57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}203m 5s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}217m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.HttpPartitionTest |
| | solr.cloud.autoscaling.sim.TestLargeCluster |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12427 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12925787/SOLR-12427.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 7626308 |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 8 2015 |
| Default Java | 1.8.0_172 |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/109/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/109/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/109/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Status 500 on Incorrect value for start and rows
>
>
> Key: SOLR-12427
> URL: https://issues.apache.org/jira/browse/SOLR-12427
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Munendra S N
>Priority: Trivial
> Attachments: SOLR-12427.patch, SOLR-12427.patch
>
>
> With SOLR-7254,
> Cases, when start and rows are negatives, was handled but the case when an
> invalid value is passed is not handled.
> Hence, Solr returns 500. It is better to return 400, as it is the client error
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12297) Create a good SolrClient for SolrCloud paving the way for async requests, HTTP2, multiplexing, and the latest & greatest Jetty features.
[
https://issues.apache.org/jira/browse/SOLR-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496880#comment-16496880
]
Shawn Heisey commented on SOLR-12297:
-
In my travels trying to get the server working on starburst, I came across a
comment on this issue:
https://github.com/eclipse/jetty.project/issues/1308#issuecomment-277940891
Where it is revealed that jetty's Http2Client *only* talks to http2 servers.
The documentation link in the comment suggests that it might only be possible
for a single client object to speak one http version, because its transport
appears to be explicitly set.
If one jetty client object can only speak either http/1.1 or http/2, then that
causes us some difficulties. SolrClient implementations and the shard handler
will need to support both versions, and explicitly decide which version to use
for all connections. If we don't do that, rolling upgrades would be
impossible. A mixed-version cluster will have to force usage of the http/1.1
client until the entire cluster is upgraded.
Asking about this on the jetty IRC channel, this is the response I got:
{noformat}
10:59 <@jmcconnell> @elyograg @sbordet was working on a unified client at one
point, I think that was targeted for Jetty 10 but he would
know the latest and greatest on that
{noformat}
> Create a good SolrClient for SolrCloud paving the way for async requests,
> HTTP2, multiplexing, and the latest & greatest Jetty features.
>
>
> Key: SOLR-12297
> URL: https://issues.apache.org/jira/browse/SOLR-12297
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
>
> Blocking or async support as well as HTTP2 compatible with multiplexing.
> Once it supports enough and is stable, replace internal usage, allowing
> async, and eventually move to HTTP2 connector and allow multiplexing. Could
> support HTTP1.1 and HTTP2 on different ports depending on state of the world
> then.
> The goal of the client itself is to work against HTTP1.1 or HTTP2 with
> minimal or no code path differences and the same for async requests (should
> initially work for both 1.1 and 2 and share majority of code).
> The client should also be able to replace HttpSolrClient and plug into the
> other clients the same way.
> I doubt it would make sense to keep ConcurrentUpdateSolrClient eventually
> though.
> I evaluated some clients and while there are a few options, I went with
> Jetty's HttpClient. It's more mature than Apache HttpClient's support (in 5
> beta) and we would have to update to a new API for Apache HttpClient anyway.
> Meanwhile, the Jetty guys have been very supportive of helping Solr with any
> issues and I like having the client and server from the same project.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 1037 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1037/ No tests ran. Build Log: [...truncated 24176 lines...] [asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [java] Processed 2214 links (1768 relative) to 3105 anchors in 246 files [echo] Validated Links & Anchors via: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr-ref-guide/bare-bones-html/ -dist-changes: [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/changes -dist-keys: [get] Getting: http://home.apache.org/keys/group/lucene.asc [get] To: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/KEYS package: -unpack-solr-tgz: -ensure-solr-tgz-exists: [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked [untar] Expanding: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/solr-8.0.0.tgz into /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked generate-maven-artifacts: resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-leve
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1899 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1899/ Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC 22 tests failed. FAILED: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded Error Message: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@20ffd67f, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@48665ba3] expected:<1> but was:<2> Stack Trace: java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@20ffd67f, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@48665ba3] expected:<1> but was:<2> at __randomizedtesting.SeedInfo.seed([B5CA5DB2C891A636:D0090BC56A320E35]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded(TestComputePlanAction.java:313) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleI
[jira] [Commented] (LUCENE-7976) Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of very large segments
[ https://issues.apache.org/jira/browse/LUCENE-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496805#comment-16496805 ] Erick Erickson commented on LUCENE-7976: Hit another failure (one in 2004), chasing now. Looks like I've messed up the hitTooLarge in some weird case. Don't think it'll be anything major, FYI. > Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of > very large segments > - > > Key: LUCENE-7976 > URL: https://issues.apache.org/jira/browse/LUCENE-7976 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, > LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, > LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch > > > We're seeing situations "in the wild" where there are very large indexes (on > disk) handled quite easily in a single Lucene index. This is particularly > true as features like docValues move data into MMapDirectory space. The > current TMP algorithm allows on the order of 50% deleted documents as per a > dev list conversation with Mike McCandless (and his blog here: > https://www.elastic.co/blog/lucenes-handling-of-deleted-documents). > Especially in the current era of very large indexes in aggregate, (think many > TB) solutions like "you need to distribute your collection over more shards" > become very costly. Additionally, the tempting "optimize" button exacerbates > the issue since once you form, say, a 100G segment (by > optimizing/forceMerging) it is not eligible for merging until 97.5G of the > docs in it are deleted (current default 5G max segment size). > The proposal here would be to add a new parameter to TMP, something like > (no, that's not serious name, suggestions > welcome) which would default to 100 (or the same behavior we have now). > So if I set this parameter to, say, 20%, and the max segment size stays at > 5G, the following would happen when segments were selected for merging: > > any segment with > 20% deleted documents would be merged or rewritten NO > > MATTER HOW LARGE. There are two cases, > >> the segment has < 5G "live" docs. In that case it would be merged with > >> smaller segments to bring the resulting segment up to 5G. If no smaller > >> segments exist, it would just be rewritten > >> The segment has > 5G "live" docs (the result of a forceMerge or optimize). > >> It would be rewritten into a single segment removing all deleted docs no > >> matter how big it is to start. The 100G example above would be rewritten > >> to an 80G segment for instance. > Of course this would lead to potentially much more I/O which is why the > default would be the same behavior we see now. As it stands now, though, > there's no way to recover from an optimize/forceMerge except to re-index from > scratch. We routinely see 200G-300G Lucene indexes at this point "in the > wild" with 10s of shards replicated 3 or more times. And that doesn't even > include having these over HDFS. > Alternatives welcome! Something like the above seems minimally invasive. A > new merge policy is certainly an alternative. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5211) updating parent as childless makes old children orphans
[ https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496790#comment-16496790 ] David Smiley commented on SOLR-5211: I propose that *if* the schema declares \_root_ then we consider this to be a parent/child enabled schema, and every doc will get it's \_root_. If it does not, then each document stands alone. The presence of child docs should cause an error here, mentioning that \_root_ must be declared in the schema. > updating parent as childless makes old children orphans > --- > > Key: SOLR-5211 > URL: https://issues.apache.org/jira/browse/SOLR-5211 > Project: Solr > Issue Type: Sub-task > Components: update >Affects Versions: 4.5, 6.0 >Reporter: Mikhail Khludnev >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-5211.patch > > > if I have parent with children in the index, I can send update omitting > children. as a result old children become orphaned. > I suppose separate \_root_ fields makes much trouble. I propose to extend > notion of uniqueKey, and let it spans across blocks that makes updates > unambiguous. > WDYT? Do you like to see a test proves this issue? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11865) Refactor QueryElevationComponent to prepare query subset matching
[ https://issues.apache.org/jira/browse/SOLR-11865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496788#comment-16496788 ] Bruno Roustant commented on SOLR-11865: --- You're right MapElevationProvider.buildElevationMap should merge in this case (which indeed should not happen since they have been merged earlier). I have created the GitHub PR ([https://github.com/apache/lucene-solr/pull/390),] to be enhanced with all your improvements. > Refactor QueryElevationComponent to prepare query subset matching > - > > Key: SOLR-11865 > URL: https://issues.apache.org/jira/browse/SOLR-11865 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SearchComponents - other >Affects Versions: master (8.0) >Reporter: Bruno Roustant >Priority: Minor > Labels: QueryComponent > Fix For: master (8.0) > > Attachments: > 0001-Refactor-QueryElevationComponent-to-introduce-Elevat.patch, > 0002-Refactor-QueryElevationComponent-after-review.patch, > 0003-Remove-exception-handlers-and-refactor-getBoostDocs.patch, > SOLR-11865.patch > > Time Spent: 20m > Remaining Estimate: 0h > > The goal is to prepare a second improvement to support query terms subset > matching or query elevation rules. > Before that, we need to refactor the QueryElevationComponent. We make it > extendible. We introduce the ElevationProvider interface which will be > implemented later in a second patch to support subset matching. The current > full-query match policy becomes a default simple MapElevationProvider. > - Add overridable methods to handle exceptions during the component > initialization. > - Add overridable methods to provide the default values for config properties. > - No functional change beyond refactoring. > - Adapt unit test. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr issue #390: Refactor QueryElevationComponent to prepare query su...
Github user bruno-roustant commented on the issue: https://github.com/apache/lucene-solr/pull/390 @dsmiley here is the PR for QueryElevationComponent. --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #390: Refactor QueryElevationComponent to prepare q...
GitHub user bruno-roustant opened a pull request: https://github.com/apache/lucene-solr/pull/390 Refactor QueryElevationComponent to prepare query subset matching [SOLR-11865] See comments in https://issues.apache.org/jira/browse/SOLR-11865 You can merge this pull request into a Git repository by running: $ git pull https://github.com/bruno-roustant/lucene-solr QueryElevationComponent Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/390.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #390 commit 5a6290359c0e12f2372c8470de636796928921cc Author: broustant Date: 2018-01-12T17:03:26Z Refactor QueryElevationComponent to introduce ElevationProvider - Refactor to introduce ElevationProvider. The current full-query match policy becomes a default simple MapElevationProvider. It can be replaced by a more efficient provider in the future, or replaced by an extending class. - Add overridable methods to handle exceptions during the component initialization. - Add overridable methods to provide the default values for config properties. - No functional change beyond refactoring. - Adapt unit test. commit e9f53315ef0dc230280e93f868055183aa09abb6 Author: broustant Date: 2018-03-30T12:04:43Z Refactor QueryElevationComponent after review commit 0bad4c66cf4ce89bc6cca3a7e631c20b23c500c4 Author: broustant Date: 2018-04-04T15:51:03Z Remove exception handlers and refactor getBoostDocs --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-5211) updating parent as childless makes old children orphans
[
https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496774#comment-16496774
]
Mikhail Khludnev edited comment on SOLR-5211 at 5/31/18 4:08 PM:
-
This what we discussed at Vegas, [~hossman]. Let's treat all docs (even
childfree) parents. Here are pros, and cons:
(+) mixing parents and childfree (not yet in tests)
(+) overwriting childfree by parent (in tests)
(-) for those who run childfree-only index its waste index space for storing
useless {{\_root_}} column, as a workaround they can enable {{}} (I'm not happy about this name).
* atomic updates for parents is not there but just possible
I'd like to tweak the tests a little. Meanwhile, the questions are:
- Are we going this way?
- what release strategy is worth to take? eg make {{ParentAlwaysDUH2}} optional
in 7.4. but default in 8.0 or turn it by default right now
was (Author: mkhludnev):
This what we discussed at Vegas, [~hossman]. Let's treat all docs (even
childfree) parents. Here are pros, and cons:
(+) mixing parents and childfree (not yet in tests)
(+) overwriting childfree by parent (in tests)
(-) for those who run childfree-only index its waste index space for storing
useless {{\_root_}} column, as a workaround they can enable {{ }} (I'm not happy about this name).
* atomic updates for parents is not there but just possible
I'd like to tweak the tests a little. Meanwhile, the questions are:
- Are we going this way?
- what release strategy is worth to take? eg make {{ParentAlwaysDUH2}} optional
in 7.4. but default in 8.0 or turn it by default right now
> updating parent as childless makes old children orphans
> ---
>
> Key: SOLR-5211
> URL: https://issues.apache.org/jira/browse/SOLR-5211
> Project: Solr
> Issue Type: Sub-task
> Components: update
>Affects Versions: 4.5, 6.0
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-5211.patch
>
>
> if I have parent with children in the index, I can send update omitting
> children. as a result old children become orphaned.
> I suppose separate \_root_ fields makes much trouble. I propose to extend
> notion of uniqueKey, and let it spans across blocks that makes updates
> unambiguous.
> WDYT? Do you like to see a test proves this issue?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5211) updating parent as childless makes old children orphans
[
https://issues.apache.org/jira/browse/SOLR-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496774#comment-16496774
]
Mikhail Khludnev commented on SOLR-5211:
This what we discussed at Vegas, [~hossman]. Let's treat all docs (even
childfree) parents. Here are pros, and cons:
(+) mixing parents and childfree (not yet in tests)
(+) overwriting childfree by parent (in tests)
(-) for those who run childfree-only index its waste index space for storing
useless {{\_root_}} column, as a workaround they can enable {{ }} (I'm not happy about this name).
* atomic updates for parents is not there but just possible
I'd like to tweak the tests a little. Meanwhile, the questions are:
- Are we going this way?
- what release strategy is worth to take? eg make {{ParentAlwaysDUH2}} optional
in 7.4. but default in 8.0 or turn it by default right now
> updating parent as childless makes old children orphans
> ---
>
> Key: SOLR-5211
> URL: https://issues.apache.org/jira/browse/SOLR-5211
> Project: Solr
> Issue Type: Sub-task
> Components: update
>Affects Versions: 4.5, 6.0
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-5211.patch
>
>
> if I have parent with children in the index, I can send update omitting
> children. as a result old children become orphaned.
> I suppose separate \_root_ fields makes much trouble. I propose to extend
> notion of uniqueKey, and let it spans across blocks that makes updates
> unambiguous.
> WDYT? Do you like to see a test proves this issue?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [More Like This] I would like to contribute
Any update on this from the community? I would like to move forward with the contribution. The first part is involving just the More Like This parameters, but the rest will involve the interesting terms retrieval and quite a big piece of testing. No hard position on this, I was thinking that extracting the parameters was a good idea to improve the readability ( I noticed a similar approach in Solr for the edismax for example : DisMaxParams, ExtendedDismaxConfiguration ... ) but if the community prefers to keep them there, it's ok and I can move to the next steps. I attach here my considerations in response to Robert ones : "First of all thanks for your feedback, much appreciated. My initial goal was to make More Like This original 1000 lines class: - More Readable - Easier to Maintain and Extend - More Testable So I started identifying the different responsibilities in the More Like This class, to separate them, in the way that if I just need to change the scoring algorithm for the interesting terms I just touch the TermScorer ect ect : You see the overall idea in these slides : https://www.slideshare.net/AlessandroBenedetti/advanced-document-similarity-with-apache-lucene I tried to modify as less as possible the logic and tests at this stage. So let me wrap my considerations under different topics : *Parameters Abstraction* I see your point for just additional indirection/abstraction ( it is exactly just that with readability in mind). My scope for this was : "We have 600 lines of code of default and parameters for the MLT. How to make them isolated, more readable and extendable ?" And I initially thought to just put them in a separate class to remove that responsibility from the original MLT . So the focus was exclusively better readability and easy maintenance at this stage. Can you elaborate why you think this is undesired ? I don't have any strong feeling regarding this bit, so I am open to suggestions with the forementioned objective in mind. *MLT Immutable* I didn't consider it , but I am completely open to do that. In such case it could be worth to add a MoreLikeThis factory that manages the parameters and create the immutable MLT object ? *MoreLikeThisQuery* It was not in the scope of this refactor but I am absolutely happy to tackle that immediately as a next step, it could give it a try to see if there is space for improvement there. *Solr Tests* I completely agree, indeed as part of my additional tests which I have ready for the sequent refactors I introducedmuch more tests Lucene side than Solr side. We can elaborate this further at the right moment " Regards -- Alessandro Benedetti Search Consultant, R&D Software Engineer, Director www.sease.io On Tue, May 22, 2018 at 1:32 PM, Alessandro Benedetti wrote: > Thank you very much! > i will update the patch name and open a Pull Request with the correct Jira > issue key! > Thank you again, this is definitely a start! > > Cheers > > -- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > www.sease.io > > On Tue, May 22, 2018 at 1:29 PM, Robert Muir wrote: > >> >> >> On Tue, May 22, 2018 at 7:03 AM, Alessandro Benedetti < >> a.benede...@sease.io> wrote: >> >>> Hi Robert, >>> thanks for the feedback. >>> I read your comment last year and you I agreed completely. >>> So I started step by step, with the refactor first. >>> >>> The first contribution is isolating a part of the refactor, so no >>> functional change in the algorithms nor a complete refactor in place. >>> I basically tried to decompose the refactor is unitary pull requests as >>> small as possible. >>> It just focused on the MLT parameters first to reduce the size of the >>> original MoreLikeThis class ( and relegating the parameter modelling >>> responsibility to a separate class) >>> >>> https://issues.apache.org/jira/browse/SOLR-12299 >>> >>> The reason I used SOLR is because the refactor affects some Solr >>> components using the MLT. >>> But I agree with you, it can (should) be moved to LUCENE ( I tried via >>> JIRA but I don't think I have the right permissions). >>> >>> Should I just create a new JIRA issue completely ( closing the SOLR one) >>> or some JIRA admin can directly move the Jira to the LUCENE project ? >>> >>> >> I moved it: https://issues.apache.org/jira/browse/LUCENE-8326 >> >> >
[jira] [Commented] (LUCENE-6687) MLT term frequency calculation bug
[
https://issues.apache.org/jira/browse/LUCENE-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496744#comment-16496744
]
Alessandro Benedetti commented on LUCENE-6687:
--
I don't have the priviledges to change the status, it should be moved to "patch
available"
> MLT term frequency calculation bug
> --
>
> Key: LUCENE-6687
> URL: https://issues.apache.org/jira/browse/LUCENE-6687
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/query/scoring, core/queryparser
>Affects Versions: 5.2.1, 6.0
> Environment: OS X v10.10.4; Solr 5.2.1
>Reporter: Marko Bonaci
>Priority: Major
> Fix For: 5.2.2
>
> Attachments: LUCENE-6687.patch, LUCENE-6687.patch,
> buggy-method-usage.png, solr-mlt-tf-doubling-bug-results.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png,
> solr-mlt-tf-doubling-bug.png, terms-accumulator.png, terms-angry.png,
> terms-glass.png, terms-how.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In {{org.apache.lucene.queries.mlt.MoreLikeThis}}, there's a method
> {{retrieveTerms}} that receives a {{Map}} of fields, i.e. a document
> basically, but it doesn't have to be an existing doc.
> !solr-mlt-tf-doubling-bug.png|height=500!
> There are 2 for loops, one inside the other, which both loop through the same
> set of fields.
> That effectively doubles the term frequency for all the terms from fields
> that we provide in MLT QP {{qf}} parameter.
> It basically goes two times over the list of fields and accumulates the term
> frequencies from all fields into {{termFreqMap}}.
> The private method {{retrieveTerms}} is only called from one public method,
> the version of overloaded method {{like}} that receives a Map: so that
> private class member {{fieldNames}} is always derived from
> {{retrieveTerms}}'s argument {{fields}}.
>
> Uh, I don't understand what I wrote myself, but that basically means that, by
> the time {{retrieveTerms}} method gets called, its parameter fields and
> private member {{fieldNames}} always contain the same list of fields.
> Here's the proof:
> These are the final results of the calculation:
> !solr-mlt-tf-doubling-bug-results.png|height=700!
> And this is the actual {{thread_id:TID0009}} document, where those values
> were derived from (from fields {{title_mlt}} and {{pagetext_mlt}}):
> !terms-glass.png|height=100!
> !terms-angry.png|height=100!
> !terms-how.png|height=100!
> !terms-accumulator.png|height=100!
> Now, let's further test this hypothesis by seeing MLT QP in action from the
> AdminUI.
> Let's try to find docs that are More Like doc {{TID0009}}.
> Here's the interesting part, the query:
> {code}
> q={!mlt qf=pagetext_mlt,title_mlt mintf=14 mindf=2 minwl=3 maxwl=15}TID0009
> {code}
> We just saw, in the last image above, that the term accumulator appears {{7}}
> times in {{TID0009}} doc, but the {{accumulator}}'s TF was calculated as
> {{14}}.
> By using {{mintf=14}}, we say that, when calculating similarity, we don't
> want to consider terms that appear less than 14 times (when terms from fields
> {{title_mlt}} and {{pagetext_mlt}} are merged together) in {{TID0009}}.
> I added the term accumulator in only one other document ({{TID0004}}), where
> it appears only once, in the field {{title_mlt}}.
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png|height=500!
> Let's see what happens when we use {{mintf=15}}:
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png|height=500!
> I should probably mention that multiple fields ({{qf}}) work because I
> applied the patch:
> [SOLR-7143|https://issues.apache.org/jira/browse/SOLR-7143].
> Bug, no?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6687) MLT term frequency calculation bug
[
https://issues.apache.org/jira/browse/LUCENE-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496736#comment-16496736
]
Alessandro Benedetti commented on LUCENE-6687:
--
I double checked and the issue was still there.
The bug would manifest when using the MLT with the "live Lucene Document" which
is basically a map passed as a parameter to the like method in the MLT ( this
is used by Solr Cloud for example and the distributed More Like This) .
A minimal patch + test is attached.
I didn't include any refactor in here.
A complete refactor with better modularity and proper testing is under way,
unfortunately proceeding slowly.
The first piece is here :
https://issues.apache.org/jira/browse/LUCENE-8326
> MLT term frequency calculation bug
> --
>
> Key: LUCENE-6687
> URL: https://issues.apache.org/jira/browse/LUCENE-6687
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/query/scoring, core/queryparser
>Affects Versions: 5.2.1, 6.0
> Environment: OS X v10.10.4; Solr 5.2.1
>Reporter: Marko Bonaci
>Priority: Major
> Fix For: 5.2.2
>
> Attachments: LUCENE-6687.patch, LUCENE-6687.patch,
> buggy-method-usage.png, solr-mlt-tf-doubling-bug-results.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png,
> solr-mlt-tf-doubling-bug.png, terms-accumulator.png, terms-angry.png,
> terms-glass.png, terms-how.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In {{org.apache.lucene.queries.mlt.MoreLikeThis}}, there's a method
> {{retrieveTerms}} that receives a {{Map}} of fields, i.e. a document
> basically, but it doesn't have to be an existing doc.
> !solr-mlt-tf-doubling-bug.png|height=500!
> There are 2 for loops, one inside the other, which both loop through the same
> set of fields.
> That effectively doubles the term frequency for all the terms from fields
> that we provide in MLT QP {{qf}} parameter.
> It basically goes two times over the list of fields and accumulates the term
> frequencies from all fields into {{termFreqMap}}.
> The private method {{retrieveTerms}} is only called from one public method,
> the version of overloaded method {{like}} that receives a Map: so that
> private class member {{fieldNames}} is always derived from
> {{retrieveTerms}}'s argument {{fields}}.
>
> Uh, I don't understand what I wrote myself, but that basically means that, by
> the time {{retrieveTerms}} method gets called, its parameter fields and
> private member {{fieldNames}} always contain the same list of fields.
> Here's the proof:
> These are the final results of the calculation:
> !solr-mlt-tf-doubling-bug-results.png|height=700!
> And this is the actual {{thread_id:TID0009}} document, where those values
> were derived from (from fields {{title_mlt}} and {{pagetext_mlt}}):
> !terms-glass.png|height=100!
> !terms-angry.png|height=100!
> !terms-how.png|height=100!
> !terms-accumulator.png|height=100!
> Now, let's further test this hypothesis by seeing MLT QP in action from the
> AdminUI.
> Let's try to find docs that are More Like doc {{TID0009}}.
> Here's the interesting part, the query:
> {code}
> q={!mlt qf=pagetext_mlt,title_mlt mintf=14 mindf=2 minwl=3 maxwl=15}TID0009
> {code}
> We just saw, in the last image above, that the term accumulator appears {{7}}
> times in {{TID0009}} doc, but the {{accumulator}}'s TF was calculated as
> {{14}}.
> By using {{mintf=14}}, we say that, when calculating similarity, we don't
> want to consider terms that appear less than 14 times (when terms from fields
> {{title_mlt}} and {{pagetext_mlt}} are merged together) in {{TID0009}}.
> I added the term accumulator in only one other document ({{TID0004}}), where
> it appears only once, in the field {{title_mlt}}.
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png|height=500!
> Let's see what happens when we use {{mintf=15}}:
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png|height=500!
> I should probably mention that multiple fields ({{qf}}) work because I
> applied the patch:
> [SOLR-7143|https://issues.apache.org/jira/browse/SOLR-7143].
> Bug, no?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6687) MLT term frequency calculation bug
[
https://issues.apache.org/jira/browse/LUCENE-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Benedetti updated LUCENE-6687:
-
Attachment: LUCENE-6687.patch
> MLT term frequency calculation bug
> --
>
> Key: LUCENE-6687
> URL: https://issues.apache.org/jira/browse/LUCENE-6687
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/query/scoring, core/queryparser
>Affects Versions: 5.2.1, 6.0
> Environment: OS X v10.10.4; Solr 5.2.1
>Reporter: Marko Bonaci
>Priority: Major
> Fix For: 5.2.2
>
> Attachments: LUCENE-6687.patch, LUCENE-6687.patch,
> buggy-method-usage.png, solr-mlt-tf-doubling-bug-results.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png,
> solr-mlt-tf-doubling-bug.png, terms-accumulator.png, terms-angry.png,
> terms-glass.png, terms-how.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In {{org.apache.lucene.queries.mlt.MoreLikeThis}}, there's a method
> {{retrieveTerms}} that receives a {{Map}} of fields, i.e. a document
> basically, but it doesn't have to be an existing doc.
> !solr-mlt-tf-doubling-bug.png|height=500!
> There are 2 for loops, one inside the other, which both loop through the same
> set of fields.
> That effectively doubles the term frequency for all the terms from fields
> that we provide in MLT QP {{qf}} parameter.
> It basically goes two times over the list of fields and accumulates the term
> frequencies from all fields into {{termFreqMap}}.
> The private method {{retrieveTerms}} is only called from one public method,
> the version of overloaded method {{like}} that receives a Map: so that
> private class member {{fieldNames}} is always derived from
> {{retrieveTerms}}'s argument {{fields}}.
>
> Uh, I don't understand what I wrote myself, but that basically means that, by
> the time {{retrieveTerms}} method gets called, its parameter fields and
> private member {{fieldNames}} always contain the same list of fields.
> Here's the proof:
> These are the final results of the calculation:
> !solr-mlt-tf-doubling-bug-results.png|height=700!
> And this is the actual {{thread_id:TID0009}} document, where those values
> were derived from (from fields {{title_mlt}} and {{pagetext_mlt}}):
> !terms-glass.png|height=100!
> !terms-angry.png|height=100!
> !terms-how.png|height=100!
> !terms-accumulator.png|height=100!
> Now, let's further test this hypothesis by seeing MLT QP in action from the
> AdminUI.
> Let's try to find docs that are More Like doc {{TID0009}}.
> Here's the interesting part, the query:
> {code}
> q={!mlt qf=pagetext_mlt,title_mlt mintf=14 mindf=2 minwl=3 maxwl=15}TID0009
> {code}
> We just saw, in the last image above, that the term accumulator appears {{7}}
> times in {{TID0009}} doc, but the {{accumulator}}'s TF was calculated as
> {{14}}.
> By using {{mintf=14}}, we say that, when calculating similarity, we don't
> want to consider terms that appear less than 14 times (when terms from fields
> {{title_mlt}} and {{pagetext_mlt}} are merged together) in {{TID0009}}.
> I added the term accumulator in only one other document ({{TID0004}}), where
> it appears only once, in the field {{title_mlt}}.
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png|height=500!
> Let's see what happens when we use {{mintf=15}}:
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png|height=500!
> I should probably mention that multiple fields ({{qf}}) work because I
> applied the patch:
> [SOLR-7143|https://issues.apache.org/jira/browse/SOLR-7143].
> Bug, no?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #389: [LUCENE-6687] not necessary nested for loop r...
GitHub user alessandrobenedetti opened a pull request: https://github.com/apache/lucene-solr/pull/389 [LUCENE-6687] not necessary nested for loop removed for terms retriev⦠Bug in term frequencies calculation for the MLT You can merge this pull request into a Git repository by running: $ git pull https://github.com/SeaseLtd/lucene-solr LUCENE-6687 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/389.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #389 commit 4cc6731adfd1d697d528c0963765fa27a5ca0e6a Author: Alessandro Benedetti Date: 2018-05-31T15:39:15Z [LUCENE-6687] not necessary nested for loop removed for terms retrieval in More Like This --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7976) Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of very large segments
[ https://issues.apache.org/jira/browse/LUCENE-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496712#comment-16496712 ] Michael McCandless commented on LUCENE-7976: Thanks [~erickerickson]; I'll look at the new iteration soon! Sounds promising :) > Make TieredMergePolicy respect maxSegmentSizeMB and allow singleton merges of > very large segments > - > > Key: LUCENE-7976 > URL: https://issues.apache.org/jira/browse/LUCENE-7976 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, > LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch, > LUCENE-7976.patch, LUCENE-7976.patch, LUCENE-7976.patch > > > We're seeing situations "in the wild" where there are very large indexes (on > disk) handled quite easily in a single Lucene index. This is particularly > true as features like docValues move data into MMapDirectory space. The > current TMP algorithm allows on the order of 50% deleted documents as per a > dev list conversation with Mike McCandless (and his blog here: > https://www.elastic.co/blog/lucenes-handling-of-deleted-documents). > Especially in the current era of very large indexes in aggregate, (think many > TB) solutions like "you need to distribute your collection over more shards" > become very costly. Additionally, the tempting "optimize" button exacerbates > the issue since once you form, say, a 100G segment (by > optimizing/forceMerging) it is not eligible for merging until 97.5G of the > docs in it are deleted (current default 5G max segment size). > The proposal here would be to add a new parameter to TMP, something like > (no, that's not serious name, suggestions > welcome) which would default to 100 (or the same behavior we have now). > So if I set this parameter to, say, 20%, and the max segment size stays at > 5G, the following would happen when segments were selected for merging: > > any segment with > 20% deleted documents would be merged or rewritten NO > > MATTER HOW LARGE. There are two cases, > >> the segment has < 5G "live" docs. In that case it would be merged with > >> smaller segments to bring the resulting segment up to 5G. If no smaller > >> segments exist, it would just be rewritten > >> The segment has > 5G "live" docs (the result of a forceMerge or optimize). > >> It would be rewritten into a single segment removing all deleted docs no > >> matter how big it is to start. The 100G example above would be rewritten > >> to an 80G segment for instance. > Of course this would lead to potentially much more I/O which is why the > default would be the same behavior we see now. As it stands now, though, > there's no way to recover from an optimize/forceMerge except to re-index from > scratch. We routinely see 200G-300G Lucene indexes at this point "in the > wild" with 10s of shards replicated 3 or more times. And that doesn't even > include having these over HDFS. > Alternatives welcome! Something like the above seems minimally invasive. A > new merge policy is certainly an alternative. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
[
https://issues.apache.org/jira/browse/LUCENE-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496701#comment-16496701
]
Michael McCandless commented on LUCENE-8341:
+1
Maybe add javadocs for the new {{DocConsumer.getHasDocValues}}?
Why did you need the {{SoftDeletesFilterCodecReader}}?
Can we also fix {{CheckIndex}} to verify the soft delete count? Hmm I guess we
cannot until/unless we record the soft deletes field in the index (LUCENE-8335).
Do we have a test case that verifies the {{softDelCount}} after new soft
deletes (by applying DV updates)?
> Record soft deletes in SegmentCommitInfo
> -
>
> Key: LUCENE-8341
> URL: https://issues.apache.org/jira/browse/LUCENE-8341
> Project: Lucene - Core
> Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8341.patch, LUCENE-8341.patch
>
>
> This change add the number of documents that are soft deletes but
> not hard deleted to the segment commit info. This is the last step
> towards making soft deletes as powerful as hard deltes since now the
> number of document can be read from commit points without opening a
> full blown reader. This also allows merge posliies to make decisions
> without requiring an NRT reader to get the relevant statistics. This
> change doesn't enforce any field to be used as soft deletes and the
> statistic
> is maintained per segment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
[ https://issues.apache.org/jira/browse/LUCENE-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-8341: Attachment: LUCENE-8341.patch > Record soft deletes in SegmentCommitInfo > - > > Key: LUCENE-8341 > URL: https://issues.apache.org/jira/browse/LUCENE-8341 > Project: Lucene - Core > Issue Type: Improvement >Affects Versions: 7.4, master (8.0) >Reporter: Simon Willnauer >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: LUCENE-8341.patch, LUCENE-8341.patch > > > This change add the number of documents that are soft deletes but > not hard deleted to the segment commit info. This is the last step > towards making soft deletes as powerful as hard deltes since now the > number of document can be read from commit points without opening a > full blown reader. This also allows merge posliies to make decisions > without requiring an NRT reader to get the relevant statistics. This > change doesn't enforce any field to be used as soft deletes and the > statistic > is maintained per segment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk1.8.0) - Build # 675 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/675/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
9 tests failed.
FAILED:
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/48)={
"replicationFactor":"2", "pullReplicas":"0",
"router":{"name":"compositeId"}, "maxShardsPerNode":"2",
"autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0",
"autoCreated":"true", "shards":{ "shard2":{ "replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true", "SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":11}, "core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":11}}, "range":"0-7fff",
"state":"active"}, "shard1":{ "stateTimestamp":"1527778875353205950",
"replicas":{ "core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true", "SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":14}, "core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":14}}, "range":"8000-",
"state":"inactive"}, "shard1_1":{ "parent":"shard1",
"stateTimestamp":"1527778875377338700", "range":"c000-",
"state":"active", "replicas":{ "core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7}}}, "shard1_0":{
"parent":"shard1", "stateTimestamp":"1527778875376842250",
"range":"8000-bfff", "state":"active", "replicas":{
"core_node7":{ "leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7}
Stack Trace:
java.util.concurrent.TimeoutException: last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/48)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
"state"
[jira] [Commented] (SOLR-12366) Avoid SlowAtomicReader.getLiveDocs -- it's slow
[ https://issues.apache.org/jira/browse/SOLR-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496675#comment-16496675 ] ASF subversion and git services commented on SOLR-12366: Commit d65f40f3852be74bf0fc5c17d8252c669ea325d8 in lucene-solr's branch refs/heads/branch_7x from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d65f40f ] * SOLR-12366: A slow "live docs" implementation was being used instead of a bitset. Affects classic faceting enum method, JSON Facets enum method, UnInvertedField faceting, GraphTermsQParser, JoinQParser. Renamed SolrIndexSearcher.getLiveDocs to getLiveDocSet. (cherry picked from commit 1e63b32) > Avoid SlowAtomicReader.getLiveDocs -- it's slow > --- > > Key: SOLR-12366 > URL: https://issues.apache.org/jira/browse/SOLR-12366 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: search >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Attachments: SOLR-12366.patch, SOLR-12366.patch, SOLR-12366.patch, > SOLR-12366.patch > > > SlowAtomicReader is of course slow, and it's getLiveDocs (based on MultiBits) > is slow as it uses a binary search for each lookup. There are various places > in Solr that use SolrIndexSearcher.getSlowAtomicReader and then get the > liveDocs. Most of these places ought to work with SolrIndexSearcher's > getLiveDocs method. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12366) Avoid SlowAtomicReader.getLiveDocs -- it's slow
[ https://issues.apache.org/jira/browse/SOLR-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496671#comment-16496671 ] ASF subversion and git services commented on SOLR-12366: Commit ce8735556d994f365e9c95c61243c352a7d50e99 in lucene-solr's branch refs/heads/master from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ce87355 ] * SOLR-12366: A slow "live docs" implementation was being used instead of a bitset. Affects classic faceting enum method, JSON Facets enum method, UnInvertedField faceting, GraphTermsQParser, JoinQParser. Renamed SolrIndexSearcher.getLiveDocs to getLiveDocSet. > Avoid SlowAtomicReader.getLiveDocs -- it's slow > --- > > Key: SOLR-12366 > URL: https://issues.apache.org/jira/browse/SOLR-12366 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: search >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Attachments: SOLR-12366.patch, SOLR-12366.patch, SOLR-12366.patch, > SOLR-12366.patch > > > SlowAtomicReader is of course slow, and it's getLiveDocs (based on MultiBits) > is slow as it uses a binary search for each lookup. There are various places > in Solr that use SolrIndexSearcher.getSlowAtomicReader and then get the > liveDocs. Most of these places ought to work with SolrIndexSearcher's > getLiveDocs method. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12387) Have cluster-wide defaults for numShards, nrtReplicas, tlogReplicas, pullReplicas
[
https://issues.apache.org/jira/browse/SOLR-12387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496667#comment-16496667
]
ASF subversion and git services commented on SOLR-12387:
Commit bc58932b9d647661df37eb2b73db5cbd16a2c6b2 in lucene-solr's branch
refs/heads/branch_7x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bc58932 ]
SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,
pullReplicas
SOLR-12389: support deeply nested json objects in clusterprops.json
> Have cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,
> pullReplicas
> -
>
> Key: SOLR-12387
> URL: https://issues.apache.org/jira/browse/SOLR-12387
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Attachments: SOLR-12387.patch
>
>
> These will be cluster properties and the commands can omit these and the
> command would pick it up from the cluster properties
>
> the cluster property names are
> {code}
> "collectionDefaults": {
> "numShards":1
> "nrtReplicas":1
> "tlogReplicas":1
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12389) Support nested properties in cluster props
[
https://issues.apache.org/jira/browse/SOLR-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496668#comment-16496668
]
ASF subversion and git services commented on SOLR-12389:
Commit bc58932b9d647661df37eb2b73db5cbd16a2c6b2 in lucene-solr's branch
refs/heads/branch_7x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bc58932 ]
SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,
pullReplicas
SOLR-12389: support deeply nested json objects in clusterprops.json
> Support nested properties in cluster props
> --
>
> Key: SOLR-12389
> URL: https://issues.apache.org/jira/browse/SOLR-12389
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Environment: cluster props API does not support nested objects .
>
>
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> A new command is added to V2 endpoint to set deeply nested objects
> example 1:
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "collectionDefaults" :{
> "numShards": 3,
> "nrtreplicas": "2 ,
> "tlogReplicas":3,
> "pullReplicas" : 2
> }}}'
> {code}
> example 2:
> unset the value of {{numShards}}
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "collectionDefaults" :{
> "numShards": null
> }}}'
> {code}
> example 2:
> unset the value of {{numShards}}
> example 3:
> unset the entire {{collectionDefaults}} and set another value for another key
> all in one command
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "autoAddReplicas" : true,
> "collectionDefaults" :null}}'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12366) Avoid SlowAtomicReader.getLiveDocs -- it's slow
[ https://issues.apache.org/jira/browse/SOLR-12366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496663#comment-16496663 ] ASF subversion and git services commented on SOLR-12366: Commit 1e63b32731bedf108aaeeb5d0a04d671f5663102 in lucene-solr's branch refs/heads/master from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1e63b32 ] * SOLR-12366: A slow "live docs" implementation was being used instead of a bitset. Affects classic faceting enum method, JSON Facets enum method, UnInvertedField faceting, GraphTermsQParser, JoinQParser. Renamed SolrIndexSearcher.getLiveDocs to getLiveDocSet. > Avoid SlowAtomicReader.getLiveDocs -- it's slow > --- > > Key: SOLR-12366 > URL: https://issues.apache.org/jira/browse/SOLR-12366 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: search >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Attachments: SOLR-12366.patch, SOLR-12366.patch, SOLR-12366.patch, > SOLR-12366.patch > > > SlowAtomicReader is of course slow, and it's getLiveDocs (based on MultiBits) > is slow as it uses a binary search for each lookup. There are various places > in Solr that use SolrIndexSearcher.getSlowAtomicReader and then get the > liveDocs. Most of these places ought to work with SolrIndexSearcher's > getLiveDocs method. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12387) Have cluster-wide defaults for numShards, nrtReplicas, tlogReplicas, pullReplicas
[
https://issues.apache.org/jira/browse/SOLR-12387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-12387:
--
Description:
These will be cluster properties and the commands can omit these and the
command would pick it up from the cluster properties
the cluster property names are
{code}
"collectionDefaults": {
"numShards":1
"nrtReplicas":1
"tlogReplicas":1
}
{code}
was:
These will be cluster properties and the commands can omit these and the
command would pick it up from the cluster properties
the cluster property names are
* {{default.numShards}}
* {{default.nrtReplicas}}
* {{default.tlogReplicas}}
* {{default.pullReplicas}}
> Have cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,
> pullReplicas
> -
>
> Key: SOLR-12387
> URL: https://issues.apache.org/jira/browse/SOLR-12387
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Attachments: SOLR-12387.patch
>
>
> These will be cluster properties and the commands can omit these and the
> command would pick it up from the cluster properties
>
> the cluster property names are
> {code}
> "collectionDefaults": {
> "numShards":1
> "nrtReplicas":1
> "tlogReplicas":1
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12387) Have cluster-wide defaults for numShards, nrtReplicas, tlogReplicas, pullReplicas
[
https://issues.apache.org/jira/browse/SOLR-12387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496655#comment-16496655
]
ASF subversion and git services commented on SOLR-12387:
Commit 12269abe349eff6655f2c3ba96bbad7667b1c641 in lucene-solr's branch
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=12269ab ]
SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,
pullReplicas
SOLR-12389: support deeply nested json objects in clusterprops.json
> Have cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,
> pullReplicas
> -
>
> Key: SOLR-12387
> URL: https://issues.apache.org/jira/browse/SOLR-12387
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Attachments: SOLR-12387.patch
>
>
> These will be cluster properties and the commands can omit these and the
> command would pick it up from the cluster properties
>
> the cluster property names are
> * {{default.numShards}}
> * {{default.nrtReplicas}}
> * {{default.tlogReplicas}}
> * {{default.pullReplicas}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12389) Support nested properties in cluster props
[
https://issues.apache.org/jira/browse/SOLR-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496656#comment-16496656
]
ASF subversion and git services commented on SOLR-12389:
Commit 12269abe349eff6655f2c3ba96bbad7667b1c641 in lucene-solr's branch
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=12269ab ]
SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas,
pullReplicas
SOLR-12389: support deeply nested json objects in clusterprops.json
> Support nested properties in cluster props
> --
>
> Key: SOLR-12389
> URL: https://issues.apache.org/jira/browse/SOLR-12389
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Environment: cluster props API does not support nested objects .
>
>
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> A new command is added to V2 endpoint to set deeply nested objects
> example 1:
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "collectionDefaults" :{
> "numShards": 3,
> "nrtreplicas": "2 ,
> "tlogReplicas":3,
> "pullReplicas" : 2
> }}}'
> {code}
> example 2:
> unset the value of {{numShards}}
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "collectionDefaults" :{
> "numShards": null
> }}}'
> {code}
> example 2:
> unset the value of {{numShards}}
> example 3:
> unset the entire {{collectionDefaults}} and set another value for another key
> all in one command
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "autoAddReplicas" : true,
> "collectionDefaults" :null}}'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (LUCENE-6687) MLT term frequency calculation bug
[
https://issues.apache.org/jira/browse/LUCENE-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro Benedetti updated LUCENE-6687:
-
Comment: was deleted
(was: I just checked the source code and I find a different logic in place, I
assume this bug was fixed long time ago.
Can anyone close this Jira ?)
> MLT term frequency calculation bug
> --
>
> Key: LUCENE-6687
> URL: https://issues.apache.org/jira/browse/LUCENE-6687
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/query/scoring, core/queryparser
>Affects Versions: 5.2.1, 6.0
> Environment: OS X v10.10.4; Solr 5.2.1
>Reporter: Marko Bonaci
>Priority: Major
> Fix For: 5.2.2
>
> Attachments: LUCENE-6687.patch, buggy-method-usage.png,
> solr-mlt-tf-doubling-bug-results.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png,
> solr-mlt-tf-doubling-bug.png, terms-accumulator.png, terms-angry.png,
> terms-glass.png, terms-how.png
>
>
> In {{org.apache.lucene.queries.mlt.MoreLikeThis}}, there's a method
> {{retrieveTerms}} that receives a {{Map}} of fields, i.e. a document
> basically, but it doesn't have to be an existing doc.
> !solr-mlt-tf-doubling-bug.png|height=500!
> There are 2 for loops, one inside the other, which both loop through the same
> set of fields.
> That effectively doubles the term frequency for all the terms from fields
> that we provide in MLT QP {{qf}} parameter.
> It basically goes two times over the list of fields and accumulates the term
> frequencies from all fields into {{termFreqMap}}.
> The private method {{retrieveTerms}} is only called from one public method,
> the version of overloaded method {{like}} that receives a Map: so that
> private class member {{fieldNames}} is always derived from
> {{retrieveTerms}}'s argument {{fields}}.
>
> Uh, I don't understand what I wrote myself, but that basically means that, by
> the time {{retrieveTerms}} method gets called, its parameter fields and
> private member {{fieldNames}} always contain the same list of fields.
> Here's the proof:
> These are the final results of the calculation:
> !solr-mlt-tf-doubling-bug-results.png|height=700!
> And this is the actual {{thread_id:TID0009}} document, where those values
> were derived from (from fields {{title_mlt}} and {{pagetext_mlt}}):
> !terms-glass.png|height=100!
> !terms-angry.png|height=100!
> !terms-how.png|height=100!
> !terms-accumulator.png|height=100!
> Now, let's further test this hypothesis by seeing MLT QP in action from the
> AdminUI.
> Let's try to find docs that are More Like doc {{TID0009}}.
> Here's the interesting part, the query:
> {code}
> q={!mlt qf=pagetext_mlt,title_mlt mintf=14 mindf=2 minwl=3 maxwl=15}TID0009
> {code}
> We just saw, in the last image above, that the term accumulator appears {{7}}
> times in {{TID0009}} doc, but the {{accumulator}}'s TF was calculated as
> {{14}}.
> By using {{mintf=14}}, we say that, when calculating similarity, we don't
> want to consider terms that appear less than 14 times (when terms from fields
> {{title_mlt}} and {{pagetext_mlt}} are merged together) in {{TID0009}}.
> I added the term accumulator in only one other document ({{TID0004}}), where
> it appears only once, in the field {{title_mlt}}.
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png|height=500!
> Let's see what happens when we use {{mintf=15}}:
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png|height=500!
> I should probably mention that multiple fields ({{qf}}) work because I
> applied the patch:
> [SOLR-7143|https://issues.apache.org/jira/browse/SOLR-7143].
> Bug, no?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6687) MLT term frequency calculation bug
[
https://issues.apache.org/jira/browse/LUCENE-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496652#comment-16496652
]
Alessandro Benedetti commented on LUCENE-6687:
--
I just checked the source code and I find a different logic in place, I assume
this bug was fixed long time ago.
Can anyone close this Jira ?
> MLT term frequency calculation bug
> --
>
> Key: LUCENE-6687
> URL: https://issues.apache.org/jira/browse/LUCENE-6687
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/query/scoring, core/queryparser
>Affects Versions: 5.2.1, 6.0
> Environment: OS X v10.10.4; Solr 5.2.1
>Reporter: Marko Bonaci
>Priority: Major
> Fix For: 5.2.2
>
> Attachments: LUCENE-6687.patch, buggy-method-usage.png,
> solr-mlt-tf-doubling-bug-results.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png,
> solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png,
> solr-mlt-tf-doubling-bug.png, terms-accumulator.png, terms-angry.png,
> terms-glass.png, terms-how.png
>
>
> In {{org.apache.lucene.queries.mlt.MoreLikeThis}}, there's a method
> {{retrieveTerms}} that receives a {{Map}} of fields, i.e. a document
> basically, but it doesn't have to be an existing doc.
> !solr-mlt-tf-doubling-bug.png|height=500!
> There are 2 for loops, one inside the other, which both loop through the same
> set of fields.
> That effectively doubles the term frequency for all the terms from fields
> that we provide in MLT QP {{qf}} parameter.
> It basically goes two times over the list of fields and accumulates the term
> frequencies from all fields into {{termFreqMap}}.
> The private method {{retrieveTerms}} is only called from one public method,
> the version of overloaded method {{like}} that receives a Map: so that
> private class member {{fieldNames}} is always derived from
> {{retrieveTerms}}'s argument {{fields}}.
>
> Uh, I don't understand what I wrote myself, but that basically means that, by
> the time {{retrieveTerms}} method gets called, its parameter fields and
> private member {{fieldNames}} always contain the same list of fields.
> Here's the proof:
> These are the final results of the calculation:
> !solr-mlt-tf-doubling-bug-results.png|height=700!
> And this is the actual {{thread_id:TID0009}} document, where those values
> were derived from (from fields {{title_mlt}} and {{pagetext_mlt}}):
> !terms-glass.png|height=100!
> !terms-angry.png|height=100!
> !terms-how.png|height=100!
> !terms-accumulator.png|height=100!
> Now, let's further test this hypothesis by seeing MLT QP in action from the
> AdminUI.
> Let's try to find docs that are More Like doc {{TID0009}}.
> Here's the interesting part, the query:
> {code}
> q={!mlt qf=pagetext_mlt,title_mlt mintf=14 mindf=2 minwl=3 maxwl=15}TID0009
> {code}
> We just saw, in the last image above, that the term accumulator appears {{7}}
> times in {{TID0009}} doc, but the {{accumulator}}'s TF was calculated as
> {{14}}.
> By using {{mintf=14}}, we say that, when calculating similarity, we don't
> want to consider terms that appear less than 14 times (when terms from fields
> {{title_mlt}} and {{pagetext_mlt}} are merged together) in {{TID0009}}.
> I added the term accumulator in only one other document ({{TID0004}}), where
> it appears only once, in the field {{title_mlt}}.
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf14.png|height=500!
> Let's see what happens when we use {{mintf=15}}:
> !solr-mlt-tf-doubling-bug-verify-accumulator-mintf15.png|height=500!
> I should probably mention that multiple fields ({{qf}}) work because I
> applied the patch:
> [SOLR-7143|https://issues.apache.org/jira/browse/SOLR-7143].
> Bug, no?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-12370) NullPointerException on MoreLikeThisComponent
[
https://issues.apache.org/jira/browse/SOLR-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496646#comment-16496646
]
Alessandro Benedetti edited comment on SOLR-12370 at 5/31/18 2:42 PM:
--
I don't think this is a bug at all.
You are mis-using the suggester component.
Including the suggester radically change your response structure, not making
it compatible with the More Like This Component .
Calling a request handler with the suggest component will return suggestions
while calling a classic request handler will return documents ( and potentially
MLT sections + spellchecking + whatever configured)
Possibly the problem is with documentation, where the suggest component should
be explained maybe a little bit better.
If you remove the suggest component you should be fine
Hope it helps.
P.S. for the future I recommend to create a Jira issue only if you are sure it
is a bug.
So first I suggest to send an email to the solr-user mailing list.
was (Author: alessandro.benedetti):
I don't think this is a bug at all.
You are mis-using the suggester component.
Including the suggester radically change your response structure, not making it
compatible with the More Like This Component .
Calling a request handler with the suggest component will return suggestions
while calling a classic request handler will return documents ( and potentially
MLT sections + spellchecking + whatever configured)
Possibly the problem is with documentation, where the suggest component should
be explained maybe a little bit better.
If you remove the suggest component you should be fine
Hope it helps.
> NullPointerException on MoreLikeThisComponent
> -
>
> Key: SOLR-12370
> URL: https://issues.apache.org/jira/browse/SOLR-12370
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: MoreLikeThis
>Affects Versions: 7.3.1
>Reporter: Gilles Bodart
>Priority: Blocker
>
> I'm trying to use the MoreLikeThis component under a suggest call, but I
> receive a npe every time (here's the stacktrace)
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.MoreLikeThisComponent.process(MoreLikeThisComponent.java:127)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> ...{code}
> and here's the config of my requestHandlers:
> {code:java}
>
>
> true
> 10
> default
> true
> default
> wordbreak
> true
> true
> 10
> true
> true
> 5
> 5
> 10
> 5
> true
> _text_
> on
> content description title
> true
> html
>
>
>
>
> suggest
> spellcheck
> mlt
> highlight
>
>
>
> {code}
> I also tried with
> {code:java}
> on{code}
> When I call
> {code:java}
> /mlt?df=_text_&q=pann&mlt.fl=_text_
> {code}
> it works fine but with
> {code:java}
> /suggest?df=_text_&q=pann&mlt.fl=_text_
> {code}
> I got the npe
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12370) NullPointerException on MoreLikeThisComponent
[
https://issues.apache.org/jira/browse/SOLR-12370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496646#comment-16496646
]
Alessandro Benedetti commented on SOLR-12370:
-
I don't think this is a bug at all.
You are mis-using the suggester component.
Including the suggester radically change your response structure, not making it
compatible with the More Like This Component .
Calling a request handler with the suggest component will return suggestions
while calling a classic request handler will return documents ( and potentially
MLT sections + spellchecking + whatever configured)
Possibly the problem is with documentation, where the suggest component should
be explained maybe a little bit better.
If you remove the suggest component you should be fine
Hope it helps.
> NullPointerException on MoreLikeThisComponent
> -
>
> Key: SOLR-12370
> URL: https://issues.apache.org/jira/browse/SOLR-12370
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: MoreLikeThis
>Affects Versions: 7.3.1
>Reporter: Gilles Bodart
>Priority: Blocker
>
> I'm trying to use the MoreLikeThis component under a suggest call, but I
> receive a npe every time (here's the stacktrace)
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.solr.handler.component.MoreLikeThisComponent.process(MoreLikeThisComponent.java:127)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> ...{code}
> and here's the config of my requestHandlers:
> {code:java}
>
>
> true
> 10
> default
> true
> default
> wordbreak
> true
> true
> 10
> true
> true
> 5
> 5
> 10
> 5
> true
> _text_
> on
> content description title
> true
> html
>
>
>
>
> suggest
> spellcheck
> mlt
> highlight
>
>
>
> {code}
> I also tried with
> {code:java}
> on{code}
> When I call
> {code:java}
> /mlt?df=_text_&q=pann&mlt.fl=_text_
> {code}
> it works fine but with
> {code:java}
> /suggest?df=_text_&q=pann&mlt.fl=_text_
> {code}
> I got the npe
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr issue #385: WIP: SOLR-12361
Github user dsmiley commented on the issue: https://github.com/apache/lucene-solr/pull/385 it's a shame we've lost some consistency in AddUpdateCommand returning Lucene docs from both cmd.getLuceneDocument and cmd.iterable()... we've removed the iterable with something computing a List of SolrInputDocument (not a Lucene Document). It would be good to keep the use of DocumentBuilder to one place, either DirectUpdateHandler2 or AddUpdateCommand but not both. Hmmm; let me know what you think. --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #385: WIP: SOLR-12361
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/385#discussion_r192117086
--- Diff: solr/core/src/java/org/apache/solr/update/AddUpdateCommand.java
---
@@ -175,69 +172,83 @@ public String getHashableId() {
return id;
}
- public boolean isBlock() {
-return solrDoc.hasChildDocuments();
+ /**
+ * @return String id to hash
+ */
+ public String getHashableId() {
+return getHashableId(solrDoc);
}
- @Override
- public Iterator iterator() {
-return new Iterator() {
- Iterator iter;
-
- {
-List all = flatten(solrDoc);
-
-String idField = getHashableId();
-
-boolean isVersion = version != 0;
-
-for (SolrInputDocument sdoc : all) {
- sdoc.setField(IndexSchema.ROOT_FIELD_NAME, idField);
- if(isVersion) sdoc.setField(CommonParams.VERSION_FIELD, version);
- // TODO: if possible concurrent modification exception (if
SolrInputDocument not cloned and is being forwarded to replicas)
- // then we could add this field to the generated lucene document
instead.
-}
-
-iter = all.iterator();
- }
+ public List computeFlattenedDocs() {
+List all = flatten(solrDoc);
- @Override
- public boolean hasNext() {
-return iter.hasNext();
- }
+String rootId = getHashableId();
- @Override
- public Document next() {
-return DocumentBuilder.toDocument(iter.next(), req.getSchema());
- }
+boolean isVersion = version != 0;
- @Override
- public void remove() {
-throw new UnsupportedOperationException();
+for (SolrInputDocument sdoc : all) {
+ if (all.size() > 1) {
--- End diff --
oooh, I see now. I think we should try to clarify this; perhaps it may
be just a matter of method naming but probably a bit more. 3 cases ought to be
better distinguished:
* an in-place update. Will have no children concerns to deal with (no
\_root_ to set, not \_version\_) to set). Presence of children here should
throw an error.
* a normal add/update of one document (detected via having no children).
Here we don't set the \_root_ or \_version_
* a normal add/update of a set of a set of documents part of a tree.
Maybe I'll take a stab at this?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12389) Support nested properties in cluster props
[
https://issues.apache.org/jira/browse/SOLR-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-12389:
--
Description:
A new command is added to V2 endpoint to set deeply nested objects
example 1:
{code}
$ curl http://localhost:8983/api/cluster -H 'Content-type: application/json' -d
'
{ "set-obj-property": {
"collectionDefaults" :{
"numShards": 3,
"nrtreplicas": "2 ,
"tlogReplicas":3,
"pullReplicas" : 2
}}}'
{code}
example 2:
unset the value of {{numShards}}
{code}
$ curl http://localhost:8983/api/cluster -H 'Content-type: application/json' -d
'
{ "set-obj-property": {
"collectionDefaults" :{
"numShards": null
}}}'
{code}
example 2:
unset the value of {{numShards}}
example 3:
unset the entire {{collectionDefaults}} and set another value for another key
all in one command
{code}
$ curl http://localhost:8983/api/cluster -H 'Content-type: application/json' -d
'
{ "set-obj-property": {
"autoAddReplicas" : true,
"collectionDefaults" :null}}'
{code}
> Support nested properties in cluster props
> --
>
> Key: SOLR-12389
> URL: https://issues.apache.org/jira/browse/SOLR-12389
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Environment: cluster props API does not support nested objects .
>
>
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> A new command is added to V2 endpoint to set deeply nested objects
> example 1:
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "collectionDefaults" :{
> "numShards": 3,
> "nrtreplicas": "2 ,
> "tlogReplicas":3,
> "pullReplicas" : 2
> }}}'
> {code}
> example 2:
> unset the value of {{numShards}}
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "collectionDefaults" :{
> "numShards": null
> }}}'
> {code}
> example 2:
> unset the value of {{numShards}}
> example 3:
> unset the entire {{collectionDefaults}} and set another value for another key
> all in one command
> {code}
> $ curl http://localhost:8983/api/cluster -H 'Content-type: application/json'
> -d '
> { "set-obj-property": {
> "autoAddReplicas" : true,
> "collectionDefaults" :null}}'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #385: WIP: SOLR-12361
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/385#discussion_r192107030
--- Diff: solr/core/src/java/org/apache/solr/update/AddUpdateCommand.java
---
@@ -175,69 +172,83 @@ public String getHashableId() {
return id;
}
- public boolean isBlock() {
-return solrDoc.hasChildDocuments();
+ /**
+ * @return String id to hash
+ */
+ public String getHashableId() {
+return getHashableId(solrDoc);
}
- @Override
- public Iterator iterator() {
-return new Iterator() {
- Iterator iter;
-
- {
-List all = flatten(solrDoc);
-
-String idField = getHashableId();
-
-boolean isVersion = version != 0;
-
-for (SolrInputDocument sdoc : all) {
- sdoc.setField(IndexSchema.ROOT_FIELD_NAME, idField);
- if(isVersion) sdoc.setField(CommonParams.VERSION_FIELD, version);
- // TODO: if possible concurrent modification exception (if
SolrInputDocument not cloned and is being forwarded to replicas)
- // then we could add this field to the generated lucene document
instead.
-}
-
-iter = all.iterator();
- }
+ public List computeFlattenedDocs() {
+List all = flatten(solrDoc);
- @Override
- public boolean hasNext() {
-return iter.hasNext();
- }
+String rootId = getHashableId();
- @Override
- public Document next() {
-return DocumentBuilder.toDocument(iter.next(), req.getSchema());
- }
+boolean isVersion = version != 0;
- @Override
- public void remove() {
-throw new UnsupportedOperationException();
+for (SolrInputDocument sdoc : all) {
+ if (all.size() > 1) {
+sdoc.setField(IndexSchema.ROOT_FIELD_NAME, rootId);
}
-};
+ if(isVersion) sdoc.setField(CommonParams.VERSION_FIELD, version);
+ // TODO: if possible concurrent modification exception (if
SolrInputDocument not cloned and is being forwarded to replicas)
+ // then we could add this field to the generated lucene document
instead.
+}
+return all;
}
private List flatten(SolrInputDocument root) {
List unwrappedDocs = new ArrayList<>();
-recUnwrapp(unwrappedDocs, root);
+recUnwrapAnonymous(unwrappedDocs, root, true);
+recUnwrapRelations(unwrappedDocs, root, true);
if (1 < unwrappedDocs.size() && !
req.getSchema().isUsableForChildDocs()) {
throw new SolrException
(SolrException.ErrorCode.BAD_REQUEST, "Unable to index docs with
children: the schema must " +
"include definitions for both a uniqueKey field and the '" +
IndexSchema.ROOT_FIELD_NAME +
"' field, using the exact same fieldType");
}
+unwrappedDocs.add(root);
return unwrappedDocs;
}
- private void recUnwrapp(List unwrappedDocs,
SolrInputDocument currentDoc) {
+ /** Extract all child documents from parent that are saved in keys. */
+ private void recUnwrapRelations(List unwrappedDocs,
SolrInputDocument currentDoc, boolean isRoot) {
+for (SolrInputField field: currentDoc.values()) {
+ Object value = field.getFirstValue();
+ // check if value is a childDocument
+ if (value instanceof SolrInputDocument) {
+Object val = field.getValue();
+if (!(val instanceof Collection)) {
+ recUnwrapRelations(unwrappedDocs, ((SolrInputDocument) val));
+ continue;
+}
+Collection childrenList = ((Collection) val);
+for (SolrInputDocument child : childrenList) {
+ recUnwrapRelations(unwrappedDocs, child);
+}
+ }
+}
+
--- End diff --
but it's not specific to the input type?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8341) Record soft deletes in SegmentCommitInfo
Simon Willnauer created LUCENE-8341: --- Summary: Record soft deletes in SegmentCommitInfo Key: LUCENE-8341 URL: https://issues.apache.org/jira/browse/LUCENE-8341 Project: Lucene - Core Issue Type: Improvement Affects Versions: 7.4, master (8.0) Reporter: Simon Willnauer Fix For: 7.4, master (8.0) This change add the number of documents that are soft deletes but not hard deleted to the segment commit info. This is the last step towards making soft deletes as powerful as hard deltes since now the number of document can be read from commit points without opening a full blown reader. This also allows merge posliies to make decisions without requiring an NRT reader to get the relevant statistics. This change doesn't enforce any field to be used as soft deletes and the statistic is maintained per segment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8091) Better nearest-neighbor queries
[
https://issues.apache.org/jira/browse/LUCENE-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand resolved LUCENE-8091.
--
Resolution: Won't Fix
I'm closing this issue in favor of LUCENE-8340 which is less generic but
probably more useful.
> Better nearest-neighbor queries
> ---
>
> Key: LUCENE-8091
> URL: https://issues.apache.org/jira/browse/LUCENE-8091
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8091.patch
>
>
> LatLonPoint.nearest is very efficient at identifying the top-k documents
> sorted by distance from a given point, by working directly on the BKD tree.
> This doesn't support filtering though, so if you need to filter by another
> property, you need to switch to querying on the filter and sorting by a
> LatLonPointSortField. Unfortunately this requires visiting all documents that
> match the filter.
> We could leverage the new {{setMinCompetitiveScore}} API introduced in
> LUCENE-4100 in order to allow for retrieval of nearest neighbors with
> arbitrary filters, by recomputing a bounding-box when a new minimum
> competitive score is set.
> In the future we could also leverage this to speed up queries that are
> boosted by distance. For instance if the final score is a weighted sum of the
> score on a text field and a distance-based score, and the minimum competitive
> score gets higher than the maximum score that may be produced on the text
> field at some point, then we could dynamically prune hits based on the
> distance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4665 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4665/ Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC 2 tests failed. FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger Error Message: number of ops expected:<2> but was:<1> Stack Trace: java.lang.AssertionError: number of ops expected:<2> but was:<1> at __randomizedtesting.SeedInfo.seed([48497AE5E134A84:674FA12CC7DC39A9]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:187) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration Error Message: did not finish processing in time Stack Trace:
[jira] [Commented] (LUCENE-8340) Allow to boost by recency
[
https://issues.apache.org/jira/browse/LUCENE-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496509#comment-16496509
]
Adrien Grand commented on LUCENE-8340:
--
Here is a patch that implements such a query on LongPoint, since this is what
users would typically use to index dates. It also requires that the field has
doc values. It computes scores as {{pivotDistance / (distance +
pivotDistance)}} very similarly to {{FeatureField.newSaturationQuery}} and
computes the range that values must belong to in {{setMinCompetitiveScore}}
using points in order to efficiently skip non-competitive documents.
Currently it works on its own, but because it can't expose useful impacts (the
only thing it knows is that the maximum score is 1), we need to do some changes
to make boolean queries as well in order to propagate
{{setMinCompetitiveScore}} calls to their sub clauses so that we could also
skip non-competitive docs when boosting full-text matches by recency.
> Allow to boost by recency
> -
>
> Key: LUCENE-8340
> URL: https://issues.apache.org/jira/browse/LUCENE-8340
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8340.patch
>
>
> I would like that we support something like
> \{{FeatureField.newSaturationQuery}} but that works with features that are
> computed dynamically like recency or geo-distance, and is still optimized for
> top-hits collection. I'm starting with recency because it makes things a bit
> easier even though I suspect that geo-distance might be a more common need.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8340) Allow to boost by recency
[
https://issues.apache.org/jira/browse/LUCENE-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-8340:
-
Attachment: LUCENE-8340.patch
> Allow to boost by recency
> -
>
> Key: LUCENE-8340
> URL: https://issues.apache.org/jira/browse/LUCENE-8340
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-8340.patch
>
>
> I would like that we support something like
> \{{FeatureField.newSaturationQuery}} but that works with features that are
> computed dynamically like recency or geo-distance, and is still optimized for
> top-hits collection. I'm starting with recency because it makes things a bit
> easier even though I suspect that geo-distance might be a more common need.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8340) Allow to boost by recency
Adrien Grand created LUCENE-8340:
Summary: Allow to boost by recency
Key: LUCENE-8340
URL: https://issues.apache.org/jira/browse/LUCENE-8340
Project: Lucene - Core
Issue Type: Improvement
Reporter: Adrien Grand
I would like that we support something like
\{{FeatureField.newSaturationQuery}} but that works with features that are
computed dynamically like recency or geo-distance, and is still optimized for
top-hits collection. I'm starting with recency because it makes things a bit
easier even though I suspect that geo-distance might be a more common need.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 72 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/72/
3 tests failed.
FAILED:
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/96)={
"replicationFactor":"2", "pullReplicas":"0",
"router":{"name":"compositeId"}, "maxShardsPerNode":"2",
"autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0",
"autoCreated":"true", "shards":{ "shard2":{ "replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true", "SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":11}, "core_node4":{
"core":"testSplitIntegration_collection_shard2_replica_n4",
"SEARCHER.searcher.maxDoc":11, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":11}}, "range":"0-7fff",
"state":"active"}, "shard1":{ "stateTimestamp":"1527766312272905800",
"replicas":{ "core_node1":{
"core":"testSplitIntegration_collection_shard1_replica_n1",
"leader":"true", "SEARCHER.searcher.maxDoc":14,
"SEARCHER.searcher.deletedDocs":0, "INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr", "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":14}, "core_node2":{
"core":"testSplitIntegration_collection_shard1_replica_n2",
"SEARCHER.searcher.maxDoc":14, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"state":"active", "type":"NRT",
"SEARCHER.searcher.numDocs":14}}, "range":"8000-",
"state":"inactive"}, "shard1_1":{ "parent":"shard1",
"stateTimestamp":"1527766312306795050", "range":"c000-",
"state":"active", "replicas":{ "core_node10":{
"leader":"true",
"core":"testSplitIntegration_collection_shard1_1_replica1",
"SEARCHER.searcher.maxDoc":6, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":6},
"core_node9":{
"core":"testSplitIntegration_collection_shard1_1_replica0",
"SEARCHER.searcher.maxDoc":6, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":6}}}, "shard1_0":{
"parent":"shard1", "stateTimestamp":"1527766312306441750",
"range":"8000-bfff", "state":"active", "replicas":{
"core_node7":{ "leader":"true",
"core":"testSplitIntegration_collection_shard1_0_replica0",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:10001_solr",
"base_url":"http://127.0.0.1:10001/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7},
"core_node8":{
"core":"testSplitIntegration_collection_shard1_0_replica1",
"SEARCHER.searcher.maxDoc":7, "SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1, "node_name":"127.0.0.1:1_solr",
"base_url":"http://127.0.0.1:1/solr";, "state":"active",
"type":"NRT", "SEARCHER.searcher.numDocs":7}
Stack Trace:
java.util.concurrent.TimeoutException: last state:
DocCollection(testSplitIntegration_collection//clusterstate.json/96)={
"replicationFactor":"2",
"pullReplicas":"0",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0",
"autoCreated":"true",
"shards":{
"shard2":{
"replicas":{
"core_node3":{
"core":"testSplitIntegration_collection_shard2_replica_n3",
"leader":"true",
"SEARCHER.searcher.maxDoc":11,
"SEARCHER.searcher.deletedDocs":0,
"INDEX.sizeInBytes":1,
"node_name":"127.0.0.1:10001_solr",
"state":"active",
"type":"NRT",
"SEARC
[JENKINS] Lucene-Solr-repro - Build # 729 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/729/ [...truncated 28 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/71/consoleText [repro] Revision: 04e1b19743e330ce66d199c4dc40bbf394be9ed7 [repro] Repro line: ant test -Dtestcase=TestIndexFileDeleter -Dtests.method=testExcInDecRef -Dtests.seed=8E9564CE40DFE608 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es -Dtests.timezone=Africa/Bamako -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=AutoscalingHistoryHandlerTest -Dtests.method=testHistory -Dtests.seed=DA7AFF85E69C76BE -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=id-ID -Dtests.timezone=JST -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testSearchRate -Dtests.seed=DA7AFF85E69C76BE -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=hr -Dtests.timezone=America/Aruba -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=ScheduledMaintenanceTriggerTest -Dtests.method=testInactiveShardCleanup -Dtests.seed=DA7AFF85E69C76BE -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ga-IE -Dtests.timezone=Asia/Bishkek -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=StreamDecoratorTest -Dtests.method=testExecutorStream -Dtests.seed=884B63D8356BE556 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sv -Dtests.timezone=Canada/Mountain -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=StreamDecoratorTest -Dtests.seed=884B63D8356BE556 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sv -Dtests.timezone=Canada/Mountain -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 76263087b5828446fa3afd05743a8383b75893fb [repro] git fetch [repro] git checkout 04e1b19743e330ce66d199c4dc40bbf394be9ed7 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]lucene/core [repro] TestIndexFileDeleter [repro]solr/solrj [repro] StreamDecoratorTest [repro]solr/core [repro] AutoscalingHistoryHandlerTest [repro] TestTriggerIntegration [repro] ScheduledMaintenanceTriggerTest [repro] ant compile-test [...truncated 147 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestIndexFileDeleter" -Dtests.showOutput=onerror -Dtests.seed=8E9564CE40DFE608 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es -Dtests.timezone=Africa/Bamako -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 113 lines...] [repro] ant compile-test [...truncated 2412 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.StreamDecoratorTest" -Dtests.showOutput=onerror -Dtests.seed=884B63D8356BE556 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sv -Dtests.timezone=Canada/Mountain -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 325 lines...] [repro] ant compile-test [...truncated 1331 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 -Dtests.class="*.AutoscalingHistoryHandlerTest|*.TestTriggerIntegration|*.ScheduledMaintenanceTriggerTest" -Dtests.showOutput=onerror -Dtests.seed=DA7AFF85E69C76BE -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=id-ID -Dtests.timezone=JST -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 11459 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 0/5 failed: org.apache.lucene.index.TestIndexFileDeleter [repro] 0/5 failed: org.apache.solr.client.solrj.io.stream.StreamDecoratorTest [repro] 1/5 failed: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest [repro] 4/5 failed: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest [repro] 4/5 failed: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration [repro] git checkout 76263087b5828446fa3afd05743a8383b75893fb [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 5 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7161) TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery assertion error
[
https://issues.apache.org/jira/browse/LUCENE-7161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496438#comment-16496438
]
Alessandro Benedetti commented on LUCENE-7161:
--
Just tried on current master all the seeds that had problem in various branches
here :
_794526110651C8E6 - OK_
25E751FED53FC993 - OK
60AAD450C5F7A579 - OK
E467DF1643BE90EA - OK
12E0331668C5EB5 - OK
3FA5C26ECE58C917- OK
116FB7FCD72BFF28 - OK
F41FAA899068BC32 - OK
C802AA860A1EAE50 - OK
I also tried myself to run the test various times, with different random seeds,
not able to reproduce any failure.
Is anybody able to reproduce this issue anymore ?
I would be more than happy to debug it and fix it.
> TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery assertion
> error
> ---
>
> Key: LUCENE-7161
> URL: https://issues.apache.org/jira/browse/LUCENE-7161
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: Michael McCandless
>Assignee: Tommaso Teofili
>Priority: Major
> Fix For: 6.7, 7.0
>
>
> I just hit this unrelated but reproducible on master
> #cc75be53f9b3b86ec59cb93896c4fd5a9a5926b2 while tweaking earth's radius:
> {noformat}
>[junit4] Suite: org.apache.lucene.queries.mlt.TestMoreLikeThis
>[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestMoreLikeThis
> -Dtests.method=testMultiFieldShouldReturnPerFieldBooleanQuery
> -Dtests.seed=794526110651C8E6 -Dtests.locale=es-HN
> -Dtests.timezone=Brazil/West -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 0.25s |
> TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery <<<
>[junit4]> Throwable #1: java.lang.AssertionError
>[junit4]> at
> __randomizedtesting.SeedInfo.seed([794526110651C8E6:1DF67ED7BBBF4E1D]:0)
>[junit4]> at
> org.apache.lucene.queries.mlt.TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery(TestMoreLikeThis.java:320)
>[junit4]> at java.lang.Thread.run(Thread.java:745)
>[junit4] 2> NOTE: test params are: codec=CheapBastard,
> sim=ClassicSimilarity, locale=es-HN, timezone=Brazil/West
>[junit4] 2> NOTE: Linux 3.13.0-71-generic amd64/Oracle Corporation
> 1.8.0_60 (64-bit)/cpus=8,threads=1,free=409748864,total=504889344
>[junit4] 2> NOTE: All tests run in this JVM: [TestMoreLikeThis]
>[junit4] Completed [1/1 (1!)] in 0.45s, 1 test, 1 failure <<< FAILURES!
>[junit4]
>[junit4]
>[junit4] Tests with failures [seed: 794526110651C8E6]:
>[junit4] -
> org.apache.lucene.queries.mlt.TestMoreLikeThis.testMultiFieldShouldReturnPerFieldBooleanQuery
> {noformat}
> Likely related to LUCENE-6954?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-EA] Lucene-Solr-7.x-Windows (64bit/jdk-11-ea+14) - Build # 621 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/621/ Java: 64bit/jdk-11-ea+14 -XX:+UseCompressedOops -XX:+UseSerialGC 13 tests failed. FAILED: org.apache.solr.common.util.TestTimeSource.testEpochTime Error Message: SimTimeSource:50.0 time diff=27735000 Stack Trace: java.lang.AssertionError: SimTimeSource:50.0 time diff=27735000 at __randomizedtesting.SeedInfo.seed([740C7D79A7885D50:4C600E5C3358FF16]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.common.util.TestTimeSource.doTestEpochTime(TestTimeSource.java:52) at org.apache.solr.common.util.TestTimeSource.testEpochTime(TestTimeSource.java:32) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.base/java.lang.Thread.run(Thread.java:832) FAILED: org.apache.solr.common.util.TestTimeSource.testEpochTime Error Message: SimTimeSource:50.0 time diff=1407 Stack Trace: java.lang.AssertionError: SimTimeSource:50.0 time diff=1407 at __randomizedt
[jira] [Created] (SOLR-12431) Remove sleep polling from code base.
Mark Miller created SOLR-12431: -- Summary: Remove sleep polling from code base. Key: SOLR-12431 URL: https://issues.apache.org/jira/browse/SOLR-12431 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: Mark Miller Both tests and main code. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
