[jira] [Commented] (SOLR-12291) Async prematurely reports completed status that causes severe shard loss
[
https://issues.apache.org/jira/browse/SOLR-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828910#comment-16828910
]
Lucene/Solr QA commented on SOLR-12291:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 4m 4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 3m 59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 3m 59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m
20s{color} | {color:green} core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
33s{color} | {color:green} test-framework in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}105m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12291 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12967330/SOLR-12291.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 8dd22bc |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/391/testReport/ |
| modules | C: solr/core solr/test-framework U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/391/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Async prematurely reports completed status that causes severe shard loss
>
>
> Key: SOLR-12291
> URL: https://issues.apache.org/jira/browse/SOLR-12291
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Backup/Restore, SolrCloud
>Reporter: Varun Thacker
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-12291.patch, SOLR-12291.patch, SOLR-12291.patch,
> SOLR-12291.patch, SOLR-12291.patch, SOLR-12291.patch, SOLR-122911.patch
>
>
> The OverseerCollectionMessageHandler sliceCmd assumes only one replica exists
> on one node
> When multiple replicas of a slice are on the same node we only track one
> replica's async request. This happens because the async requestMap's key is
> "node_name"
> I discovered this when [~alabax] shared some logs of a restore issue, where
> the second replica got added before the first replica had completed it's
> restorecore action.
> While looking at the logs I noticed that the overseer never called
> REQUESTSTATUS for the restorecore action , almost as if it had missed
> tracking that particular async request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10860) in-place updates currently throw NumberFormatException instead of a Bad Request SolrException for bad input
[ https://issues.apache.org/jira/browse/SOLR-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Munendra S N updated SOLR-10860: Attachment: SOLR-10860.patch > in-place updates currently throw NumberFormatException instead of a Bad > Request SolrException for bad input > --- > > Key: SOLR-10860 > URL: https://issues.apache.org/jira/browse/SOLR-10860 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Tomás Fernández Löbbe >Assignee: Ishan Chattopadhyaya >Priority: Major > Attachments: SOLR-10860.patch, SOLR-10860.patch, SOLR-10860.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10860) in-place updates currently throw NumberFormatException instead of a Bad Request SolrException for bad input
[ https://issues.apache.org/jira/browse/SOLR-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828900#comment-16828900 ] Munendra S N commented on SOLR-10860: - [^SOLR-10860.patch] Rebased to the latest master > in-place updates currently throw NumberFormatException instead of a Bad > Request SolrException for bad input > --- > > Key: SOLR-10860 > URL: https://issues.apache.org/jira/browse/SOLR-10860 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Tomás Fernández Löbbe >Assignee: Ishan Chattopadhyaya >Priority: Major > Attachments: SOLR-10860.patch, SOLR-10860.patch, SOLR-10860.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12914) Solr crashes in /terms request handler
[
https://issues.apache.org/jira/browse/SOLR-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828895#comment-16828895
]
Munendra S N commented on SOLR-12914:
-
This issue is same as SOLR-13337
> Solr crashes in /terms request handler
> --
>
> Key: SOLR-12914
> URL: https://issues.apache.org/jira/browse/SOLR-12914
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 7.5
>Reporter: Vadim Miller
>Priority: Major
> Labels: terms
> Attachments: terms.patch
>
>
> TermsComponent class always tries to fetch all terms from all shards for a
> further processing. There is {{java.lang.OutOfMemoryError}} exception if
> the resulting list is too long. Solr stops working on this shard after this
> exception, only restart helps.
> There is a very common use case when the full terms list is not required: a
> client needs to see next N terms in alphabetically sorted list starting with
> a given value. Usually, this is needed for some autocomplete field on a page.
> Example URL:
>
> {{[http://localhost:8983/solr/mycollection/terms?terms.fl=fulltext=index=cat=50]}}
>
> In this example TermsComponent needs to fetch only 50 terms from each shard
> starting with a value provided in {{terms.lower}} URL parameter. So, it
> should not reset TermsParams.TERMS_LIMIT parameter when generates a shard
> query in createSmartShardQuery() method.
> The patch is attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13432) Add .toString methods to BitDocSet and SortedIntDocSet
[ https://issues.apache.org/jira/browse/SOLR-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828896#comment-16828896 ] Shalin Shekhar Mangar commented on SOLR-13432: -- Slight modifications to use size() instead of size in both classes because the cached size value in BitDocSet may not be correct in case it is backed by a FixedBitSet. > Add .toString methods to BitDocSet and SortedIntDocSet > -- > > Key: SOLR-13432 > URL: https://issues.apache.org/jira/browse/SOLR-13432 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: SOLR-13432.patch, SOLR-13432.patch > > > Add .toString methods to BitDocSet and SortedIntDocSet. The motivation here > is that the "showItems" property on FastLRUCache tries to show the items > inside the cache. But the output is not useful because these two classes > (used as values for filter cache) do not implement toString so the > Object.toString is called. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13432) Add .toString methods to BitDocSet and SortedIntDocSet
[ https://issues.apache.org/jira/browse/SOLR-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-13432: - Attachment: SOLR-13432.patch > Add .toString methods to BitDocSet and SortedIntDocSet > -- > > Key: SOLR-13432 > URL: https://issues.apache.org/jira/browse/SOLR-13432 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: SOLR-13432.patch, SOLR-13432.patch > > > Add .toString methods to BitDocSet and SortedIntDocSet. The motivation here > is that the "showItems" property on FastLRUCache tries to show the items > inside the cache. But the output is not useful because these two classes > (used as values for filter cache) do not implement toString so the > Object.toString is called. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13432) Add .toString methods to BitDocSet and SortedIntDocSet
[ https://issues.apache.org/jira/browse/SOLR-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828894#comment-16828894 ] Shalin Shekhar Mangar commented on SOLR-13432: -- Here's a simple patch that adds toString method to both classes with size (number of docs) and ramUsed (human readable value of ramBytesUsed()). > Add .toString methods to BitDocSet and SortedIntDocSet > -- > > Key: SOLR-13432 > URL: https://issues.apache.org/jira/browse/SOLR-13432 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: SOLR-13432.patch > > > Add .toString methods to BitDocSet and SortedIntDocSet. The motivation here > is that the "showItems" property on FastLRUCache tries to show the items > inside the cache. But the output is not useful because these two classes > (used as values for filter cache) do not implement toString so the > Object.toString is called. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13432) Add .toString methods to BitDocSet and SortedIntDocSet
Shalin Shekhar Mangar created SOLR-13432: Summary: Add .toString methods to BitDocSet and SortedIntDocSet Key: SOLR-13432 URL: https://issues.apache.org/jira/browse/SOLR-13432 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Reporter: Shalin Shekhar Mangar Assignee: Shalin Shekhar Mangar Fix For: 8.1, master (9.0) Add .toString methods to BitDocSet and SortedIntDocSet. The motivation here is that the "showItems" property on FastLRUCache tries to show the items inside the cache. But the output is not useful because these two classes (used as values for filter cache) do not implement toString so the Object.toString is called. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13432) Add .toString methods to BitDocSet and SortedIntDocSet
[ https://issues.apache.org/jira/browse/SOLR-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-13432: - Attachment: SOLR-13432.patch > Add .toString methods to BitDocSet and SortedIntDocSet > -- > > Key: SOLR-13432 > URL: https://issues.apache.org/jira/browse/SOLR-13432 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Shalin Shekhar Mangar >Assignee: Shalin Shekhar Mangar >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: SOLR-13432.patch > > > Add .toString methods to BitDocSet and SortedIntDocSet. The motivation here > is that the "showItems" property on FastLRUCache tries to show the items > inside the cache. But the output is not useful because these two classes > (used as values for filter cache) do not implement toString so the > Object.toString is called. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 347 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/347/ 6 tests failed. FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test Error Message: acoll: 1556501333538 bcoll: 1556501333648 Stack Trace: java.lang.AssertionError: acoll: 1556501333538 bcoll: 1556501333648 at __randomizedtesting.SeedInfo.seed([1343863FEBDC9C97:9B17B9E54520F16F]:0) at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.solr.cloud.MultiThreadedOCPTest.testFillWorkQueue(MultiThreadedOCPTest.java:116) at org.apache.solr.cloud.MultiThreadedOCPTest.test(MultiThreadedOCPTest.java:71) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at
[jira] [Commented] (SOLR-13320) add a param ignoreDuplicates=true to updates to not overwrite existing docs
[
https://issues.apache.org/jira/browse/SOLR-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828873#comment-16828873
]
Noble Paul commented on SOLR-13320:
---
[~tomasflobbe]
Well, no. IIRC {{DocBasedVersionConstraintsProcessor}} can skip the docs based
on the _{{version_}} field in the document (not from a request param)
> add a param ignoreDuplicates=true to updates to not overwrite existing docs
> ---
>
> Key: SOLR-13320
> URL: https://issues.apache.org/jira/browse/SOLR-13320
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> Updates should have an option to ignore duplicate documents and drop them if
> an option {{ignoreDuplicates=true}} is specified
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13320) add a param ignoreDuplicates=true to updates to not overwrite existing docs
[
https://issues.apache.org/jira/browse/SOLR-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828861#comment-16828861
]
Tomás Fernández Löbbe commented on SOLR-13320:
--
With {{DocBasedVersionConstraintsProcessor}} you can tell Solr to skip
documents that have a higher (or equal) version than the one you are trying to
add (see {{ignoreOldUpdates}}). Isn't that what you need?
> add a param ignoreDuplicates=true to updates to not overwrite existing docs
> ---
>
> Key: SOLR-13320
> URL: https://issues.apache.org/jira/browse/SOLR-13320
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
>
> Updates should have an option to ignore duplicate documents and drop them if
> an option {{ignoreDuplicates=true}} is specified
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
msokolov commented on a change in pull request #657: LUCENE-8781: FST direct
arc addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279216926
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FSTEnum.java
##
@@ -141,11 +141,65 @@ protected void doSeekCeil() throws IOException {
//System.out.println(" cycle upto=" + upto + " arc.label=" + arc.label
+ " (" + (char) arc.label + ") vs targetLabel=" + targetLabel);
if (arc.bytesPerArc != 0 && arc.label != -1) {
+// Arcs are in an array
Review comment:
After all, this revision includes break-up into smaller functions; it's
definitely easier to read, and I guess could feed into some restructuring into
classes later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-8.x - Build # 173 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-8.x/173/ 1 tests failed. FAILED: org.apache.solr.security.AuditLoggerIntegrationTest.testAsyncQueueDrain Error Message: Failed waiting for 3 callbacks after 30 seconds Stack Trace: java.lang.AssertionError: Failed waiting for 3 callbacks after 30 seconds at __randomizedtesting.SeedInfo.seed([F4F92CA55A0CAED9:45E6D381EACF3D06]:0) at org.junit.Assert.fail(Assert.java:88) at org.apache.solr.security.AuditLoggerIntegrationTest.waitForAuditEventCallbacks(AuditLoggerIntegrationTest.java:229) at org.apache.solr.security.AuditLoggerIntegrationTest.waitForAuditEventCallbacks(AuditLoggerIntegrationTest.java:221) at org.apache.solr.security.AuditLoggerIntegrationTest.assertThreeAdminEvents(AuditLoggerIntegrationTest.java:252) at org.apache.solr.security.AuditLoggerIntegrationTest.testAsyncQueueDrain(AuditLoggerIntegrationTest.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at
[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 1321 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1321/ No tests ran. Build Log: [...truncated 23468 lines...] [asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [java] Processed 2526 links (2067 relative) to 3355 anchors in 253 files [echo] Validated Links & Anchors via: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr-ref-guide/bare-bones-html/ -dist-changes: [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/changes package: -unpack-solr-tgz: -ensure-solr-tgz-exists: [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked [untar] Expanding: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/solr-9.0.0.tgz into /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked generate-maven-artifacts: resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail:
[GitHub] [lucene-solr] msokolov commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
msokolov commented on a change in pull request #657: LUCENE-8781: FST direct
arc addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279211690
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FST.java
##
@@ -632,36 +641,85 @@ long addNode(Builder builder,
Builder.UnCompiledNode nodeIn) throws IOExce
if (doFixedArray) {
final int MAX_HEADER_SIZE = 11; // header(byte) + numArcs(vint) +
numBytes(vint)
assert maxBytesPerArc > 0;
- // 2nd pass just "expands" all arcs to take up a fixed
- // byte size
+ // 2nd pass just "expands" all arcs to take up a fixed byte size
+
+ // If more than 1/2 of the "slots" would be occupied, write an arc array
that may have holes in it
+ // so that we can address the arcs directly by label without binary
search
+ // TODO: There's no need to write the labels in this case; we can just
store the first label in the header
+ // TODO: tune this heuristic?
+ // TODO: write multiple direct blocks with gaps between?
+ int labelRange = nodeIn.arcs[nodeIn.numArcs - 1].label -
nodeIn.arcs[0].label + 1;
+ boolean writeDirectly = builder.useDirectArcAddressing && labelRange > 0
&& labelRange < 4 * nodeIn.numArcs;
//System.out.println("write int @pos=" + (fixedArrayStart-4) + "
numArcs=" + nodeIn.numArcs);
// create the header
// TODO: clean this up: or just rewind+reuse and deal with it
byte header[] = new byte[MAX_HEADER_SIZE];
ByteArrayDataOutput bad = new ByteArrayDataOutput(header);
// write a "false" first arc:
- bad.writeByte(ARCS_AS_FIXED_ARRAY);
- bad.writeVInt(nodeIn.numArcs);
+ if (writeDirectly) {
+bad.writeByte(ARCS_AS_DIRECT_ARRAY);
+bad.writeVInt(labelRange);
+ } else {
+bad.writeByte(ARCS_AS_FIXED_ARRAY);
+bad.writeVInt(nodeIn.numArcs);
+ }
bad.writeVInt(maxBytesPerArc);
int headerLen = bad.getPosition();
final long fixedArrayStart = startAddress + headerLen;
- // expand the arcs in place, backwards
- long srcPos = builder.bytes.getPosition();
- long destPos = fixedArrayStart + nodeIn.numArcs*maxBytesPerArc;
- assert destPos >= srcPos;
- if (destPos > srcPos) {
-builder.bytes.skipBytes((int) (destPos - srcPos));
-for(int arcIdx=nodeIn.numArcs-1;arcIdx>=0;arcIdx--) {
- destPos -= maxBytesPerArc;
- srcPos -= builder.reusedBytesPerArc[arcIdx];
- //System.out.println(" repack arcIdx=" + arcIdx + " srcPos=" +
srcPos + " destPos=" + destPos);
- if (srcPos != destPos) {
-//System.out.println(" copy len=" +
builder.reusedBytesPerArc[arcIdx]);
-assert destPos > srcPos: "destPos=" + destPos + " srcPos=" +
srcPos + " arcIdx=" + arcIdx + " maxBytesPerArc=" + maxBytesPerArc + "
reusedBytesPerArc[arcIdx]=" + builder.reusedBytesPerArc[arcIdx] + "
nodeIn.numArcs=" + nodeIn.numArcs;
-builder.bytes.copyBytes(srcPos, destPos,
builder.reusedBytesPerArc[arcIdx]);
+ if (writeDirectly) {
Review comment:
OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
msokolov commented on a change in pull request #657: LUCENE-8781: FST direct
arc addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279211075
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FST.java
##
@@ -632,36 +641,85 @@ long addNode(Builder builder,
Builder.UnCompiledNode nodeIn) throws IOExce
if (doFixedArray) {
final int MAX_HEADER_SIZE = 11; // header(byte) + numArcs(vint) +
numBytes(vint)
assert maxBytesPerArc > 0;
- // 2nd pass just "expands" all arcs to take up a fixed
- // byte size
+ // 2nd pass just "expands" all arcs to take up a fixed byte size
+
+ // If more than 1/2 of the "slots" would be occupied, write an arc array
that may have holes in it
Review comment:
Right it should have said 1/4, and I agree a symbol of some kind is called
for. I think I'll add a constant, because rIght now the Builder has no (other)
settable members; in practice every setting is passed on the constructor: it's
not actually a Builder in the builder-pattern sense, and adding a constructor
arg will be kind of intrusive given there is not yet any call for it elsewhere.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-master #2549: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-master/2549/ No tests ran. Build Log: [...truncated 18103 lines...] BUILD FAILED /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/build.xml:673: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/build.xml:209: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/lucene/build.xml:408: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/lucene/common-build.xml:1648: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-master/lucene/common-build.xml:581: Error deploying artifact 'org.apache.lucene:lucene-test-framework:jar': Error installing artifact's metadata: repository metadata for: 'artifact org.apache.lucene:lucene-test-framework' could not be retrieved from repository: apache.snapshots.https due to an error: Error transferring file: Connection refused (Connection refused) Total time: 9 minutes 2 seconds Build step 'Invoke Ant' marked build as failure Email was triggered for: Failure - Any Sending email for trigger: Failure - Any - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
msokolov commented on a change in pull request #657: LUCENE-8781: FST direct
arc addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279210936
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FSTEnum.java
##
@@ -141,11 +141,65 @@ protected void doSeekCeil() throws IOException {
//System.out.println(" cycle upto=" + upto + " arc.label=" + arc.label
+ " (" + (char) arc.label + ") vs targetLabel=" + targetLabel);
if (arc.bytesPerArc != 0 && arc.label != -1) {
+// Arcs are in an array
Review comment:
I was thinking of introducing multiple Arc classes so that the (now three)
different kinds of Arc decoding methods can be modeled explicitly with function
overrides and the branching between different implementations done using
virtual function dispatch rather than inline conditionals. That would likely
subsume any refactoring here, so I'd like to postpone it, but I definitely
agree with the temptation. Still, if you think it would make the review easier,
I'd be happy to dive into that, but overall it will probably lead to more code
changes making direct comparison difficult, and I was thinking to do a separate
a refactoring commit with no behavior changes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12291) Async prematurely reports completed status that causes severe shard loss
[
https://issues.apache.org/jira/browse/SOLR-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828831#comment-16828831
]
Lucene/Solr QA commented on SOLR-12291:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 45m 43s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.ReindexCollectionTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12291 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12967309/SOLR-12291.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 8dd22bc |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | LTS |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/390/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/390/testReport/ |
| modules | C: solr/core U: solr/core |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/390/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Async prematurely reports completed status that causes severe shard loss
>
>
> Key: SOLR-12291
> URL: https://issues.apache.org/jira/browse/SOLR-12291
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Backup/Restore, SolrCloud
>Reporter: Varun Thacker
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-12291.patch, SOLR-12291.patch, SOLR-12291.patch,
> SOLR-12291.patch, SOLR-12291.patch, SOLR-122911.patch
>
>
> The OverseerCollectionMessageHandler sliceCmd assumes only one replica exists
> on one node
> When multiple replicas of a slice are on the same node we only track one
> replica's async request. This happens because the async requestMap's key is
> "node_name"
> I discovered this when [~alabax] shared some logs of a restore issue, where
> the second replica got added before the first replica had completed it's
> restorecore action.
> While looking at the logs I noticed that the overseer never called
> REQUESTSTATUS for the restorecore action , almost as if it had missed
> tracking that particular async request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] msokolov commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
msokolov commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279210601 ## File path: lucene/core/src/java/org/apache/lucene/util/fst/Builder.java ## @@ -69,6 +69,8 @@ private final IntsRefBuilder lastInput = new IntsRefBuilder(); + final boolean useDirectArcAddressing; Review comment: FST reads this so it can't be private. This is following the same pattern as some existing settings like `allowArrayArcs`, which by the way should probably be final but isn't. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Solr-reference-guide-master - Build # 15491 - Failure
Build: https://builds.apache.org/job/Solr-reference-guide-master/15491/ Log: Started by timer [EnvInject] - Loading node environment variables. Building remotely on websites1 (git-websites svn-websites) in workspace /home/jenkins/jenkins-slave/workspace/Solr-reference-guide-master No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url > https://gitbox.apache.org/repos/asf/lucene-solr.git # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Fetching upstream changes from https://gitbox.apache.org/repos/asf/lucene-solr.git > git --version # timeout=10 > git fetch --tags --progress > https://gitbox.apache.org/repos/asf/lucene-solr.git > +refs/heads/*:refs/remotes/origin/* ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from https://gitbox.apache.org/repos/asf/lucene-solr.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:894) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1810) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --progress https://gitbox.apache.org/repos/asf/lucene-solr.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: fatal: unable to access 'https://gitbox.apache.org/repos/asf/lucene-solr.git/': gnutls_handshake() failed: Error in the pull function. at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2042) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1761) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$400(CliGitAPIImpl.java:72) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:442) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 37.48.69.226/37.48.69.226:52074 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:955) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.GeneratedMethodAccessor927.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy115.execute(Unknown Source) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:892) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1161) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1192) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
[jira] [Closed] (LUCENE-8774) Performance improvement for update?optimize=true
[ https://issues.apache.org/jira/browse/LUCENE-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley closed LUCENE-8774. > Performance improvement for update?optimize=true > > > Key: LUCENE-8774 > URL: https://issues.apache.org/jira/browse/LUCENE-8774 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: trunk, 7.2.1 >Reporter: Johann Abraham >Priority: Major > Fix For: LUCENE-8558 > > Attachments: > 0001-Performance-improvement-for-update-optimize-true.patch > > > Running optimize on a document set that has a large number of docValue fields > grows linearly with the number of docValue fields. > We discovered this because our corpus has few documents but a very large > number of dynamic docValue fields. > The cause is in the FilterFieldInfos constructor, a linear search of the > fielterFields occurs due to the filterFields.contains(...) function call. > Changing the underlying Collection type from an ArrayList to a LinkedHashSet > decreased our optimize time from 60 minutes to < 1 minute. Patch attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8774) Performance improvement for update?optimize=true
[ https://issues.apache.org/jira/browse/LUCENE-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley resolved LUCENE-8774. -- Resolution: Duplicate Fix Version/s: LUCENE-8558 Thanks for your efforts but this issue was already identified and fixed in LUCENE-8558 released in v7.6. > Performance improvement for update?optimize=true > > > Key: LUCENE-8774 > URL: https://issues.apache.org/jira/browse/LUCENE-8774 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: trunk, 7.2.1 >Reporter: Johann Abraham >Priority: Major > Fix For: LUCENE-8558 > > Attachments: > 0001-Performance-improvement-for-update-optimize-true.patch > > > Running optimize on a document set that has a large number of docValue fields > grows linearly with the number of docValue fields. > We discovered this because our corpus has few documents but a very large > number of dynamic docValue fields. > The cause is in the FilterFieldInfos constructor, a linear search of the > fielterFields occurs due to the filterFields.contains(...) function call. > Changing the underlying Collection type from an ArrayList to a LinkedHashSet > decreased our optimize time from 60 minutes to < 1 minute. Patch attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8780) Improve ByteBufferGuard in Java 11
[
https://issues.apache.org/jira/browse/LUCENE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828826#comment-16828826
]
Mike Sokolov commented on LUCENE-8780:
--
I don't have a good theory, but I was curious so I ran a few tests, and one
thing I saw is that if you limit to a single searcher thread, you see only the
negative side of this distribution, or at least it becomes more negative.
> Improve ByteBufferGuard in Java 11
> --
>
> Key: LUCENE-8780
> URL: https://issues.apache.org/jira/browse/LUCENE-8780
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/store
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
> Labels: Java11
> Attachments: LUCENE-8780.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In LUCENE-7409 we added {{ByteBufferGuard}} to protect MMapDirectory from
> crushing the JVM with SIGSEGV when you close and unmap the mmapped buffers of
> an IndexInput, while another thread is accessing it.
> The idea was to do a volatile write access to flush the caches (to trigger a
> full fence) and set a non-volatile boolean to true. All accesses would check
> the boolean and stop the caller from accessing the underlying ByteBuffer.
> This worked most of the time, until the JVM optimized away the plain read
> access to the boolean (you can easily see this after some runtime of our
> by-default ignored testcase).
> With master on Java 11, we can improve the whole thing. Using VarHandles you
> can use any access type when reading or writing the boolean. After reading
> Doug Lea's expanation and some
> testing, I was no longer able to crush my JDK (even after running for minutes
> unmapping bytebuffers).
> The apraoch is the same, we do a full-fenced write (standard volatile write)
> when we unmap, then we yield the thread (to finish in-flight reads in other
> threads) and then unmap all byte buffers.
> On the test side (read access), instead of using a plain read, we use the new
> "opaque read". Opaque reads are the same as plain reads, there are only
> different order requirements. Actually the main difference is explained by
> Doug like this: "For example in constructions in which the only modification
> of some variable x is for one thread to write in Opaque (or stronger) mode,
> X.setOpaque(this, 1), any other thread spinning in
> while(X.getOpaque(this)!=1){} will eventually terminate. Note that this
> guarantee does NOT hold in Plain mode, in which spin loops may (and usually
> do) infinitely loop -- they are not required to notice that a write ever
> occurred in another thread if it was not seen on first encounter." - And
> that's waht we want to have: We don't want to do volatile reads, but we want
> to prevent the compiler from optimizing away our read to the boolean. So we
> want it to "eventually" see the change. By the much stronger volatile write,
> the cache effects should be visible even faster (like in our Java 8 approach,
> just now we improved our read side).
> The new code is much slimmer (theoretically we could also use a AtomicBoolean
> for that and use the new method {{getOpaque()}}, but I wanted to prevent
> extra method calls, so I used a VarHandle directly).
> It's setup like this:
> - The underlying boolean field is a private member (with unused
> SuppressWarnings, as its unused by the java compiler), marked as volatile
> (that's the recommendation, but in reality it does not matter at all).
> - We create a VarHandle to access this boolean, we never do this directly
> (this is why the volatile marking does not affect us).
> - We use VarHandle.setVolatile() to change our "invalidated" boolean to
> "true", so enforcing a full fence
> - On the read side we use VarHandle.getOpaque() instead of VarHandle.get()
> (like in our old code for Java 8).
> I had to tune our test a bit, as the VarHandles make it take longer until it
> actually crushes (as optimizations jump in later). I also used a random for
> the reads to prevent the optimizer from removing all the bytebuffer reads.
> When we commit this, we can disable the test again (it takes approx 50 secs
> on my machine).
> I'd still like to see the differences between the plain read and the opaque
> read in production, so maybe [~mikemccand] or [~rcmuir] can do a comparison
> with nightly benchmarker?
> Have fun, maybe [~dweiss] has some ideas, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc
addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279208083
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FSTEnum.java
##
@@ -141,11 +141,65 @@ protected void doSeekCeil() throws IOException {
//System.out.println(" cycle upto=" + upto + " arc.label=" + arc.label
+ " (" + (char) arc.label + ") vs targetLabel=" + targetLabel);
if (arc.bytesPerArc != 0 && arc.label != -1) {
+// Arcs are in an array
Review comment:
I feel tempted to split those very large methods into smaller units
(methods); it'd really make them clearer.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc
addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279208192
##
File path: lucene/core/src/test/org/apache/lucene/util/fst/Test2BFST.java
##
@@ -50,12 +50,12 @@ public void test() throws Exception {
for(int iter=0;iter<1;iter++) {
// Build FST w/ NoOutputs and stop when nodeCount > 2.2B
- {
+if (false) {
Review comment:
should be reverted (introduces dead code block).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc
addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279208389
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FST.java
##
@@ -632,36 +641,85 @@ long addNode(Builder builder,
Builder.UnCompiledNode nodeIn) throws IOExce
if (doFixedArray) {
final int MAX_HEADER_SIZE = 11; // header(byte) + numArcs(vint) +
numBytes(vint)
assert maxBytesPerArc > 0;
- // 2nd pass just "expands" all arcs to take up a fixed
- // byte size
+ // 2nd pass just "expands" all arcs to take up a fixed byte size
+
+ // If more than 1/2 of the "slots" would be occupied, write an arc array
that may have holes in it
Review comment:
Is this "1/2" consistent with the condition below (4 * nodeIn.numArcs)?
Also, I think it'd be better to have the proportion as at least a constant or
(better?) a configurable setting on the builder (not necessarily a constructor
attribute).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc
addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279207910
##
File path:
lucene/codecs/src/java/org/apache/lucene/codecs/blocktreeords/OrdsSegmentTermsEnum.java
##
@@ -1082,12 +1082,14 @@ private InputOutput getByOutput(long targetOrd) throws
IOException {
if (FST.targetHasArcs(arc)) {
// System.out.println(" targetHasArcs");
result.grow(1+upto);
-
+if (arc.target < 0 || arc.target > Integer.MAX_VALUE) {
Review comment:
What's the point of this block (and isn't it effectively dead code)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc
addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279208432
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FST.java
##
@@ -632,36 +641,85 @@ long addNode(Builder builder,
Builder.UnCompiledNode nodeIn) throws IOExce
if (doFixedArray) {
final int MAX_HEADER_SIZE = 11; // header(byte) + numArcs(vint) +
numBytes(vint)
assert maxBytesPerArc > 0;
- // 2nd pass just "expands" all arcs to take up a fixed
- // byte size
+ // 2nd pass just "expands" all arcs to take up a fixed byte size
+
+ // If more than 1/2 of the "slots" would be occupied, write an arc array
that may have holes in it
+ // so that we can address the arcs directly by label without binary
search
+ // TODO: There's no need to write the labels in this case; we can just
store the first label in the header
+ // TODO: tune this heuristic?
+ // TODO: write multiple direct blocks with gaps between?
+ int labelRange = nodeIn.arcs[nodeIn.numArcs - 1].label -
nodeIn.arcs[0].label + 1;
+ boolean writeDirectly = builder.useDirectArcAddressing && labelRange > 0
&& labelRange < 4 * nodeIn.numArcs;
//System.out.println("write int @pos=" + (fixedArrayStart-4) + "
numArcs=" + nodeIn.numArcs);
// create the header
// TODO: clean this up: or just rewind+reuse and deal with it
byte header[] = new byte[MAX_HEADER_SIZE];
ByteArrayDataOutput bad = new ByteArrayDataOutput(header);
// write a "false" first arc:
- bad.writeByte(ARCS_AS_FIXED_ARRAY);
- bad.writeVInt(nodeIn.numArcs);
+ if (writeDirectly) {
+bad.writeByte(ARCS_AS_DIRECT_ARRAY);
Review comment:
Perhaps change constants to something reflecting reality here --
ARCS_AS_ARRAY_DENSE, ARCS_AS_ARRAY_WITH_GAPS?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc
addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279208474
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FST.java
##
@@ -632,36 +641,85 @@ long addNode(Builder builder,
Builder.UnCompiledNode nodeIn) throws IOExce
if (doFixedArray) {
final int MAX_HEADER_SIZE = 11; // header(byte) + numArcs(vint) +
numBytes(vint)
assert maxBytesPerArc > 0;
- // 2nd pass just "expands" all arcs to take up a fixed
- // byte size
+ // 2nd pass just "expands" all arcs to take up a fixed byte size
+
+ // If more than 1/2 of the "slots" would be occupied, write an arc array
that may have holes in it
+ // so that we can address the arcs directly by label without binary
search
+ // TODO: There's no need to write the labels in this case; we can just
store the first label in the header
+ // TODO: tune this heuristic?
+ // TODO: write multiple direct blocks with gaps between?
+ int labelRange = nodeIn.arcs[nodeIn.numArcs - 1].label -
nodeIn.arcs[0].label + 1;
+ boolean writeDirectly = builder.useDirectArcAddressing && labelRange > 0
&& labelRange < 4 * nodeIn.numArcs;
//System.out.println("write int @pos=" + (fixedArrayStart-4) + "
numArcs=" + nodeIn.numArcs);
// create the header
// TODO: clean this up: or just rewind+reuse and deal with it
byte header[] = new byte[MAX_HEADER_SIZE];
ByteArrayDataOutput bad = new ByteArrayDataOutput(header);
// write a "false" first arc:
- bad.writeByte(ARCS_AS_FIXED_ARRAY);
- bad.writeVInt(nodeIn.numArcs);
+ if (writeDirectly) {
+bad.writeByte(ARCS_AS_DIRECT_ARRAY);
+bad.writeVInt(labelRange);
+ } else {
+bad.writeByte(ARCS_AS_FIXED_ARRAY);
+bad.writeVInt(nodeIn.numArcs);
+ }
bad.writeVInt(maxBytesPerArc);
int headerLen = bad.getPosition();
final long fixedArrayStart = startAddress + headerLen;
- // expand the arcs in place, backwards
- long srcPos = builder.bytes.getPosition();
- long destPos = fixedArrayStart + nodeIn.numArcs*maxBytesPerArc;
- assert destPos >= srcPos;
- if (destPos > srcPos) {
-builder.bytes.skipBytes((int) (destPos - srcPos));
-for(int arcIdx=nodeIn.numArcs-1;arcIdx>=0;arcIdx--) {
- destPos -= maxBytesPerArc;
- srcPos -= builder.reusedBytesPerArc[arcIdx];
- //System.out.println(" repack arcIdx=" + arcIdx + " srcPos=" +
srcPos + " destPos=" + destPos);
- if (srcPos != destPos) {
-//System.out.println(" copy len=" +
builder.reusedBytesPerArc[arcIdx]);
-assert destPos > srcPos: "destPos=" + destPos + " srcPos=" +
srcPos + " arcIdx=" + arcIdx + " maxBytesPerArc=" + maxBytesPerArc + "
reusedBytesPerArc[arcIdx]=" + builder.reusedBytesPerArc[arcIdx] + "
nodeIn.numArcs=" + nodeIn.numArcs;
-builder.bytes.copyBytes(srcPos, destPos,
builder.reusedBytesPerArc[arcIdx]);
+ if (writeDirectly) {
Review comment:
Huge methods... move to a submethod?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279207963 ## File path: lucene/core/src/java/org/apache/lucene/util/fst/Builder.java ## @@ -69,6 +69,8 @@ private final IntsRefBuilder lastInput = new IntsRefBuilder(); + final boolean useDirectArcAddressing; Review comment: private final (?) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc addressing
dweiss commented on a change in pull request #657: LUCENE-8781: FST direct arc
addressing
URL: https://github.com/apache/lucene-solr/pull/657#discussion_r279208143
##
File path: lucene/core/src/java/org/apache/lucene/util/fst/FSTEnum.java
##
@@ -281,10 +335,81 @@ protected void doSeekFloor() throws IOException {
//System.out.println(" cycle upto=" + upto + " arc.label=" + arc.label
+ " (" + (char) arc.label + ") targetLabel=" + targetLabel + " isLast?=" +
arc.isLast() + " bba=" + arc.bytesPerArc);
if (arc.bytesPerArc != 0 && arc.label != FST.END_LABEL) {
-// Arcs are fixed array -- use binary search to find
-// the target.
-
+// arcs are in an array
+
final FST.BytesReader in = fst.getBytesReader();
+
+if (arc.arcIdx == Integer.MIN_VALUE) {
+ // The array is addressed directly by label and may contain holes.
+ in.setPosition(arc.posArcsStart);
+ in.skipBytes(1);
+ int firstLabel = fst.readLabel(in);
+ int targetOffset = targetLabel - firstLabel;
+ if (targetOffset < 0) {
+//System.out.println(" before first"); Very first arc is after our
target TODO: if each
+// arc could somehow read the arc just before, we can save this
re-scan. The ceil case
+// doesn't need this because it reads the next arc instead:
+while(true) {
+ // First, walk backwards until we find a first arc
+ // that's before our target label:
+ fst.readFirstTargetArc(getArc(upto-1), arc, fstReader);
+ if (arc.label < targetLabel) {
+// Then, scan forwards to the arc just before
+// the targetLabel:
+while(!arc.isLast() && fst.readNextArcLabel(arc, in) <
targetLabel) {
+ fst.readNextArc(arc, fstReader);
+}
+pushLast();
+return;
+ }
+ upto--;
+ if (upto == 0) {
+return;
+ }
+ targetLabel = getTargetLabel();
+ arc = getArc(upto);
+}
+ } else {
+if (targetOffset >= arc.numArcs) {
+ arc.nextArc = arc.posArcsStart - arc.bytesPerArc * (arc.numArcs
- 1);
+ fst.readNextRealArc(arc, in);
+ assert arc.isLast();
+ assert arc.label < targetLabel: "arc.label=" + arc.label + " vs
targetLabel=" + targetLabel;
+ pushLast();
+ return;
+}
+arc.nextArc = arc.posArcsStart - arc.bytesPerArc * targetOffset;
+fst.readNextRealArc(arc, in);
+if (arc.label == targetLabel) {
+ // found -- copy pasta from below
Review comment:
pasta is good, but paste is better.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8774) Performance improvement for update?optimize=true
[
https://issues.apache.org/jira/browse/LUCENE-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828821#comment-16828821
]
Lucene/Solr QA commented on LUCENE-8774:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 35m
30s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8774 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12966488/0001-Performance-improvement-for-update-optimize-true.patch
|
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP
Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 8dd22bc |
| ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 |
| Default Java | LTS |
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/183/testReport/ |
| modules | C: lucene/core U: lucene/core |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/183/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Performance improvement for update?optimize=true
>
>
> Key: LUCENE-8774
> URL: https://issues.apache.org/jira/browse/LUCENE-8774
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
>Affects Versions: trunk, 7.2.1
>Reporter: Johann Abraham
>Priority: Major
> Attachments:
> 0001-Performance-improvement-for-update-optimize-true.patch
>
>
> Running optimize on a document set that has a large number of docValue fields
> grows linearly with the number of docValue fields.
> We discovered this because our corpus has few documents but a very large
> number of dynamic docValue fields.
> The cause is in the FilterFieldInfos constructor, a linear search of the
> fielterFields occurs due to the filterFields.contains(...) function call.
> Changing the underlying Collection type from an ArrayList to a LinkedHashSet
> decreased our optimize time from 60 minutes to < 1 minute. Patch attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13394) Change default GC from CMS to G1
[ https://issues.apache.org/jira/browse/SOLR-13394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828813#comment-16828813 ] Ishan Chattopadhyaya commented on SOLR-13394: - bq. Ishan Chattopadhyaya - Was this supposed to be changed on the 8.x branch or just on master (9.x) where the switch to JDK 11 was made? I intended this for Solr 8.1 (Java 8+). Do you suggest otherwise and restrict to 9x? > Change default GC from CMS to G1 > > > Key: SOLR-13394 > URL: https://issues.apache.org/jira/browse/SOLR-13394 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Ishan Chattopadhyaya >Assignee: Ishan Chattopadhyaya >Priority: Major > Fix For: 8.1 > > Attachments: SOLR-13394.patch > > Time Spent: 40m > Remaining Estimate: 0h > > CMS has been deprecated in new versions of Java > (http://openjdk.java.net/jeps/291). This issue is to switch Solr default from > CMS to G1. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-master - Build # 3315 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/3315/ 2 tests failed. FAILED: org.apache.solr.client.solrj.impl.CloudHttp2SolrClientTest.preferLocalShardsTest Error Message: java.lang.NullPointerException Stack Trace: org.apache.solr.client.solrj.SolrServerException: java.lang.NullPointerException at __randomizedtesting.SeedInfo.seed([2861E2AE1800AF6D:D4AC7A17E6D73897]:0) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:954) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:769) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207) at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:239) at org.apache.solr.client.solrj.impl.CloudHttp2SolrClientTest.preferLocalShardsTest(CloudHttp2SolrClientTest.java:426) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828801#comment-16828801
]
ASF subversion and git services commented on SOLR-5970:
---
Commit 8dd22bc0564b0e2da02ff9e7c34cad66e76c7a05 in lucene-solr's branch
refs/heads/master from Ishan Chattopadhyaya
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8dd22bc ]
SOLR-5970: Fix precommit
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Fix For: 8.1
>
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828802#comment-16828802
]
ASF subversion and git services commented on SOLR-5970:
---
Commit 980472aa47a4f5393b085efd188326c0908f6bd6 in lucene-solr's branch
refs/heads/branch_8x from Ishan Chattopadhyaya
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=980472a ]
SOLR-5970: Fix precommit
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Fix For: 8.1
>
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on a change in pull request #647: SOLR-12638: Added docs in the ref-guide for nested atomic updates
dsmiley commented on a change in pull request #647: SOLR-12638: Added docs in the ref-guide for nested atomic updates URL: https://github.com/apache/lucene-solr/pull/647#discussion_r279205550 ## File path: solr/solr-ref-guide/src/indexing-nested-documents.adoc ## @@ -51,6 +51,13 @@ With the exception of in-place updates, the whole block must be updated or delet Consequently, the field need not be defined in the schema and probably shouldn't be as it would be confusing. There is no child document field type, at least not yet. +[NOTE] Review comment: I think it's okay to include a small modification here to error out if `_route_` is not provided. I'm assuming it's required only for atomic updates of child documents -- that is, when the user provides a child document as the root of the update. I don't think it is required when updating a root doc to manipulate (add/update/delete) a child doc? I wonder if we can tell the difference -- when the user makes this mistake. And if `_route_` is provided unnecessarily, ideally we would verify that its value is correct. What I wonder about is what happens when `_root_` is not stored/docValues? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-12) - Build # 7916 - Still unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7916/ Java: 64bit/jdk-12 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC 1 tests failed. FAILED: org.apache.solr.security.AuditLoggerIntegrationTest.testAsyncQueueDrain Error Message: Failed waiting for 3 callbacks after 30 seconds Stack Trace: java.lang.AssertionError: Failed waiting for 3 callbacks after 30 seconds at __randomizedtesting.SeedInfo.seed([6955CA8C2EDB1C8F:D84A35A89E188F50]:0) at org.junit.Assert.fail(Assert.java:88) at org.apache.solr.security.AuditLoggerIntegrationTest.waitForAuditEventCallbacks(AuditLoggerIntegrationTest.java:229) at org.apache.solr.security.AuditLoggerIntegrationTest.waitForAuditEventCallbacks(AuditLoggerIntegrationTest.java:221) at org.apache.solr.security.AuditLoggerIntegrationTest.assertThreeAdminEvents(AuditLoggerIntegrationTest.java:252) at org.apache.solr.security.AuditLoggerIntegrationTest.testAsyncQueueDrain(AuditLoggerIntegrationTest.java:132) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:567) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Resolved] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ishan Chattopadhyaya resolved SOLR-5970.
Resolution: Fixed
Fix Version/s: 8.1
Thanks [~gerlowskija], [~keshareenv]! Lets spin up new issues for broader fix
involving status for other collections API responses.
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Fix For: 8.1
>
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8756) MLT queries ignore custom term frequencies
[ https://issues.apache.org/jira/browse/LUCENE-8756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828796#comment-16828796 ] Olli Kuonanoja commented on LUCENE-8756: Related to https://issues.apache.org/jira/browse/LUCENE-7854 [~mikemccand] what is your take on this? > MLT queries ignore custom term frequencies > -- > > Key: LUCENE-8756 > URL: https://issues.apache.org/jira/browse/LUCENE-8756 > Project: Lucene - Core > Issue Type: Bug > Components: core/queryparser >Affects Versions: 7.0, 7.0.1, 7.1, 7.2, 7.2.1, 7.3, 7.4, 7.3.1, 7.5, 7.6, > 7.7, 7.7.1, 8.0 >Reporter: Olli Kuonanoja >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The MLT queries ignore any custom term frequencies for the like-texts and > uses a hard-coded frequency of 1 per occurrence. I have prepared a test-case > to demonstrate the issue and a fix proposal > https://github.com/ollik1/lucene-solr/commit/9dbbce2af26698cec1ac82a526d9cee60a880678 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ishan Chattopadhyaya reassigned SOLR-5970:
--
Assignee: Ishan Chattopadhyaya (was: Jason Gerlowski)
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Ishan Chattopadhyaya
>Priority: Major
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828794#comment-16828794
]
ASF subversion and git services commented on SOLR-5970:
---
Commit 855c7d6bb50f5120bead983044d2f1a4e3a44cbf in lucene-solr's branch
refs/heads/branch_8x from Ishan Chattopadhyaya
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=855c7d6 ]
SOLR-5970: Return correct status upon collection creation failure
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828793#comment-16828793
]
ASF subversion and git services commented on SOLR-5970:
---
Commit dd9899b1c16aa06fd7acb43d2f43bcac44a97776 in lucene-solr's branch
refs/heads/master from Ishan Chattopadhyaya
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=dd9899b ]
SOLR-5970: Return correct status upon collection creation failure
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8756) MLT queries ignore custom term frequencies
[ https://issues.apache.org/jira/browse/LUCENE-8756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olli Kuonanoja updated LUCENE-8756: --- Labels: pull-request-available (was: ) > MLT queries ignore custom term frequencies > -- > > Key: LUCENE-8756 > URL: https://issues.apache.org/jira/browse/LUCENE-8756 > Project: Lucene - Core > Issue Type: Bug > Components: core/queryparser >Affects Versions: 7.0, 7.0.1, 7.1, 7.2, 7.2.1, 7.3, 7.4, 7.3.1, 7.5, 7.6, > 7.7, 7.7.1, 8.0 >Reporter: Olli Kuonanoja >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The MLT queries ignore any custom term frequencies for the like-texts and > uses a hard-coded frequency of 1 per occurrence. I have prepared a test-case > to demonstrate the issue and a fix proposal > https://github.com/ollik1/lucene-solr/commit/9dbbce2af26698cec1ac82a526d9cee60a880678 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-12248) Grouping in SolrCloud fails if indexed="false" docValues="true" and stored="false"
[ https://issues.apache.org/jira/browse/SOLR-12248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya resolved SOLR-12248. - Resolution: Fixed Fix Version/s: 8.1 Thanks [~munendrasn], [~sstults]. > Grouping in SolrCloud fails if indexed="false" docValues="true" and > stored="false" > -- > > Key: SOLR-12248 > URL: https://issues.apache.org/jira/browse/SOLR-12248 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6.2 >Reporter: Erick Erickson >Assignee: Ishan Chattopadhyaya >Priority: Minor > Fix For: 8.1 > > Attachments: SOLR-12248.patch, SOLR-12248.patch > > > In SolrCloud _only_ (it works in stand-alone mode), a field defined as: > indexed="false" docValues="true" stored="false" /> > will fail with the following error: > java.lang.NullPointerException > org.apache.solr.schema.BoolField.toExternal(BoolField.java:131) > org.apache.solr.schema.BoolField.toObject(BoolField.java:142) > org.apache.solr.schema.BoolField.toObject(BoolField.java:51) > org.apache.solr.search.grouping.endresulttransformer.GroupedEndResultTransformer.transform(GroupedEndResultTransformer.java:72) > org.apache.solr.handler.component.QueryComponent.groupedFinishStage(QueryComponent.java:830) > org.apache.solr.handler.component.QueryComponent.finishStage(QueryComponent.java:793) > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:435) > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173) > . > . > curiously enough it succeeds with a field identically defined except for > stored="true" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4647) Grouping is broken on docvalues-only fields
[ https://issues.apache.org/jira/browse/SOLR-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828791#comment-16828791 ] ASF subversion and git services commented on SOLR-4647: --- Commit a877632aaf86f5eeeba50201e7974f3329f5ccb9 in lucene-solr's branch refs/heads/branch_8x from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a877632 ] SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields > Grouping is broken on docvalues-only fields > --- > > Key: SOLR-4647 > URL: https://issues.apache.org/jira/browse/SOLR-4647 > Project: Solr > Issue Type: Bug > Components: search >Affects Versions: 4.2 >Reporter: Adrien Grand >Priority: Major > Labels: newdev > Attachments: SOLR-4647.patch > > > There are a few places where grouping uses > FieldType.toObject(SchemaField.createField(String, float)) to translate a > String field value to an Object. The problem is that createField returns null > when the field is neither stored nor indexed, even if it has doc values. > An option to fix it could be to use the ValueSource instead to resolve the > Object value (similarily to NumericFacets). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12248) Grouping in SolrCloud fails if indexed="false" docValues="true" and stored="false"
[ https://issues.apache.org/jira/browse/SOLR-12248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828790#comment-16828790 ] ASF subversion and git services commented on SOLR-12248: Commit a877632aaf86f5eeeba50201e7974f3329f5ccb9 in lucene-solr's branch refs/heads/branch_8x from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a877632 ] SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields > Grouping in SolrCloud fails if indexed="false" docValues="true" and > stored="false" > -- > > Key: SOLR-12248 > URL: https://issues.apache.org/jira/browse/SOLR-12248 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6.2 >Reporter: Erick Erickson >Assignee: Ishan Chattopadhyaya >Priority: Minor > Attachments: SOLR-12248.patch, SOLR-12248.patch > > > In SolrCloud _only_ (it works in stand-alone mode), a field defined as: > indexed="false" docValues="true" stored="false" /> > will fail with the following error: > java.lang.NullPointerException > org.apache.solr.schema.BoolField.toExternal(BoolField.java:131) > org.apache.solr.schema.BoolField.toObject(BoolField.java:142) > org.apache.solr.schema.BoolField.toObject(BoolField.java:51) > org.apache.solr.search.grouping.endresulttransformer.GroupedEndResultTransformer.transform(GroupedEndResultTransformer.java:72) > org.apache.solr.handler.component.QueryComponent.groupedFinishStage(QueryComponent.java:830) > org.apache.solr.handler.component.QueryComponent.finishStage(QueryComponent.java:793) > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:435) > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173) > . > . > curiously enough it succeeds with a field identically defined except for > stored="true" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4647) Grouping is broken on docvalues-only fields
[ https://issues.apache.org/jira/browse/SOLR-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Chattopadhyaya resolved SOLR-4647. Resolution: Duplicate > Grouping is broken on docvalues-only fields > --- > > Key: SOLR-4647 > URL: https://issues.apache.org/jira/browse/SOLR-4647 > Project: Solr > Issue Type: Bug > Components: search >Affects Versions: 4.2 >Reporter: Adrien Grand >Priority: Major > Labels: newdev > Attachments: SOLR-4647.patch > > > There are a few places where grouping uses > FieldType.toObject(SchemaField.createField(String, float)) to translate a > String field value to an Object. The problem is that createField returns null > when the field is neither stored nor indexed, even if it has doc values. > An option to fix it could be to use the ValueSource instead to resolve the > Object value (similarily to NumericFacets). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12248) Grouping in SolrCloud fails if indexed="false" docValues="true" and stored="false"
[ https://issues.apache.org/jira/browse/SOLR-12248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828788#comment-16828788 ] ASF subversion and git services commented on SOLR-12248: Commit 4b49bd99ca20cee2263a3cd3c115c7d00b98f043 in lucene-solr's branch refs/heads/master from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4b49bd9 ] SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields > Grouping in SolrCloud fails if indexed="false" docValues="true" and > stored="false" > -- > > Key: SOLR-12248 > URL: https://issues.apache.org/jira/browse/SOLR-12248 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 6.6.2 >Reporter: Erick Erickson >Assignee: Ishan Chattopadhyaya >Priority: Minor > Attachments: SOLR-12248.patch, SOLR-12248.patch > > > In SolrCloud _only_ (it works in stand-alone mode), a field defined as: > indexed="false" docValues="true" stored="false" /> > will fail with the following error: > java.lang.NullPointerException > org.apache.solr.schema.BoolField.toExternal(BoolField.java:131) > org.apache.solr.schema.BoolField.toObject(BoolField.java:142) > org.apache.solr.schema.BoolField.toObject(BoolField.java:51) > org.apache.solr.search.grouping.endresulttransformer.GroupedEndResultTransformer.transform(GroupedEndResultTransformer.java:72) > org.apache.solr.handler.component.QueryComponent.groupedFinishStage(QueryComponent.java:830) > org.apache.solr.handler.component.QueryComponent.finishStage(QueryComponent.java:793) > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:435) > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173) > . > . > curiously enough it succeeds with a field identically defined except for > stored="true" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4647) Grouping is broken on docvalues-only fields
[ https://issues.apache.org/jira/browse/SOLR-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828789#comment-16828789 ] ASF subversion and git services commented on SOLR-4647: --- Commit 4b49bd99ca20cee2263a3cd3c115c7d00b98f043 in lucene-solr's branch refs/heads/master from Ishan Chattopadhyaya [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4b49bd9 ] SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields > Grouping is broken on docvalues-only fields > --- > > Key: SOLR-4647 > URL: https://issues.apache.org/jira/browse/SOLR-4647 > Project: Solr > Issue Type: Bug > Components: search >Affects Versions: 4.2 >Reporter: Adrien Grand >Priority: Major > Labels: newdev > Attachments: SOLR-4647.patch > > > There are a few places where grouping uses > FieldType.toObject(SchemaField.createField(String, float)) to translate a > String field value to an Object. The problem is that createField returns null > when the field is neither stored nor indexed, even if it has doc values. > An option to fix it could be to use the ValueSource instead to resolve the > Object value (similarily to NumericFacets). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-BadApples-Tests-8.x - Build # 88 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-8.x/88/
1 tests failed.
FAILED:
org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
Error Message:
Doc with id=4 not found in
http://127.0.0.1:43791/solr/outOfSyncReplicasCannotBecomeLeader-false due to:
Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=4 not found in
http://127.0.0.1:43791/solr/outOfSyncReplicasCannotBecomeLeader-false due to:
Path not found: /id; rsp={doc=null}
at
__randomizedtesting.SeedInfo.seed([ADD9D2460AD3871A:D332F256C9B48820]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.solr.cloud.TestCloudConsistency.assertDocExists(TestCloudConsistency.java:283)
at
org.apache.solr.cloud.TestCloudConsistency.assertDocsExistInAllReplicas(TestCloudConsistency.java:267)
at
org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:138)
at
org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[jira] [Comment Edited] (LUCENE-8780) Improve ByteBufferGuard in Java 11
[
https://issues.apache.org/jira/browse/LUCENE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828039#comment-16828039
]
Uwe Schindler edited comment on LUCENE-8780 at 4/28/19 3:32 PM:
Thats the result after 20 runs of wikimediumall with 6 searcher threads (with
ParallelGC) on Mike's lucenebench:
{noformat}
use java command /home/jenkins/tools/java/64bit/jdk-11.0.2/bin/java -server
-Xms2g -Xmx2g -XX:+UseParallelGC -Xbatch
JAVA:
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
OS:
Linux serv1.sd-datasolutions.de 4.18.0-17-generic #18~18.04.1-Ubuntu SMP Fri
Mar 15 15:27:12 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[...]
Report after iter 19:
TaskQPS orig StdDev QPS patch StdDev
Pct diff
IntNRQ 30.88 (0.6%) 26.33 (0.8%)
-14.7% ( -16% - -13%)
PKLookup 107.70 (2.7%) 94.31 (2.9%)
-12.4% ( -17% - -7%)
AndHighHigh 10.76 (11.5%) 10.17 (3.3%)
-5.4% ( -18% - 10%)
Fuzzy2 45.10 (7.7%) 43.21 (9.0%)
-4.2% ( -19% - 13%)
LowSloppyPhrase7.28 (16.8%)6.98 (6.3%)
-4.2% ( -23% - 22%)
OrHighNotLow 783.24 (7.1%) 751.37 (2.5%)
-4.1% ( -12% -5%)
OrHighNotHigh 934.39 (6.5%) 896.38 (2.1%)
-4.1% ( -11% -4%)
Respell 45.36 (10.6%) 43.65 (7.0%)
-3.8% ( -19% - 15%)
OrNotHighHigh 779.95 (3.8%) 752.28 (1.8%)
-3.5% ( -8% -2%)
HighSloppyPhrase 10.37 (12.8%) 10.03 (3.5%)
-3.3% ( -17% - 14%)
LowPhrase 11.60 (8.9%) 11.23 (1.7%)
-3.2% ( -12% -8%)
LowTerm 1694.00 (8.9%) 1642.34 (5.5%)
-3.0% ( -16% - 12%)
MedTerm 1292.82 (9.3%) 1253.69 (8.2%)
-3.0% ( -18% - 15%)
AndHighMed 71.41 (9.9%) 69.77 (7.5%)
-2.3% ( -17% - 16%)
OrNotHighMed 634.32 (7.2%) 620.67 (7.5%)
-2.2% ( -15% - 13%)
Prefix3 110.65 (14.9%) 108.55 (8.7%)
-1.9% ( -22% - 25%)
OrHighLow 347.02 (4.3%) 340.51 (9.9%)
-1.9% ( -15% - 12%)
OrNotHighLow 591.61 (5.5%) 580.60 (9.0%)
-1.9% ( -15% - 13%)
OrHighNotMed 1258.21 (1.8%) 1237.28 (5.0%)
-1.7% ( -8% -5%)
Fuzzy1 91.79 (4.3%) 90.77 (11.1%)
-1.1% ( -15% - 14%)
OrHighMed 10.29 (7.9%) 10.25 (11.8%)
-0.4% ( -18% - 20%)
Wildcard 52.28 (6.3%) 52.21 (6.8%)
-0.1% ( -12% - 13%)
OrHighHigh8.16 (6.9%)8.22 (9.3%)
0.8% ( -14% - 18%)
AndHighLow 563.89 (9.1%) 569.31 (15.3%)
1.0% ( -21% - 27%)
HighPhrase 15.88 (9.3%) 16.04 (13.0%)
1.0% ( -19% - 25%)
MedPhrase 14.84 (9.0%) 15.15 (12.8%)
2.1% ( -18% - 26%)
HighSpanNear2.16 (9.8%)2.21 (10.1%)
2.3% ( -16% - 24%)
MedSloppyPhrase 18.48 (15.4%) 18.96 (18.9%)
2.6% ( -27% - 43%)
MedSpanNear 17.75 (3.8%) 18.31 (10.0%)
3.1% ( -10% - 17%)
HighTerm 1031.00 (9.9%) 1068.12 (17.1%)
3.6% ( -21% - 33%)
LowSpanNear8.22 (5.5%)8.53 (13.3%)
3.7% ( -14% - 23%)
HighTermDayOfYearSort9.78 (11.0%) 10.25 (18.2%)
4.8% ( -21% - 38%)
HighTermMonthSort 23.40 (26.5%) 27.11 (32.1%)
15.9% ( -33% - 101%)
{noformat}
The total runtime of each run did not change, always approx 280s per run
patched and unpatched. Not sure how to interpret this.
was (Author: thetaphi):
Thats the result after 20 runs with 6 searcher threads (with ParallelGC) on
Mike's lucenebench:
{noformat}
use java command /home/jenkins/tools/java/64bit/jdk-11.0.2/bin/java -server
-Xms2g -Xmx2g -XX:+UseParallelGC -Xbatch
JAVA:
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
OS:
Linux serv1.sd-datasolutions.de 4.18.0-17-generic #18~18.04.1-Ubuntu SMP Fri
Mar 15 15:27:12 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[...]
Report after iter 19:
TaskQPS orig StdDev QPS patch StdDev
Pct
[jira] [Comment Edited] (LUCENE-8780) Improve ByteBufferGuard in Java 11
[
https://issues.apache.org/jira/browse/LUCENE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828039#comment-16828039
]
Uwe Schindler edited comment on LUCENE-8780 at 4/28/19 3:31 PM:
Thats the result after 20 runs with 6 searcher threads (with ParallelGC) on
Mike's lucenebench:
{noformat}
use java command /home/jenkins/tools/java/64bit/jdk-11.0.2/bin/java -server
-Xms2g -Xmx2g -XX:+UseParallelGC -Xbatch
JAVA:
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
OS:
Linux serv1.sd-datasolutions.de 4.18.0-17-generic #18~18.04.1-Ubuntu SMP Fri
Mar 15 15:27:12 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[...]
Report after iter 19:
TaskQPS orig StdDev QPS patch StdDev
Pct diff
IntNRQ 30.88 (0.6%) 26.33 (0.8%)
-14.7% ( -16% - -13%)
PKLookup 107.70 (2.7%) 94.31 (2.9%)
-12.4% ( -17% - -7%)
AndHighHigh 10.76 (11.5%) 10.17 (3.3%)
-5.4% ( -18% - 10%)
Fuzzy2 45.10 (7.7%) 43.21 (9.0%)
-4.2% ( -19% - 13%)
LowSloppyPhrase7.28 (16.8%)6.98 (6.3%)
-4.2% ( -23% - 22%)
OrHighNotLow 783.24 (7.1%) 751.37 (2.5%)
-4.1% ( -12% -5%)
OrHighNotHigh 934.39 (6.5%) 896.38 (2.1%)
-4.1% ( -11% -4%)
Respell 45.36 (10.6%) 43.65 (7.0%)
-3.8% ( -19% - 15%)
OrNotHighHigh 779.95 (3.8%) 752.28 (1.8%)
-3.5% ( -8% -2%)
HighSloppyPhrase 10.37 (12.8%) 10.03 (3.5%)
-3.3% ( -17% - 14%)
LowPhrase 11.60 (8.9%) 11.23 (1.7%)
-3.2% ( -12% -8%)
LowTerm 1694.00 (8.9%) 1642.34 (5.5%)
-3.0% ( -16% - 12%)
MedTerm 1292.82 (9.3%) 1253.69 (8.2%)
-3.0% ( -18% - 15%)
AndHighMed 71.41 (9.9%) 69.77 (7.5%)
-2.3% ( -17% - 16%)
OrNotHighMed 634.32 (7.2%) 620.67 (7.5%)
-2.2% ( -15% - 13%)
Prefix3 110.65 (14.9%) 108.55 (8.7%)
-1.9% ( -22% - 25%)
OrHighLow 347.02 (4.3%) 340.51 (9.9%)
-1.9% ( -15% - 12%)
OrNotHighLow 591.61 (5.5%) 580.60 (9.0%)
-1.9% ( -15% - 13%)
OrHighNotMed 1258.21 (1.8%) 1237.28 (5.0%)
-1.7% ( -8% -5%)
Fuzzy1 91.79 (4.3%) 90.77 (11.1%)
-1.1% ( -15% - 14%)
OrHighMed 10.29 (7.9%) 10.25 (11.8%)
-0.4% ( -18% - 20%)
Wildcard 52.28 (6.3%) 52.21 (6.8%)
-0.1% ( -12% - 13%)
OrHighHigh8.16 (6.9%)8.22 (9.3%)
0.8% ( -14% - 18%)
AndHighLow 563.89 (9.1%) 569.31 (15.3%)
1.0% ( -21% - 27%)
HighPhrase 15.88 (9.3%) 16.04 (13.0%)
1.0% ( -19% - 25%)
MedPhrase 14.84 (9.0%) 15.15 (12.8%)
2.1% ( -18% - 26%)
HighSpanNear2.16 (9.8%)2.21 (10.1%)
2.3% ( -16% - 24%)
MedSloppyPhrase 18.48 (15.4%) 18.96 (18.9%)
2.6% ( -27% - 43%)
MedSpanNear 17.75 (3.8%) 18.31 (10.0%)
3.1% ( -10% - 17%)
HighTerm 1031.00 (9.9%) 1068.12 (17.1%)
3.6% ( -21% - 33%)
LowSpanNear8.22 (5.5%)8.53 (13.3%)
3.7% ( -14% - 23%)
HighTermDayOfYearSort9.78 (11.0%) 10.25 (18.2%)
4.8% ( -21% - 38%)
HighTermMonthSort 23.40 (26.5%) 27.11 (32.1%)
15.9% ( -33% - 101%)
{noformat}
The total runtime of each run did not change, always approx 280s per run
patched and unpatched. Not sure how to interpret this.
was (Author: thetaphi):
Thats the result after 20 runs with 6 searcher threads (with ParallelGC) on
Mike's lucenebench:
{noformat}
use java command /home/jenkins/tools/java/64bit/jdk-11.0.2/bin/java -server
-Xms2g -Xmx2g -XX:+UseParallelGC -Xbatch
JAVA:
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
OS:
Linux serv1.sd-datasolutions.de 4.18.0-17-generic #18~18.04.1-Ubuntu SMP Fri
Mar 15 15:27:12 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[...]
Report after iter 19:
TaskQPS orig StdDev QPS patch StdDev
Pct diff
[jira] [Commented] (LUCENE-8780) Improve ByteBufferGuard in Java 11
[
https://issues.apache.org/jira/browse/LUCENE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828039#comment-16828039
]
Uwe Schindler commented on LUCENE-8780:
---
Thats the result after 20 runs with 6 searcher threads (with ParallelGC) on
Mike's lucenebench:
{noformat}
use java command /home/jenkins/tools/java/64bit/jdk-11.0.2/bin/java -server
-Xms2g -Xmx2g -XX:+UseParallelGC -Xbatch
JAVA:
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
OS:
Linux serv1.sd-datasolutions.de 4.18.0-17-generic #18~18.04.1-Ubuntu SMP Fri
Mar 15 15:27:12 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
[...]
Report after iter 19:
TaskQPS orig StdDev QPS patch StdDev
Pct diff
IntNRQ 30.88 (0.6%) 26.33 (0.8%)
-14.7% ( -16% - -13%)
PKLookup 107.70 (2.7%) 94.31 (2.9%)
-12.4% ( -17% - -7%)
AndHighHigh 10.76 (11.5%) 10.17 (3.3%)
-5.4% ( -18% - 10%)
Fuzzy2 45.10 (7.7%) 43.21 (9.0%)
-4.2% ( -19% - 13%)
LowSloppyPhrase7.28 (16.8%)6.98 (6.3%)
-4.2% ( -23% - 22%)
OrHighNotLow 783.24 (7.1%) 751.37 (2.5%)
-4.1% ( -12% -5%)
OrHighNotHigh 934.39 (6.5%) 896.38 (2.1%)
-4.1% ( -11% -4%)
Respell 45.36 (10.6%) 43.65 (7.0%)
-3.8% ( -19% - 15%)
OrNotHighHigh 779.95 (3.8%) 752.28 (1.8%)
-3.5% ( -8% -2%)
HighSloppyPhrase 10.37 (12.8%) 10.03 (3.5%)
-3.3% ( -17% - 14%)
LowPhrase 11.60 (8.9%) 11.23 (1.7%)
-3.2% ( -12% -8%)
LowTerm 1694.00 (8.9%) 1642.34 (5.5%)
-3.0% ( -16% - 12%)
MedTerm 1292.82 (9.3%) 1253.69 (8.2%)
-3.0% ( -18% - 15%)
AndHighMed 71.41 (9.9%) 69.77 (7.5%)
-2.3% ( -17% - 16%)
OrNotHighMed 634.32 (7.2%) 620.67 (7.5%)
-2.2% ( -15% - 13%)
Prefix3 110.65 (14.9%) 108.55 (8.7%)
-1.9% ( -22% - 25%)
OrHighLow 347.02 (4.3%) 340.51 (9.9%)
-1.9% ( -15% - 12%)
OrNotHighLow 591.61 (5.5%) 580.60 (9.0%)
-1.9% ( -15% - 13%)
OrHighNotMed 1258.21 (1.8%) 1237.28 (5.0%)
-1.7% ( -8% -5%)
Fuzzy1 91.79 (4.3%) 90.77 (11.1%)
-1.1% ( -15% - 14%)
OrHighMed 10.29 (7.9%) 10.25 (11.8%)
-0.4% ( -18% - 20%)
Wildcard 52.28 (6.3%) 52.21 (6.8%)
-0.1% ( -12% - 13%)
OrHighHigh8.16 (6.9%)8.22 (9.3%)
0.8% ( -14% - 18%)
AndHighLow 563.89 (9.1%) 569.31 (15.3%)
1.0% ( -21% - 27%)
HighPhrase 15.88 (9.3%) 16.04 (13.0%)
1.0% ( -19% - 25%)
MedPhrase 14.84 (9.0%) 15.15 (12.8%)
2.1% ( -18% - 26%)
HighSpanNear2.16 (9.8%)2.21 (10.1%)
2.3% ( -16% - 24%)
MedSloppyPhrase 18.48 (15.4%) 18.96 (18.9%)
2.6% ( -27% - 43%)
MedSpanNear 17.75 (3.8%) 18.31 (10.0%)
3.1% ( -10% - 17%)
HighTerm 1031.00 (9.9%) 1068.12 (17.1%)
3.6% ( -21% - 33%)
LowSpanNear8.22 (5.5%)8.53 (13.3%)
3.7% ( -14% - 23%)
HighTermDayOfYearSort9.78 (11.0%) 10.25 (18.2%)
4.8% ( -21% - 38%)
HighTermMonthSort 23.40 (26.5%) 27.11 (32.1%)
15.9% ( -33% - 101%)
{noformat}
The total runtime of each run did not change. Not sure how to interpret this.
> Improve ByteBufferGuard in Java 11
> --
>
> Key: LUCENE-8780
> URL: https://issues.apache.org/jira/browse/LUCENE-8780
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/store
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
> Labels: Java11
> Attachments: LUCENE-8780.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In LUCENE-7409 we added {{ByteBufferGuard}} to protect MMapDirectory from
> crushing the JVM with SIGSEGV when you close and unmap the mmapped buffers of
> an IndexInput, while another thread is accessing it.
> The idea was
[GitHub] [lucene-solr] moshebla commented on a change in pull request #647: SOLR-12638: Added docs in the ref-guide for nested atomic updates
moshebla commented on a change in pull request #647: SOLR-12638: Added docs in the ref-guide for nested atomic updates URL: https://github.com/apache/lucene-solr/pull/647#discussion_r279198791 ## File path: solr/solr-ref-guide/src/indexing-nested-documents.adoc ## @@ -51,6 +51,13 @@ With the exception of in-place updates, the whole block must be updated or delet Consequently, the field need not be defined in the schema and probably shouldn't be as it would be confusing. There is no child document field type, at least not yet. +[NOTE] Review comment: > /home/internet/Projects/lucene-solr/solr/package/solr-7.7.2-SNAPSHOT.tgz Currently we only throw an error if the wrong route param was supplied when updating a non root doc. Should we throw an error when no _route_ param is supplied? I could work on that if it is deemed necessary. WDYT? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4056) Japanese Tokenizer (Kuromoji) cannot build UniDic dictionary
[
https://issues.apache.org/jira/browse/LUCENE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828033#comment-16828033
]
Kazuaki Hiraga commented on LUCENE-4056:
[~Tomoko Uchida] I am going to prepare a patch. So, let's work together to fix
the issue.
> Japanese Tokenizer (Kuromoji) cannot build UniDic dictionary
>
>
> Key: LUCENE-4056
> URL: https://issues.apache.org/jira/browse/LUCENE-4056
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/analysis
>Affects Versions: 3.6
> Environment: Solr 3.6
> UniDic 1.3.12 for MeCab (unidic-mecab1312src.tar.gz)
>Reporter: Kazuaki Hiraga
>Priority: Major
>
> I tried to build a UniDic dictionary for using it along with Kuromoji on Solr
> 3.6. I think UniDic is a good dictionary than IPA dictionary, so Kuromoji for
> Lucene/Solr should support UniDic dictionary as standalone Kuromoji does.
> The following is my procedure:
> Modified build.xml under lucene/contrib/analyzers/kuromoji directory and run
> 'ant build-dict', I got the error as the below.
> build-dict:
> [java] dictionary builder
> [java]
> [java] dictionary format: UNIDIC
> [java] input directory:
> /home/kazu/Work/src/solr/brunch_3_6/lucene/build/contrib/analyzers/kuromoji/unidic-mecab1312src
> [java] output directory:
> /home/kazu/Work/src/solr/brunch_3_6/lucene/contrib/analyzers/kuromoji/src/resources
> [java] input encoding: utf-8
> [java] normalize entries: false
> [java]
> [java] building tokeninfo dict...
> [java] parse...
> [java] sort...
> [java] Exception in thread "main" java.lang.AssertionError
> [java] encode...
> [java] at
> org.apache.lucene.analysis.ja.util.BinaryDictionaryWriter.put(BinaryDictionaryWriter.java:113)
> [java] at
> org.apache.lucene.analysis.ja.util.TokenInfoDictionaryBuilder.buildDictionary(TokenInfoDictionaryBuilder.java:141)
> [java] at
> org.apache.lucene.analysis.ja.util.TokenInfoDictionaryBuilder.build(TokenInfoDictionaryBuilder.java:76)
> [java] at
> org.apache.lucene.analysis.ja.util.DictionaryBuilder.build(DictionaryBuilder.java:37)
> [java] at
> org.apache.lucene.analysis.ja.util.DictionaryBuilder.main(DictionaryBuilder.java:82)
> And the diff of build.xml:
> ===
> --- build.xml (revision 1338023)
> +++ build.xml (working copy)
> @@ -28,19 +28,31 @@
>
>
>
> +
>
>
> -
> +
> +
> +
> +
> + value="/home/kazu/Work/src/nlp/unidic/_archive"/>
> +
>
> +
> +
> +
> +
>
>
>
> @@ -58,7 +70,8 @@
>
>
>
> -
> +
> + tofile="${build.dir}/${dict.src.file}"/>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11035) (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload
[ https://issues.apache.org/jira/browse/SOLR-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828031#comment-16828031 ] ASF subversion and git services commented on SOLR-11035: Commit 16203c266823efa9afe09c65f78ac533d1bf06d4 in lucene-solr's branch refs/heads/branch_8x from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=16203c2 ] SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload. More fixes, bad test. > (at least) 2 distinct failures possible when clients attempt searches during > SolrCore reload > > > Key: SOLR-11035 > URL: https://issues.apache.org/jira/browse/SOLR-11035 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Hoss Man >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-11035-bandaid.patch, SOLR-11035-bandaid.patch, > SOLR-11035.patch, log.txt, log.txt > > > If a SolrCore is reloaded, there are (at least) 2 distinct types of failures > that clients may observe when executing updates + queries while the reload is > in progress... > * documents may appear missing during queries > * queries may fail with "SolrException: openNewSearcher called on closed core" > Details to follow in comment... -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11035) (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload
[ https://issues.apache.org/jira/browse/SOLR-11035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16828028#comment-16828028 ] ASF subversion and git services commented on SOLR-11035: Commit a6262af84253c1e377c0c7a32d0277edb5dd2d6e in lucene-solr's branch refs/heads/master from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a6262af ] SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload. More fixes, bad test. > (at least) 2 distinct failures possible when clients attempt searches during > SolrCore reload > > > Key: SOLR-11035 > URL: https://issues.apache.org/jira/browse/SOLR-11035 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Hoss Man >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-11035-bandaid.patch, SOLR-11035-bandaid.patch, > SOLR-11035.patch, log.txt, log.txt > > > If a SolrCore is reloaded, there are (at least) 2 distinct types of failures > that clients may observe when executing updates + queries while the reload is > in progress... > * documents may appear missing during queries > * queries may fail with "SolrException: openNewSearcher called on closed core" > Details to follow in comment... -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1834 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1834/ 1 tests failed. FAILED: org.apache.solr.cloud.CollectionsAPISolrJTest.testCreateCollWithDefaultClusterPropertiesNewFormat Error Message: expected:<[2]> but was:<[null]> Stack Trace: org.junit.ComparisonFailure: expected:<[2]> but was:<[null]> at __randomizedtesting.SeedInfo.seed([3D33197E7E01D78F:503BFF9FCB56DD03]:0) at org.junit.Assert.assertEquals(Assert.java:115) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.solr.cloud.CollectionsAPISolrJTest.testCreateCollWithDefaultClusterPropertiesNewFormat(CollectionsAPISolrJTest.java:237) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.base/java.lang.Thread.run(Thread.java:834) Build Log: [...truncated 14544 lines...] [junit4] Suite: org.apache.solr.cloud.CollectionsAPISolrJTest [junit4] 2> Creating dataDir:
[jira] [Commented] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827955#comment-16827955
]
Kesharee Nandan Vishwakarma commented on SOLR-5970:
---
Adding patch for master
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5970) Create collection API always has status 0
[
https://issues.apache.org/jira/browse/SOLR-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kesharee Nandan Vishwakarma updated SOLR-5970:
--
Attachment: SOLR-5970.patch
> Create collection API always has status 0
> -
>
> Key: SOLR-5970
> URL: https://issues.apache.org/jira/browse/SOLR-5970
> Project: Solr
> Issue Type: Bug
>Reporter: Abraham Elmahrek
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-5970-test.patch, SOLR-5970.patch, SOLR-5970.patch,
> SOLR-5970_branch_8x.patch, bad.jar, schema.xml, solrconfig.xml
>
>
> The responses below are from a successful create collection API
> (https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CreateormodifyanAliasforaCollection)
> call and an unsuccessful create collection API call. It seems the 'status'
> is always 0.
> Success:
> {u'responseHeader': {u'status': 0, u'QTime': 4421}, u'success': {u'':
> {u'core': u'test1_shard1_replica1', u'responseHeader': {u'status': 0,
> u'QTime': 3449
> Failure:
> {u'failure':
> {u'':
> u"org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:Error
> CREATEing SolrCore 'test43_shard1_replica1': Unable to create core:
> test43_shard1_replica1 Caused by: Could not find configName for collection
> test43 found:[test1]"},
> u'responseHeader': {u'status': 0, u'QTime': 17149}
> }
> It seems like the status should be 400 or something similar for an
> unsuccessful attempt?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12291) Async prematurely reports completed status that causes severe shard loss
[ https://issues.apache.org/jira/browse/SOLR-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Khludnev updated SOLR-12291: Attachment: (was: SOLR-12291.patch) > Async prematurely reports completed status that causes severe shard loss > > > Key: SOLR-12291 > URL: https://issues.apache.org/jira/browse/SOLR-12291 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Backup/Restore, SolrCloud >Reporter: Varun Thacker >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-12291.patch, SOLR-12291.patch, SOLR-12291.patch, > SOLR-12291.patch, SOLR-12291.patch, SOLR-122911.patch > > > The OverseerCollectionMessageHandler sliceCmd assumes only one replica exists > on one node > When multiple replicas of a slice are on the same node we only track one > replica's async request. This happens because the async requestMap's key is > "node_name" > I discovered this when [~alabax] shared some logs of a restore issue, where > the second replica got added before the first replica had completed it's > restorecore action. > While looking at the logs I noticed that the overseer never called > REQUESTSTATUS for the restorecore action , almost as if it had missed > tracking that particular async request. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-8.x-MacOSX (64bit/jdk-11.0.2) - Build # 108 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-MacOSX/108/ Java: 64bit/jdk-11.0.2 -XX:-UseCompressedOops -XX:+UseSerialGC 1 tests failed. FAILED: org.apache.solr.update.processor.CategoryRoutedAliasUpdateProcessorTest.testInvalidMustMatch Error Message: Error from server at http://127.0.0.1:58098/solr: no core retrieved for testInvalidMustMatch Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:58098/solr: no core retrieved for testInvalidMustMatch at __randomizedtesting.SeedInfo.seed([AF4A00D44C9C80AB:F76E98689224114E]:0) at org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException.create(BaseHttpSolrClient.java:66) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:626) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368) at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1068) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:837) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:769) at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1274) at org.apache.solr.update.processor.RoutedAliasUpdateProcessorTest.createConfigSet(RoutedAliasUpdateProcessorTest.java:115) at org.apache.solr.update.processor.CategoryRoutedAliasUpdateProcessorTest.testInvalidMustMatch(CategoryRoutedAliasUpdateProcessorTest.java:271) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at
[GitHub] [lucene-solr] uschindler opened a new pull request #658: LUCENE-8780: Improve ByteBufferGuard with Java 11 VarHandles and new …
uschindler opened a new pull request #658: LUCENE-8780: Improve ByteBufferGuard with Java 11 VarHandles and new … URL: https://github.com/apache/lucene-solr/pull/658 …memory model This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8780) Improve ByteBufferGuard in Java 11
[
https://issues.apache.org/jira/browse/LUCENE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827872#comment-16827872
]
Uwe Schindler commented on LUCENE-8780:
---
I created a pull request for easier review and perf testing (easy chackout of
branch from my github repo): https://github.com/apache/lucene-solr/pull/658
> Improve ByteBufferGuard in Java 11
> --
>
> Key: LUCENE-8780
> URL: https://issues.apache.org/jira/browse/LUCENE-8780
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/store
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
> Labels: Java11
> Attachments: LUCENE-8780.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In LUCENE-7409 we added {{ByteBufferGuard}} to protect MMapDirectory from
> crushing the JVM with SIGSEGV when you close and unmap the mmapped buffers of
> an IndexInput, while another thread is accessing it.
> The idea was to do a volatile write access to flush the caches (to trigger a
> full fence) and set a non-volatile boolean to true. All accesses would check
> the boolean and stop the caller from accessing the underlying ByteBuffer.
> This worked most of the time, until the JVM optimized away the plain read
> access to the boolean (you can easily see this after some runtime of our
> by-default ignored testcase).
> With master on Java 11, we can improve the whole thing. Using VarHandles you
> can use any access type when reading or writing the boolean. After reading
> Doug Lea's expanation and some
> testing, I was no longer able to crush my JDK (even after running for minutes
> unmapping bytebuffers).
> The apraoch is the same, we do a full-fenced write (standard volatile write)
> when we unmap, then we yield the thread (to finish in-flight reads in other
> threads) and then unmap all byte buffers.
> On the test side (read access), instead of using a plain read, we use the new
> "opaque read". Opaque reads are the same as plain reads, there are only
> different order requirements. Actually the main difference is explained by
> Doug like this: "For example in constructions in which the only modification
> of some variable x is for one thread to write in Opaque (or stronger) mode,
> X.setOpaque(this, 1), any other thread spinning in
> while(X.getOpaque(this)!=1){} will eventually terminate. Note that this
> guarantee does NOT hold in Plain mode, in which spin loops may (and usually
> do) infinitely loop -- they are not required to notice that a write ever
> occurred in another thread if it was not seen on first encounter." - And
> that's waht we want to have: We don't want to do volatile reads, but we want
> to prevent the compiler from optimizing away our read to the boolean. So we
> want it to "eventually" see the change. By the much stronger volatile write,
> the cache effects should be visible even faster (like in our Java 8 approach,
> just now we improved our read side).
> The new code is much slimmer (theoretically we could also use a AtomicBoolean
> for that and use the new method {{getOpaque()}}, but I wanted to prevent
> extra method calls, so I used a VarHandle directly).
> It's setup like this:
> - The underlying boolean field is a private member (with unused
> SuppressWarnings, as its unused by the java compiler), marked as volatile
> (that's the recommendation, but in reality it does not matter at all).
> - We create a VarHandle to access this boolean, we never do this directly
> (this is why the volatile marking does not affect us).
> - We use VarHandle.setVolatile() to change our "invalidated" boolean to
> "true", so enforcing a full fence
> - On the read side we use VarHandle.getOpaque() instead of VarHandle.get()
> (like in our old code for Java 8).
> I had to tune our test a bit, as the VarHandles make it take longer until it
> actually crushes (as optimizations jump in later). I also used a random for
> the reads to prevent the optimizer from removing all the bytebuffer reads.
> When we commit this, we can disable the test again (it takes approx 50 secs
> on my machine).
> I'd still like to see the differences between the plain read and the opaque
> read in production, so maybe [~mikemccand] or [~rcmuir] can do a comparison
> with nightly benchmarker?
> Have fun, maybe [~dweiss] has some ideas, too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12291) Async prematurely reports completed status that causes severe shard loss
[
https://issues.apache.org/jira/browse/SOLR-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827852#comment-16827852
]
Lucene/Solr QA commented on SOLR-12291:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} SOLR-12291 does not apply to master. Rebase required? Wrong
Branch? See
https://wiki.apache.org/solr/HowToContribute#Creating_the_patch_file for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-12291 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12967296/SOLR-12291.patch |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/389/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Async prematurely reports completed status that causes severe shard loss
>
>
> Key: SOLR-12291
> URL: https://issues.apache.org/jira/browse/SOLR-12291
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Backup/Restore, SolrCloud
>Reporter: Varun Thacker
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-12291.patch, SOLR-12291.patch, SOLR-12291.patch,
> SOLR-12291.patch, SOLR-12291.patch, SOLR-122911.patch
>
>
> The OverseerCollectionMessageHandler sliceCmd assumes only one replica exists
> on one node
> When multiple replicas of a slice are on the same node we only track one
> replica's async request. This happens because the async requestMap's key is
> "node_name"
> I discovered this when [~alabax] shared some logs of a restore issue, where
> the second replica got added before the first replica had completed it's
> restorecore action.
> While looking at the logs I noticed that the overseer never called
> REQUESTSTATUS for the restorecore action , almost as if it had missed
> tracking that particular async request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-12) - Build # 23997 - Still unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23997/
Java: 64bit/jdk-12 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.TestCloudRecovery2.test
Error Message:
Timeout waiting to see state for collection=collection1
:DocCollection(collection1//collections/collection1/state.json/14)={
"pullReplicas":"0", "replicationFactor":"2", "shards":{"shard1":{
"range":"8000-7fff", "state":"active", "replicas":{
"core_node3":{ "core":"collection1_shard1_replica_n1",
"base_url":"http://127.0.0.1:42939/solr;,
"node_name":"127.0.0.1:42939_solr", "state":"active",
"type":"NRT", "force_set_state":"false", "leader":"true"},
"core_node4":{ "core":"collection1_shard1_replica_n2",
"base_url":"http://127.0.0.1:42637/solr;,
"node_name":"127.0.0.1:42637_solr", "state":"down",
"type":"NRT", "force_set_state":"false",
"router":{"name":"compositeId"}, "maxShardsPerNode":"2",
"autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"} Live
Nodes: [127.0.0.1:42637_solr, 127.0.0.1:42939_solr] Last available state:
DocCollection(collection1//collections/collection1/state.json/14)={
"pullReplicas":"0", "replicationFactor":"2", "shards":{"shard1":{
"range":"8000-7fff", "state":"active", "replicas":{
"core_node3":{ "core":"collection1_shard1_replica_n1",
"base_url":"http://127.0.0.1:42939/solr;,
"node_name":"127.0.0.1:42939_solr", "state":"active",
"type":"NRT", "force_set_state":"false", "leader":"true"},
"core_node4":{ "core":"collection1_shard1_replica_n2",
"base_url":"http://127.0.0.1:42637/solr;,
"node_name":"127.0.0.1:42637_solr", "state":"down",
"type":"NRT", "force_set_state":"false",
"router":{"name":"compositeId"}, "maxShardsPerNode":"2",
"autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError:
Timeout waiting to see state for collection=collection1
:DocCollection(collection1//collections/collection1/state.json/14)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{"shard1":{
"range":"8000-7fff",
"state":"active",
"replicas":{
"core_node3":{
"core":"collection1_shard1_replica_n1",
"base_url":"http://127.0.0.1:42939/solr;,
"node_name":"127.0.0.1:42939_solr",
"state":"active",
"type":"NRT",
"force_set_state":"false",
"leader":"true"},
"core_node4":{
"core":"collection1_shard1_replica_n2",
"base_url":"http://127.0.0.1:42637/solr;,
"node_name":"127.0.0.1:42637_solr",
"state":"down",
"type":"NRT",
"force_set_state":"false",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}
Live Nodes: [127.0.0.1:42637_solr, 127.0.0.1:42939_solr]
Last available state:
DocCollection(collection1//collections/collection1/state.json/14)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{"shard1":{
"range":"8000-7fff",
"state":"active",
"replicas":{
"core_node3":{
"core":"collection1_shard1_replica_n1",
"base_url":"http://127.0.0.1:42939/solr;,
"node_name":"127.0.0.1:42939_solr",
"state":"active",
"type":"NRT",
"force_set_state":"false",
"leader":"true"},
"core_node4":{
"core":"collection1_shard1_replica_n2",
"base_url":"http://127.0.0.1:42637/solr;,
"node_name":"127.0.0.1:42637_solr",
"state":"down",
"type":"NRT",
"force_set_state":"false",
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at
__randomizedtesting.SeedInfo.seed([D7D0562621C4E2F5:5F8469FC8F388F0D]:0)
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:310)
at
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:288)
at
org.apache.solr.cloud.TestCloudRecovery2.test(TestCloudRecovery2.java:106)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at
