[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844528#comment-16844528
]
Ishan Chattopadhyaya commented on LUCENE-8785:
--
Updated to 8.1, thanks!
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.2, 8.1, master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844061#comment-16844061
]
Andrzej Bialecki commented on LUCENE-8785:
---
[~ichattopadhyaya] this issue has fixVersion = 8.1.1 but it looks like it was
included in 8.1.0. Could you please confirm this and set the fixVersion
accordingly? Thanks!
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.2, master (9.0), 8.2, 8.1.1
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835894#comment-16835894
]
Simon Willnauer commented on LUCENE-8785:
-
{quote} Please feel free to commit this to the release branch. In case of a
re-spin, I'll pick this change up. {quote}
[~ichattopadhyaya] done. Thanks.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.2, master (9.0), 8.2, 8.1.1
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835892#comment-16835892
]
ASF subversion and git services commented on LUCENE-8785:
-
Commit e31be1ee532189996c8979973ff74cab3626283f in lucene-solr's branch
refs/heads/branch_8_1 from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e31be1e ]
LUCENE-8785: Ensure threadstates are locked before iterating (#664)
Ensure new threadstates are locked before retrieving the
number of active threadstates. This causes assertion errors
and potentially broken field attributes in the IndexWriter when
IndexWriter#deleteAll is called while actively indexing.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.2, master (9.0), 8.2, 8.1.1
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835799#comment-16835799
]
Ishan Chattopadhyaya commented on LUCENE-8785:
--
bq. This reverts commit 2af15a6e725b2c548bb8ded2ba67935ef592823d. We are
currently
releasing this branch and it's unclear if we respin. In the case of a respin
we can backport this commit easily from the stable branch.
Please feel free to commit this to the release branch. In case of a re-spin,
I'll pick this change up.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.2, master (9.0), 8.2, 8.1.1
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835500#comment-16835500
]
Michael McCandless commented on LUCENE-8785:
Thank you [~simonw]! Love how open-source works ;) Lucene gets better.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.2, master (9.0), 8.2, 8.1.1
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835472#comment-16835472
]
ASF subversion and git services commented on LUCENE-8785:
-
Commit bd663649a48971f0a2fc2ada82471961e5a2507b in lucene-solr's branch
refs/heads/branch_8_1 from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=bd66364 ]
Revert "LUCENE-8785: Ensure threadstates are locked before iterating (#664)"
This reverts commit 2af15a6e725b2c548bb8ded2ba67935ef592823d. We are currently
releasing this branch and it's unclear if we respin. In the case of a respin
we can backport this commit easily from the stable branch.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.2, master (9.0), 8.2, 8.1.1
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835467#comment-16835467
]
ASF subversion and git services commented on LUCENE-8785:
-
Commit d2d3371a354e9534f09e067bfb0a690c3f2260ab in lucene-solr's branch
refs/heads/branch_7_7 from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d2d3371 ]
LUCENE-8785: Ensure threadstates are locked before iterating (#664)
Ensure new threadstates are locked before retrieving the
number of active threadstates. This causes assertion errors
and potentially broken field attributes in the IndexWriter when
IndexWriter#deleteAll is called while actively indexing.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.1, 8.0.1, 8.1, master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835458#comment-16835458
]
ASF subversion and git services commented on LUCENE-8785:
-
Commit 2af15a6e725b2c548bb8ded2ba67935ef592823d in lucene-solr's branch
refs/heads/branch_8_1 from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=2af15a6 ]
LUCENE-8785: Ensure threadstates are locked before iterating (#664)
Ensure new threadstates are locked before retrieving the
number of active threadstates. This causes assertion errors
and potentially broken field attributes in the IndexWriter when
IndexWriter#deleteAll is called while actively indexing.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.1, 8.0.1, 8.1, master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835454#comment-16835454
]
ASF subversion and git services commented on LUCENE-8785:
-
Commit a1560f26c1c245f8e7a4377155ee8bcaa805e554 in lucene-solr's branch
refs/heads/branch_8x from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=a1560f2 ]
LUCENE-8785: Ensure threadstates are locked before iterating (#664)
Ensure new threadstates are locked before retrieving the
number of active threadstates. This causes assertion errors
and potentially broken field attributes in the IndexWriter when
IndexWriter#deleteAll is called while actively indexing.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.1, 8.0.1, 8.1, master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16835451#comment-16835451
]
ASF subversion and git services commented on LUCENE-8785:
-
Commit e8d88a5b54b268b370f4bbc6f8a61148a62067c9 in lucene-solr's branch
refs/heads/master from Simon Willnauer
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=e8d88a5 ]
LUCENE-8785: Ensure threadstates are locked before iterating (#664)
Ensure new threadstates are locked before retrieving the
number of active threadstates. This causes assertion errors
and potentially broken field attributes in the IndexWriter when
IndexWriter#deleteAll is called while actively indexing.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Assignee: Simon Willnauer
>Priority: Minor
> Fix For: 7.7.1, 8.0.1, 8.1, master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834467#comment-16834467
]
Simon Willnauer commented on LUCENE-8785:
-
{quote}
If there is another thread coming in after we locked the existent threadstates
we just issue a new one.
Yuck
{quote}
I looked at the code again and we actually lock the threadstates for this
purpose. I implemented this in LUCENE-8639. The issue here is in-fact a race
condition since we request the number of active threadstates before we lock new
ones. It's a classic one-line fix. I referenced a PR for this. [~mikemccand]
would you take a look
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832487#comment-16832487
]
Dawid Weiss commented on LUCENE-8785:
-
bq. I want to understand why, in such a rather edgy case a user can't do
exactly this.
The difference exists for people who manage several commits in the past. A
deleteAll leaves an empty commit, but other threads and applications can still
access the commit history (and can do so, even concurrently while deleteAll is
being called).
Don't know whether this is particularly common (perhaps not), but it's a fairly
valid use case (creating an "empty" commit by wiping any existing content).
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Priority: Minor
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832336#comment-16832336
]
Simon Willnauer commented on LUCENE-8785:
-
{quote} I realize neither ES nor Solr expose deleteAll but I don't think that's
a valid argument to remove it from Lucene.
{quote}
huh, I don't think that's a valid argument either, I just re-read my comments
- sorry if you felt I was alluding to es or solr here. My argument is that if
you want to do that you should construct a new IndexWriter instead of calling
deleteAll(). Given this comment on the javadocs:
{noformat}
Essentially a call to {@link #deleteAll()} is equivalent to creating a new
{@link IndexWriter} with {@link OpenMode#CREATE}
{noformat}
I want to understand why, in such a rather edgy case a user can't do exactly
this. There is no race, no confusion it's very simple from a semantics
perspective. Currently there are 2 ways and one if confusing. I think we should
move towards removing the second way.
{quote}And for some reason the index is reset once per week, but the devs want
to allow searching of the old index while the new index is (slowly) built up.
But if something goes badly wrong, they need to be able to rollback (the
deleteAll and all subsequently added docs) to the last commit and try again
later. If instead it succeeds, then a refresh/commit will switch to the new
index atomically.
{quote}
Well, there are tons of ways to do that no? I mean you can have 2 directories?
Yes it causes some engineering effort but the semantics would be cleaner even
for the app that does what you explain.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Priority: Minor
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832036#comment-16832036
]
Michael McCandless commented on LUCENE-8785:
{quote}If there is another thread coming in after we locked the existent
threadstates we just issue a new one.
{quote}
Yuck :(
{quote}I think we can just do what deleteAll() does today except of not
dropping the schema on the floor?
{quote}
The thing is, I think erasing schema while under transaction is a useful
feature of Lucene. I realize neither ES nor Solr expose deleteAll but I don't
think that's a valid argument to remove it from Lucene ;)
{quote}I want to understand the usecase for this. I can see how somebody wants
to drop all docs but basically droping all IW state on the floor is difficult
in my eyes.
{quote}
Well, imagine a user searching documents with diverse/varying fields, maybe
arriving from an external (not controlled by the developer) source. And for
some reason the index is reset once per week, but the devs want to allow
searching of the old index while the new index is (slowly) built up. But if
something goes badly wrong, they need to be able to rollback (the {{deleteAll}}
and all subsequently added docs) to the last commit and try again later. If
instead it succeeds, then a refresh/commit will switch to the new index
atomically.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Priority: Minor
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831635#comment-16831635
]
Simon Willnauer commented on LUCENE-8785:
-
> But at the point we call clear() haven't we already blocked all indexing
> threads?
no, it might look like we do that but we don't. We block and lock all threads
up that that point in time. If there is another thread coming in after we
locked the existent threadstates we just issue a new one.
> I also dislike deleteAll() and you're right a user could deleteByQuery using
> MatchAllDocsQuery; can we make that close-ish as efficient as deleteAll() is
> today?
I think we can just do what deleteAll() does today except of not dropping the
schema on the floor?
> Though indeed that would preserve the schema, while deleteAll() let's you
> delete docs, delete schema, all under transaction (the change is not visible
> until commit).
I want to understand the usecase for this. I can see how somebody wants to drop
all docs but basically droping all IW state on the floor is difficult in my
eyes.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Priority: Minor
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831618#comment-16831618
]
Michael McCandless commented on LUCENE-8785:
But at the point we call {{clear()}} haven't we already blocked all indexing
threads?
I also dislike {{deleteAll()}} and you're right a user could deleteByQuery
using MatchAllDocsQuery; can we make that close-ish as efficient as
{{deleteAll()}} is today? Though indeed that would preserve the schema, while
{{deleteAll()}} let's you delete docs, delete schema, all under transaction
(the change is not visible until commit). I'm torn on just removing that ...
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Priority: Minor
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8785) TestIndexWriterDelete.testDeleteAllNoDeadlock failure
[
https://issues.apache.org/jira/browse/LUCENE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831612#comment-16831612
]
Simon Willnauer commented on LUCENE-8785:
-
[~mikemccand] I think this is caused by the fact that we simply call _clear()_
during _IW#deleteAll()_. If this happens concurrently to the a document being
indexed this assertion can trip. I personally always disliked the complexity of
_IW#deleteAll_ and from my perspective we should remove this method entirely
and ask users to open a new IW if they want to drop all the information
including the _schema_. We can still fast-path a _MatchAllQuery_ through
something like this as we do today (which is a problem IMO since it drops all
fields map info which it shouldn't?). IMO if you want a fresh index start from
scratch but to delete all docs go and run DeleteByQueyr and keep the schema.
> TestIndexWriterDelete.testDeleteAllNoDeadlock failure
> -
>
> Key: LUCENE-8785
> URL: https://issues.apache.org/jira/browse/LUCENE-8785
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
>Affects Versions: 7.6
> Environment: OpenJDK 1.8.0_202
>Reporter: Michael McCandless
>Priority: Minor
>
> I was running Lucene's core tests on an {{i3.16xlarge}} EC2 instance (64
> cores), and hit this random yet spooky failure:
> {noformat}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterDelete -Dtests.method=testDeleteAllNoDeadLock
> -Dtests.seed=952BE262BA547C1 -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-YE -Dtests.timezone=Europe/Lisbon -Dtests.as\
> serts=true -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.16s J3 | TestIndexWriterDelete.testDeleteAllNoDeadLock
> <<<
> [junit4] > Throwable #1:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=36, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexWriterDelete]
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1:3A4B5138AB66FD97]:0)
> [junit4] > Caused by: java.lang.RuntimeException:
> java.lang.IllegalArgumentException: field number 0 is already mapped to field
> name "null", not "content"
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([952BE262BA547C1]:0)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:332)
> [junit4] > Caused by: java.lang.IllegalArgumentException: field number
> 0 is already mapped to field name "null", not "content"
> [junit4] > at
> org.apache.lucene.index.FieldInfos$FieldNumbers.verifyConsistent(FieldInfos.java:310)
> [junit4] > at
> org.apache.lucene.index.FieldInfos$Builder.getOrAdd(FieldInfos.java:415)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.getOrAddField(DefaultIndexingChain.java:650)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:428)
> [junit4] > at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:297)
> [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:450)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1291)
> [junit4] > at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1264)
> [junit4] > at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:159)
> [junit4] > at
> org.apache.lucene.index.TestIndexWriterDelete$1.run(TestIndexWriterDelete.java:326){noformat}
> It does *not* reproduce unfortunately ... but maybe there is some subtle
> thread safety issue in this code ... this is a hairy part of Lucene ;)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
