Simon Willnauer created LUCENE-8754:
---------------------------------------
Summary: SegmentInfo#toString can cause
ConcurrentModificationException
Key: LUCENE-8754
URL: https://issues.apache.org/jira/browse/LUCENE-8754
Project: Lucene - Core
Issue Type: Improvement
Reporter: Simon Willnauer
A recent change increased the likelihood for this issue to show up but it can
already happen before since we are using the attributes map in the
StoredFieldsFormat for quite some time. I found this issue due to a test
failure on our CI:
{noformat}
13:11:56 [junit4] Suite: org.apache.lucene.index.TestIndexSorting
13:11:56 [junit4] 2> apr 05, 2019 8:11:53 AM
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
13:11:56 [junit4] 2> WARNING: Uncaught exception in thread:
Thread[Thread-507,5,TGRP-TestIndexSorting]
13:11:56 [junit4] 2> java.util.ConcurrentModificationException
13:11:56 [junit4] 2> at
__randomizedtesting.SeedInfo.seed([7C25B308F180203B]:0)
13:11:56 [junit4] 2> at
java.util.HashMap$HashIterator.nextNode(HashMap.java:1442)
13:11:56 [junit4] 2> at
java.util.HashMap$EntryIterator.next(HashMap.java:1476)
13:11:56 [junit4] 2> at
java.util.HashMap$EntryIterator.next(HashMap.java:1474)
13:11:56 [junit4] 2> at
java.util.AbstractMap.toString(AbstractMap.java:554)
13:11:56 [junit4] 2> at
org.apache.lucene.index.SegmentInfo.toString(SegmentInfo.java:222)
13:11:56 [junit4] 2> at
org.apache.lucene.index.SegmentCommitInfo.toString(SegmentCommitInfo.java:345)
13:11:56 [junit4] 2> at
org.apache.lucene.index.SegmentCommitInfo.toString(SegmentCommitInfo.java:364)
13:11:56 [junit4] 2> at java.lang.String.valueOf(String.java:2994)
13:11:56 [junit4] 2> at
java.lang.StringBuilder.append(StringBuilder.java:131)
13:11:56 [junit4] 2> at
java.util.AbstractMap.toString(AbstractMap.java:557)
13:11:56 [junit4] 2> at
java.util.Collections$UnmodifiableMap.toString(Collections.java:1493)
13:11:56 [junit4] 2> at java.lang.String.valueOf(String.java:2994)
13:11:56 [junit4] 2> at
java.lang.StringBuilder.append(StringBuilder.java:131)
13:11:56 [junit4] 2> at
org.apache.lucene.index.TieredMergePolicy.findForcedMerges(TieredMergePolicy.java:628)
13:11:56 [junit4] 2> at
org.apache.lucene.index.IndexWriter.updatePendingMerges(IndexWriter.java:2181)
13:11:56 [junit4] 2> at
org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2154)
13:11:56 [junit4] 2> at
org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1988)
13:11:56 [junit4] 2> at
org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1939)
13:11:56 [junit4] 2> at
org.apache.lucene.index.TestIndexSorting$UpdateRunnable.run(TestIndexSorting.java:1851)
13:11:56 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
13:11:56 [junit4] 2>
13:11:56 [junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestIndexSorting -Dtests.method=testConcurrentUpdates
-Dtests.seed=7C25B308F180203B -Dtests.slow=true -Dtest
{noformat}
The issue is that we update the attributes map (also we similarly do the same
for diagnostics but it's not necessarily causing the issue since the
diagnostics map is never modified) during the merge process but access it in
the merge policy when looking at running merges and there we call toString on
SegmentCommitInfo which happens without any synchronization. This is
technically unsafe publication but IW is a mess along those lines and real
fixes would require significant changes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
