[
https://issues.apache.org/jira/browse/KAFKA-7297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16741921#comment-16741921
]
Zhanxiang (Patrick) Huang commented on KAFKA-7297:
--------------------------------------------------
Sorry for being late to the discussion.
[~lindong] I think we will still have the same issue if we only lock the read
but not copy during reads because writes can take effect after we get back the
iterators in the read and release the lock but before we access the iterators.
If I understand it correctly, the weak consistency provided by
ConcurrentSkipListMap doesn't prevent returning additional entry if we don't
copy on reads because _"it may (but are not guaranteed to) reflect any
modifications subsequent to construction"_. That being said, I think copying is
unavoidable in either approach. I am more in favor of copy on writes
([~ijuma]'s proposal) than copy on reads since reads are more frequent than
writes.
> Both read/write access to Log.segments should be protected by lock
> ------------------------------------------------------------------
>
> Key: KAFKA-7297
> URL: https://issues.apache.org/jira/browse/KAFKA-7297
> Project: Kafka
> Issue Type: Improvement
> Reporter: Dong Lin
> Assignee: Zhanxiang (Patrick) Huang
> Priority: Major
>
> Log.replaceSegments() updates segments in two steps. It first adds new
> segments and then remove old segments. Though this operation is protected by
> a lock, other read access such as Log.logSegments does not grab lock and thus
> these methods may return an inconsistent view of the segments.
> As an example, say Log.replaceSegments() intends to replace segments [0,
> 100), [100, 200) with a new segment [0, 200). In this case if Log.logSegments
> is called right after the new segments are added, the method may return
> segments [0, 200), [100, 200) and messages in the range [100, 200) may be
> duplicated if caller choose to enumerate all messages in all segments
> returned by the method.
> The solution is probably to protect read/write access to Log.segments with
> read/write lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
