yujun777 commented on code in PR #69850:
URL: https://github.com/apache/doris/pull/69850#discussion_r4423992258
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/stream/OlapTableStream.java:
##########
@@ -318,6 +375,7 @@ public void
unprotectedUpdateStreamUpdate(AbstractTableStreamUpdate update, Long
partitionOffset.put(entry.getKey(), entry.getValue());
partitionConsumptionTime.put(entry.getKey(), ts);
}
+ unprotectedRecordPartitionStreamStates(next.keySet());
Review Comment:
Fixed in a9e7634806c, both halves of the finding:
1. **The range and the counters now come from one read.**
`OlapTableStreamWrapper` snapshots the partitions' `PartitionStreamState` at
the same time it builds `outputUpdateMap` (constructor, and
`installCloudReadStates` for cloud read states), with the stream read lock held
across both, and the estimate reads that snapshot instead of asking the stream
later. An offset advance can no longer pair the range of an old offset with the
counters of a newer one.
2. **The map read back from an image is made concurrent.**
`OlapTableStream.gsonPostProcess()` replaces whatever map the deserialization
built with a `ConcurrentHashMap`, and `partitionStates()` also converts on
first use, so the planner's lock-free reads and the offset advance's
replacement do not race on a plain map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
