github-actions[bot] commented on code in PR #65688:
URL: https://github.com/apache/doris/pull/65688#discussion_r3594491471
##########
fe/fe-core/src/main/java/org/apache/doris/job/offset/jdbc/JdbcSourceOffsetProvider.java:
##########
@@ -286,11 +294,13 @@ public void fetchRemoteMeta(Map<String, String>
properties) throws Exception {
}
Map<String, String> newEndOffset = parseCdcResponseData(
result.getResponse(), new TypeReference<Map<String,
String>>() {});
- // null→value also counts as a change: upstream may have advanced
while fetch was blocked.
- if (endBinlogOffset == null ||
!endBinlogOffset.equals(newEndOffset)) {
- hasMoreData = true;
+ synchronized (splitsLock) {
+ // null→value also counts as a change: upstream may have
advanced while fetch was blocked.
+ if (endBinlogOffset == null ||
!endBinlogOffset.equals(newEndOffset)) {
+ hasMoreData = true;
+ }
+ endBinlogOffset = newEndOffset;
Review Comment:
[P2] Make fetch publication participate in stale-state validation
Putting the assignment under `splitsLock` does not ensure that the
asynchronously fetched value is current, and it interacts badly with the new
reference-identity guard in both directions:
1. If a compare snapshots `E1`, an unchanged periodic fetch is deserialized
as `E2`; this assignment replaces the reference even though the values are
equal. A valid zero comparison then looks stale, returns `true`, and can
dispatch an empty load instead of idling.
2. If a fetch starts at `E`, then a task commits `C > E` and the negative
compare branch promotes EndOffset to `C`, the older fetch can finish afterward
and lower EndOffset back to `E`, recreating the exact `CurrentOffset >
EndOffset` state this change is fixing.
The BE forwards these requests on a concurrent heavy-work pool, so both
completion orders are possible. Please snapshot/version current and end state
around `fetchEndOffset`, preserve the existing map reference for a value-equal
response, and reject a response made stale by intervening progress. The focused
tests should cover both the equal-replacement and reverse-completion
interleavings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
