github-actions[bot] commented on code in PR #68270:
URL: https://github.com/apache/doris/pull/68270#discussion_r4256607860
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertJob.java:
##########
@@ -1087,6 +1120,19 @@ public void replayOnUpdated(StreamingInsertJob
replayJob) {
}
try {
modifyPropertiesInternal(replayJob.getProperties());
+ if (replayJob.getOffsetProviderPersist() != null) {
+ setOffsetProviderPersist(replayJob.getOffsetProviderPersist());
+ if (tvfType != null && !Config.isCloudMode()) {
+
offsetProvider.restoreFromPersistInfo(offsetProviderPersist);
Review Comment:
[P1] Keep status journals from rewinding committed TVF offsets
JobManager.alterJobStatus() releases the job lock before
logUpdateOperation(). After RESUME publishes PENDING, an existing timer can
start the first INSERT and queue COMMITTED(B) before afterCommitted() advances
offsetProviderPersist from A. The unlocked RESUME record can then be serialized
behind that transaction with persist=A: with batched edit logs and
enable_txn_log_outside_lock=false, both records can be serialized while the
commit thread awaits the flush. Replaying COMMITTED(B) followed by this update
now restores A into the non-cloud TVF provider;
S3SourceOffsetProvider.replayIfNeed() preserves it on follower promotion, so
the already committed file range is ingested again and duplicates rows in a
DUPLICATE KEY target. The base kept currentOffset=B for that same live
promotion without an image reload. Serialize status offset snapshots
consistently with commit callbacks, or distinguish explicit ALTER offset resets
from ordinary updates so stale status records cannot overwri
te committed progress while intentional rewinds remain supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
