Copilot commented on code in PR #64301:
URL: https://github.com/apache/doris/pull/64301#discussion_r3385358100
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingMultiTblTask.java:
##########
@@ -84,6 +85,8 @@ public class StreamingMultiTblTask extends
AbstractStreamingTask {
private long filteredRows = 0L;
private long loadedRows = 0L;
private long runningBackendId;
+ long lastScannedRows = -1;
+ long lastProgressMs = 0;
Review Comment:
`runningBackendId` is written in the task execution thread (during
`sendWriteRequest`) and read from the scheduler thread in
`fetchProgress()/getTimeoutReason()`; without a happens-before edge this can be
observed as `0` and disable the progress heartbeat. Also,
`lastScannedRows/lastProgressMs` look intended to be runtime-only heartbeat
state (per PR description) but they are currently non-`transient`, so they will
be persisted by Gson and can affect replay/rolling-upgrade behavior.
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingMultiTblTask.java:
##########
@@ -361,23 +365,61 @@ private String getFrontendAddress() {
return Env.getCurrentEnv().getMasterHost() + ":" +
Env.getCurrentEnv().getMasterHttpPort();
}
- public boolean isTimeout() {
+ boolean isTimeout(StreamingTaskProgress progress) {
if (startTimeMs == null) {
// It's still pending, waiting for scheduling.
return false;
}
+ long now = System.currentTimeMillis();
+ if (progress != null && progress.getScannedRows() > lastScannedRows) {
Review Comment:
`isTimeout()` computes `elapsed` from `lastProgressMs`, but for tasks
deserialized from older images/journals (or if `lastProgressMs` is otherwise
left at its default), `lastProgressMs` will be `0` and the task will
immediately time out when `progress` is null/unavailable. Initialize
`lastProgressMs` from `startTimeMs` before using it to keep backward
compatibility with the old start-time-based behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
