This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 32136ca4e93 [fix](routine load) write edit log when rescheduled job
(#40728)
32136ca4e93 is described below
commit 32136ca4e9366e18b947f055c507718a7001a73c
Author: hui lai <[email protected]>
AuthorDate: Sat Sep 14 16:28:33 2024 +0800
[fix](routine load) write edit log when rescheduled job (#40728)
```
2024-09-11 20:00:53,079 ERROR (replayer|105)
[RoutineLoadManager.replayChangeRoutineLoadJob():836] should not happened
org.apache.doris.common.DdlException: errCode = 2, detailMessage = Could
not transform PAUSED to PAUSED
at
org.apache.doris.load.routineload.RoutineLoadJob.checkStateTransform(RoutineLoadJob.java:855)
~[doris-fe.jar:1.2-SNAPSHOT]
at
org.apache.doris.load.routineload.RoutineLoadJob.unprotectUpdateState(RoutineLoadJob.java:1407)
~[doris-fe.jar:1.2-SNAPSHOT]
at
org.apache.doris.load.routineload.RoutineLoadJob.updateState(RoutineLoadJob.java:1394)
~[doris-fe.jar:1.2-SNAPSHOT]
at
org.apache.doris.load.routineload.RoutineLoadManager.replayChangeRoutineLoadJob(RoutineLoadManager.java:834)
~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.persist.EditLog.loadJournal(EditLog.java:717)
~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.replayJournal(Env.java:2913)
~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env$4.runOneCycle(Env.java:2675)
~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116)
~[doris-fe.jar:1.2-SNAPSHOT]
```
`unprotectNeedReschedule()` will change job state to
`JobState.NEED_SCHEDULE` without `logOpRoutineLoadJob`.If job is paused
then rescheduled and paused finally, the record of two consecutive edit
logs will be 'PAUSED', the correct
replay sequence should be: `PAUSED` -> `NEED_SCHEDULE` ->` PAUSED`.
Therefore, it is need to write edit log when rescheduled job.
---
.../src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java | 2 +-
.../test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java | 1 -
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
index 42032ebfc96..e12b215a0e4 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java
@@ -1514,7 +1514,7 @@ public abstract class RoutineLoadJob
.add("msg", "Job need to be rescheduled")
.build());
unprotectUpdateProgress();
- executeNeedSchedule();
+ unprotectUpdateState(JobState.NEED_SCHEDULE, null, false);
}
} finally {
writeUnlock();
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java
b/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java
index 863cc6807c7..6f3dd2eaaa8 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/load/routineload/RoutineLoadJobTest.java
@@ -290,7 +290,6 @@ public class RoutineLoadJobTest {
};
RoutineLoadJob routineLoadJob = new KafkaRoutineLoadJob();
- Deencapsulation.setField(routineLoadJob, "state",
RoutineLoadJob.JobState.RUNNING);
Deencapsulation.setField(routineLoadJob, "progress", kafkaProgress);
routineLoadJob.update();
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
