maks3201 opened a new issue, #66562:
URL: https://github.com/apache/doris/issues/66562
## Version
master (4.2-SNAPSHOT), also affects branch-4.1 (4.1.3)
## Environment
Compute-storage-decoupled mode (cloud mode), MySQL CDC streaming job
(cdc_stream TVF).
## Reproduction Steps
```sql
-- 1. Create a CDC streaming job in initial mode (snapshot + binlog)
CREATE JOB my_cdc_job
ON SCHEDULER EVERY 10 SECONDS
PROPERTIES (...)
SOURCE PROPERTIES ("offset" = "initial", ...)
DO INSERT INTO target_table SELECT * FROM cdc_stream(...);
-- 2. Wait for the job to complete the snapshot phase and enter binlog
streaming.
-- SHOW JOB STATUS shows offset at e.g. {"file":"binlog.000003","pos":154}
-- 3. Pause the job
PAUSE JOB WHERE jobName = 'my_cdc_job';
-- 4. ALTER the offset back to initial to force a fresh snapshot
ALTER JOB 'my_cdc_job' SOURCE PROPERTIES ("offset" = "initial");
-- 5. Resume the job
RESUME JOB WHERE jobName = 'my_cdc_job';
```
## Expected Behavior
After RESUME, the job starts a fresh snapshot from scratch — fetching split
definitions from the BE and dispatching snapshot tasks, as if it were a newly
created job with `offset=initial`.
## Observed Behavior
After RESUME, the job dispatches a streaming task using the **old binlog
position** (`binlog.000003:154`) instead of starting a fresh snapshot. The
operator's intent to reset the job is silently ignored.
This happens because `JdbcSourceOffsetProvider` retains the previous binlog
position (`currentOffset`, `binlogOffsetPersist`) and split progress in memory
across the ALTER. On the next scheduler tick:
1. `replayIfNeed()` restores `currentOffset` from the stale
`offsetProviderPersist`
2. `noMoreSplits()` returns `true` (the old split progress indicates
completion)
3. `getNextOffset()` returns the stale binlog offset
4. The task is dispatched with the wrong starting position
## Root Cause
`StreamingInsertJob.alterJob()` updates `sourceProperties` (including
`offset=initial`) but does **not** clear the offset provider's cached state.
The provider's in-memory fields (`currentOffset`, `binlogOffsetPersist`,
`chunkHighWatermarkMap`, split progress, etc.) still reflect the old
binlog-streaming phase.
## Impact
Any manual CDC recovery workflow that relies on PAUSE → ALTER offset=initial
→ RESUME will silently fail to reset. The operator believes they restarted the
job from scratch; in reality the job continues from the stale binlog position,
which may have expired or become invalid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
