Doris-Breakwater commented on issue #66562: URL: https://github.com/apache/doris/issues/66562#issuecomment-5210054245
Breakwater-GitHub-Analysis-Slot: slot_c2143d87d931 Initial triage: **not reproducible from the supplied SQL against the referenced upstream code; clarification is required before treating this as a confirmed bug.** No labels were attached when this issue was ingested. Verified facts: - The reproduction mixes the two CDC job forms: a `cdc_stream(...)` TVF job and a `FROM MYSQL (...) TO DATABASE ...` source job. `SOURCE PROPERTIES`/`FROM MYSQL` belongs to the latter form. - On both the inspected `master` (`4e3c1b84`) and `branch-4.1` (`e0115cac`) revisions, changing `offset` in source properties is explicitly rejected by [`AlterJobCommand`](https://github.com/apache/doris/blob/4e3c1b84dd5936b8d751e0a95719350e8cd8e980/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/AlterJobCommand.java#L224-L229). - The supported CDC ALTER route is top-level `PROPERTIES('offset' = '<JSON offset>')`; named modes such as `initial` are explicitly rejected by [`JdbcSourceOffsetProvider.validateAlterOffset()`](https://github.com/apache/doris/blob/4e3c1b84dd5936b8d751e0a95719350e8cd8e980/fe/fe-core/src/main/java/org/apache/doris/job/offset/jdbc/JdbcSourceOffsetProvider.java#L493-L499). The MySQL CDC regression test covers both rejections: [`test_streaming_mysql_job_special_offset.groovy`](https://github.com/apache/doris/blob/4e3c1b84dd5936b8d751e0a95719350e8cd8e980/regression-test/suites/job_p0/streaming_job/cdc/test_streaming_mysql_job_special_offset.groovy#L175-L189). - Consequently, the ALTER statement shown in this issue should return an error and should not reach `StreamingInsertJob.alterJob()`. A silent successful ALTER followed by stale dispatch would contradict the upstream validation path. The proposed stale-state mechanism is nevertheless plausible **if there is another build or ALTER form that actually accepts `initial`**. Merely refreshing source/TVF properties does not reset the provider. In particular, the TVF replay path retains an existing binlog `currentOffset` and intentionally does nothing when that offset is already in the binlog phase ([`JdbcTvfSourceOffsetProvider.replayIfNeed()`](https://github.com/apache/doris/blob/4e3c1b84dd5936b8d751e0a95719350e8cd8e980/fe/fe-core/src/main/java/org/apache/doris/job/offset/jdbc/JdbcTvfSourceOffsetProvider.java#L321-L380)). This establishes a code-level risk for any future or downstream path that permits a named-mode reset without a dedicated reset operation, but it does not establish that the supplied upstream reproduction is reachable. Please provide the following to disambiguate: 1. The complete, literal `CREATE JOB` and `ALTER JOB` statements (with secrets redacted), including whether this is a TVF job or a `FROM MYSQL ... TO DATABASE` job. 2. The exact FE result/error returned by the ALTER statement. 3. `SELECT VERSION()`, the build/commit hash, and whether the binary contains downstream patches. 4. `SHOW JOB`/`jobs("type"="insert")` rows immediately before ALTER and immediately after RESUME, including status, current offset, and execute SQL/source properties where available. 5. FE logs covering PAUSE, ALTER, RESUME, `modifyPropertiesInternal`, provider replay, and the first created task's running offset. Recommended next step: - If the literal statement is rejected as upstream currently specifies, treat this as a feature request for an explicit “restart snapshot” operation rather than an offset-update bug. Such an operation must atomically reset in-memory and persisted provider state, rebuild/delete snapshot split metadata as appropriate, reset cloud progress, and define duplicate-data/target-table semantics. - If the reporter can show that one of the referenced upstream builds accepts `initial`, first identify the validation bypass and add a regression test for PAUSE -> reset -> RESUME (including FE restart/cloud replay). The fix should use a provider-level reset API rather than only changing `sourceProperties`; otherwise `currentOffset`, `binlogOffsetPersist`, split queues/progress, and high-watermark state can remain mutually inconsistent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
