JNSimba opened a new pull request, #64728:
URL: https://github.com/apache/doris/pull/64728
### What problem does this PR solve?
For JDBC (MySQL/PostgreSQL) CDC streaming jobs in `initial`/`snapshot` mode,
the remote
CDC reader — which creates the PostgreSQL replication slot and publication —
was only
initialized lazily on the first runtime split fetch. Two problems followed:
1. **CREATE JOB succeeded even when the source side was unusable.** If reader
initialization failed at runtime (e.g. the source user lacks privilege to
`CREATE PUBLICATION`), the job was already created and simply stayed
paused with no
progress and no clear failure at creation time.
2. **The real remote error was masked.** The cdc client returns failures as a
`{code, data:"<message>"}` envelope over HTTP 200. FE force-deserialized
the response
straight into the success payload type, so on failure the error string
was coerced
into e.g. `List<SnapshotSplit>`, producing a misleading type-mismatch
that hid the
original cause.
### What is changed?
- `initOnCreate` now opens the remote reader for **every** startup mode (not
only the
non-splitting ones), so connectivity / auth / slot / publication problems
fail
`CREATE JOB` synchronously instead of surfacing later as a silently paused
job.
- Added an envelope-aware parse that checks the response `code` before
deserializing,
and routed `fetchEndOffset` / `compareOffset` / `fetchSplits` through it
so the
original remote error is propagated to the job's failure reason.
- `initSourceReader` is now `protected` so the async-split unit test can
stub it.
### Check List
- [x] Checkstyle passed on the changed files (main + test).
- [x] Unit tests added for the response-envelope parsing and the CREATE JOB
fail-fast path.
- [ ] fe-core unit tests / regression to be exercised by CI (the local
workspace lacks the thrift toolchain).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
