JNSimba opened a new pull request, #70651: URL: https://github.com/apache/doris/pull/70651
### Background Doris streaming jobs consume source changes and recover from persisted offsets. These regression suites cover CDC ingestion, S3 task retries, and FE restart recovery. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Test Problem Summary: The auto-resume suite reads the wrong failure-counter row and can swallow a failed check, while the FE restart suite references an undefined status variable and does not verify newly arriving data after recovery. Existing CDC checks also lack explicit rollback and successive committed changes to the same key. Strengthen these seven suites with bounded waits, committed markers, and complete expected-output comparisons; no production behavior changes. ### Details - **Change map:** 337 changed lines: 205 in seven regression suites (+183/-22) and 132 added lines in seven expected-output files. No production code, protocol, dependency, or documentation changes. - **Failure:** Auto-resume checks can continue after a timeout, restart validation can fail on its PENDING branch, and status/counter checks alone do not establish recovered data correctness. - **Cause:** Incorrect counter indexing and string comparisons, a missing exception rethrow, an undefined variable, and missing phase-specific data checks. - **Scenarios:** Roll back INSERT/UPDATE/DELETE on MySQL, PostgreSQL, and OceanBase, then commit a marker and compare full rows. While paused, commit UPDATE/DELETE/INSERT on the same complete key and delete another key; resume and check the final contents. For S3 recovery, verify existing rows before and after FE restart, then upload a new file and verify both the offset and all 21 rows. - **What existing tests miss:** Rolled-back changes leaking downstream, stale values or resurrected deletes after successive key changes, and a recovered job that retains its state but no longer consumes new input. - **Fails when broken:** Missing markers, incorrect values or key sets, unexpected offsets, failed waits, and output mismatches all fail the suite. Fault injection is removed in finally; each restart run uses an isolated object prefix and preserves the shared fixtures. - **Stability evidence:** Each of the seven suites passed five complete runs in the verified modes, with zero failed, fatal, or skipped suites in those 35 successful runs. PostgreSQL incremental waits now use bounded progress polling instead of two fixed one-minute sleeps. ### Testing - Built ASAN FE/BE and the CDC client; built the regression framework including its unit tests. - In a Linux development environment, ran the MySQL/PostgreSQL/OceanBase basic suites, both composite-primary-key suites, and auto-resume five times each. Ran FE restart in `not_cloud` mode five times. All 35 successful rounds used normal expected-output comparisons. - S3 tests used an isolated MinIO bucket containing byte-identical copies of the existing CSV fixtures. Restart repetitions used otherwise identical per-run suite copies with unique names to retain their Docker environments and logs. - Expected outputs were initially derived from fixture and transaction data, then verified by the normal regression runs. All 14 tested case/output files match the submitted versions. - Cloud restart was attempted but blocked before data verification: FoundationDB could not initialize inotify because the host resource limit was exhausted. No Cloud data checks are claimed as passing. - `git diff --check` passed. ### Release note None. ### Check List (For Author) - Behavior changed: No production behavior changes. - Does this need documentation: No. - Pre-submission status: Partially verified: all seven suites passed five runs in the modes listed above; FE restart in Cloud mode remains blocked by the test environment. ### AI Disclosure - Tool: Codex (GPT-6), with remote Codex for validation. - Generated: Regression test changes, expected-output baselines, and this description. - Human review: Final reviewer approval is pending. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
