goutamadwant commented on code in PR #11897: URL: https://github.com/apache/seatunnel/pull/11897#discussion_r3818873327
########## docs/en/design/upgrade-compatibility-stip.md: ########## @@ -0,0 +1,267 @@ +# STIP: Upgrade Compatibility Program (Phase 2) + +**STIP Number**: TBD +**Status**: Draft +**Author**: zhang-arvin +**Created**: 2026-08-20 +**Related Issues**: [#11356](https://github.com/apache/seatunnel/issues/11356), [#11239](https://github.com/apache/seatunnel/issues/11239) +**Related PRs**: [#11301](https://github.com/apache/seatunnel/pull/11301) (Phase 1) + +## Abstract + +Phase 1 (PR #11301) added a scheduled and manually-triggered cross-version restore workflow +with two representative scenarios. This STIP defines Phase 2: turning that initial workflow +into a **maintainable upgrade-compatibility program** with a documented compatibility contract, +expanded scenario coverage, and clear operational policies. + +## Motivation + +SeaTunnel has multiple stateful upgrade surfaces that can break silently across versions: + +- Checkpoint / savepoint restore +- Source enumerator state restore +- Sink writer state restore +- CDC resume / restore after upgrade +- Serialized job graph and config compatibility + +PR #11301 provides an important first step, but it does not yet answer the release-grade question: +**What upgrade guarantees do we actually verify before shipping changes that affect persisted runtime state?** + +The remaining gaps include: + +1. Scenario coverage is intentionally small (2 scenarios) +2. Not all high-risk state surfaces are represented +3. No clear policy for when upgrade checks must run on PRs +4. Failure output does not clearly identify which stage broke +5. No documented compatibility contract for the community + +## Scope + +### In Scope + +- Define the minimum supported upgrade scenarios that must stay green on `dev` +- Expand scenario coverage for stateful source, sink, CDC, and checkpoint/restore paths +- Define trigger policy: when the workflow runs (schedule, manual, PR-gated) +- Make failures actionable: identify whether the break happened during state creation, restore, resume, or post-restore validation +- Document the compatibility contract the workflow asserts +- English and Chinese documentation updates + +### Out of Scope (Phase 2) + +- Exhaustive coverage of every connector and every historical version +- Turning every PR into a full compatibility matrix run +- Claiming stronger guarantees than the workflow actually verifies + +## Design + +### 1. Supported Version Matrix + +The upgrade compatibility workflow tests **N-1 → dev** restore paths, where N-1 is the +most recent stable SeaTunnel release at the time the workflow runs. + +| Component | Version Policy | Rationale | +|-----------|---------------|-----------| +| Old release (savepoint source) | Latest stable release (e.g., 2.3.13) | Represents the most common user upgrade path | +| Current build (restore target) | `dev` branch HEAD | Catches regressions before they ship | +| Future versions | Add N-2 as optional manual trigger | Lower priority; N-1 covers majority of users | + +**Version sourcing**: The old release binary is downloaded from Apache mirrors +(`downloads.apache.org` / `archive.apache.org`). The workflow parameter `old_version` +defaults to the latest stable release and is updated by maintainers after each release. + +### 2. Scenario Selection Strategy + +Scenarios are selected based on **risk × coverage**: each scenario should exercise a +stateful surface that has historically produced regressions. + +#### Phase 2 Minimum Scenario Set + +| # | Scenario | State Surface Exercised | Risk Level | +|---|----------|------------------------|------------| +| 1 | `generic-fake-localfile` (existing) | Zeta checkpoint, LocalFile sink writer state | Medium | +| 2 | `mysql-cdc-multitable-localfile` (existing) | CDC source enumerator + reader state, multi-table restore | High | +| 3 | `kafka-source-localfile` | Source split enumerator state, Kafka offset restore | Medium | +| 4 | `jdbc-sink-postgres` | JDBC sink writer state, exactly-once semantics | Medium | +| 5 | `mysql-cdc-to-jdbc-sink` | Full CDC pipeline: source state + sink writer state | High | + +#### Scenario Directory Structure + +Each scenario lives under `tools/upgrade_compatibility/scenarios/<name>/` with: + +``` +<name>/ +├── seatunnel.yaml # Engine config template (uses __CHECKPOINT_DIR__) +├── job.conf # Streaming job template (uses __SINK_DIR__) +├── assert.conf # Batch assertion job template +├── plugin_config # Connector artifacts for old release +├── setup.sh # Optional: external service setup (e.g., Docker) +├── teardown.sh # Optional: external service teardown +└── endless # Optional: marker for streaming (cancel-after-savepoint) +``` + +### 3. High-Risk State Surfaces + +The following code areas are classified as **compatibility-sensitive**. Changes to these +areas should trigger the upgrade compatibility workflow: + +| Surface | Modules | Examples | +|---------|---------|----------| +| Checkpoint serialization | `seatunnel-api`, `seatunnel-engine` | `CheckpointState`, `StateSerializer` | +| Source enumerator state | `seatunnel-api`, connector source modules | `SourceSplitEnumerator` state | +| Sink writer state | `seatunnel-api`, connector sink modules | `SinkWriter` state, transaction state | +| CDC offset models | CDC connector modules | `LsnOffset`, `ScnOffset`, `BinlogOffset` | +| Job graph / config | `seatunnel-api`, `seatunnel-core` | `JobConfig`, `Action`, `SeaTunnelConfig` | +| Serialization framework | `seatunnel-api` | `Serializable` contract changes | + +### 4. Trigger / Gating Policy + +The workflow uses a **progressive gating** model: + +#### Phase 2a: Current State (already implemented) + +``` +schedule: daily at 18:00 UTC (dev branch) +workflow_dispatch: manual trigger with old_version + scenario inputs +``` + +#### Phase 2b: PR-Selective Trigger + +Add a `pull_request` trigger with **path filters** for compatibility-sensitive areas: + +```yaml +on: + pull_request: + paths: + - 'seatunnel-api/**' + - 'seatunnel-engine/**' + - 'seatunnel-core/**' + - 'seatunnel-connectors-v2/connector-cdc-**/**' Review Comment: This filter misses the connector code exercised by the scenario matrix. `seatunnel-connectors-v2/connector-cdc-**/**` matches no tracked file because CDC is under `seatunnel-connectors-v2/connector-cdc/**`. Kafka, JDBC, File, and Fake are also absent, so state changes in every current or planned connector scenario can skip the PR check. I verified zero matches for the documented CDC pattern versus 334 for the actual CDC path. Please list the real scenario module paths, at least `connector-cdc/**`, `connector-kafka/**`, `connector-jdbc/**`, `connector-file/**`, and `connector-fake/**`, and mirror the correction in the Chinese document. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
