corgy-w opened a new issue, #13217:
URL: https://github.com/apache/seatunnel/issues/13217

   ### Search before asking
   
   - [x] I searched the existing feature and design issues. No open issue/PR 
covers unifying default-value handling across the DDL pipeline. (#11790 STIP-36 
is observability-only and does not touch default-value semantics.)
   
   ### Description
   
   ## Summary
   
   Unify how column default values flow through the DDL pipeline:
   source parsing → SeaTunnel column model → sink DDL rendering, for both the
   auto create-table path and the CDC schema-evolution path.
   
   Today the three stages are implemented independently and behave 
inconsistently,
   so defaults are silently lost or rendered as wrong SQL depending on the
   source/sink dialect pair.
   
   ## Current behavior
   
   ### 1. Source → SeaTunnel model (reading)
   
   - JDBC MySQL catalog reads `COLUMN_DEFAULT` from `information_schema`
     (`MySqlCatalog.buildColumn`). On MySQL 8, expression defaults such as
     `DEFAULT CURRENT_TIMESTAMP` are `NULL` in this column (the expression lives
     in `EXTRA`), so they are lost.
   - MySQL CDC keeps special defaults as expression strings via the Debezium
     parser (`MySqlTypeUtils`), so the JDBC and CDC paths disagree on the same
     table.
   - PostgreSQL/Oracle CDC keep raw expression text (e.g. `now()`,
     `nextval(...)`, `sysdate`); `PostgresCatalog` deliberately drops defaults
     containing `regclass`.
   
   ### 2. SeaTunnel model
   
   - `Column.defaultValue` is a bare `Object` with no literal-vs-expression
     semantics and no source-dialect information, so downstream code cannot
     reliably re-render it.
   
   ### 3. SeaTunnel model → sink DDL (rendering)
   
   - Create-table path: most `XxxCreateTableSqlBuilder`s (MySQL, SQL Server,
     Oracle, OceanBase, ...) never emit `DEFAULT` at all — only DuckDB does.
     Defaults are silently dropped when auto-creating tables (`save_mode` /
     multi-table sync).
   - Schema-evolution path: `JdbcDialect.applySchemaChange` only renders
     `DEFAULT` when the source and sink dialect are identical (`sameCatalog`);
     cross-database default sync is skipped, with an existing TODO
     ("Support for cross-database default values for ddl statements") and a
     special case that skips TIMESTAMP.
   - Rendering logic is scattered across five hook methods
     (`supportDefaultValue`, `needsQuotesWithDefaultValue`,
     `isSpecialDefaultValue`, `quotesDefaultValue`, `sqlClauseWithDefaultValue`)
     with per-dialect overrides. The base `isSpecialDefaultValue` only
     understands MySQL special defaults, so expressions from other sources
     (e.g. PostgreSQL `now()`) are quoted like string literals and produce
     invalid DDL.
   
   ## Proposal
   
   Introduce one unified representation and one entry point per stage:
   
   1. **Unified model (seatunnel-api).** Add a `DefaultValue` abstraction
      carried by `Column` (additive to the existing `defaultValue` field):
      `kind` (`LITERAL` | `EXPRESSION`), the typed literal value for `LITERAL`,
      and a normalized expression text for `EXPRESSION`. Backward compatible;
      no existing config changes.
   
   2. **One reading function (source → model).** All JDBC catalogs and CDC
      sources populate `DefaultValue` consistently. For MySQL, also fix the
      MySQL 8 gap by reading `EXTRA` / the generation expression (or `SHOW
      CREATE TABLE`) so `CURRENT_TIMESTAMP` is no longer lost.
   
   3. **One conversion function (source dialect → target dialect).** A central
      `DefaultValueConverter` that maps
      `source dialect + DefaultValue` → `target dialect expression`.
      - Literals are always supported (serialized per target type: quoting,
        escaping, date/time formats).
      - Expressions use a per-dialect mapping table (e.g. MySQL
        `CURRENT_TIMESTAMP` → PostgreSQL `CURRENT_TIMESTAMP`, Oracle `SYSDATE`
        → MySQL `CURRENT_TIMESTAMP`).
      - Unmappable expressions fall back to skip + WARN instead of emitting
        invalid DDL.
   
   4. **One rendering function (model → target DDL).** A single render entry
      used by BOTH the create-table builders and the schema-evolution ALTER
      statements. Each dialect implements only "canonical expression → native
      SQL", replacing the five scattered hook methods.
   
   ## Proposed milestones
   
   1. Add the `DefaultValue` model in seatunnel-api (additive, backward
      compatible).
   2. Make create-table builders emit `DEFAULT` for literals (fixes silent loss
      in the common case).
   3. Remove the `sameCatalog` restriction in schema evolution and route
      through the converter.
   4. Add per-dialect expression mapping tables and normalization
      (MySQL/PostgreSQL/Oracle first; other dialects fall back to
      literal-only).
   5. E2E coverage: MySQL → PostgreSQL and MySQL → MySQL for both create-table
      and CDC schema-change paths; update docs (en/zh).
   
   ## Affected modules
   
   - seatunnel-api (`Column` / catalog model)
   - seatunnel-connectors-v2 (`connector-jdbc`, `connector-cdc-*`)
   
   ## Related issues
   
   - #11790 (STIP-36, schema evolution observability — different scope)
   
   ## Are you willing to submit a PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ## Code of Conduct
   
   - [x] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to