corgy-w opened a new issue, #13217:
URL: https://github.com/apache/seatunnel/issues/13217
### Search before asking
- [x] I searched the existing feature and design issues. No open issue/PR
covers unifying default-value handling across the DDL pipeline. (#11790 STIP-36
is observability-only and does not touch default-value semantics.)
### Description
## Summary
Unify how column default values flow through the DDL pipeline:
source parsing → SeaTunnel column model → sink DDL rendering, for both the
auto create-table path and the CDC schema-evolution path.
Today the three stages are implemented independently and behave
inconsistently,
so defaults are silently lost or rendered as wrong SQL depending on the
source/sink dialect pair.
## Current behavior
### 1. Source → SeaTunnel model (reading)
- JDBC MySQL catalog reads `COLUMN_DEFAULT` from `information_schema`
(`MySqlCatalog.buildColumn`). On MySQL 8, expression defaults such as
`DEFAULT CURRENT_TIMESTAMP` are `NULL` in this column (the expression lives
in `EXTRA`), so they are lost.
- MySQL CDC keeps special defaults as expression strings via the Debezium
parser (`MySqlTypeUtils`), so the JDBC and CDC paths disagree on the same
table.
- PostgreSQL/Oracle CDC keep raw expression text (e.g. `now()`,
`nextval(...)`, `sysdate`); `PostgresCatalog` deliberately drops defaults
containing `regclass`.
### 2. SeaTunnel model
- `Column.defaultValue` is a bare `Object` with no literal-vs-expression
semantics and no source-dialect information, so downstream code cannot
reliably re-render it.
### 3. SeaTunnel model → sink DDL (rendering)
- Create-table path: most `XxxCreateTableSqlBuilder`s (MySQL, SQL Server,
Oracle, OceanBase, ...) never emit `DEFAULT` at all — only DuckDB does.
Defaults are silently dropped when auto-creating tables (`save_mode` /
multi-table sync).
- Schema-evolution path: `JdbcDialect.applySchemaChange` only renders
`DEFAULT` when the source and sink dialect are identical (`sameCatalog`);
cross-database default sync is skipped, with an existing TODO
("Support for cross-database default values for ddl statements") and a
special case that skips TIMESTAMP.
- Rendering logic is scattered across five hook methods
(`supportDefaultValue`, `needsQuotesWithDefaultValue`,
`isSpecialDefaultValue`, `quotesDefaultValue`, `sqlClauseWithDefaultValue`)
with per-dialect overrides. The base `isSpecialDefaultValue` only
understands MySQL special defaults, so expressions from other sources
(e.g. PostgreSQL `now()`) are quoted like string literals and produce
invalid DDL.
## Proposal
Introduce one unified representation and one entry point per stage:
1. **Unified model (seatunnel-api).** Add a `DefaultValue` abstraction
carried by `Column` (additive to the existing `defaultValue` field):
`kind` (`LITERAL` | `EXPRESSION`), the typed literal value for `LITERAL`,
and a normalized expression text for `EXPRESSION`. Backward compatible;
no existing config changes.
2. **One reading function (source → model).** All JDBC catalogs and CDC
sources populate `DefaultValue` consistently. For MySQL, also fix the
MySQL 8 gap by reading `EXTRA` / the generation expression (or `SHOW
CREATE TABLE`) so `CURRENT_TIMESTAMP` is no longer lost.
3. **One conversion function (source dialect → target dialect).** A central
`DefaultValueConverter` that maps
`source dialect + DefaultValue` → `target dialect expression`.
- Literals are always supported (serialized per target type: quoting,
escaping, date/time formats).
- Expressions use a per-dialect mapping table (e.g. MySQL
`CURRENT_TIMESTAMP` → PostgreSQL `CURRENT_TIMESTAMP`, Oracle `SYSDATE`
→ MySQL `CURRENT_TIMESTAMP`).
- Unmappable expressions fall back to skip + WARN instead of emitting
invalid DDL.
4. **One rendering function (model → target DDL).** A single render entry
used by BOTH the create-table builders and the schema-evolution ALTER
statements. Each dialect implements only "canonical expression → native
SQL", replacing the five scattered hook methods.
## Proposed milestones
1. Add the `DefaultValue` model in seatunnel-api (additive, backward
compatible).
2. Make create-table builders emit `DEFAULT` for literals (fixes silent loss
in the common case).
3. Remove the `sameCatalog` restriction in schema evolution and route
through the converter.
4. Add per-dialect expression mapping tables and normalization
(MySQL/PostgreSQL/Oracle first; other dialects fall back to
literal-only).
5. E2E coverage: MySQL → PostgreSQL and MySQL → MySQL for both create-table
and CDC schema-change paths; update docs (en/zh).
## Affected modules
- seatunnel-api (`Column` / catalog model)
- seatunnel-connectors-v2 (`connector-jdbc`, `connector-cdc-*`)
## Related issues
- #11790 (STIP-36, schema evolution observability — different scope)
## Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
## Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]