taozhiyi13 commented on issue #12923: URL: https://github.com/apache/seatunnel/issues/12923#issuecomment-6573529094
@DanielLeens Thanks for the clarification. I agree that the two remaining points require explicit compatibility decisions. I will incorporate the following contracts into the STIP body. ## 1. MySQL JDBC `GEOMETRY` is an incompatible schema change Previously, when MySQL JDBC Source directly selected a generic `GEOMETRY` column, SeaTunnel exposed it as `BYTES`, with the JDBC binary value stored in a `byte[]`. After this change, the same column is exposed as `GEOMETRY(GEOMETRY)`, with a `Geometry` value containing the unsigned SRID and standard WKB. This is a user-visible, backward-incompatible schema change. Existing JDBC jobs that directly select a generic `GEOMETRY` column and expect `BYTES` will not remain schema-compatible. The change is intentional. The previous representation could transport opaque bytes, but it did not preserve a shared geometry kind or SRID contract for downstream components. Retaining an automatic `BYTES` fallback would also make the JDBC and CDC spatial contracts inconsistent. Therefore, the implementation will not silently downgrade `GEOMETRY` to `BYTES` or `STRING`. A JDBC job that still requires a binary schema must request it explicitly: ```sql SELECT ST_AsWKB(column_name) AS column_name FROM table_name ``` This produces a `BYTES` column containing standard WKB. It preserves the binary-schema migration path, but not necessarily the exact historical MySQL internal byte layout. The incompatible-change scope is limited to the generic MySQL JDBC `GEOMETRY` mapping. Concrete spatial types could previously fail during schema conversion, while MySQL-CDC had no supported end-to-end contract for non-null spatial values; those paths are new capabilities rather than compatible historical contracts. The implementation PR will include: 1. migration guidance in the English and Chinese MySQL JDBC Source documentation; 2. entries in both language versions of `incompatible-changes.md`; 3. a before/after job-schema test covering `BYTES -> GEOMETRY(GEOMETRY)`; 4. a migration test proving that `ST_AsWKB(column)` is still discovered and emitted as `BYTES`. Existing jobs must either adopt the new `GEOMETRY` contract or explicitly use `ST_AsWKB` to retain a binary schema. ## 2. Spatial jobs require a strict full-cluster upgrade The first version will use a strict full-cluster upgrade boundary rather than introducing a new Zeta capability-negotiation protocol. `GeometryType` is transported in the job action graph, and `Geometry` is transported as a `SeaTunnelRow` field through `RecordSerializer`. An old master or worker does not contain these classes and cannot deserialize the new schema or row value. The upgrade contract is therefore: - the client, master, and all workers must be upgraded to a compatible version containing `GeometryType` and `Geometry` before a spatial job is submitted or restored; - mixed-version execution of a spatial job is unsupported; - a spatial checkpoint or savepoint created by the new version can only be restored on a fully upgraded cluster; - downgrading a spatial job after it has produced new state is unsupported; - an unsupported component must not reinterpret `GEOMETRY` as `BYTES` or `STRING`. If this boundary is violated, failure may occur while deserializing the job action, deployed task, row field, or checkpoint schema. This STIP does not describe such a failure as graceful plan-time or pre-dispatch rejection. Non-spatial jobs do not contain `GeometryType` or `Geometry`. This proposal does not change their existing schema, row, or checkpoint representations. The implementation PR will verify: 1. a mixed-version boundary cannot deserialize the new spatial action or row and does not silently coerce it; 2. a fully upgraded cluster can checkpoint and restore a spatial CDC job; 3. an old version cannot restore checkpoint state containing `GeometryType`; 4. representative historical non-spatial payloads remain readable by the new version. A general Zeta capability-negotiation mechanism may be proposed separately if mixed-version execution is required in the future. ## Mailing-list discussion After updating the STIP body with these decisions, I will start a `[DISCUSS][STIP-42]` thread on `[email protected]` covering the shared API, JDBC migration contract, and Zeta upgrade boundary. I will link the mailing-list thread back to this issue, while keeping the STIP as the detailed source of truth. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
