taozhiyi13 opened a new issue, #12832: URL: https://github.com/apache/seatunnel/issues/12832
### Search before asking - [x] I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. ### Description SeaTunnel currently lacks end-to-end support for synchronizing MySQL spatial columns to OpenSearch. We would like to support the following data path: ```text MySQL spatial columns → MySQL-CDC / Debezium → SeaTunnel → Elasticsearch/OpenSearch sink ``` The feature should support MySQL spatial types including: - `GEOMETRY` - `POINT` - `LINESTRING` - `POLYGON` - `MULTIPOINT` - `MULTILINESTRING` - `MULTIPOLYGON` - `GEOMETRYCOLLECTION` MySQL-CDC should preserve the spatial value, spatial type, and SRID during both the initial snapshot and incremental binlog phases. SeaTunnel should retain these spatial semantics through the pipeline instead of treating the value as ordinary binary data. ### OpenSearch type mapping OpenSearch represents geographic data using `geo_point` and `geo_shape`. The expected mapping is: - MySQL `POINT` → OpenSearch `geo_point` - Other MySQL spatial types → OpenSearch `geo_shape` ### SRID handling MySQL spatial values can use different valid SRIDs. MySQL-CDC and its underlying Debezium data path should preserve any MySQL-valid SRID without restricting it based on the target system. The source and sink responsibilities should be clearly separated: - MySQL-CDC and Debezium should preserve the original geometry value and any valid source SRID. - SeaTunnel should carry the geometry and SRID through the pipeline without silently transforming or discarding the spatial reference information. - The SRID restriction applies only when writing to OpenSearch. - OpenSearch `geo_point` and `geo_shape` do not store an SRID and interpret geographic coordinates as WGS84 longitude and latitude. - Spatial values with SRID `4326` can therefore be written directly to OpenSearch. - Spatial values with another SRID must not be silently interpreted as WGS84. They should either be explicitly transformed to WGS84 before reaching the sink or be rejected by the OpenSearch sink with a clear error. The internal representation, validation stage, and possible coordinate transformation mechanism can be discussed in the implementation PR. ### Usage Scenario Applications commonly store locations, routes, areas, and geofences in MySQL spatial columns. Users need MySQL-CDC to synchronize both existing data and subsequent binlog changes to OpenSearch for location-based search, proximity search, and spatial relationship queries. ### Related issues There was a previously closed issue, #8548, which attempted to convert MySQL geometry values emitted by Debezium into the existing SeaTunnel `BYTES` type. It did not introduce spatial types in SeaTunnel or support synchronizing spatial data to OpenSearch `geo_point` and `geo_shape` fields. Therefore, it does not cover the feature requested in this issue. ### Are you willing to submit a PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
