VISHAL B created FLINK-41300:
--------------------------------
Summary: [cdc][mysql] MySQL pipeline connector fails the job when
a GEOMETRY/POLYGON value cannot be exported to GeoJSON
Key: FLINK-41300
URL: https://issues.apache.org/jira/browse/FLINK-41300
Project: Flink
Issue Type: Bug
Components: Flink CDC
Environment: Flink 2.2.1, flink-cdc-pipeline-connector-mysql 3.6.0,
MySQL (AWS RDS), Kubernetes operator.
Reporter: VISHAL B
With the MySQL pipeline connector 3.6.0 (Flink 2.2.1), a table with a POLYGON
column holding a self-intersecting polygon makes the source fail while reading
it, in both snapshot and binlog reads.
The same code is in release-3.7 and master.
Stack:
IllegalArgumentException: Failed to convert Struct\{wkb=[B@23763a6} to geometry
JSON.
at MySqlEventDeserializer.convertToString(MySqlEventDeserializer.java:199)
Caused by: com.esri.core.geometry.GeometryException: corrupted geometry
at
OperatorExportToGeoJsonCursor.exportPolygonToGeoJson_(OperatorExportToGeoJsonCursor.java:192)
at OGCGeometry.asGeoJson(OGCGeometry.java:128)
at MySqlEventDeserializer.convertToString(MySqlEventDeserializer.java:183)
Cause: OGCGeometry.fromBinary(...) accepts the WKB, but asGeoJson() throws for
a non-simple polygon.
convertToString catches it and rethrows, so the job fails and restarts on the
same record forever.
Reproduce: create a table with a POLYGON column, insert a self-intersecting
polygon (for example a zero-area triangle or a bow-tie ring; ST_IsValid = 0),
then start a snapshot or binlog read.
Expected: a way to keep the job running, for example a source option to emit
NULL (with a WARN log naming the table and column) for geometry values that
cannot be converted. The default can stay "fail".
Also: the exception message contains only the byte-array reference, with no
table, column or primary key, so the bad row cannot be found from the logs.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
