SEZ9 commented on issue #13091: URL: https://github.com/apache/seatunnel/issues/13091#issuecomment-6568037024
Thanks for the detailed report and the reproducible setup. As you describe, none of the split conditions (range, or `ABS(CRC32(col) % n) = ?` for the hash split) match NULL, and no split reads `col IS NULL`, so rows with a NULL split column are dropped silently. Since #12597 only covered `connector-cdc-base`, the JDBC source splitter in `connector-jdbc` needs its own fix. A possible direction: (a) have `ChunkSplitter#findSplitKey` skip nullable unique-index columns and fall back to a non-nullable column or a single split; (b) when the chosen or user-configured `partition_column` is nullable, add one extra split reading `col IS NULL` so those rows are read exactly once; (c) at minimum, log a warning when splitting on a nullable column. A test in `JdbcMysqlSplitIT` with a NULL-containing unique key would guard against regressions. Since you're willing to submit a PR, please go ahead — feel free to ping me for review. <!-- streview-comment:2214 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
