SEPURI-SAI-KRISHNA commented on issue #13091: URL: https://github.com/apache/seatunnel/issues/13091#issuecomment-6568512558
Thanks @SEZ9. The PR is #13092 and it takes direction (b): when the split column is nullable and a probe finds a NULL in the rows to be read, the splitter adds one split that reads `col IS NULL`, so those rows are read exactly once. It covers the automatically chosen unique key and a user `partition_column` alike, and `JdbcMysqlSplitIT#testNullSplitKeyRowsAreReadOnce` guards it on MySQL. Why not (a) or (c) on their own: - (a) would change the splits of every table whose unique key is nullable, even when it holds no NULL, and would not help a nullable `partition_column`, so (b) is needed anyway. - (c) would make the loss visible but still drop the rows. With (b), a column declared `NOT NULL` or without NULL values keeps exactly the splits it has today. @DanielLeens has reviewed it, and the latest commit addresses his two points. It is ready for your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
