SEZ9 commented on issue #13091: URL: https://github.com/apache/seatunnel/issues/13091#issuecomment-6593603781
@SEPURI-SAI-KRISHNA thanks for the detailed write-up, and for linking #13092. Direction (b) is the right call for this issue. Your reasoning matches how I see it: (a) alone would reshuffle splits for every table with a nullable unique key even when no NULL is present and still leaves a nullable `partition_column` uncovered, and (c) only surfaces the loss without fixing it. Adding a single `col IS NULL` split only when the probe actually finds NULL rows, so that `NOT NULL` columns and NULL-free columns keep exactly today's splits, is the behaviour I was hoping for, and it brings the JDBC source in line with what was already done for the CDC sources. I will pick up the review of #13092. A few things that would help me get through it quickly, so please confirm in the PR (or here) if they are not already covered: 1. The NULL probe runs once per table/split-column, not per chunk, so there is no extra round trip per split on large tables. 2. The `IS NULL` split is generated in the shared `ChunkSplitter` path and the generated SQL is dialect-neutral, so the fix applies to all JDBC dialects and not only MySQL even though the IT (`JdbcMysqlSplitIT#testNullSplitKeyRowsAreReadOnce`) runs on MySQL. 3. The test asserts both that the NULL rows are read and that no row is read twice (the "exactly once" part), since that is the regression this issue is about. Nothing else is blocking from my side; once those are confirmed I will leave my review on the PR. <!-- streview-comment:2229 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
