SEZ9 commented on PR #13092:
URL: https://github.com/apache/seatunnel/pull/13092#issuecomment-6870060209
@SEPURI-SAI-KRISHNA Thanks for the rework in `65703fe498`. Following the CDC
split-column selection instead of adding a NULL split sounds like a simpler
shape, and if the probe query, the `IS NOT NULL` predicates,
`isNullableSplitKey` and the `JdbcSourceSplit` change are all gone, the earlier
points on those should be covered. I'll confirm that against the diff on the
next pass.
Two things I'd like to clarify before approving:
1. **How nullability is determined for the auto-selected key.** When the
split key is chosen automatically, how does the connector decide that a primary
key or unique key column is nullable? In particular, what happens for
`query`-based reads, where a column declared NOT NULL can still produce NULL
rows (e.g. after an outer join)? If such a column is picked as the split key,
those rows would be skipped silently, which is the same risk as before in a
different place. Falling back to the documented explicit-`partition_column`
behaviour ("rows with NULL are not read") or stating in the docs that the
auto-selection check only covers table reads would both work; I just want the
behaviour to be deliberate and written down.
2. **`incompatible-changes.md` wording.** With the new design the
user-visible change is: a job that previously auto-split on a nullable primary
key or unique key column now either uses a different column or reads the table
as a single split (with a WARN), while a job with an explicit
`partition_column` keeps its old behaviour. Please make sure the entry says
exactly that, and that the EN/ZH split-key sections no longer reference the
removed NULL split or its interaction with the partition bounds.
The `JdbcMysqlSplitIT` case you describe sounds like a good regression guard
for point 1; I'll look at it together with the rest of the diff. Thanks again
for the quick turnaround.
<!-- streview-comment:2326 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]