fatmanverse opened a new issue, #12821: URL: https://github.com/apache/seatunnel/issues/12821
### Search before asking - [x] I had searched in the issues and found no similar issues. Follow-up of #12799 / #12803 (Jdbc sink), raised during review of #12803. ### What happened The ClickHouse sink buffers changelog rows in `ReduceBufferedBatchStatementExecutor` the same way the Jdbc sink did before #12803: ```java // connector-clickhouse/.../sink/client/executor/ReduceBufferedBatchStatementExecutor.java @NonNull private final LinkedHashMap<SeaTunnelRow, Pair<Boolean, SeaTunnelRow>> buffer = new LinkedHashMap<>(); ... SeaTunnelRow key = keyExtractor.apply(record); buffer.put(key, Pair.of(changeFlag, value)); ... deleteExecutor.addToBatch(entry.getKey()); ``` The map key is the extracted `primary_key` row. `SeaTunnelRow#equals` compares the fields with `Arrays.deepEquals`, so all rows whose key contains NULL are equal and collapse into one buffer entry. Only the last of them is written, and the others are dropped before any SQL runs. This executor is used when `primary_key` is configured and the table engine is not in the Log family (`JdbcBatchStatementExecutorBuilder#build`). A key column can hold NULL when it is `Nullable(...)` in ClickHouse (MergeTree with `allow_nullable_key = 1`), or when the upstream column is nullable and the value is NULL. ### How to reproduce I have not run this against a ClickHouse server. The analysis is from code reading, and the code path is the same as the Jdbc one verified in #12799. 1. Create a table with a nullable key, for example `ENGINE = MergeTree ORDER BY code SETTINGS allow_nullable_key = 1` with `code Nullable(Int32)`. 2. Write several rows with `code = NULL` through the ClickHouse sink with `primary_key = "code"` in one batch. 3. Expected: all rows are written. Actual (by the code path): only the last NULL-key row of each batch is written. ### Expected behavior Rows whose key contains NULL are not merged with each other in the sink buffer. The same rule as #12803 would apply: a row with a NULL key field is buffered under its own identity, and rows with a complete key keep the last-write-wins reduction. ### SeaTunnel Version dev (3.0.1-SNAPSHOT) ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
