esnhysythh commented on issue #4702: URL: https://github.com/apache/paimon/issues/4702#issuecomment-4955571652
> But as I tried, it does need only the primary key when deleting a record, eg. > > delete from some_table where id = 123; Yes, this works in `paimon-flink-connector`, but the reason is that it first performs a query to fetch the existing row values, and then writes a row with the same column values but with `RowKind.DELETE`. The relevant implementation is here: `paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/DeleteAction.java` So all columns are filled with their previous values from the Paimon table. For `NOT NULL` columns, they will naturally not be null, so the validation issue does not occur in this path. However, I do not think this necessarily means the current behavior is reasonable. The extra query cost is not negligible, especially when Paimon is used as a lake table and the data may be stored on remote OSS/S3/HDFS. In my understanding, the main impact of this issue is on changelog generation: the generated delete changelog may no longer contain the previous values. This does not affect the final table semantics, but it does lose some information in the changelog. For tables using the `deduplicate` merge engine, I think this validation is unnecessary. So I believe this is indeed a real issue. --- 是的，在 `paimon-flink-connector` 中，这种写法是可以工作的。但原因是它会先执行一次查询，查出待删除行的已有列值，然后再写入一行列值相同、但 `RowKind` 为 `DELETE` 的数据。 相关实现可以参考： `paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/DeleteAction.java` 也就是说，所有列都会被填充为 Paimon 表中已有的值。对于 `NOT NULL` 列来说，它们自然不会为空，因此在这个路径下不会触发校验问题。 但是，我认为这并不说明当前行为一定是合理的。额外的一次查询成本并不可忽略，尤其是当 Paimon 作为数据湖表使用，并且数据存储在远端 OSS/S3/HDFS 上时。 我的理解是，这个问题主要影响 changelog 生成：生成的 delete changelog 可能不再包含删除前的旧值。这不会影响最终表语义，但确实会让 changelog 丢失部分信息。 对于使用 `deduplicate` merge engine 的表，我认为这个校验是不必要的。 因此，我认为这里确实存在一个实际问题。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
