ZZZxDong commented on PR #8837: URL: https://github.com/apache/paimon/pull/8837#issuecomment-5090773683
@JingsongLi Thanks for the review! Summarizing the current state since the comments above are scattered: > For fast paths, only fill in the primary key (PK); set all non-PK fields to NULL; however, the validation also allows partial updates and aggregates. / Non-null, non-PK fields will cause an error... / `partial-update.remove-record-on-sequence-group` also requires values... / the literal extraction does not exclude NULL values... All four points are addressed, plus two more cases that CI surfaced along the way: - The fast path is now gated to **DEDUPLICATE** tables with **no NOT NULL non-pk columns** and **no pk index definitions** (index entries are built from real field values, so NULL-filled -D rows would leave deletions invisible to index lookups). NULL literals in the condition also fall back to preserve SQL three-valued logic. - CI further revealed that incremental / audit_log reads expect the old field values on -D rows, which the fast path cannot provide by design. Since this affects the changelog contract for any deduplicate table, the fast path is now **opt-in**: off by default, enabled via `spark.paimon.delete.point-delete.enabled`, with the trade-off documented on the option. **Default behavior is completely unchanged.** The remaining Flink 1.x Others failure is unrelated to this Spark-only change (that suite also fails intermittently on master). PTAL when you have time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
