junmuz commented on PR #9822:
URL: https://github.com/apache/paimon/pull/9822#issuecomment-5860400336
@JingsongLi Thanks for the review.
1. I see your point, and I am of the opinion if someone uses it inside for
filtering and aggregation then it is an incorrect configuration. I have also
updated the documentation to not use the metadata field with aggregation or
filtering. Also, it is quite difficult unless someone chooses to use it that
way, because now they have to setup additional table property as well as
exposing the metadata column. We have similar behavior with sequence fields as
well if someone configures ts as sequence field, and uses `SELECT MAX(ts) as ts
from source_table; then retraction doesn't work.
2. I’ve made changelog-producer.metadata-field-prefix immutable after the
table has snapshots. The stored changelog field uses the prefix with the source
field ID, so its physical identity stays stable across source field renames.
Spark and Flink can use the same logical metadata name through their respective
APIs (Spark as a column and Flink as a metadata key) without engine-specific
names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
