leaves12138 commented on PR #9283: URL: https://github.com/apache/paimon/pull/9283#issuecomment-5337003666
@JingsongLi Thanks for raising both points. 1. I checked how the other lake formats handle build provenance. Iceberg stores `IcebergBuild.fullVersion()` in snapshot summary as `iceberg-version`; Hudi and Delta similarly keep writer/engine version information in commit metadata rather than encoding it in the commit user. I updated this PR to follow that model: `commitUser` is unchanged, and provenance is stored separately as nullable `Snapshot.fullVersion`, formatted as `<project-version>-<git-sha>` and exposed through `$snapshots.full_version`. 2. I agree that the mechanism must be consistent across languages. Java, Flink, and Spark all create snapshots through `FileStoreCommitImpl`, so they share the same implementation. PyPaimon has an independent Python `Snapshot` / `FileStoreCommit` path, so the JVM-generated build resource does not cover it automatically. The snapshot field itself is language-neutral and old snapshots remain readable, but non-JVM writers such as PyPaimon still need to populate the same field from their own build metadata. This is the remaining cross-language gap; it should be addressed before treating the feature as fully available across languages. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
