soumya-ghosh commented on issue #1053: URL: https://github.com/apache/iceberg-python/issues/1053#issuecomment-2832536611
@kevinjqliu I agree to your point that while time-travelling to older snapshots, metadata tables should adhere to schema as of that snapshot. > I did a test to see the behavior in Spark, observations in [gist](https://gist.github.com/soumya-ghosh/77bccf0fe77926da0b6e96432021879a). It appears that in Spark constructs the readable_metrics column by considering the current schema (which maybe a bug). In output files of attached gist, we can see that columns like lower_bound, upper_bound, value_counts are adhering to schema as they are obtained from the data files itself, whereas readable_metrics is not as it is deriving its structure from current schema. My opinion is we should merge this PR and initiate a discussion / PR to correct this behavior in main Iceberg. WDYT, @kevinjqliu ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
