dramaticlly opened a new pull request, #14380: URL: https://github.com/apache/iceberg/pull/14380
Today the lastUpdatedMillis in TableMetadata is used to sequence the metadata log entry on build https://github.com/apache/iceberg/blob/f65558ec30a27fb26ed85ba0e3dc4e498f2c046b/core/src/main/java/org/apache/iceberg/TableMetadata.java#L392 This PR allow customization of lastUpdateTimestampMilli when building the TableMetadata, as currently the table metadata last updated timestamp is derived from different sources for data change and metadata change For data change when there's snapshot involved: - addSnapshot: https://github.com/apache/iceberg/blob/f65558ec30a27fb26ed85ba0e3dc4e498f2c046b/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1258 - set(snapshot)Ref: https://github.com/apache/iceberg/blob/f65558ec30a27fb26ed85ba0e3dc4e498f2c046b/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1317 For metadata change such as schema/partition spec/sort order/properties update, without produce the snapshot: - if timestamp is unset, it will use the system clock on build https://github.com/apache/iceberg/blob/f65558ec30a27fb26ed85ba0e3dc4e498f2c046b/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1543 - or special case here when update metadataLocation in https://github.com/apache/iceberg/blob/f65558ec30a27fb26ed85ba0e3dc4e498f2c046b/core/src/main/java/org/apache/iceberg/TableMetadata.java#L1017 (more context in #11151 ) This can be potentially problematic for REST based catalog, where snapshot is produced by the process engine and table metadata is updated on the catalog, where 2 system might have different clock. Today, there's ability to tolerate the clock screw within one minute, but this might not be enough for more complex scenario when add back-dated snapshot. Provides a dedicated builder method will give more flexibility to the needs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
