chandrasekhar-188k commented on code in PR #17286:
URL: https://github.com/apache/iceberg/pull/17286#discussion_r3609686912
##########
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##########
@@ -66,6 +66,16 @@ public class TableMetadata implements Serializable {
private static final long ONE_MINUTE = TimeUnit.MINUTES.toMillis(1);
+ /**
+ * System property that controls the allowed clock-skew tolerance (in
milliseconds) used when
+ * validating that snapshot-log and metadata-log entries are monotonically
increasing in time.
+ * Defaults to 1 minute if unset or invalid.
+ */
+ static final String CLOCK_SKEW_TOLERANCE_MS_PROPERTY =
+ "iceberg.table-metadata.clock-skew-tolerance-ms";
+
+ private static final long DEFAULT_CLOCK_SKEW_TOLERANCE_MS = ONE_MINUTE;
Review Comment:
in deployments where multiple compute engines (for example, Spark, Flink,
Trino, or custom applications) operate on the same Iceberg table from different
clusters, temporary clock skew greater than one minute can occur despite
otherwise healthy deployments. In such cases, valid commits may fail because
the metadata timestamp appears to be too far in the future relative to the
committing process. we have observed this issue in some of our deployments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
