rdblue commented on issue #1109:
URL: https://github.com/apache/iceberg/issues/1109#issuecomment-642912064
Yeah, unfortunately this can happen for a number of reasons.
I considered the option of modifying `lastUpdatedMillis` so that it is
always strictly increasing, but that could lead to some really bad behaviors.
For example, if a node is misconfigured and its timestamp is a year in the
future, we don't want to either make up a timestamp like `max(lastUpdated+1,
now)` because that perpetuates the problem. We also can't just wait until `now
> lastUpdated` because that might be a long time and would DOS the table. What
I concluded was that we might have timestamps that are off and that has to be
okay.
Given that, I think that the current check for `lastUpdatedMillis` in
`TableMetadata` should be considered the bug. Since we know this can happen, we
should allow for some reasonable clock skew in that check.
How about changing the check to allow some out-of-order log entries, but
nothing more than 1s?
```java
HistoryEntry last = null;
for (HistoryEntry logEntry : snapshotLog) {
if (last != null) {
Preconditions.checkArgument(
(logEntry.timestampMillis() - last.timestampMillis()) >= -1000,
"[BUG] Expected sorted snapshot log entries.");
}
last = logEntry;
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
