danny0405 commented on PR #7627: URL: https://github.com/apache/hudi/pull/7627#issuecomment-1530862011
> we should also make the stateTransition time as the default for "timestamp as of" or time travel query I'm not sure we should do that, because it is a semantics change, by using the timestamp, we are actually indicating the instant version number(yeah the timestamp is another format of version id), by using stateTransition time, we are using the completion time, that's a huge different conceptially, we should be conservative about that. The tricky part left that needs to be addressed is like you said, the handing of archivied instants, we do not support effieicnt filtering by state transition time on the archived timeline, and we do not support a state transition time versioning on the files layout(`_hoodie_commit_time` is the txn start time, whichi is a inconsistency). We need to think through about this part. So in Flink side, I just drop the support for switching the start/end offset by using state transition time, instead, the state transition time is used as a internal attribute to help find those instants written by multiplr writers that we previously missed to read. But still, we need to solve the archiving issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
