singhpk234 commented on code in PR #5160:
URL: https://github.com/apache/iceberg/pull/5160#discussion_r911616713
##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/IcebergSource.java:
##########
@@ -173,7 +174,19 @@ public Optional<String>
extractTimeTravelVersion(CaseInsensitiveStringMap option
@Override
public Optional<String> extractTimeTravelTimestamp(CaseInsensitiveStringMap
options) {
- return Optional.ofNullable(PropertyUtil.propertyAsString(options,
"timestampAsOf", null));
+ String timestampAsOf = PropertyUtil.propertyAsString(options,
"timestampAsOf", null);
+ if (timestampAsOf == null) {
+ return Optional.empty();
+ }
+
+ try {
+ // timestamp provided should be at a seconds precision.
+ // TODO: remove once https://issues.apache.org/jira/browse/SPARK-39633
is resolved
+ long timestampAsOfAsLong = Long.parseLong(timestampAsOf);
+ return
Optional.of(DateTimeUtil.formatTimestampMillisWithLocalTime(timestampAsOfAsLong
* 1000));
Review Comment:
Makes sense, calling spark api's via iceberg will also not be ok i think
then as this now indirectly puts the translation to iceberg (as which spark api
to use and to provide zoneids etc). Should I then add a TODO in the code with
the issue linking it and add the UT added in this PR in ignore as soon as we
upgrade to 3.3.1 we will have it (The PR for
[SPARK-39633](https://issues.apache.org/jira/browse/SPARK-39633) is merged in
upstream).
Meanwhile we have existing ways to TT via dataframe options we can specify
[as-of-timestamp](https://iceberg.apache.org/docs/latest/spark-queries/#time-travel)
in milliseconds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
