[
https://issues.apache.org/jira/browse/HIVE-26431?focusedWorklogId=795990&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-795990
]
ASF GitHub Bot logged work on HIVE-26431:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Jul/22 08:37
Start Date: 28/Jul/22 08:37
Worklog Time Spent: 10m
Work Description: szlta commented on code in PR #3479:
URL: https://github.com/apache/hive/pull/3479#discussion_r931934208
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java:
##########
@@ -105,8 +106,18 @@ public void initialize(@Nullable Configuration
configuration, Properties serDePr
} else {
try {
Table table = IcebergTableUtil.getTable(configuration,
serDeProperties);
- // always prefer the original table schema if there is one
- this.tableSchema = table.schema();
+ if (serDeProperties.containsKey(serdeConstants.AS_OF_VERSION)) {
+ this.tableSchema = SnapshotUtil.schemaFor(table,
+
Long.valueOf(serDeProperties.getProperty(serdeConstants.AS_OF_VERSION)));
+ serDeProperties.remove(serdeConstants.AS_OF_VERSION);
Review Comment:
Let's explain in a comment why we need to remove this from serdeProperties,
and perhaps also why we need serdeProperties to carry this information nstead
of job configuration in the first place.
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java:
##########
@@ -347,7 +351,15 @@ final public Deserializer getDeserializer(boolean
skipConfError) {
final public Deserializer getDeserializerFromMetaStore(boolean
skipConfError) {
try {
- return HiveMetaStoreUtils.getDeserializer(SessionState.getSessionConf(),
tTable, metaTable, skipConfError);
+ String asOfTimestamp = null;
+ if (getAsOfTimestamp() != null) {
+ ZoneId zoneId = SessionState.get() == null ? new
HiveConf().getLocalTimeZone() :
+ SessionState.get().getConf().getLocalTimeZone();
+ TimestampTZ time =
TimestampTZUtil.parse(PlanUtils.stripQuotes(getAsOfTimestamp()), zoneId);
+ asOfTimestamp = Long.toString(time.toEpochMilli());
Review Comment:
This calculation of Iceberg As Of information from string to long is found
in multiple places, I think we should refactor this logic into a utility class
of ql perhaps.
Issue Time Tracking
-------------------
Worklog Id: (was: 795990)
Time Spent: 20m (was: 10m)
> Use correct schema for iceberg time travel queries
> --------------------------------------------------
>
> Key: HIVE-26431
> URL: https://issues.apache.org/jira/browse/HIVE-26431
> Project: Hive
> Issue Type: Improvement
> Reporter: László Pintér
> Assignee: László Pintér
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For time travel queries we currently always use the latest schema in
> IcebergSerDe, however since schemas are now versioned, we could use the
> schema which was active at the time in the past.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
