[
https://issues.apache.org/jira/browse/HIVE-25453?focusedWorklogId=645256&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-645256
]
ASF GitHub Bot logged work on HIVE-25453:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Sep/21 13:03
Start Date: 01/Sep/21 13:03
Worklog Time Spent: 10m
Work Description: szlta commented on a change in pull request #2586:
URL: https://github.com/apache/hive/pull/2586#discussion_r700191437
##########
File path:
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java
##########
@@ -158,8 +167,11 @@ private LlapRecordReader(MapWork mapWork, JobConf job,
FileSplit split,
rbCtx = ctx != null ? ctx : LlapInputFormat.createFakeVrbCtx(mapWork);
isAcidScan = AcidUtils.isFullAcidScan(jobConf);
- TypeDescription schema = OrcInputFormat.getDesiredRowTypeDescr(
- job, isAcidScan, Integer.MAX_VALUE);
+
+ String icebergOrcSchema =
job.get(ColumnProjectionUtils.ICEBERG_ORC_SCHEMA_STRING);
Review comment:
Yeah it's unfortunate. This is the equivalent of non-LLAP but vectorized
ORC case at
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java#L80
Hive sets IOConstants.SCHEMA_EVOLUTION_COLUMNS and
SCHEMA_EVOLUTION_COLUMNS_TYPES during compile and pushes this down to
execution. These are simple string representations of Hive types so not ORC
specific, and relates to LOGICAL schema.
Iceberg - in order to support a broader schema evolution - produces the FILE
schema based on file info and logical type info, and the result is an ORC
TypeDescription instance. (See VectorizedReadUtils.handleIcebergProjection) I
have found no easy way to transform this object back into Hive types, the
conversion only exists in the other direction:
OrcInputFormat.typeDescriptionsFromHiveTypeProperty()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 645256)
Time Spent: 2.5h (was: 2h 20m)
> Add LLAP IO support for Iceberg ORC tables
> ------------------------------------------
>
> Key: HIVE-25453
> URL: https://issues.apache.org/jira/browse/HIVE-25453
> Project: Hive
> Issue Type: New Feature
> Reporter: Ádám Szita
> Assignee: Ádám Szita
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
