[
https://issues.apache.org/jira/browse/HIVE-26397?focusedWorklogId=792658&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-792658
]
ASF GitHub Bot logged work on HIVE-26397:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Jul/22 12:49
Start Date: 19/Jul/22 12:49
Worklog Time Spent: 10m
Work Description: pvary commented on code in PR #3445:
URL: https://github.com/apache/hive/pull/3445#discussion_r924461104
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -334,24 +338,32 @@ public boolean supportsPartitionTransform() {
public List<PartitionTransformSpec>
getPartitionTransformSpec(org.apache.hadoop.hive.ql.metadata.Table hmsTable) {
TableDesc tableDesc = Utilities.getTableDesc(hmsTable);
Table table = IcebergTableUtil.getTable(conf, tableDesc.getProperties());
- return table.spec().fields().stream().map(f -> {
- PartitionTransformSpec spec = new PartitionTransformSpec();
- spec.setColumnName(table.schema().findColumnName(f.sourceId()));
- // right now the only way to fetch the transform type and its params is
through the toString() call
- String transformName = f.transform().toString().toUpperCase();
- // if the transform name contains '[' it means it has some config params
- if (transformName.contains("[")) {
- spec.setTransformType(PartitionTransformSpec.TransformType
- .valueOf(transformName.substring(0, transformName.indexOf("["))));
- spec.setTransformParam(Optional.of(Integer
- .valueOf(transformName.substring(transformName.indexOf("[") + 1,
transformName.indexOf("]")))));
- } else {
-
spec.setTransformType(PartitionTransformSpec.TransformType.valueOf(transformName));
- spec.setTransformParam(Optional.empty());
- }
+ return table.spec().fields().stream().map(f ->
+ getTransformSpec(table, f.transform().toString().toUpperCase(),
f.sourceId())
+ ).collect(Collectors.toList());
+ }
+
+ private List<PartitionTransformSpec> getSortTransformSpec(Table table) {
Review Comment:
Should we rename the `PartitionTransformSpec` to something more appropriate?
Are we able to handle every sort mode defined by Iceberg?
Issue Time Tracking
-------------------
Worklog Id: (was: 792658)
Time Spent: 20m (was: 10m)
> Honour Iceberg sort orders when writing a table
> -----------------------------------------------
>
> Key: HIVE-26397
> URL: https://issues.apache.org/jira/browse/HIVE-26397
> Project: Hive
> Issue Type: Improvement
> Reporter: László Pintér
> Assignee: László Pintér
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Iceberg specification defines sort orders. We should consider this when
> writing to an Iceberg table through Hive.
> See: https://iceberg.apache.org/spec/#sort-orders
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
