kokila-19 commented on code in PR #6603:
URL: https://github.com/apache/hive/pull/6603#discussion_r3594686105
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##########
@@ -580,11 +580,9 @@ private static Map<String, String>
getPartishSummary(Partish partish, Table tabl
PartitionStatisticsFile statsFile =
IcebergTableUtil.getPartitionStatsFile(table, snapshot.snapshotId());
if (statsFile != null) {
Types.StructType partitionType = Partitioning.partitionType(table);
- Schema recordSchema = PartitionStatsHandler.schema(partitionType,
TableUtil.formatVersion(table));
-
- try (CloseableIterable<PartitionStats> recordIterator =
PartitionStatsHandler.readPartitionStatsFile(
- recordSchema, table.io().newInputFile(statsFile.path()))) {
- PartitionStats partitionStats = Iterables.tryFind(recordIterator,
stats -> {
+ try (CloseableIterable<PartitionStatistics> recordIterator =
Review Comment:
https://github.com/apache/iceberg/blob/5f3d3c560a956b8ca3e45c525a353c77f8a1f9b4/core/src/main/java/org/apache/iceberg/BasePartitionStatisticsScan.java#L76
Yes, With the introduction of **newPartitionStatisticsScan()**, Iceberg
handles reading partition statistics file internally in
**BasePartitionStatisticsScan**, we no longer need statsFile for I/O.
The only reason we still call **getPartitionStatsFile()** is the condition
check, to preserve existing behavior when no stats file exists in table
metadata.
I’m thinking of replacing the statsFile variable with inline
getPartitionStatsFile() != null or partitionStatisticsFiles(), so we don't have
to fetch/hold statsFile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
