[
https://issues.apache.org/jira/browse/IMPALA-13666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914897#comment-17914897
]
ASF subversion and git services commented on IMPALA-13666:
----------------------------------------------------------
Commit 5371e0c6df3e329398712af3ebb739465b947454 in impala's branch
refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5371e0c6d ]
IMPALA-13666: Provide a non-null fileMetadataStats for HdfsPartition
IMPALA-13154 added the method getFileMetadataStats() to
HdfsPartition.java that would return the file metadata statistics. The
method requires the corresponding HdfsPartition instance to have a
non-null field of 'fileMetadataStats_'.
This patch revises two existing constructors of HdfsPartition to provide
a non-null value for 'fileMetadataStats'. This makes it easier for a
third party extension to set up and update the field of
'fileMetadataStats_'. A third party extension has to update the field of
'fileMetadataStats_' if it would like to use this field to get the size
of the partition since all three fields in 'fileMetadataStats_' are
defaulted to 0.
A new constructor was also added for HdfsPartition that allows a third
party extension to provide their own FileMetadataStats when
instantiating an HdfsPartition. To facilitate instantiating a
FileMetadataStats, a new constructor was added for FileMetadataStats
that takes in a List of FileDescriptor's to construct a
FileMetadataStats.
Change-Id: I7e690729fcaebb1e380cc61f2b746783c86dcbf7
Reviewed-on: http://gerrit.cloudera.org:8080/22340
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add a new constructor for ImpalaHdfsPartition after IMPALA-13154
> ----------------------------------------------------------------
>
> Key: IMPALA-13666
> URL: https://issues.apache.org/jira/browse/IMPALA-13666
> Project: IMPALA
> Issue Type: Task
> Reporter: Fang-Yu Rao
> Assignee: Fang-Yu Rao
> Priority: Major
>
> IMPALA-13154 added the method {{getFileMetadataStats()}} to
> [HdfsPartition.java|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java]
> that would return the file metadata statistics. The method requires the
> corresponding {{HdfsPartition}} instance to have a non-null field of
> '{{{}fileMetadataStats_{}}}'.
> However, currently in order to set up the field of
> '{{{}fileMetadataStats_{}}}' when instantiating an {{{}HdfsPartition{}}}, we
> also need to provide the value of the field '{{{}id_{}}}', which should be
> unique across the whole catalog for each partition. Since it's possible that
> we may not always know what value we should be using for '{{{}id_{}}}' when
> instantiating an {{HdfsPartition}} via a third party extension, if would be
> good that we add a new constructor for {{HdfsPartition}} to allow us to set
> up the field of '{{{}fileMetadataStats_{}}}' when instantiating an
> {{HdfsPartition}} without having to provide the value of the field
> '{{{}id_{}}}'.
> An alternative would be revising the existing constructors of
> {{HdfsPartition}} to provide a non-null value for
> '{{{}fileMetadataStats_{}}}'.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
