[
https://issues.apache.org/jira/browse/IMPALA-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18068784#comment-18068784
]
ASF subversion and git services commented on IMPALA-14803:
----------------------------------------------------------
Commit a04b8184009e619dba7022abddd2e0fc5f569ab8 in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a04b81840 ]
IMPALA-14803: Speed up Iceberg table minimal topic updates
Before this patch only HdfsTable was optimized for minimal topic
updates and IcebergTable created Thrift objects that included
all file descriptors. Most of the info in the Thrift object
is not used later in minimal topic mode (see
CatalogServiceCatalog#getMinimalObjectForV2). The patch
changes all table types other than HdfsTable to create
minimal Thrift representation.
The minimal topic update Thrift generation for an 1M file Iceberg
table decreased from around 0.5s to <1ms. I don't expect
noticable change for other table types as only HdfsTable and
IcebergTable have file descriptor lists.
Change-Id: If037a855dc5d2c4292347b34ebe0221477eae2fe
Reviewed-on: http://gerrit.cloudera.org:8080/24064
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Avoid converting whole Iceberg table to Thrift during catalog topic update
> --------------------------------------------------------------------------
>
> Key: IMPALA-14803
> URL: https://issues.apache.org/jira/browse/IMPALA-14803
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
