[
https://issues.apache.org/jira/browse/IMPALA-8438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16841757#comment-16841757
]
ASF subversion and git services commented on IMPALA-8438:
---------------------------------------------------------
Commit 2e7f689e26510a66e701fb7767a6ee2e7a097963 in impala's branch
refs/heads/master from Yongzhi Chen
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2e7f689 ]
IMPALA-8438: Store WriteId and ValidWriteId list for table and partition
This happens when tables load metadata from HMS.
Add MetastoreShim functions to support HMS3 only functions.
Add validwriteIdlists to query profile through timeline.
Tests:
Manually tests HMS2 and HMS3, using log files to check
Unit tests against HMS3
ToDo:
WriteId and valid writeIds can be fetched in other time, need
more study on that.
Profile example:
Query Compilation: 5s057ms
- Metadata load started: 63.006ms (63.006ms)
- Metadata load finished. loaded-tables=2/2...: 4s801ms (4s738ms)
- Loaded ValidWriteIdLists:
acid.insert_only_no_partitions:6:9223372036854775807::
acid.insert_only_with_partitions:3:9223372036854775807::
: 4s921ms (120.580ms)
- Analysis finished: 4s929ms (8.013ms)
Change-Id: I6edbd64424edf0ba88af110ab8b958a1966b8b54
Reviewed-on: http://gerrit.cloudera.org:8080/13215
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> List valid writeIds for a ACID table
> ------------------------------------
>
> Key: IMPALA-8438
> URL: https://issues.apache.org/jira/browse/IMPALA-8438
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
> Reporter: Yongzhi Chen
> Assignee: Yongzhi Chen
> Priority: Critical
> Labels: impala-acid
>
> Before listing the partitions of a table, fetch and store the list of valid
> (committed) writeIds for the table. This will be used later during
> planning/refresh.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
