[
https://issues.apache.org/jira/browse/IMPALA-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162754#comment-17162754
]
ASF subversion and git services commented on IMPALA-9859:
---------------------------------------------------------
Commit ee3f0538736d34ec39f4c647b4dd6adbf1a8469a in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ee3f053 ]
IMPALA-9964: Fill file descriptors properly in setFileMetadataFromFS
IMPALA-9859 added separate fields for insert and delete file
descriptors. They are needed for full ACID tables.
I did not set these in CatalogServiceCatalog.setFileMetadataFromFS
which could result in a NullPointerException in CatalogdMetaProvider.
During the fix I found another bug related to delete delta files. In
AcidUtils we did not filter them based on the valid write id list. I
fixed this issue as well in this commit.
Added unit tests about the issues.
Change-Id: I2927171cf426597c86766fb83d565c5e57025c73
Reviewed-on: http://gerrit.cloudera.org:8080/16206
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Milestone 4: Read updated tables
> --------------------------------
>
> Key: IMPALA-9859
> URL: https://issues.apache.org/jira/browse/IMPALA-9859
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
>
> Hive ACID supports row-level DELETE and UPDATE operations on a table. It
> achieves it via assigning a unique row-id for each row, and maintaining two
> sets of files in a table. The first set is in the delta directories, they
> contain the INSERTed rows. The second set of files are in the delete-delta
> directories, they contain the DELETEd rows.
> _Note: UPDATE operations are implemented via DELETE+INSERT._
> In the filesystem it looks like e.g.:
> {noformat}
> full_acid/delta_0000001_0000001_0000/0000_0
> full_acid/delete_delta_0000002_0000002_0000/0000_0
> {noformat}
> During scanning we need to return INSERTed rows minus DELETEd rows. One way
> of doing that is to create an ANTI JOIN between INSERT and DELETE events.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
