[
https://issues.apache.org/jira/browse/HIVE-24428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241588#comment-17241588
]
Zoltan Haindrich commented on HIVE-24428:
-----------------------------------------
apparently similar things may happen at 2 places:
* metastore might bump into issues adding the same partition for a table
* during filemove operations issues might happen
addressed them both separately
* because other apis may also use the metastore api - I've added a some simple
table level locking to the metastore side
* the filemove operation(s) from now will acquire a SEMI_SHARED lock for the
actual table while the file move happens - this will lock out multiple writers
from the same partition
> Concurrent add_partitions requests may lead to data loss
> --------------------------------------------------------
>
> Key: HIVE-24428
> URL: https://issues.apache.org/jira/browse/HIVE-24428
> Project: Hive
> Issue Type: Bug
> Reporter: Zoltan Haindrich
> Assignee: Zoltan Haindrich
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> in case multiple clients are adding partitions to the same table - when the
> same partition is being added there is a chance that the data dir is removed
> after the other client have already written its data
> https://github.com/apache/hive/blob/5e96b14a2357c66a0640254d5414bc706d8be852/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L3958
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
