[
https://issues.apache.org/jira/browse/HIVE-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319761#comment-15319761
]
Chaoyu Tang commented on HIVE-13959:
------------------------------------
[~ychena] Thanks for the review. For your analysis and questions, please see
below:
Yes -- one WriteEntity map to one List<HiveLockObj>
Yes -- These list of HiveLockObj are all created during acquireLocks related to
the query.
Yes -- In the releaseLocks code, lockObj.getObj() return HiveLockObject
The problem is here: List<HiveLock> locks = lockMgr.getLocks(lockObj.getObj(),
false, true); it returns all locks under the pathName which might not related
to this MoveTask query:
{code}
The getLocks method in ZookeeperHiveLockManager:
private static List<HiveLock> getLocks(HiveConf conf,
HiveLockObject key, String parent, boolean verifyTablePartition, boolean
fetchData)
throws LockException {
List<HiveLock> locks = new ArrayList<HiveLock>();
List<String> children;
boolean recurse = true;
String commonParent;
try {
if (key != null) {
commonParent = "/" + parent + "/" + key.getName();
children = curatorFramework.getChildren().forPath(commonParent);
/* ==> this call returns all locks under commonParent, say
db/cdhpart/LOCK-SHARE-000000, db/cdhpart/LOCK-SHARE-000001 for pathNames
db/cdhpart */
recurse = false;
}
else {
commonParent = "/" + parent;
children = curatorFramework.getChildren().forPath(commonParent);
}
} catch (Exception e) {
// no locks present
return locks;
}
{code}
For an example, if we run query1 in one session "insert overwrite table cdhpart
partition (level1= 'l1', level2='l2', level3 = 'l3', level4) select key, value,
level4 from cdhsrc;" and query2 in the other session concurrently "select *
from cdhpart where level1 = 'l1'"
query1 and query2 both have its own znode (lock) under pathNames (db/cdhpart/)
say LOCK-SHARE-000000, LOCK-SHARE-000001 respectively, the getLocks for
HiveLockObject key with its getName() value db/cdhpart/ will return both
LOCK-SHARE-000000, LOCK-SHARE-000001. But LOCK-SHARE-000001 is not in the
ctx.getHiveLocks(), the lock list for the query1, so
ctx.getHiveLocks().remove() returns false because the HiveLockObjectData.equals
always return false due to the different queryStr/queryId, therefore
lockMgr.unlock(lock) should not be called to unlock the LOCK-SHARE-000001 for
query2.
> MoveTask should only release its query associated locks
> -------------------------------------------------------
>
> Key: HIVE-13959
> URL: https://issues.apache.org/jira/browse/HIVE-13959
> Project: Hive
> Issue Type: Bug
> Components: Locking
> Reporter: Chaoyu Tang
> Assignee: Chaoyu Tang
> Attachments: HIVE-13959.patch, HIVE-13959.patch
>
>
> releaseLocks in MoveTask releases all locks under a HiveLockObject pathNames.
> But some of locks under this pathNames might be for other queries and should
> not be released.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
