[
https://issues.apache.org/jira/browse/HIVE-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864457#action_12864457
]
John Sichi commented on HIVE-1293:
----------------------------------
Regarding the implementation of lock acquisition/release in terms of zookeeper,
can you elaborate on how you are proposing to map the lock hierarchy into the
znode hierarchy?
It sounds like the znode paths will correspond to
database.table.partition.subpartition... paths, which is good.
However, does the lock acquisition recipe actually need to reflect the
hierarchy, which I think is what you are proposing? In other words, can't we
just come up with a flat list of object locks to take (including both
table-level and partition-level), sort them, and then acquire them each
independently using the non-hierarchical recipe (except as you mention with
failfast instead of wait)? If any fail, then delete them all before
re-entering the retry loop.
Assuming the sorting matches the compound naming scheme, this should guarantee
hierarchical lock acquisition order within each table.
Also, I do not understand the part below.
"The 'X' lock for table T is specified as follows:
...
#
# For all parent znodes of T, call getChildren() without setting the watch
flag."
Do you mean "for the parent znode of T" rather than "all parent znodes of T",
and this is supposed to apply for case where T is actually a partition?
> Concurreny Model for Hive
> -------------------------
>
> Key: HIVE-1293
> URL: https://issues.apache.org/jira/browse/HIVE-1293
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Namit Jain
>
> Concurrency model for Hive:
> Currently, hive does not provide a good concurrency model. The only
> guanrantee provided in case of concurrent readers and writers is that
> reader will not see partial data from the old version (before the write) and
> partial data from the new version (after the write).
> This has come across as a big problem, specially for background processes
> performing maintenance operations.
> The following possible solutions come to mind.
> 1. Locks: Acquire read/write locks - they can be acquired at the beginning of
> the query or the write locks can be delayed till move
> task (when the directory is actually moved). Care needs to be taken for
> deadlocks.
> 2. Versioning: The writer can create a new version if the current version is
> being read. Note that, it is not equivalent to snapshots,
> the old version can only be accessed by the current readers, and will be
> deleted when all of them have finished.
> Comments.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
