[
https://issues.apache.org/jira/browse/HIVE-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864450#action_12864450
]
Edward Capriolo commented on HIVE-1293:
---------------------------------------
I see you have made mention to zookeeper. Do we really need zookeeper to
implement locking when we already have a transactional MetaStore via JPOX
(Derby/MySQL)? From my prospecting on the management side and from a
development side having to integrate and understand the Zookeeper API for
simple low performance locking seems overkill.
For example, I understand that zookeeper has facilities to auto-expire old
locks and other features that make it seem ideal in a use case, but I think the
visibility into it is rather low. If our locks were implemented in the
meta-store, clearing them would be easy to do thought hive commands or with
JDBC. What tools would we have to investigate Hive-Zookeeper-Locking?
I say this because of experience with hbase-zookeeper. Other the 'hbase status'
zookeeper is a total black box and makes troubleshooting that component of
hbase difficult. I would not want to see hive go down the same road.
No matter which implementation we go with we should have DDL commands like
'show locks' 'clear locks'.
> Concurreny Model for Hive
> -------------------------
>
> Key: HIVE-1293
> URL: https://issues.apache.org/jira/browse/HIVE-1293
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: Namit Jain
>
> Concurrency model for Hive:
> Currently, hive does not provide a good concurrency model. The only
> guanrantee provided in case of concurrent readers and writers is that
> reader will not see partial data from the old version (before the write) and
> partial data from the new version (after the write).
> This has come across as a big problem, specially for background processes
> performing maintenance operations.
> The following possible solutions come to mind.
> 1. Locks: Acquire read/write locks - they can be acquired at the beginning of
> the query or the write locks can be delayed till move
> task (when the directory is actually moved). Care needs to be taken for
> deadlocks.
> 2. Versioning: The writer can create a new version if the current version is
> being read. Note that, it is not equivalent to snapshots,
> the old version can only be accessed by the current readers, and will be
> deleted when all of them have finished.
> Comments.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
