Hi Alan 1.When writer takes exclusive lock , hive won't allow to write anyone(even the session which holds lock) to write in table. Do I need to pass lock handle to read query or I am missing here something. 2.Or you mean to insert using hadoop filesystem not using hive ?
On Tue, Sep 23, 2014 at 8:13 PM, Alan Gates <ga...@hortonworks.com> wrote: > Shushant, > > Creating a patched jar that would include the lock functionality you want > is unlikely to work. Wouldn't the following workflow work for you: > > 1. Writer locks the table explicitly via LOCK TABLE > 2. Writer inserts > 3. Writer unlocks the table explicitly via UNLOCK TABLE > > If you're using ZK for your locking I think the client dying (as opposed > to ending the session) should cause the lock to expire. If not, you may > have to assure the unlock happens in your application. Hope that helps. > > Alan. > > Shushant Arora <shushantaror...@gmail.com> > September 20, 2014 at 8:00 > Hi Alan > > I have 0.10 version of hive deployed in my org's cluster, I cannot update > that because of org's policy. > How can I achieve exclusive lock functionality while inserting in dynamic > partition on hive 0.10 ? > Does calling hive scripts via some sort of java api with patched jar > included will help ? > Moreover hive does not release locks in 0.10 when hive session is killed . > User has to explicitly unlock a table. > Can i specify any sort of max expiry time while taking a lock. > > Thanks > Shushant > > > Alan Gates <ga...@hortonworks.com> > September 20, 2014 at 7:41 > Up until Hive 0.13 locks in Hive were really advisory only, since as you > note any user can remove any other user's lock. In Hive 0.13 a new type of > locking was introduced, see > https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-LockManager > This new locking is automatic and ignores both LOCK and UNLOCK commands. > Note that it is off by default, you have to configure Hive to use the new > DbTxnManager to get turn on this locking. In 0.13 it still has the bug you > describe as far as acquiring the wrong lock for dynamic partitioning, but I > believe I've fixed that in 0.14. > > Alan. > > > Shushant Arora <shushantaror...@gmail.com> > September 20, 2014 at 5:39 > > Hive version 0.9 and later has a bug > > > > While inserting in a hive table Hive takes an exclusive lock. But if table > is partitioned , and insert is in dynamic partition , it will take shared > lock on table but if all partitions are static then hive takes exclusive > lock on partitions in which data is being inserted > > and shared lock on table. > > https://issues.apache.org/jira/browse/HIVE-3509 > > > 1.What if I want to take exclusive lock on table while inserting in > dynamic partition ? > > > I tried to take explicit lock using : > > LOCK TABLE tablename EXCLUSIVE; > > > But it made table to be disabled. > > I cannot even read from table anymore even is same session until I do > > unlock table tablename in another session; > > > 2. moreover whats lock level in hive , I mean any user can remove any > other users lock. that too seems buggy. > > > Thanks > > Shushant > > > > > -- > Sent with Postbox <http://www.getpostbox.com> > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
