Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "Hive/Locking" page has been changed by NamitJain. http://wiki.apache.org/hadoop/Hive/Locking?action=diff&rev1=13&rev2=14 -------------------------------------------------- The rational behind the lock mode to acquire is as follows: - For a non-partitioned table, the lock modes are pretty intutive. When the table is being read, a S + For a non-partitioned table, the lock modes are pretty intuitive. When the table is being read, a S lock is acquired, whereas an X lock is acquired for all other operations (insert into the table, alter table of any kind etc.) @@ -50, +50 @@ ||'''alter table T1 add partition P1'''||'''S on T1, X on T1.P1'''|| ||'''alter table T1 drop partition P1'''||'''S on T1, X on T1.P1'''|| ||'''alter table T1 touch partition P1'''||'''S on T1, X on T1.P1'''|| - ||'''alter table T1 set serdeProperties '''||'''S on T1'''|| + ||'''alter table T1 set serdeproperties '''||'''S on T1'''|| ||'''alter table T1 set serializer '''||'''S on T1'''|| ||'''alter table T1 set file format '''||'''S on T1'''|| ||'''alter table T1 set tblproperties '''||'''X on T1'''|| @@ -63, +63 @@ number of partitions may not be known, an exclusive lock is taken on the table, or the prefix that is known. + The lock modes are hierarchical in nature - if 'S' lock is acquired on T, implicitly 'S' lock is acquired + on all partitions of T. + Two new configurable parameters will be added to decide the number of retries for the lock and the wait time between each retry. If the number of retries are really high, it can lead to a live lock. Look at ZooKeeper (http://hadoop.apache.org/zookeeper/docs/r3.1.2/recipes.html#sc_recipes_Locks) to @@ -70, +73 @@ the lock request will be denied. The existing locks will be released, and all of them will be retried after the retry interval. + The recipe listed above will not work as specified, because of the hierarchical nature of locks. + The 'S' lock for table T are specified as follows: + * Call create( ) to create a node with pathname "/warehouse/T/read-". This is the lock node use later in the protocol. Make sure to set ephemeral flag. + * Call getChildren( ) on the lock node without setting the watch flag. + * If there are a child with a pathname starting with "write-", then the lock cannot be acquired. Delete the node created in the first step and return. + * For all sub-directories of T, if there is a child with a pathname starting with "write-", the lock cannot be acquired. Delete the node created in the first step and return. + * Otherwise the lock is granted. + + The 'X' lock for table T are specified as follows: + * Call create( ) to create a node with pathname "/warehouse/T/write-". This is the lock node use later in the protocol. Make sure to set the ephemeral flag. + * Call getChildren( ) on the lock node without setting the watch flag. + * If there are a child with a pathname starting with "read-", then the lock cannot be acquired. Delete the node created in the first step and return. + * For all sub-directories of T, if there is a child with a pathname starting with "read-" or "write-", the lock cannot be acquired. Delete the node created in the first step and return. + * Otherwise the lock is granted. + + The proposed scheme starves the writers for readers. In case of long readers, it may lead to starvation for writers. + The default hive behavior will not be changed, and concurrency will not be supported.
