Eugene Koifman commented on HIVE-21115:

Isn't there a direct SQL path somewhere that modifies HMS objects w/o using 
Could this be expressed via some on-update trigger instead?

> Add support for object versions in metastore
> --------------------------------------------
>                 Key: HIVE-21115
>                 URL: https://issues.apache.org/jira/browse/HIVE-21115
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Vihang Karajgaonkar
>            Priority: Major
> Currently, metastore objects are identified uniquely by their names (eg. 
> catName, dbName and tblName for a table is unique). Once a table or partition 
> is created it could be altered in many ways. There is no good way currently 
> to identify the version of the object once it is altered. For example, 
> suppose there are two clients (Hive and Impala) using the same metastore. 
> Once some alter operations are performed by a client, another client which 
> wants to do a alter operation has no good way to know if the object which it 
> has is the same as the one stored in metastore. Metastore updates the 
> {{transient_lastDdlTime}} every time there is a DDL operation on the object. 
> However, this value cannot be relied for all the clients since after 
> HIVE-1768 metastore updates the value only when it is not set in the 
> parameters. It is possible that a client which alters the object state, does 
> not remove the {{transient_lastDdlTime}} and metastore will not update it. 
> Secondly, if there is a clock skew between multiple HMS instances when HMS-HA 
> is configured, time values cannot be relied on to find out the sequence of 
> alter operations on a given object.
> This JIRA propose to use JDO versioning support by Datanucleus  
> http://www.datanucleus.org/products/accessplatform_4_2/jdo/versioning.html to 
> generate a incrementing sequence number every time a object is altered. The 
> value of this object can be set as one of the values in the parameters. The 
> advantage of using Datanucleus the versioning can be done across HMS 
> instances as part of the database transaction and it should work for all the 
> supported databases.
> In theory such a version can be used to detect if the client is presenting a 
> object which is "stale" when issuing a alter request. Metastore can choose to 
> reject such a alter request since the client may be caching a old version of 
> the object and any alter operation on such stale object can potentially 
> overwrite previous operations. However, this is can be done in a separate 

This message was sent by Atlassian JIRA

Reply via email to