Vihang Karajgaonkar commented on HIVE-19040:

+1 to the idea of having native expressions support on metastore side. 

> get_partitions_by_expr() implementation  in HiveMetaStore causes backward 
> incompatibility easily
> ------------------------------------------------------------------------------------------------
>                 Key: HIVE-19040
>                 URL: https://issues.apache.org/jira/browse/HIVE-19040
>             Project: Hive
>          Issue Type: Improvement
>          Components: Standalone Metastore
>    Affects Versions: 2.0.0
>            Reporter: Aihua Xu
>            Priority: Major
> In the HiveMetaStore implementation of {{public PartitionsByExprResult 
> get_partitions_by_expr(PartitionsByExprRequest req) throws TException}} , an 
> expression is serialized into byte array from the client side and passed 
> through  PartitionsByExprRequest. Then HMS will deserialize back into the 
> expression and filter the partitions by it.
> Such partition filtering expression can contain various UDFs. If there are 
> some changes to one of the UDFs between different Hive versions, HS2 on the 
> older version will serialize the expression in old format which won't be able 
> to be deserialized by HMS on the newer version.  One example of that is, 
> GenericUDFIn class adds {{transient}}  to the field constantInSet which will 
> cause such incompatibility.
> One approach I'm thinking of is, instead of converting the expression object 
> to byte array, we can pass the expression string directly. 

This message was sent by Atlassian JIRA

Reply via email to