[
https://issues.apache.org/jira/browse/HIVE-19040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416098#comment-16416098
]
Sergey Shelukhin edited comment on HIVE-19040 at 3/27/18 6:59 PM:
------------------------------------------------------------------
My point is, metastore only calls Hive code via the proxy; so metastore version
is not a concern, it's the compat between the caller of the API (HS2) and the
Hive jar deployed with metastore.
So, how big of a problem this is depends on the model for the latter... will
this JAR be shipped with metastore? seems unlikely. User will have to copy it
somehow. If they install new HS2 but don't copy a new jar, then they can have
issues. If they just upgrade the metastore it shouldn't matter.
If we wanted to just fail over to client side filtering, like it should do now
in case of failure, we could add version to the API and to the proxy.
It may also be possible to make a better API... it's not great that UDF is
serialized as part of ExprNodeDesc.
was (Author: sershe):
My point is, metastore only calls Hive code via the proxy; so metastore version
is not a concern, it's the compat between the caller of the API (HS2) and the
Hive jar deployed with metastore.
So, how big of a problem this is depends on the model for the latter... will
this JAR be shipped with metastore? seems unlikely. User will have to copy it
somehow. If they install new HS2 but don't copy a new jar, then they can have
issues. If they just upgrade the metastore it shouldn't matter.
> get_partitions_by_expr() implementation in HiveMetaStore causes backward
> incompatibility easily
> ------------------------------------------------------------------------------------------------
>
> Key: HIVE-19040
> URL: https://issues.apache.org/jira/browse/HIVE-19040
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Affects Versions: 2.0.0
> Reporter: Aihua Xu
> Priority: Major
>
> In the HiveMetaStore implementation of {{public PartitionsByExprResult
> get_partitions_by_expr(PartitionsByExprRequest req) throws TException}} , an
> expression is serialized into byte array from the client side and passed
> through PartitionsByExprRequest. Then HMS will deserialize back into the
> expression and filter the partitions by it.
> Such partition filtering expression can contain various UDFs. If there are
> some changes to one of the UDFs between different Hive versions, HS2 on the
> older version will serialize the expression in old format which won't be able
> to be deserialized by HMS on the newer version. One example of that is,
> GenericUDFIn class adds {{transient}} to the field constantInSet which will
> cause such incompatibility.
> One approach I'm thinking of is, instead of converting the expression object
> to byte array, we can pass the expression string directly.
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
