[
https://issues.apache.org/jira/browse/HIVE-21596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HIVE-21596:
-------------------------------
Description:
{{HiveMetastoreClient}} currently depends on the fact that both the client and
server versions are the same. Additionally, since the server APIs are backwards
compatible, it is possible for a older client (eg. 2.1.0 client version) to
connect to a newer server (eg. 3.1.0 server version) without any issues. This
is useful in setups where HMS is deployed in a remote mode and clients connect
to it remotely.
It would be a good improvement if a newer version {{HiveMetastoreClient }} can
connect to the a older server version. When a newer client is talking to a
older server following things can happen:
1. Client invokes a RPC to the older server which doesn't exist.
In such a case, thrift will throw {{Invalid method name}} exception which
should be automatically be handled by the clients since each API already throws
TException.
2. Client invokes a RPC using thrift objects which has new fields added.
When a new field is added to a thrift object, the server does not deserialize
the field in the first place since it does not know about that field id. So the
wire-compatibility exists already. However, the client side application should
understand the implications of such a behavior. In such cases, it would be
better for the client to throw exception by checking the server version which
was added in HIVE-21484
3. If the newer client has re-implemented a certain API, for example, using a
newer more efficient thrift API, but an older thrift API also exists which can
provide the same functionality. In this case, the new client will start seeing
exception {{Invalid method name}} since the older server does not have such a
method. This can be handled on the client side by making sure that the newer
implementation is conditional to the server version, and falling back to the
older (maybe less-efficient) one when necessary. Which means client should
check the server version and invoke the new implementation only if the server
version supports the newer API. (On a side note, it would be great if metastore
also gives information of which APIs are supported for a given version)
One of the real world use-case of such a feature is in Impala which wants to
have capability to talk to both HMS 2.x and HMS 3.x. But other applications
like Spark (or third party applications which want to support multiple HMS
versions) may also find this useful.
was:
{{HiveMetastoreClient}} currently depends on the fact that both the client and
server versions are the same. Additionally, since the server APIs are backwards
compatible, it is possible for a older client (eg. 2.1.0 client version) to
connect to a newer server (eg. 3.1.0 server version) without any issues. This
is useful in setups where HMS is deployed in a remote mode and clients connect
to it remotely.
It would be a good improvement if a newer version {{HiveMetastoreClient }} can
connect to the a older server version. When a newer client is talking to a
older server following things can happen:
1. Client invokes a RPC to the older server which doesn't exist.
In such a case, thrift will throw {{Invalid method name}} exception which
should be automatically be handled by the clients since each API already throws
TException.
2. Client invokes a RPC using thrift objects which has new fields added.
When a new field is added to a thrift object, the server does not deserialize
the field in the first place since it does not know about that field id. So the
wire-compatibility exists already. However, the client side application should
understand the implications of such a behavior. In such cases, it would be
better for the client to throw exception by checking the server version which
was added in HIVE-21484
3. If the newer client has re-implemented a certain API, for example, using
newer thrift API the client will start seeing exception {{Invalid method name}}
since the older server does not have such a method.
This can be handled on the client side by making sure that the newer
implementation is conditional to the server version. Which means client should
check the server version and invoke the new implementation only if the server
version supports the newer API. (On a side note, it would be great if metastore
also gives information of which APIs are supported for a given version)
One of the real world use-case of such a feature is in Impala which wants to
have capability to talk to both HMS 2.x and HMS 3.x. But other applications
like Spark (or third party applications which want to support multiple HMS
versions) may also find this useful.
> HiveMetastoreClient should be able to connect to older metastore servers
> ------------------------------------------------------------------------
>
> Key: HIVE-21596
> URL: https://issues.apache.org/jira/browse/HIVE-21596
> Project: Hive
> Issue Type: Improvement
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
>
> {{HiveMetastoreClient}} currently depends on the fact that both the client
> and server versions are the same. Additionally, since the server APIs are
> backwards compatible, it is possible for a older client (eg. 2.1.0 client
> version) to connect to a newer server (eg. 3.1.0 server version) without any
> issues. This is useful in setups where HMS is deployed in a remote mode and
> clients connect to it remotely.
> It would be a good improvement if a newer version {{HiveMetastoreClient }}
> can connect to the a older server version. When a newer client is talking to
> a older server following things can happen:
> 1. Client invokes a RPC to the older server which doesn't exist.
> In such a case, thrift will throw {{Invalid method name}} exception which
> should be automatically be handled by the clients since each API already
> throws TException.
> 2. Client invokes a RPC using thrift objects which has new fields added.
> When a new field is added to a thrift object, the server does not deserialize
> the field in the first place since it does not know about that field id. So
> the wire-compatibility exists already. However, the client side application
> should understand the implications of such a behavior. In such cases, it
> would be better for the client to throw exception by checking the server
> version which was added in HIVE-21484
> 3. If the newer client has re-implemented a certain API, for example, using a
> newer more efficient thrift API, but an older thrift API also exists which
> can provide the same functionality. In this case, the new client will start
> seeing exception {{Invalid method name}} since the older server does not have
> such a method. This can be handled on the client side by making sure that the
> newer implementation is conditional to the server version, and falling back
> to the older (maybe less-efficient) one when necessary. Which means client
> should check the server version and invoke the new implementation only if the
> server version supports the newer API. (On a side note, it would be great if
> metastore also gives information of which APIs are supported for a given
> version)
> One of the real world use-case of such a feature is in Impala which wants to
> have capability to talk to both HMS 2.x and HMS 3.x. But other applications
> like Spark (or third party applications which want to support multiple HMS
> versions) may also find this useful.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
