Hi All, I was looking into the gRPC support in the standalone-metastore and it seems metastore does not expose the gRPC endpoint currently.
I opened HIVE-30063 to track this discussion: https://issues.apache.org/jira/browse/HIVE-30063 <https://issues.apache.org/jira/browse/HIVE-30063?utm_source=chatgpt.com> however there exist a grpc proto definition : standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/hive_metastore.proto Currently both the IDL files hive_metastore.proto and hive_metastore.thrift appear to be maintained and updated separately. This has caused the proto file to be out of sync from the thrift definition. Additionally a lot of things/features in the gRPC proto are not one to one mapped to the thrift like the 1. *string*: in thrift the string can have values like null, empty string (""); but the proto string is default to empty string only, unless it's specified as an optional field in proto. 2. *List<Object>*: in thrift this can have values like null, empty list; but for the proto the default value is empty list always and it cannot have the null value, to have workaround this to box them in other object like ListObject which contain the List<Object> inside as single field. 3. *Input parameters*: A Thrift service method can define multiple parameters directly, whereas a protobuf RPC accepts a single request message. The Thrift parameters therefore need to be wrapped in one proto request object. There are many such instances where the GRPC and thrift mapping do not match one to one and need some handling to allow them. I tried to find some open source tooling if something is present that converts the thrift to a proto file and found 2 projects. One is in the python <https://github.com/AnneYang720/thrift2proto> and other in go <https://github.com/YYCoder/protobuf-thrift> but I could not find an established Java-native Thrift-to-Proto generator suitable for integration into Hive's build.. The two projects also do not currently generate a valid proto definition from Hive's Thrift IDL and have issues handling some of the cases mentioned above. Given that there is no currently available solution for the thrift to proto file one possible approach would be to implement a Hive-owned Thrift-to-Proto generator. that creates/updates the proto file on build time when thrift is updated. so the approach flow will be hive_metastore.thrift -> [thrift to proto generator] -> [validate generated proto] -> hive_metastore.proto . and this proto file can be used to setup a gRPC endpoint for the metastore in future to support the gRPC calls. Before starting any implementation work on this, I want to get the feedback on the direction of the approach for this. Does generating the protobuf definition from the Thrift IDL seem like the right direction? More broadly, is there interest in supporting a gRPC endpoint for the standalone metastore? If this makes sense I can work on the proposal and implementation for this. Thanks, Udit Kumar
