[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644088#comment-16644088 ] Alexander Kolbasov commented on HIVE-18712: --- I have some experimental work-in-progress on this at [gthub hmsv2api|https://github.com/akolb1/hmsv2api]. This includes grpc specs, simple prototype implementation on the server side and Java code that can be used on HMS side to use it. I have a working HMS that uses this API as a prototype. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: metastore.proto, metastore_index.html, > metastore_swagger.json, metastore_swagger.yaml > > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16444260#comment-16444260 ] Andrew Sherman commented on HIVE-18712: --- +1 for server side filtering everywhere > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: metastore.proto, metastore_index.html, > metastore_swagger.json, metastore_swagger.yaml > > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443384#comment-16443384 ] Alexander Kolbasov commented on HIVE-18712: --- I attached a file metastore.proto which we can use as the starting point. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: metastore.proto > > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443383#comment-16443383 ] Alexander Kolbasov commented on HIVE-18712: --- A few thoughts about the nature of the newish APIs. * It would be difficult to convert a lot of existing apps to new APIs so we need to preserve compatibility with existing HMS APIs. * We would like to separate pure metadata operations from the rest of the work done by HMS. This means that we may want to define "metadata API" and let HMS use it. * Currently this most closely matches ObjectStore layer. Note that ObjectStore doesn't use Kerberos to talk to the underlying DB so we can define gRPC non-kerberos API that talks to metadata service. What would be a good way for us to start working on this and share the work? Should we work in a branch off branch-3 or set up another git repo or use some other tools? Any suggestions? > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416854#comment-16416854 ] Alexander Kolbasov commented on HIVE-18712: --- Looks like gRPC doesn't support Kerberos. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16413321#comment-16413321 ] Alexander Kolbasov commented on HIVE-18712: --- gRPC requires more or less recent Guava version. Luckily, branch-3 uses Guava version 19.0 which works with gRPC. Also, maps are only supported in protobuf version 3. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410707#comment-16410707 ] Alexander Kolbasov commented on HIVE-18712: --- Would using TLS encryption be an issue? For example, if we use Kerberos for authentication and TLS for encryption? > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410705#comment-16410705 ] Alexander Kolbasov commented on HIVE-18712: --- I am not an expert on this, but I know of at least one product that uses gRPC and Kerberos and it works. I'll try to get more details. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409462#comment-16409462 ] Lars Francke commented on HIVE-18712: - gRPC is great but it does not support Kerberos authentication and I assume we won't get away from Kerberos anytime soon. We'd need to do custom code (sending a ticket as metadata) etc. also encryption would only really be possible using TLS and not on the Kerberos/SASL layer. I'm not _against_ gRPC but it's something we'd need to think about. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409266#comment-16409266 ] Peter Vary commented on HIVE-18712: --- That streaming API is sold me the gRcp :) > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407497#comment-16407497 ] Alexander Kolbasov commented on HIVE-18712: --- Another argument towards gRpc - it natively supports streaming APIs which fits nicely to things like getting tons of partitions. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407422#comment-16407422 ] Alexander Kolbasov commented on HIVE-18712: --- Let's clarify what kind of objects do we currently have: # Catalog (new thing in flight) - collection of databases # Database - collection of tables # Table - collection of partitions # Partition - bunch of stuff # Schemas - where do these fit? [~alangates] any comment? I would actually suggest to add a notion of "namespace' as a top-level abstraction. Everything should belong to some namespace, there is no sharing between namespaces, they can use different backends if the implementation chooses, they may represent different servers, different clusters, etc. ANy thoughts on this? > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399927#comment-16399927 ] Alexander Kolbasov commented on HIVE-18712: --- I think that another useful thing should be considered in the new API is the notion of session ID that can be used to correlate multiple HMS operations with each other and with other Hive operations. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[
https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384404#comment-16384404
]
Alexander Kolbasov commented on HIVE-18712:
---
As we start thinking about updating Hive Metastore APIs, we need to immediately
think about the object ownership issue.
Here is, for example, how Database object is defined:
{code:java}
enum PrincipalType {
USER = 1,
ROLE = 2,
GROUP = 3,
}{code}
{code:java}
// namespace for tables
struct Database {
1: string name,
...
5: optional PrincipalPrivilegeSet privileges,
6: optional string ownerName,
7: optional PrincipalType ownerType
} {code}
Is this really the model we would like to carry forward? Does USER/ROLE/GROUP
type adequately describe the ownership model we want to have? Would this still
apply in mixed cloud-like environments?
More over, do we really need the notion of the owner in the first place? We do
need an initial set of permissions for the object, but do we really need the
concept of an owner?
If we are defining the model from scratch, what do you think is a useful model
here?
> Design HMS Api v2
> -
>
> Key: HIVE-18712
> URL: https://issues.apache.org/jira/browse/HIVE-18712
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
>
> This is an umbrella Jira covering the design of Hive Metastore API v2.
> It is supposed to be a placeholder for discussion and design documents.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384113#comment-16384113 ] Vihang Karajgaonkar commented on HIVE-18712: I think its a great idea to have v2 APIs. I think we should to make sure that APIs are extensible for the future. A lot of the our current APIs are not very extensible which caused everyone to add newer APIs which could have easily been implemented by extending the older ones. +1 to have request/response objects for the API instead of non-extensible input and output arguments. I think one problem with the current APIs, is that there is no way to filter objects on the server side. For instance, it should be possible to get objects based on some filter criteria. Additionally, I think it would be good to have some way to fetch objects in batches rather than getting all the objects in one go. This adds a lot of memory pressure on HMS when clients requests huge number of objects. I like the partition expression proxy API which enables clients to push down filters when querying partitions. We should have similar generic way to return only stuff which is needed by the clients. Another interesting idea to consider is if we can have some kind of version handshake between client and server. This helps with cross-version compatibility. I am not sure how other systems handle these. But it would be good to take a look at that. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384097#comment-16384097 ] Alexander Kolbasov commented on HIVE-18712: --- Looks like there are some tools (e.g. [gRPC-REST|[https://grpc.io/blog/coreos])] that allow building both gRPC and REST interface at the same time which is really nice. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[
https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372561#comment-16372561
]
Peter Vary commented on HIVE-18712:
---
I think separating the FS changes, and the MetaData changes is especially
important when we consider no native tables - AFAIK currently we use
{{HiveMetaDataHook}}'s, to ensure to update / notify the system which
physically stores the data. Also I think one of our goals with the MetaStore
separation is to integrate more systems. So I definitely would vote for a
solution, where there is a clean and well defined way to handle the storage of
the data.
> Design HMS Api v2
> -
>
> Key: HIVE-18712
> URL: https://issues.apache.org/jira/browse/HIVE-18712
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Priority: Major
>
> This is an umbrella Jira covering the design of Hive Metastore API v2.
> It is supposed to be a placeholder for discussion and design documents.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372486#comment-16372486 ] Alexander Kolbasov commented on HIVE-18712: --- I also think that we should consider semantics of the v2 API. For things like createSomething, I think we should consider separating metadata only changes from file system changes - e.g. currently createDatabase not only updates metadata but also creates directory in the file system and there are some APIs which can involve huge amount of file system ops (e.g. database/table renames), especially on S3-type filesystems. We should consider splitting APIs into metadata-specific and filesystem-specific calls (and may be even move file-system specific calls outside of metadata). > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372479#comment-16372479 ] Alexander Kolbasov commented on HIVE-18712: --- Speaking of REST style APIs, we have something similar available via WebHcat - is anyone using these? If not, do we know why? > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368915#comment-16368915 ] Peter Vary commented on HIVE-18712: --- I am not against the Rest API, but I think it worth to mention that using any typed RPC request (thrift or gRCP) could help to ensure API compatibility. I do not know how strict we would like to be with API backward compatibility, and how we would like to enforce it, but extra effort is needed to ensure this in case of the Rest API > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368069#comment-16368069 ] Alexander Kolbasov commented on HIVE-18712: --- I think documentation part is orthogonal - it is possible to document thrift as well. For the use of REST we may need to consider things like max message sizes but it would be good to avoid huge messages anyway. Anyone has any thoughts on gRPC? Note that in the case of REST style API it is very easy to add extra args later, so the whole Request/Response argument is a bit less relevant. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367679#comment-16367679 ] Eugene Koifman commented on HIVE-18712: --- +1 for consistent use of Request/Response objects Thirft is not very well documented - one argument in favor of more common REST > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364714#comment-16364714 ] Thejas M Nair commented on HIVE-18712: -- Thanks for creating this. The API's can certainly use some cleanup. These API's might be used in other projects as well, so we need to have some deprecation cycle for this. There are many API's that do very similar things. I think lot of those would not have been needed if we had followed the pattern of using a single *Request object as argument and *Response argument as response. It would have been possible to evolve API by adding members to the Request/Response objects instead of exploding the number of functions. (HiveServer2 thrift API has seen better usage of this pattern). I also see demand for HTTP based API's. This would allow the use of an HTTP gateway, specially in cloud environments. Supporting Thrift over HTTP is an option for that in the short term. Longer term, a REST API would be good to have. The webhcat DDL API's could be moved to metastore server, instead of keeping it in a separate server (we should also stop forking a "hcat cli" process to get the results!). > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363425#comment-16363425 ] Alexander Kolbasov commented on HIVE-18712: --- A few things to consider: # What parts of the current API work really well # What parts are obsolete, do not work or work poorly There are several other considerations: # Transport/Encoding: ** Thrift ** HTTP ** gRPC ** Something else? # Security ** Kerberos ** SSL ** Something else? # Compatibility # API evolution It is also worth establishing higher-level design principles. I would argue that the following are useful: * API should be able to deal efficiently with very large amounts of objects. * It should have clear failure model * It should not be tailored to any specific implementation language and should work reasonably well with at least 3 different ones * It should support metadata needs of existing known consumers Please treat items above as a starting point of discussion - I am sure there will be lots and lots of various opinions on the subject. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
