Its great to hear that you are considering integrating Metastore to store metadata for Drill. Couple of things to note regarding the timeline though. As pointed by Alan the work being done in HIVE-17159 is on the master branch which currently points to Hive 3.0 The latest release of Hive is 2.3 which is forked from branch-2 where this work is not available at the moment. So unless we have a Hive 3.0 release in December you may not be able to consume the standalone metastore right away. That said if you decide to develop/design against existing Metastore public API, it should not be too hard to rebase to latest Hive libraries when it is released. Also, it may be important to note that there will be a few metastore APIs which may still need Hive jars in the runtime, but based on our understanding these APIs are exclusively needed by Hive and other components like Drill may not need them.
Hope that helps. Thanks, Vihang On Wed, Nov 1, 2017 at 7:10 AM, Alan Gates <alanfga...@gmail.com> wrote: > Excellent, I am glad to hear that this could be useful for you. > > After discussing the proposal in the community [1] we agreed that the first > step should be to break out the metastore as a separately releasable module > inside Hive. You can see the progress of that work at HIVE-17159[2]. With > this change Drill we be able to use the metastore without needing all of > Hive. Once that is complete we will take up the discussion of whether and > how to make it a separate TLP. Open source communities being what they > are, I cannot give you precise dates on when the separately releasable > module will be available. At the current rate of progress it should be > done by the end of this year. > > Regarding compatibility, we are working hard to keep the thrift interface > completely backwards compatible. So if you develop a client against the > current Hive metastore it should just work against the standalone > metastore. There will be small changes to metastore listeners (plugins > that can be run inside the metastore) that may require changes to user code > or recompilation of the listeners. But even there we are trying to keep > the changes to a minimum. > > Alan. > > 1. If you go to https://lists.apache.org/list.html?dev@hive.apache.org and > do an advanced search over the last 6 months for messages with a subject of > "[DISCUSS] Separating out the metastore as its own TLP” you will see all > the mails. Sorry, I can’t figure out how to get ponymail to give me a > single link to the whole thread. > 2. https://issues.apache.org/jira/browse/HIVE-17159 > > On Tue, Oct 31, 2017 at 4:14 PM, Padma Penumarthy <ppenumar...@mapr.com> > wrote: > > > Hi, > > > > We are exploring the possibility of using hive metastore as metastore for > > Apache Drill, to store and > > manage our metadata. While investigating, I came across the metastore TLP > > proposal, > > https://cwiki.apache.org/confluence/display/Hive/Metastore+TLP+Proposal. > > > > I want to find out what is the timeline for this to be available to end > > users ? > > when will it be production ready ? > > If we integrate with hive and design/develop using current metastore > > client API, do we run > > into risk of incompatibility in the future ? Our goal is to eventually > > move to using hive metastore only for the > > reasons correctly identified in your wiki i.e. users do not have to > > install hive to be able to use Drill. > > > > Any input/feedback you can provide to us will be very helpful for us to > > figure out our plans. > > > > Thanks > > Padma > > > > > > >
