Besides the fact that the refactoring required is significant, I don't think this is possible to do quickly since:
1) Hive (unlike Pig) requires a metastore 2) Hive releases can't depend on an incubator project It's worth pointing out that Howl is already using Hive's CLI+DDL (not just metastore). That's a huge amount of code. In biological terms, Howl has the same DNA as Hive (plus some new Howl-specific genes on a separate plugin chromosome), but only a subset of the Hive genes are expressed when running Howl; the rest are just junk DNA from Howl's perspective. It's not clear yet that refactoring is worth the effort even in the end state. We can achieve the desired compatibility by keeping the current approach but removing the Hive code copy from Howl, instead creating a dependency from Howl to Hive. In this case, graduating to become a Hive subproject might be the correct exit from the incubator. If we do go ahead with pulling the metastore out of Hive, it might make most sense for Howl to become its own TLP rather than a subproject. In the incubator proposal, we have mentioned these issues, but we've attempted to avoid prejudicing any decision. Instead, we'd like to assess the pros and cons (including effort required and impact expected) for both approaches as part of the incubation process. I don't have any voting rights on Pig but obviously I'm +1 on the proposal for incubation. JVS On Feb 3, 2011, at 8:52 AM, Jay Booth wrote: > Food for thought, what if the metastore were moved to Howl more > aggressively? It seems like the end state everyone's aiming for is > that Hive and Pig share Howl as a metastore layer, which makes all > kinds of sense.. would it increase the chances of long term success > if you guys just went for it and introduced the Hive->Howl dependency > as soon as possible? It would probably create some short term > disruption but it could be more healthy for Howl assuming that things > were worked out, design choices could be validated faster, you have > that end-to-end "it works" thing going, etc. > > On Thu, Feb 3, 2011 at 11:43 AM, Ashutosh Chauhan <hashut...@apache.org> > wrote: >> +1 >> >> On Wed, Feb 2, 2011 at 13:18, Alan Gates <ga...@yahoo-inc.com> wrote: >>> Howl is a table management system built to provide metadata and storage >>> management across data processing tools in Hadoop (Pig, Hive, MapReduce, >>> ...). You can learn more details at http://wiki.apache.org/pig/Howl. For >>> the last six months the code has been hosted at github. The Howl team would >>> like to move the project into the Apache Incubator. You can see the >>> proposal for the project at http://wiki.apache.org/incubator/HowlProposal. >>> >>> In order to be accepted as an Incubator project Howl needs a Sponsoring >>> project. I propose that we, the Pig project, sponsor Howl. By sponsoring >>> Howl we are saying that we believe it is a good fit for the ASF and that we >>> will assist the Howl project to succeed. You can read full details of >>> sponsoring a project at >>> http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sponsor. >>> >>> Our bylaws don't explicitly cover such a vote, but I think lazy majority >>> should be reasonable. All votes are welcome, PMC member votes will be >>> binding. >>> >>> Clearly I'm +1. >>> >>> Alan. >>> >>
