Alan, 1. Contribs being removed from hadoop is due to a. inactivity and b. test failures. Since Howl will be actively worked on, and will be well-tested as a production deployment, I am sure it will not be objected to.
2. That was when Yahoo! was producing it's own distribution, thus not having dependencies on apache releases. With the recent announcements, that would change, no ? - milind On Feb 2, 2011, at 8:58 PM, Alan Gates wrote: > I see a couple blockers that prevent this from being a contrib project of > Hadoop: > > 1) The Hadoop project is actively trying to remove the contrib projects it > has, see http://tinyurl.com/6yl25jz. I doubt it's interested in any new ones. > > 2) The Hadoop project is producing a release every 2 or 3 years currently. > As a new project Howl will be wanting to release every 2 or 3 months for a > while. Being tied to something as slow moving as Hadoop for releases would > make it hard for Howl get releases out the door. > > Alan. > > On Feb 2, 2011, at 4:57 PM, Milind Bhandarkar wrote: > >> I feel that Howl should start as a contrib to Hadoop, and move to be a >> subproject of Hadoop once there is sufficient adoption, rather than going >> the incubator way. My reasons are as follows: >> >> 1. Howl is aimed at providing abstractions for facilitating interoperability >> between various systems built *on top of Hadoop*, and should not limit >> itself to Pig, Hive, and native MapReduce. So, any system that is hadoop >> compatible should be able to use Howl as a metadata store. >> >> 2. Having Howl as contrib of Hadoop will ensure that the input and output >> formats, compression codecs, underlying storage APIs etc remain in sync from >> release to release, and users do not have to worry about whether version x >> of Howl is compatible with version y of Hadoop or not. >> >> 3. Pig, Hive, Cascading, .. are all already dependent on Hadoop. Including >> Howl as Hadoop contrib means they do not add any more dependencies. >> >> 4. The roadmap of Howl includes authentication and authorization support. It >> is a standard industry practice that metadata security mechanisms match >> those for data security. Thus, a significant code can be shared with >> hadoop's authorization and authentication. >> >> 5. Hadoop-compatible file systems provide an abstraction over underlying >> storage systems. Howl currently provides a table abstraction over the file >> system. In future, when Hadoop provides blockpool abstraction (as part of >> federation), Howl will be able to take advantage of that and optimize. >> >> 6. Howl roadmap currently does not contain multi-tenancy features such as >> quotas. Since there is a strong correlation between number of tables, number >> of partitions in Howl and number of directories and files in HDFS, it could >> be streamlined if Howl is part of Hadoop. >> >> Thoughts ? >> >> - milind >> >> >> On Feb 2, 2011, at 1:18 PM, Alan Gates wrote: >> >>> Howl is a table management system built to provide metadata and storage >>> management across data processing tools in Hadoop (Pig, Hive, MapReduce, >>> ...). You can learn more details at http://wiki.apache.org/pig/Howl. For >>> the last six months the code has been hosted at github. The Howl team >>> would like to move the project into the Apache Incubator. You can see the >>> proposal for the project at http://wiki.apache.org/incubator/HowlProposal. >>> >>> In order to be accepted as an Incubator project Howl needs a Sponsoring >>> project. I propose that we, the Pig project, sponsor Howl. By sponsoring >>> Howl we are saying that we believe it is a good fit for the ASF and that we >>> will assist the Howl project to succeed. You can read full details of >>> sponsoring a project at >>> http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Sponsor. >>> >>> Our bylaws don't explicitly cover such a vote, but I think lazy majority >>> should be reasonable. All votes are welcome, PMC member votes will be >>> binding. >>> >>> Clearly I'm +1. >>> >>> Alan. >> >> --- >> Milind Bhandarkar >> [email protected] >> >> >> > --- Milind Bhandarkar [email protected]
