Thanks bruno : thats a good point i think. so Ive created a spreadsheet we can use to track who is maintaining what in bigtop:
https://docs.google.com/spreadsheets/d/1ohjbkbLLP7NzOoZEyfhOkCmBSjUcOo8hmuFpkwbE4KM/ Can you folks help me to fill it out ? >From there then we will have simple, structured , data driven way to define the BOM. On Thu, Oct 16, 2014 at 3:42 AM, Bruno Mahé <[email protected]> wrote: > I would still keep Ubuntu/OpenSuse. > As nice as docker is, people still deploy Apache Hadoop in many different > ways. > > Regarding Apache Flume, what is the issue? > We are using it at work and are happy with it. It just works. > I submitted a patch to Apache Bigtop to improve a small thing in the last > month or so, but aside from that, it (again) just works. > > Generally speaking, I would keep the work and process as simple as > possible. > We can also look at what the GNU/Linux distributions are doing. > > Thanks, > Bruno > > > On 10/15/2014 09:09 PM, Konstantin Boudnik wrote: > >> I am not sure we can drop Ubuntu all together - after all it seems to be >> the >> most used dev platform (and may be more?). >> >> We may request the help from the upstreams, hopefully it will work. >> >> Patching upstream will require a lot of QE resources from our community. >> Do we >> have it? Same with refs to the upstream trunks - in my experience trunks >> are >> usually a pile of commits which might or might not be even compilable. >> Hence, >> we'll have to do all the QE at the bigtop focal point. I am personally >> won't >> subscribe for that. >> >> If the community decides that we'd better keep in Pig, Flume, etc - I am >> totally fine with that. However, 0.8.0 experience demonstrated that >> there's >> not much help in keeping these alive. Am I misreading something? >> >> Cos >> >> On Tue, Oct 14, 2014 at 04:52PM, Ruijing Guo wrote: >> >>> 1. I am thinking bigtop may start to support Centos 7 docker image and >>> deprecate Ubuntu/OpenSuse. docker image may be unified package for >>> bigtop. >>> Bigtop can reduce effort to support different platform and add more >>> effort >>> to resolve incompatibility of upstream. >>> >>> 2. It is right way/goal for upstream to manage package. Bigtop may >>> request >>> one committer from upstream to join bigtop until bigtop contribute >>> package >>> to upstream. >>> >>> 3. bigtop may add distribution reference as goal so that bigtop can >>> attract >>> more developers/vendors to join the project: >>> >>> *The primary goal of Bigtop is to build a community around the packaging, >>> interoperability testing and redistribution of Hadoop-related projects.* >>> >>> BigTop may patch from upstream and Bigtop cannot be released with build >>> or >>> security issues. >>> >>> 4. Most distribution still includes pig, mahout and flume. If bigtop >>> retire >>> these component, I am afraid that more developers/vendors may leave >>> bigtop >>> project. >>> >>> 5. bigtop may start to resolve incompatibility of upstream. Bigtop may >>> refer to upstream trunk branch. If build break due to upstream, bigtop >>> may >>> create blocker JIRA in upstream. >>> >>> On Tue, Oct 14, 2014 at 12:09 PM, Jay Vyas <[email protected]> >>> wrote: >>> >>> To simplify our lives I guess we can just produce packages for the 3 >>>> most >>>> common distros...?Is that a good idea? I.e. Centos 7, Ubuntu 14, and >>>> openSuse 13. >>>> >>>> Or are others important? >>>> Just thinking out loud don't want to leave anyone out! >>>> >>>> On Oct 12, 2014, at 11:25 PM, Roman Shaposhnik <[email protected]> >>>>> >>>> wrote: >>>> >>>>> On Sat, Oct 11, 2014 at 6:58 PM, Konstantin Boudnik <[email protected]> >>>>>> >>>>> wrote: >>>> >>>>> Looks like it is going slow. so I will continue. To start with... >>>>>> >>>>>> I am proposing the following set of supported platforms (similar to >>>>>> >>>>> last time) >>>> >>>>> OS: >>>>>> CentOS6, CentOS7, Fedora 20 >>>>>> >>>>> Should we be really targeting both? >>>>> >>>>> SLES12, OpenSUSE 13.1 >>>>>> Ubuntu 14.04 LTS >>>>>> >>>>>> Java: >>>>>> OpenJDK 7 >>>>>> >>>>>> Components: >>>>>> >>>>>> Zookeeper 3.4.6 (or later?) >>>>>> Hadoop 2.6.x >>>>>> HBase 0.98.x (latest; I am not sure if 1.0 will be ready in the 2 >>>>>> >>>>> months?) >>>> >>>>> That's a great question for Andrew. >>>>> >>>>> Hive 0.14 >>>>>> Sqoop 1.99.4 >>>>>> Oozie 4.0.1 >>>>>> Giraph 1.1.0 >>>>>> Groovy 2.3 >>>>>> Hue 3.6.0 >>>>>> DataFU 1.0.0 >>>>>> Solr 4.6.0 >>>>>> Crunch 0.10.0 >>>>>> Spark 1.1.0 >>>>>> Phoenix 4.1.0 >>>>>> Tomcat 6.0.36 >>>>>> jsvc 1.0.15 >>>>>> What other new stuff do we feel like adding? >>>>>> >>>>> This looks like a reasonable list. How about we put it into the JIRA >>>>> as a current plan of record? >>>>> >>>>> To retire: >>>>>> Pig (is there enough interest in the community to keep it on?) >>>>>> >>>>> I'd rather keep it. >>>>> >>>>> Whirr 0.8.2 (seems to be headed to the Attic) >>>>>> >>>>> +1 to dropping it >>>>> >>>>> Mahout 0.9 (unless we have a version working w/ Hadoop 2) >>>>>> >>>>> It seems Mahout has migrated to SPARK 100%. If that >>>>> works -- I'd be interested in maintaining it. >>>>> >>>>> Flume 1.5.0.1 (the project seems to be winding down?) >>>>>> >>>>> Flume still seems to be pretty widely used and historically >>>>> it hasn't caused us that much trouble. Keeping? >>>>> >>>>> Thanks, >>>>> Roman. >>>>> >>>> >>> >>> -- >>> Thanks, >>> -Ruijing >>> >> > -- jay vyas
