Thanks for the comments, everyone! If there's still no objection in a few days, I'm going to update BIGTOP-3123's description.
> 'contrib' module will be easier for us to maintain traditional distros and > cnb. Agreed. I think that merging the cnb branch later will be a hard work too. > What do you think about the components? > Is there a list of components you'd like to upgrade? I'd like to upgrade the following components: - Zookeeper: 3.4.13 - Hadoop: 3.2.1 (or 2.10.0 if packaging Hadoop 3 is too hard, as Olaf mentioned before) - HBase: 2.2.2 - Hive: 3.1.2 - Tez: 0.9.2 - Spark: 2.4.4 (or 3.0.0, if GA is released before long) - Phoenix: 5.0.0 - Kafka: 2.3.1 - Ignite: 2.7.6 - Zeppelin: 0.8.2 My Teammates and I are trying to package them, and all of them are successfully built anyway. But we have not tested them yet, and I'm sure many problems will be found from now, just as Olaf already came across on Hadoop 3... :) > the community was lean to the direction of having important component better > supported > instead of spending resources for 20~30 components. Totally agreed. If we succeed to package Hadoop 3, I'd like to drop inactive components which can't be built with it, at least for now. > Just one concern about the puppet recipes compatibility across multiple > puppet versions Exactly. I think it's difficult to support Puppet 3, 4 and 5 with a single manifest or config file, so I'm thinking to create a new "puppet5" directory beside the existing "puppet" directory and put the manifests and config files for Puppet 5 into it (and when we drop the distros using Puppet 3 and 4 completely in the future, we can drop the existing "puppet" directory and promote "puppet5" to "puppet"). If it doesn't work as expected, I'll ask you the possibility to drop old versions in the next release again. > one source of problems was using puppet-forge for instance for puppet-stdlib > and puppet-apt, Yeah, puppet modules seem to be installed into /usr/share/puppet/modules via apt on Debian 9 and Ubuntu 16.04, while /etc/puppet/modules via `puppet module install` on CentOS 7, as you said. Maybe we have to consolidate them somehow, or specify both of them for `--modulepath` (I'm not sure if it works though), or choose either of them in accordance with the distro. Kengo Seki <[email protected]> On Thu, Nov 21, 2019 at 3:17 PM Olaf Flebbe <[email protected]> wrote: > > hi > > one source of problems was using puppet-forge for instance for puppet-stdlib > and puppet-apt, since they require rather new versions. look out for 'puppet > module install ....'. While all distros using apt do have matching > prepackaged versions in their repository. > > other was different search paths of all these versions. we never fixed that > consistently. > > olaf > > Von meinem iPad gesendet > > > Am 21.11.2019 um 03:43 schrieb Jun HE <[email protected]>: > > > > I'm fine with the new distros list. Just one concern about the puppet > > recipes compatibility across multiple puppet versions (3.8.5 for > > Ubuntu-16.04, 4.8.2 for Debian-9, and 5.x for other new distros). I didn't > > do any investigation yet. If such issues arise, I'll vote for drop distros > > with older puppet. > > > > Evans Ye <[email protected]> 于2019年11月21日周四 上午1:51写道： > > > >> Fine by me for the OS side. > >> What do you think about the components? Is there a list of components you'd > >> like to upgrade? > >> We can target a subset of current supported matrix as we previously > >> discussed about this and the community was lean to the direction of having > >> important component better supported instead of spending resources for > >> 20~30 components. > >> > >> Youngwoo Kim (김영우) <[email protected]> 於 2019年11月20日 週三 上午9:41寫道： > >> > >>> Kengo, > >>> > >>> Looks good to me. I think puppet on CentOS 8 would be fine. > >>> > >>> On Cloud Native Bigtop, I believe we should consider that components as a > >>> 'contrib' at this point. > >>> I'm considering about Jay's idea, making 'CNB' on master as a contrib > >>> module. A development branch is good but on our "two-tracks" development, > >>> 'contrib' module will be easier for us to maintain traditional distros > >> and > >>> cnb. > >>> > >>> Thanks, > >>> Youngwoo > >>> > >>>> On Wed, Nov 20, 2019 at 9:28 AM Kengo Seki <[email protected]> wrote: > >>> > >>>> Hi folks, > >>>> > >>>> I'd like to discuss the target distros for the next 1.5.0 release [1], > >>>> because over 1.5 years have passed since Ubuntu 18.04 was released > >>>> and the next LTS will be released within half a year. In addition, > >>>> Fedora 26 and openSUSE 42.3 have already been EOL'd. > >>>> > >>>> (I understand the "Cloud Native Bigtop" project is going on > >>>> and am really looking forward to it, but my customers still requires > >>>> the traditional software stack :) > >>>> > >>>> Based on the past discussion [2], here's my proposal: > >>>> > >>>> - Add Debian 10, Fedora 31 and Ubuntu 18.04 as the target distros > >>>> and use the puppet package provided by each distro, so that > >>>> we can support all CPU architectures (x86_64, aarch64, and ppc64le). > >>>> Their puppet versions are 5.4.0 (ubuntu) and 5.5.10 (debian and > >>> fedora). > >>>> > >>>> Keep Debian 9 and Ubuntu 16.04 since they are still in the support > >>>> period. > >>>> > >>>> Drop Fedora 26 since it has reached to the EOL on 2018-05-29. > >>>> > >>>> - Add CentOS 8. Unfortunately, that version doesn't seem to > >>>> provide the distro's puppet package, even including EPEL. > >>>> Even though, I'd like to support it since that distro > >>>> (and RHEL8) are widely used especially in enterprise systems. > >>>> So, as the next best option, how about using Puppet 5.5 provided by > >>>> Puppetlabs and only supporting the x86_64 architecture on this > >> version? > >>>> > >>>> Keep CentOS 7 since it's still in the support period. > >>>> > >>>> - Drop openSUSE 42.3 since it has reached to the EOL on 2019-07-01 > >>>> and don't add a new version of that distro, as discussed in [2]. > >>>> > >>>> To summarize the above, the supported distros and their versions > >>>> in the 1.5.0 release are as follows: > >>>> > >>>> - CentOS 7, 8 (8 is only supported on x86_64) > >>>> - Debian 9, 10 > >>>> - Fedora 31 > >>>> - Ubuntu 16.04, 18.04 > >>>> > >>>> Does this sound reasonable? I'd appreciate any comments or suggestions. > >>>> > >>>> (Honestly, I'd actually like to drop CentOS 7, Debian 9, and Ubuntu > >>> 16.04, > >>>> so that we can consolidate the Puppet version to 5.x. > >>>> But it may be too aggressive for users.) > >>>> > >>>> [1]: https://issues.apache.org/jira/browse/BIGTOP-3123 > >>>> [2]: > >>>> > >>> > >> https://lists.apache.org/thread.html/26e14cf36e9cfd61e0de581ed83bf305565c2e65234f1ce3bfb97628@%3Cdev.bigtop.apache.org%3E > >>>> > >>>> Kengo Seki <[email protected]> > >>>> > >>> > >>
