Hi Mikhail,
Let's discuss this one-by-one: puppet versioning is a long standing problem, since every distribution had its own version of puppet bundled. Additionially the packages from puppetlabs do not support all the range of architectures. I doublechecked and it seems puppetlabs now supports ppc64le starting with puppet5, but still misses to support aarch64 aka arm64 . http://apt.puppetlabs.com/pool/xenial/puppet5/p/puppet-agent/ <http://apt.puppetlabs.com/pool/xenial/puppet5/p/puppet-agent/> Mixing puppetlabs and OS supplied packages on different platforms poses a problem: Binaries and Config Files are in different places, creating a huge task to write and test platform independent scripts. So let's stick with OS supplied packages, since these support all archs (amd64/ppc64le/aarch64) OS & prepackaged puppet Debian 9 | 4.8.2 ubuntu 16.04 | 3.8.5 fedora 26 | 4 ? *** EOL *** rhel-7 / epel | 3.6.2 opensuse 42.3 | 3.8.7 opensuse 15 | ----- puppet not included any more --- Right now we are struggeling to get the mixed puppet3 and puppet4 systems running NEW: ubuntu 18.04 | puppet 5.4.0 NEW: fedora 29 | 5.5.1 NEW: Debian buster (testing) | 5.5.6 Supporting fedora > 26 is complicated since fedora includes an own hadoop distribution incompatble with bigtop. Debian buster is still testing with no release date announced AFAIK. IMHO puppet5 can only be done if we drop fedora (because of its own hadoop), opensuse (lack of puppet5), ubunt16.04 (lack of puppet5 for aarch64), move to Debian testing and use puppetlab's puppet5 for RHEL-7 . We are dropping a lot of problems--and customers :-) Do you have suggestions how to mitigate this problem? Olaf > Am 05.10.2018 um 19:46 schrieb Evans Ye <[email protected]>: > > Hi Mikhail, > > Thank you for sharing your survey of supporting Ubuntu 18.04 in bigtop. I’d > like to take a step back and ask some really dumb questions if you dont > mind ;) > > 1. What’s the trigger that you wanna have support for Ubuntu 18.04? Is it > easier to make a shift to Ubuntu 18.04 if we wait for it to be more mature? > 2. The refactor is a total package. So if we upgrade the whole build system > for Ubuntu 18.04. Does the Debian side or even the RPM side gonna be > compatible for those OS specific tools? > > Evans > > Mikhail Epikhin <[email protected]>於 2018年10月5日 週五，下午2:03寫道： > >> Hi folks! >> >> I wrote this post on github https://github.com/apache/bigtop/pull/374, >> but, it will be better to move this discussion to mail. >> >> TL;DR >> Just before support ubuntu 18.04 we should do some major things, and >> before doing it we should reach an agreement: >> >> * Support puppet 5, current version is 3. >> * Rewrite templating and code generation of initrd scripts and support >> systemd services. >> * Use aptly insteadof reprepro for debian-based distros? >> >> Okey, now longread: >> >> Ubuntu 18.04 supports only puppet 5, but in bigtop we are using puppet 3. >> I've tried to using manifests from third version, but puppet fifth have new >> keyword like site, and some configs clashes with puppet 5. I fixed that, >> but, i don't know, maybe some problems are still there. Living with >> manifests for two versions is not good idea, so we need to choice one of >> them and migrate all code to newest deploy system. >> >> Ubuntu 18.04 have magic converting tool from sysvinit to systemd services, >> but it doesn't work with current initd scritps. I successfully tried to >> write some of systemd services, but this way have a couple of shades: >> a) We should generate this services as we generate initd services, but the >> current way is a bit of weird. A lot of bash templating, and current "bash >> templates" a not usable for new systemd services, just because the same >> things in both ways are very different. Working with locks, pids, >> controlling aliveness of a process, controlling user and group. I've tried >> to change current bash templates, but the code looks ugly. >> b) A lot of components. And each components have a lot of services. I just >> can't write a templating script for all of this scripts, because every >> service have own specific things in bash methods, and u cannot convert it >> automatically, only by reading your eyes. >> c) Current initrd scripts have custom features like rollingupgrade and >> init in hdfs-namenode. Systemd doesn't support that, and the right ways is >> -- create new systemd service only for that! For each custom feature and >> action -- new systemd service. A lot of services. >> >> This is not blocker issue, but the execution of ./gradlew apt on builded >> components doesn't support *.ddeb artifacts. Newest reprepro doesn't >> support this files, and the problem looks like this >> https://github.com/ros-infrastructure/buildfarm_deployment/issues/186 and >> we have three ways: >> a) skip these artifacts, weird way; >> b) use new version of reprepro not from ubuntu repositories, which is also >> weird; >> c) use tool like https://www.aptly.info, which is more supportable and >> powerful in features. >> >> If anyone have a time for support 18.04 it would be great for doing it >> together with splitting this pull request to sub tasks and discussion it. >> >> So, anyone?:) >> >> --- >> Mikhail Epikhin >>
