Luca, Thank you for the detailed checks! That's really helpful and informative, so I added some of your findings to the cwiki. https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix
Kengo Seki <[email protected]> On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <[email protected]> wrote: > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <[email protected]> wrote: > > > > This is the vote for release 1.5.0 of Apache Bigtop. > > > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop > > (Community vote) > > I completed my tests, the new release didn't raise any blocking issue. > My tests were for this use case: > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal hosts. > - HDFS and Yarn HA setup (including journal nodes). > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client > node on Debian 10. > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages > without removing the old ones first). > > I have tested an in place upgrade, trying to rollback/re-deploy before > the HDFS finalize step, all good. Some interesting things that I > noticed or had to fix in my config after the upgrade: > - the HDFS Namenodes seem to be smarter when dealing with a rollback. > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow > the upstream guidelines and explicitly issue commands to rollback the > hdfs state, but when testing this release it was not needed anymore. > I'll dig a big more into this, but overall the upgrade/rollback > procedure seemed smoother. > - our Hadoop clusters never used the > "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE > while running map-reduce jobs, like described in > https://issues.apache.org/jira/browse/YARN-8056. Once I set the value, > everything worked as expected. > - while upgrading the packages, I had a little hiccup with oozie since > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so > it needed a downgrade. Nothing really major, just mentioning it. > > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0: > - verified .asc/.shaXXX, all good. > - all debian packages (9 and 10 versions) worked fine, and Kengo > Seki's gpg signature was ok as well. > > Thanks a lot for this release! > > Luca
