Correction sorry - the oozie packages are ok, a downgrade was raised when installing the bigtop 1.5 packages since on bigtop 1.4 I have a version of oozie (4.3.0-3) with some extra patches in it (IIRC due to https://issues.apache.org/jira/browse/BIGTOP-3330).
Luca On Mon, Dec 14, 2020 at 2:28 AM Kengo Seki <[email protected]> wrote: > > Luca, > > Thank you for the detailed checks! > That's really helpful and informative, so I added some of your > findings to the cwiki. > https://cwiki.apache.org/confluence/display/BIGTOP/Overview+of+Bigtop+1.5.0+Support+Matrix > > Kengo Seki <[email protected]> > > On Sun, Dec 13, 2020 at 5:40 PM Luca Toscano <[email protected]> wrote: > > > > On Mon, Dec 7, 2020 at 12:47 AM Kengo Seki <[email protected]> wrote: > > > > > > This is the vote for release 1.5.0 of Apache Bigtop. > > > > > > [x] +1, accept RC0 as the official 1.5.0 release of Apache Bigtop > > > > (Community vote) > > > > I completed my tests, the new release didn't raise any blocking issue. > > My tests were for this use case: > > - Hadoop test cluster with Hive/Oozie/Spark/Jupyter/Hue on bare metal hosts. > > - HDFS and Yarn HA setup (including journal nodes). > > - Bigtop 1.4 for Debian 9 for the majority of the nodes, and a client > > node on Debian 10. > > - In place upgrade from Bigtop 1.4 to 1.5 (upgrading the packages > > without removing the old ones first). > > > > I have tested an in place upgrade, trying to rollback/re-deploy before > > the HDFS finalize step, all good. Some interesting things that I > > noticed or had to fix in my config after the upgrade: > > - the HDFS Namenodes seem to be smarter when dealing with a rollback. > > While testing CDH -> Bigtop 1.4 (hdfs 2.6.0 -> 2.8.5) I had to follow > > the upstream guidelines and explicitly issue commands to rollback the > > hdfs state, but when testing this release it was not needed anymore. > > I'll dig a big more into this, but overall the upgrade/rollback > > procedure seemed smoother. > > - our Hadoop clusters never used the > > "yarn.resourcemanager.webapp.address.$rm-id" settings in yarn-site.xml > > (that seem to have a good default), but on Bigtop 1.5 it lead to NPE > > while running map-reduce jobs, like described in > > https://issues.apache.org/jira/browse/YARN-8056. Once I set the value, > > everything worked as expected. > > - while upgrading the packages, I had a little hiccup with oozie since > > the version on Bigtop 1.4 is higher than 1.5 (4.3.0-3 vs 4.3.0-1), so > > it needed a downgrade. Nothing really major, just mentioning it. > > > > About https://dist.apache.org/repos/dist/dev/bigtop/bigtop-1.5.0-RC0: > > - verified .asc/.shaXXX, all good. > > - all debian packages (9 and 10 versions) worked fine, and Kengo > > Seki's gpg signature was ok as well. > > > > Thanks a lot for this release! > > > > Luca
