Hi Kengo, After my testing, I find that the version of Bigtop3.0.0's bigtop-ambari-mpack is *not* compatible with this version of Ambari (2.7.5), also is *not *compatible with the previous one (2.6.1, provided by Bigtop 1.5.0). "Ambari 2.7.5 and Mpack built from the master branch" is true. So, we should note that "In the next version of Bigtop(3.0.1), the bigtop-ambari-mpack is compatible with the version of Ambari(2.7.5, provided by Bigtop3.0.1) , but the components are provided by Bigtop 1.5.0"
peng lee. Kengo Seki <[email protected]> 于2022年3月22日周二 07:08写道： > Hi Yuqi and Peng, > > Sorry for my late response. > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 3.0 release > > -> it works and deploys Bigtop 1.5 stack > > For ensuring this, I tried to deploy a cluster using Ambari 2.7.5 and > Mpack both provided by Bigtop 3.0.0, > but unfortunately I came across the following error in the last step > of the deployment via Ambari UI. > > ``` > stderr: > Traceback (most recent call last): > File > "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", > line 39, in <module> > AfterInstallHook().execute() > File > "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", > line 352, in execute > method(env) > File > "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", > line 32, in hook > setup_stack_symlinks(self.stroutfile) > File > "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py", > line 53, in setup_stack_symlinks > stack_packages = > stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL) > File > "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", > line 185, in get_packages > raise Fail("The stack packages are not defined on the command. > Unable to load packages for the stack-select tool") > resource_management.core.exceptions.Fail: The stack packages are not > defined on the command. Unable to load packages for the stack-select > tool > ``` > > This is because Ambari 2.7.5 and Mpack requires BIGTOP-3592 and > BIGTOP-3591 respectively to work with each other. > So I assume Peng meant "Ambari 2.7.5 and Mpack built from the master > branch" (not the official 3.0.0 release), > which these changes were merged into. > > Based on the discussion so far, I've updated the Bigtop wiki. Let me > know if I misunderstand something. > https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+3.0.0+Release > (diff: > https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=191334594&selectedPageVersions=4&selectedPageVersions=3 > ) > > I'll also write a similar note when publishing the 3.0.1 release. > > Kengo Seki <[email protected]> > > On Fri, Mar 4, 2022 at 11:24 PM 李帅 <[email protected]> wrote: > > > > Oh Kengo and yuqi. It's my fault. I am so sorry about that. Actually > the > > Ambari 2.7.5 from Bigtop 3.0 and the Mpack from the Bigtop 3.0, they are > > work, but their components are from Bigtop 1.5. So > > > > > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 3.0 release > > > -> it works and deploys Bigtop 1.5 stack > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月4日周五 08:21写道： > > > > > Hi Peng, > > > > > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 1.5 release > > > > -> it works and deploys Bigtop 1.5 stack > > > > > > Thanks! To be precise, the above combination is going to deploy Bigtop > 1.4 > > > stack by default > > > (selecting "Use Public Repository" on Ambari's Web UI shows the > repository > > > URL of 1.4.0), > > > and if users rewrite the repo URL, it can deploy 1.5 stack, right? > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > On Thu, Mar 3, 2022 at 11:13 PM 李帅 <[email protected]> wrote: > > > > > > > Hi, Kengo > > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 1.5 release > > > > -> it works and deploys Bigtop 1.5 stack > > > > > > > > > > > > Kengo Seki <[email protected]> 于2022年3月3日周四 19:20写道： > > > > > > > > > > The bgtp-ambari-mpack is works well with the Ambari 2.7.5 which > come > > > > from > > > > > > Bigtop 3.0, but it's components(Hadoop and Zookeeper) are all > come > > > from > > > > > Bigtop 1.5. > > > > > > > > > > Oh I see, that's what I originally thought so. > > > > > The mpack of Bigtop 3.0 is intended to deploy Bigtop 1.5 stack, > though > > > > it's > > > > > a bit misleading :) > > > > > It should be mentioned in the release notes. > > > > > > > > > > Let me organize the combinations of the conditions. My > understanding is > > > > as > > > > > follows, are these correct? > > > > > > > > > > * Ambari 2.6.1 (from Bigtop 1.5) w/ Mpack from the 3.0.0 release > > > > > -> It works and deploys Bigtop 1.5 stack. > > > > > > > > > > * Ambari 2.6.1 (from Bigtop 1.5) w/ Mpack from the upcoming 3.0.1 > > > release > > > > > -> Same as the above. > > > > > (Or, we can merge BIGTOP-3591 into branch-3.0 before the > release. > > > > > In this case, this combination doesn't work) > > > > > > > > > > * Ambari 2.6.1 (from Bigtop 1.5) w/ Mpack from the (currently > > > unreleased) > > > > > 3.1.0 release > > > > > -> It doesn't work, since BIGTOP-3591 has already been merged. > > > > > > > > > > * Ambari 2.7.5 (from Bigtop 3.0) w/ Mpack from the 3.0.0 release > > > > > -> It doesn't work, since BIGTOP-3591 is not merged yet. > > > > > > > > > > * Ambari 2.7.5 (from Bigtop 3.0) w/ Mpack from the upcoming 3.0.1 > > > release > > > > > -> Same as the above. > > > > > (Or, we can merge BIGTOP-3591 into branch-3.0 before the 3.0.1 > > > > > release. > > > > > In this case, this combination works and deploys Bigtop 1.5.0 > > > > stack) > > > > > > > > > > * Ambari 2.7.5 (from Bigtop 3.0) w/ Mpack from the (currently > > > unreleased) > > > > > 3.1.0+ release > > > > > -> It works, and is supposed to deploy Bigtop 3.0 stack after > merging > > > > > Yuqi's upcoming PR. > > > > > But if we release 3.1.0 based on the current state of the > master > > > > > branch, it deploys Bigtop 1.5.0 stack. > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > On Thu, Mar 3, 2022 at 6:03 PM 李帅 <[email protected]> > wrote: > > > > > > > > > > > Hi Kengo, > > > > > > I am sorry that my preceding words look ambiguous. > > > > > > The bgtp-ambari-mpack is works well with the Ambari 2.7.5 which > come > > > > from > > > > > > Bigtop 3.0, but it's components(Hadoop and Zookeeper) are all > come > > > from > > > > > > Bigtop 1.5. > > > > > > thanks, > > > > > > peng lee. > > > > > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月3日周四 16:32写道： > > > > > > > > > > > > > Thanks for the comment, Yuqi and Lee! (sorry if I mistook your > > > name) > > > > > > > I'll update the release note later. > > > > > > > > > > > > > > Let me make sure just one point, Bigtop 1.5 provides Ambari > 2.6.1 > > > > (not > > > > > > > 2.7.5), > > > > > > > so I understood "mpack works with Ambari 2.7.5 installed via > Bigtop > > > > > 1.5" > > > > > > in > > > > > > > your comments > > > > > > > is actually "... Ambari 2.6.1 installed via Bigtop 1.5". > Correct me > > > > if > > > > > > I'm > > > > > > > wrong :) > > > > > > > > > > > > > > > I'd like to start to work on it and add the Mpack Bigtop-3.0 > > > > support > > > > > on > > > > > > > the > > > > > > > next minor release. > > > > > > > > > > > > > > Great, I'm looking forward to it! I'll try it once the PR is > > > > submitted. > > > > > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 3, 2022 at 2:52 PM 李帅 <[email protected]> > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > Actually, bigtop-ambari-mpack provided as a > > > > > > > > component of Bigtop 3.0 > > > > > > > > works with Ambari 2.7.5 installed via Bigtop 1.5, but not > with > > > > Ambari > > > > > > > > 2.7.5/Bigtop 3.0. > > > > > > > > thanks. > > > > > > > > > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月3日周四 07:19写道： > > > > > > > > > > > > > > > > > Thank you for the comment! > > > > > > > > > Let me confirm, you mean that bigtop-ambari-mpack which is > > > > provided > > > > > > as > > > > > > > a > > > > > > > > > component of Bigtop 3.0 > > > > > > > > > works with Ambari 2.6.1 installed via Bigtop 1.5, but not > with > > > > > Ambari > > > > > > > > > 2.7.5/Bigtop 3.0, right? > > > > > > > > > If my understanding is correct, that information is > significant > > > > and > > > > > > I'm > > > > > > > > > going to reflect it to the release note. > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+3.0.0+Release > > > > > > > > > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 2, 2022 at 12:48 PM 李帅 < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > > > hi，Maybe we should note that the bgtp-ambari-mpack is not > > > work > > > > > well > > > > > > > > with > > > > > > > > > > bigtop3.x, but it can work well with bigtop 1.5 > > > > > > > > > > > > > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月1日周二 21:25写道： > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > As discussed in [1], I'd like to publish the 3.0.1 > release > > > > > based > > > > > > on > > > > > > > > > > > branch-3.0 soon, > > > > > > > > > > > which mainly focuses on addressing the Log4Shell > > > > > vulnerabilities. > > > > > > > > > > > > > > > > > > > > > > I've created all CI jobs which are required to that > > > release, > > > > > > > > > > > but some of the CI nodes are currently blacklisted > from the > > > > ASF > > > > > > > > sites. > > > > > > > > > > > Luca has already asked the ASF Infra to remove that > > > > restriction > > > > > > > [2], > > > > > > > > > > > but they still seem to be banned as of now. > > > > > > > > > > > > > > > > > > > > > > Once they become available again, I'm going to start > the > > > > > release > > > > > > > > > process. > > > > > > > > > > > (but it will take a few weeks due to long > > > build/test/transfer > > > > > > time, > > > > > > > > as > > > > > > > > > > > usual) > > > > > > > > > > > So let me know if there are any fixes that you'd like > to > > > > > include > > > > > > > into > > > > > > > > > it. > > > > > > > > > > > > > > > > > > > > > > [1]: > > > > > > > > > https://lists.apache.org/thread/vor809oqqcjxso98nxrml5s3lthrlws5 > > > > > > > > > > > [2]: > > > > > > > > > > > > > https://issues.apache.org/jira/browse/BIGTOP-3644#comment-17498543 > > > > > > > > > > > > > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
