Hi Peng, Thank you so much for testing!
> I find that the version of Bigtop3.0.0's bigtop-ambari-mpack is *not* > compatible with this version of Ambari (2.7.5), also is *not* compatible with > the previous one (2.6.1, provided by Bigtop 1.5.0). I also confirmed that combination doesn't work. The Bigtop stack doesn't appear in the candidate list though the Mpack is successfully imported. > "In the next version of Bigtop(3.0.1), the bigtop-ambari-mpack is compatible > with the version of Ambari(2.7.5, provided by Bigtop 3.0.1), but the > components are provided by Bigtop 1.5.0" To make it true, we have to merge BIGTOP-3591 and BIGTOP-3592 into branch-3.0 (to be released as 3.0.1) in addition to master. Considering that we currently don't have any usable combination of Ambari and Mpack in the 3.0.0 release, we should do that. In addition, I found some problems about Ambari and Mpack on the master branch. I'll file them into JIRA later. Kengo Seki <[email protected]> On Wed, Mar 23, 2022 at 2:07 PM 李帅 <[email protected]> wrote: > > Hi Kengo, > After my testing, I find that the version of Bigtop3.0.0's > bigtop-ambari-mpack is *not* compatible with this version of Ambari > (2.7.5), also is *not *compatible with the previous one (2.6.1, provided by > Bigtop 1.5.0). > "Ambari 2.7.5 and Mpack built from the master branch" is true. > So, we should note that "In the next version of Bigtop(3.0.1), the > bigtop-ambari-mpack is compatible with the version of Ambari(2.7.5, > provided by Bigtop3.0.1) , but the components are provided by Bigtop 1.5.0" > > peng lee. > > Kengo Seki <[email protected]> 于2022年3月22日周二 07:08写道： > > > Hi Yuqi and Peng, > > > > Sorry for my late response. > > > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 3.0 release > > > -> it works and deploys Bigtop 1.5 stack > > > > For ensuring this, I tried to deploy a cluster using Ambari 2.7.5 and > > Mpack both provided by Bigtop 3.0.0, > > but unfortunately I came across the following error in the last step > > of the deployment via Ambari UI. > > > > ``` > > stderr: > > Traceback (most recent call last): > > File > > "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", > > line 39, in <module> > > AfterInstallHook().execute() > > File > > "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", > > line 352, in execute > > method(env) > > File > > "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/hook.py", > > line 32, in hook > > setup_stack_symlinks(self.stroutfile) > > File > > "/var/lib/ambari-agent/cache/stack-hooks/after-INSTALL/scripts/shared_initialization.py", > > line 53, in setup_stack_symlinks > > stack_packages = > > stack_select.get_packages(stack_select.PACKAGE_SCOPE_INSTALL) > > File > > "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", > > line 185, in get_packages > > raise Fail("The stack packages are not defined on the command. > > Unable to load packages for the stack-select tool") > > resource_management.core.exceptions.Fail: The stack packages are not > > defined on the command. Unable to load packages for the stack-select > > tool > > ``` > > > > This is because Ambari 2.7.5 and Mpack requires BIGTOP-3592 and > > BIGTOP-3591 respectively to work with each other. > > So I assume Peng meant "Ambari 2.7.5 and Mpack built from the master > > branch" (not the official 3.0.0 release), > > which these changes were merged into. > > > > Based on the discussion so far, I've updated the Bigtop wiki. Let me > > know if I misunderstand something. > > https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+3.0.0+Release > > (diff: > > https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=191334594&selectedPageVersions=4&selectedPageVersions=3 > > ) > > > > I'll also write a similar note when publishing the 3.0.1 release. > > > > Kengo Seki <[email protected]> > > > > On Fri, Mar 4, 2022 at 11:24 PM 李帅 <[email protected]> wrote: > > > > > > Oh Kengo and yuqi. It's my fault. I am so sorry about that. Actually > > the > > > Ambari 2.7.5 from Bigtop 3.0 and the Mpack from the Bigtop 3.0, they are > > > work, but their components are from Bigtop 1.5. So > > > > > > > > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 3.0 release > > > > -> it works and deploys Bigtop 1.5 stack > > > > > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月4日周五 08:21写道： > > > > > > > Hi Peng, > > > > > > > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 1.5 release > > > > > -> it works and deploys Bigtop 1.5 stack > > > > > > > > Thanks! To be precise, the above combination is going to deploy Bigtop > > 1.4 > > > > stack by default > > > > (selecting "Use Public Repository" on Ambari's Web UI shows the > > repository > > > > URL of 1.4.0), > > > > and if users rewrite the repo URL, it can deploy 1.5 stack, right? > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > On Thu, Mar 3, 2022 at 11:13 PM 李帅 <[email protected]> wrote: > > > > > > > > > Hi, Kengo > > > > > * Ambari 2.7.5(from Bigtop 3.0) w/ Mpack from the 1.5 release > > > > > -> it works and deploys Bigtop 1.5 stack > > > > > > > > > > > > > > > Kengo Seki <[email protected]> 于2022年3月3日周四 19:20写道： > > > > > > > > > > > > The bgtp-ambari-mpack is works well with the Ambari 2.7.5 which > > come > > > > > from > > > > > > > Bigtop 3.0, but it's components(Hadoop and Zookeeper) are all > > come > > > > from > > > > > > Bigtop 1.5. > > > > > > > > > > > > Oh I see, that's what I originally thought so. > > > > > > The mpack of Bigtop 3.0 is intended to deploy Bigtop 1.5 stack, > > though > > > > > it's > > > > > > a bit misleading :) > > > > > > It should be mentioned in the release notes. > > > > > > > > > > > > Let me organize the combinations of the conditions. My > > understanding is > > > > > as > > > > > > follows, are these correct? > > > > > > > > > > > > * Ambari 2.6.1 (from Bigtop 1.5) w/ Mpack from the 3.0.0 release > > > > > > -> It works and deploys Bigtop 1.5 stack. > > > > > > > > > > > > * Ambari 2.6.1 (from Bigtop 1.5) w/ Mpack from the upcoming 3.0.1 > > > > release > > > > > > -> Same as the above. > > > > > > (Or, we can merge BIGTOP-3591 into branch-3.0 before the > > release. > > > > > > In this case, this combination doesn't work) > > > > > > > > > > > > * Ambari 2.6.1 (from Bigtop 1.5) w/ Mpack from the (currently > > > > unreleased) > > > > > > 3.1.0 release > > > > > > -> It doesn't work, since BIGTOP-3591 has already been merged. > > > > > > > > > > > > * Ambari 2.7.5 (from Bigtop 3.0) w/ Mpack from the 3.0.0 release > > > > > > -> It doesn't work, since BIGTOP-3591 is not merged yet. > > > > > > > > > > > > * Ambari 2.7.5 (from Bigtop 3.0) w/ Mpack from the upcoming 3.0.1 > > > > release > > > > > > -> Same as the above. > > > > > > (Or, we can merge BIGTOP-3591 into branch-3.0 before the 3.0.1 > > > > > > release. > > > > > > In this case, this combination works and deploys Bigtop 1.5.0 > > > > > stack) > > > > > > > > > > > > * Ambari 2.7.5 (from Bigtop 3.0) w/ Mpack from the (currently > > > > unreleased) > > > > > > 3.1.0+ release > > > > > > -> It works, and is supposed to deploy Bigtop 3.0 stack after > > merging > > > > > > Yuqi's upcoming PR. > > > > > > But if we release 3.1.0 based on the current state of the > > master > > > > > > branch, it deploys Bigtop 1.5.0 stack. > > > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > > > > On Thu, Mar 3, 2022 at 6:03 PM 李帅 <[email protected]> > > wrote: > > > > > > > > > > > > > Hi Kengo, > > > > > > > I am sorry that my preceding words look ambiguous. > > > > > > > The bgtp-ambari-mpack is works well with the Ambari 2.7.5 which > > come > > > > > from > > > > > > > Bigtop 3.0, but it's components(Hadoop and Zookeeper) are all > > come > > > > from > > > > > > > Bigtop 1.5. > > > > > > > thanks, > > > > > > > peng lee. > > > > > > > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月3日周四 16:32写道： > > > > > > > > > > > > > > > Thanks for the comment, Yuqi and Lee! (sorry if I mistook your > > > > name) > > > > > > > > I'll update the release note later. > > > > > > > > > > > > > > > > Let me make sure just one point, Bigtop 1.5 provides Ambari > > 2.6.1 > > > > > (not > > > > > > > > 2.7.5), > > > > > > > > so I understood "mpack works with Ambari 2.7.5 installed via > > Bigtop > > > > > > 1.5" > > > > > > > in > > > > > > > > your comments > > > > > > > > is actually "... Ambari 2.6.1 installed via Bigtop 1.5". > > Correct me > > > > > if > > > > > > > I'm > > > > > > > > wrong :) > > > > > > > > > > > > > > > > > I'd like to start to work on it and add the Mpack Bigtop-3.0 > > > > > support > > > > > > on > > > > > > > > the > > > > > > > > next minor release. > > > > > > > > > > > > > > > > Great, I'm looking forward to it! I'll try it once the PR is > > > > > submitted. > > > > > > > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 3, 2022 at 2:52 PM 李帅 <[email protected]> > > > > wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > Actually, bigtop-ambari-mpack provided as a > > > > > > > > > component of Bigtop 3.0 > > > > > > > > > works with Ambari 2.7.5 installed via Bigtop 1.5, but not > > with > > > > > Ambari > > > > > > > > > 2.7.5/Bigtop 3.0. > > > > > > > > > thanks. > > > > > > > > > > > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月3日周四 07:19写道： > > > > > > > > > > > > > > > > > > > Thank you for the comment! > > > > > > > > > > Let me confirm, you mean that bigtop-ambari-mpack which is > > > > > provided > > > > > > > as > > > > > > > > a > > > > > > > > > > component of Bigtop 3.0 > > > > > > > > > > works with Ambari 2.6.1 installed via Bigtop 1.5, but not > > with > > > > > > Ambari > > > > > > > > > > 2.7.5/Bigtop 3.0, right? > > > > > > > > > > If my understanding is correct, that information is > > significant > > > > > and > > > > > > > I'm > > > > > > > > > > going to reflect it to the release note. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+3.0.0+Release > > > > > > > > > > > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 2, 2022 at 12:48 PM 李帅 < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > hi，Maybe we should note that the bgtp-ambari-mpack is not > > > > work > > > > > > well > > > > > > > > > with > > > > > > > > > > > bigtop3.x, but it can work well with bigtop 1.5 > > > > > > > > > > > > > > > > > > > > > > Kengo Seki <[email protected]> 于 2022年3月1日周二 21:25写道： > > > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > > > As discussed in [1], I'd like to publish the 3.0.1 > > release > > > > > > based > > > > > > > on > > > > > > > > > > > > branch-3.0 soon, > > > > > > > > > > > > which mainly focuses on addressing the Log4Shell > > > > > > vulnerabilities. > > > > > > > > > > > > > > > > > > > > > > > > I've created all CI jobs which are required to that > > > > release, > > > > > > > > > > > > but some of the CI nodes are currently blacklisted > > from the > > > > > ASF > > > > > > > > > sites. > > > > > > > > > > > > Luca has already asked the ASF Infra to remove that > > > > > restriction > > > > > > > > [2], > > > > > > > > > > > > but they still seem to be banned as of now. > > > > > > > > > > > > > > > > > > > > > > > > Once they become available again, I'm going to start > > the > > > > > > release > > > > > > > > > > process. > > > > > > > > > > > > (but it will take a few weeks due to long > > > > build/test/transfer > > > > > > > time, > > > > > > > > > as > > > > > > > > > > > > usual) > > > > > > > > > > > > So let me know if there are any fixes that you'd like > > to > > > > > > include > > > > > > > > into > > > > > > > > > > it. > > > > > > > > > > > > > > > > > > > > > > > > [1]: > > > > > > > > > > > https://lists.apache.org/thread/vor809oqqcjxso98nxrml5s3lthrlws5 > > > > > > > > > > > > [2]: > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/BIGTOP-3644#comment-17498543 > > > > > > > > > > > > > > > > > > > > > > > > Kengo Seki <sekikn@ <[email protected]>apache.org> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
