> Multiple versions of the same package (deb/rpm) can not be installed at the > same time. I dn't think, we'll required to install multiple versions same time. Even this can be handled. Anyway Bigtop will be used to create the RPM's not for deploying the cluster with ambari now.
> I feel hdp-select/bdp-select should belongs to Ambari > since it is assuming the convention specific to HDP/BDP/Ambari. > We can easily add packaging for the code under Ambari > (as ambari-bdp-select for example) based on the existing stuff under > bigtop-packages/*/ambari. IMO, as components rpm's are build with bigtop, having it in bigtop will be perfect choice. As we need to change the spec's. On 03/07/22, 9:56 AM, "Masatake Iwasaki" <[email protected]> wrote: > For example, assuming that the following stack is installed: > > /usr/hdp/SOME_VERSION/hadoop/... > /spark/... > /zookeeper/... Multiple versions of the same package (deb/rpm) can not be installed at the same time. IIRC, HDP uses convention in which the product version is part of package name for addressing this. like:: $ rpm -qi hadoop_2_6_1_0_129 Name : hadoop_2_6_1_0_129 Version : 2.7.3.2.6.1.0 ... > So if we accept bdp-select, we also have to develop a new feature to > change install directories of the packages, > keeping the current default for avoiding user-facing incompatibility. > Otherwise bdp-select itself is useless. I don't like to bring the awkward package name convention to Bigtop at least by default. I feel hdp-select/bdp-select should belongs to Ambari since it is assuming the convention specific to HDP/BDP/Ambari. We can easily add packaging for the code under Ambari (as ambari-bdp-select for example) based on the existing stuff under bigtop-packages/*/ambari. Masatake Iwasaki On 2022/07/03 7:17, Kengo Seki wrote: > Thank you for proposing this, Viraj and Brahma. > This proposal may feel sudden to someone, so let me explain some context, > mainly to the Bigtop community. > > Ambari went to Attic once due to inactivity, > but some of its users and developers are reviving it now. > From the Bigtop community, Roman and Evans undertook the role of > initial PMC members, > and Jun, Yuqi, Ganesh, Masatake, and I are also going to support it. > > The default stack of Ambari was HDP, but it has gone behind paywall, > so we have to add an alternative stack in the next release of Ambari. > Viraj and Brahma are planning to leverage Bigtop packages for that purpose, > and that stack is tentatively named BDP (BigData Platform). > > We would also like to take over a feature called hdp-select, > which enables users to switch the current stack easily with symlink. > For example, assuming that the following stack is installed: > > /usr/hdp/SOME_VERSION/hadoop/... > /spark/... > /zookeeper/... > > /etc/hadoop, /var/lib/hadoop, /usr/lib/hadoop are symlinks of > /usr/hdp/SOME_VERSION/hadoop/etc, /usr/hdp/SOME_VERSION/hadoop/var/lib, > /usr/hdp/SOME_VERSION/hadoop/usr/lib respectively. > > Then the new stack is installed under /usr/hdp/NEW_VERSION, > users can run hdp-select to switch the symlinks to the new directories > for upgrading the stack. > (Is my understanding correct? Correct me if I'm wrong, Viraj and Brahma) > > hdp-select was provided as a RPM/DEB package just like other packages > HDP provided, > so it is user-friendly to provide bdp-select (the BDP version of > hdp-select) as well. > This is the background of Viraj's proposal. > > But there is a problem here to leverage Bigtop packages in this manner. > Bigtop respects Linux's Filesystem Hierarchy Standard, > so it doesn't install files into a single place (/usr/hdp/VERSION in > the example above), > but directly into /etc, /usr, /var and so on. > So if we accept bdp-select, we also have to develop a new feature to > change install directories of the packages, > keeping the current default for avoiding user-facing incompatibility. > Otherwise bdp-select itself is useless. > But it will be a tough work, because installation paths are hard-coded > in the RPM specs and DEB packaging scripts now. > > Based on the above, how about creating a new branch for adding hdp-select > and developing the feature to switch installation paths? > After finishing the work on that branch and ensuring that it works expectedly, > we can merge it into master, I think. > > Kengo Seki <[email protected]> > > On Thu, Jun 30, 2022 at 4:21 PM Battula, Brahma Reddy > <[email protected]> wrote: >> >> Thanks @Viraj Jasani to start discussing on this. >> >> IMO, For upcoming ambari release(2.7.7) , we should have select other than HDP so that people can start using the ambari without HDP. >> >> I am +1(non-binding) on this approach to have selects in bigtop code. May be for long term , we can think for tarballs or mpacks which can go in mabari trunk. >> >> >> On 29/06/22, 10:22 AM, "Viraj Jasani" <[email protected]> wrote: >> >> Hi Ambari/Bigtop dev, >> >> As per the new roadmap of Apache Ambari, we would like to propose moving >> certain scripts (previously known as hdp-select and conf-select) to Bigtop >> so that their rpm installation could be managed independently. >> These scripts are a basic necessity in the Ambari framework for the >> installation of various Bigdata packages. The only major changes they would >> receive is when we onboard new services and components to Ambari, else they >> usually do not receive updates. In the past, we used to get hdp-select rpm >> downloaded and installed from HDP repositories. >> >> We have still not officially finalized the new replacement name for >> hdp-select, it would likely be bdp-select (bdp: BigData Platform). However, >> the purpose of bdp-select and conf-select remains the same. >>
