Thanks for the reply, Brahma! > We'll use following repo, how other components in Apache are hosted.
Using that site for distribution has no problem. My concern is, do we have the source tree for building HDP packages and does it still work without the artifacts behind the paywall? According to ASF's release policy [1], we must publish the corresponding source releases whenever we publish binary artifacts, such as RPM and DEB packages for HDP. [1]: https://www.apache.org/legal/release-policy.html#compiled-packages Kengo Seki <[email protected]> On Mon, Jun 20, 2022 at 6:19 PM Battula, Brahma Reddy <[email protected]> wrote: > > Hi Kengo Seki, > > Thanks for pitching here.. We'll use following repo, how other components in > Apache are hosted. Any other suggestions..? > > https://dlcdn.apache.org/ > > > > On 20/06/22, 2:32 PM, "Kengo Seki" <[email protected]> wrote: > > Oh, I've just noticed that I submitted an additional question into the > wrong thread mistakenly. > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fj6kc2h1lphvjvsblcthcl5yzzmjk3tb5&data=05%7C01%7Cbbattula%40visa.com%7C8f8816799a4f4de1f58608da529b9c42%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913125540570268%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kdYhCFVQVGowhmsHAagNOJFngJWkqmdL0bNQ03JBEGc%3D&reserved=0 > is > originally intended as a reply to this thread. Sorry for the > confusion. > > Kengo Seki <[email protected]> > > On Mon, Jun 20, 2022 at 11:31 AM Kengo Seki <[email protected]> wrote: > > > > Thank you so much for writing the draft and commenting on it, Brahma > and Viraj! > > Firstly, a minor comment: > > > > > Trunk Code : Parallel we can work for the trunk code, this we might > give as 2.8. > > > * Working on bigtop mpack > > > > This means adding a new built-in stack which uses the Bigtop > > repository, and not developing Mpack on the Bigtop side, right? > > If so, "bigtop mpack" sounds confusing a bit. > > > > > hdp-select can stay on branch-2.7 and it can be moved to bigtop for > trunk branch. Thoughts? > > > > Sounds good to me. During the 2.7.x releases, users can use the HDP > > stack by default. In addition, they can also add the Bigtop stack > > through the Mpack, as Zhiguo mentioned in [1]. > > Since 2.8 or 3.0, we will drop HDP and users can use the built-in > > Bigtop stack by default. It also allows us to drop the Mpack on the > > Bigtop side. > > > > Regarding the built-in Bigtop stack developed on trunk, the current > > BIGTOP stack in Ambari [2] may be a good start point to be upgraded. > > I ensured that Ambari 2.7.6 was successfully built with the > > `-Dstack.distribution=BIGTOP` option, though Bigtop 0.8 is too old to > > use for now. > > > > [1]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fljnyymp2g9fg7cq4z7sfxd36bh8x7nlm&data=05%7C01%7Cbbattula%40visa.com%7C8f8816799a4f4de1f58608da529b9c42%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913125540570268%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Zf6225sm1LNTsTEuXCkl3F1NiEsuN1afC9oFEsCpAoU%3D&reserved=0 > > [2]: > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fambari%2Ftree%2Frelease-2.7.6%2Fambari-server%2Fsrc%2Fmain%2Fresources%2Fstacks%2FBIGTOP%2F0.8&data=05%7C01%7Cbbattula%40visa.com%7C8f8816799a4f4de1f58608da529b9c42%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913125540570268%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kffTwEi%2FvLaQe7CjLzQQJVwc7K9nwZ3FZ4yJlsPr%2BFA%3D&reserved=0 > > > > Kengo Seki <[email protected]> > > > > On Mon, Jun 20, 2022 at 9:26 AM Viraj Jasani <[email protected]> wrote: > > > > > > Thanks for the nice draft, Brahma! > > > > > > IMHO, as we narrow down to 2.7.7 release, it might be still worth > sticking > > > to HDP i.e. plugin the new stack for HDP for 2.7 release line. For > 2.8.0 > > > (or 3.0.0), we can add entire new stack (BDP sounds good one) and > start off > > > with 2.8.0/3.0.0 stack version. > > > > > > hdp-select can stay on branch-2.7 and it can be moved to bigtop for > trunk > > > branch. Thoughts? > > > > > > Similarly, we can start-off with "python 3 only" support in trunk, > whereas > > > 2.7 release line can continue with python 2 to reduce complexities for > > > maintenance releases on 2.7. > > > > > > All other points look good. Once again, thank you for drafting the > roadmap > > > and starting with this thread. > > > > > > > > > On Sun, Jun 19, 2022 at 10:07 AM Battula, Brahma Reddy > > > <[email protected]> wrote: > > > > > > > Hi All, > > > > > > > > As Ambari back now, let’s start discussing roadmap. I will be > closely > > > > tracking for Jira, GitHub(with Jenkins) and wiki to work normal. > > > > Based on the conclusion of this thread will update wiki[1]. > > > > > > > > As 2.7.6 is released recently, we need to release 2.7.7 > > > > > > > > > > > > 1. Stack Management. There are two approaches for this. Open to > discuss > > > > on following.. > > > > * Entirely new stack > > > > > > > > i. > > > > Define new stack name. (BDP: BigDataPlatform, BDS:BigDataStack, > > > > OSS:OpenSourceStack,TDP: TuskerDataPlatform…) > > > > > > > > ii. > > > > Define stack versions > > > > (zk-3.5.9,Hadoop-3.2.3,hive-3.1.3,tez-0.10.1,spark-3.2..) > > > > > > > > iii. > Based > > > > above two we need to have select in bigtop code. > > > > > > > > * Using the existing for current release. > > > > > > > > i. > > > > Plugin the new stack into HDP self > > > > > > > > ii. > > > > Still, we can have hdp select > > > > > > > > > > > > > > > > I prefer the planA but planB takes less time. Users can easily > adopt it > > > > upgrade their stack. > > > > > > > > > > > > > > > > Note: Ideally bigtop mpack can be long term approach as stack id > decoupled > > > > now, but currently it’s not ready for latest stack (upgrades can be > risk). > > > > We’ll work in trunk for same. If this works shortly, we can choose. > > > > > > > > > > > > > > > > 1. Upgrade : > > > > * Most of users will be deployed with HDP > > > > versions(HDP-2.6.5,HDP-3.1.0), we might need to support the upgrade > to 2.7.7 > > > > 2. Other work > > > > * Any critical/blocker CVE’s Fixes > > > > * CVEs Fixes : we’ll try to target some CVE fixes > > > > * Review the open PR’s and Jira’s > > > > > > > > > > > > > > > > Trunk Code : Parallel we can work for the trunk code, this we might > give > > > > as 2.8. > > > > > > > > * Removal of python2 support > > > > * Removal of hard dependency on HDP > > > > * Working on bigtop mpack > > > > > > > > Can add more based on your inputs… > > > > > > > > > > > > Any thoughts on this..? Please feel free to add anything which I > missed. > > > > > > > > > > > > > > > > Reference: > > > > > > > > 1. > > > > > https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fpages%2Fviewpage.action%3FpageId%3D30755705&data=05%7C01%7Cbbattula%40visa.com%7C8f8816799a4f4de1f58608da529b9c42%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C637913125540570268%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8HISa3GK%2FDRU6%2BBkFg86WWXOVMOf3QOx36volDkTlWE%3D&reserved=0 > > > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
