Hello Brahma, Thank you for your insightful feedback. Your suggestion regarding the new Hadoop version 3.3.6 is indeed interesting, considering it addresses a plethora of bugs and introduces many improvements. It seems to be a promising option.
Our Ambari depends on Bigtop, and specifically, Ambari 2.9 corresponds to Bigtop 3.3. The various component versions can be found here: https://issues.apache.org/jira/browse/BIGTOP-3909. Notably, the version of Hadoop is 3.3.5, Spark is at 3.3.2, and Hive sits at 3.1.3. The primary version choices for Bigtop 3.3 have been settled upon after considerable discussion. The Bigtop community has conducted extensive testing of Hadoop 3.3.5, Hive 3.1.3, and Spark 3.3.2. A significant amount of work has been done on adapting Hadoop 3.3.5 to other components, such as Hive Ranger, including wide-ranging compatibility fixes and bug fixes. Unfortunately, the current timing prevents us from participating in the discussion about Bigtop 3.3. Apache Bigtop community members have already done substantial work on their chosen version for Bigtop 3.3. But it may be beneficial for us to join in these discussions earlier during the upcoming Bigtop version. Regarding your specific queries: 1-3：We would certainly consider participating in future version discussions alongside the Bigtop community. 4.We do support Namenode federation, albeit with a small user base and limited feedback presently. Agreed, offering support for multiple standby Namenodes sounds like a worthy idea. 5.Regarding Ambari scalability, could you kindly specify what are you mainly referring to? 6.In Ambari, it appears that support for Region server grouping requires just a simple configuration. HBase table management, RegionServer Group management can be performed using commands in the HBase shell. 7.Hive 4 repairs an enormous number of bugs, perhaps even thousands! However, its usage is currently limited, and it still presents compatibility issues with the Hadoop 3 ecosystem. It might be prudent to observe its progress and possibly discuss it during the next Bigtop version planning. I hope this addresses your concerns, and please feel free to reach out if you need any further clarification. Best Regards, [Your Name] > On Aug 16, 2023, at 15:36, Battula, Brahma Reddy <bbatt...@visa.com.INVALID> > wrote: > > Thanks Jialiang Cai for initiating this discussion. Good to see features > which you mentioned which are aligned with my initial roadmap discussion. > > IMO, we should go with two options. > > i) As 2.8 is cancelled , we can re-create the same branch and continue to > work on the some of the following you mentioned and delivery the release.. > ii) Create branch-3 (As Some point we need to move) and plan alpha, beta > releases as there so many major changes are planned. So, that we can plan > only bugfixes with 2.8...etc > > > Even we need to consider, how ambari metrics releases will be streamlined > with ambari release. > > > Queries: > 1) can't we use latest Hadoop version which is Hadoop-3.3.6..? > 2) what about spark version, any chance to consider 3.4.0.? > 3) Can we've hbase to 2.5 and zookeeper to 3.8..? > 4) Can we namenode (RBF)/Yarn federation.? Multiple standby namenodes.? > 5) Ambari scalability > 6) Region server grouping for HBase. > 7) Not urgent : Hive also we might need to revisit, as 4.0.0 beta is > available.? > > > Regards, > Brahma > > > > > On 16/08/23, 12:36 PM, "Lei Yao" <yao...@apache.org > <mailto:yao...@apache.org> <mailto:yao...@apache.org > <mailto:yao...@apache.org>>> wrote: > > > Thanks Jialiang Cai for driving this work. > I agree on this Ambari 2.9.0 plan. We need more contribution and PMC's > attention. > After all, we need to release a stable version based on Bigtop. > BTW, it will be greatly appreciated if someone could provide some virtual > machines on cloud. We can use them as a test environment or a public yum > repository > > > Best Regards. > Lei Yao > > > On 2023/08/16 05:43:50 Jialiang Cai wrote: >> Due to the failed release of Ambari 2.8, and the fact that there are indeed >> many areas in Ambari 2.8 that need to be improved for deployment, we plan to >> rectify the 2.8 bugs and enhance some features in the 2.9 release. Since no >> one in the community is currently promoting Ambari 2.9, I'm initiating a >> discussion on the Ambari 2.9.0 Roadmap. >> >> The contributors to Ambari 2.9 are currently very few, only 2-3 people, so >> we have temporarily set the following features that are under development: >> >> 1. Adaption of Hadoop 3.3.5 to the Bigtop stack and integration with Ambari. >> 2. Support for the merged Ranger 2.4. >> 3. Integration of the Ambari infrastructure (merged). >> 4. Upgrading service versions to Bigtop 3.3.0. >> 5. Support for Phoenix integration both in Ambari and Bigtop. >> 6. Adding YARN TimelineService V2 and Registry DNS support (currently under >> review). >> 7. In-progress efforts to incorporate Logsearch support. >> 8. Integration of Ambari Infra (merged). >> 9. Exploring support for openEuler OS with Ambari. >> 10. Future plans for adding Knox support. >> 11. Incorporating a YARN view in Ambari Views. >> 12. Addressing the continuous increase of WebSocket connections in the >> Ambari web UI. >> 13. Ongoing discussions about upgrading from Python 2 to Python 3. >> >> Our goal is for Ambari 2.9 to be the first stable big data version after HDP >> becomes proprietary. We also aim to provide users with pre-compiled RPMs and >> convenient installation methods. >> >> Here is the issue for discussion: >> https://issues.apache.org/jira/browse/AMBARI-25951 >> <https://issues.apache.org/jira/browse/AMBARI-25951> >> <https://issues.apache.org/jira/browse/AMBARI-25951 >> <https://issues.apache.org/jira/browse/AMBARI-25951>> >> >> We welcome everyone's discussion and opinions on the Ambari 2.9.0 Roadmap. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@ambari.apache.org >> <mailto:dev-unsubscr...@ambari.apache.org> >> <mailto:dev-unsubscr...@ambari.apache.org >> <mailto:dev-unsubscr...@ambari.apache.org>> >> For additional commands, e-mail: dev-h...@ambari.apache.org >> <mailto:dev-h...@ambari.apache.org> <mailto:dev-h...@ambari.apache.org >> <mailto:dev-h...@ambari.apache.org>> >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@ambari.apache.org > <mailto:dev-unsubscr...@ambari.apache.org> > <mailto:dev-unsubscr...@ambari.apache.org > <mailto:dev-unsubscr...@ambari.apache.org>> > For additional commands, e-mail: dev-h...@ambari.apache.org > <mailto:dev-h...@ambari.apache.org> <mailto:dev-h...@ambari.apache.org > <mailto:dev-h...@ambari.apache.org>> > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@ambari.apache.org > <mailto:dev-unsubscr...@ambari.apache.org> > For additional commands, e-mail: dev-h...@ambari.apache.org > <mailto:dev-h...@ambari.apache.org>
