Gentle reminder that we need to determine next steps here. We have an updated proposal on this thread. Do we need a VOTE or can we move forward directly to adjusting the members, etc?
Thanks! --larry On Thu, Oct 20, 2022 at 3:26 PM larry mccay <lmc...@apache.org> wrote: > @Justin Mclean <justinmcl...@me.com> - any insights on next steps here? > > > On Tue, Oct 18, 2022 at 5:44 PM larry mccay <lmc...@apache.org> wrote: > >> Very good, here is the latest revision with updated Mentors. >> Sunil and I have been added to the IPMC as well. >> Welcome Madhawa and thanks for stepping up as a Mentor for Livy! >> >> Abstract >> >> Livy is a web service that exposes a REST interface for managing long >> running Apache Spark contexts in your cluster. With Livy, new applications >> can be built on top of Apache Spark that require fine grained interaction >> with many Spark contexts [1]. >> >> While this project has been well regarded and used in many contexts as >> the defacto standard API to Spark environments, it has been incubating for >> over 5 years without graduation to a TLP and it has become difficult to >> impossible for fixes and improvements to be contributed as the current >> community seems to have moved on. >> >> There has been discussion regarding retirement of this podling where >> there seems to be some increasing interest in joining and reviving the >> community [2]. >> >> The intent of this proposal is to avoid retiring a well regarded, >> actively used and rather mature project by reviving the PPMC and community >> with new folks that have a vested interest in the project and health of the >> community. >> Proposal >> >> We propose to revive the PPMC with a set of contributors and maintainers >> as mentors, PPMC members and committers. >> >> The retirement DISCUSS thread [2] has shown a growing interest in >> providing new committers and bringing improvements and fixes from >> organization’s internally maintained forks back to a revived community. >> >> General Approach to Revival: >> >> - >> >> Add new Mentors >> - >> >> Larry McCay, lmc...@apache.org , Cloudera >> - >> >> Sunil Govindan, sun...@apache.org, Cloudera >> - >> >> Jean-Baptiste Onofré, jbono...@apache.org, Talend >> - >> >> Madhawa Gunasekara, madhaw...@gmail.com, Independent >> >> >> >> - >> >> Add new Committers/PPMC >> - >> >> Larry McCay, lmc...@apache.org, Cloudera >> - >> >> Vinod Kumar Vavilapalli, vino...@cloudera.com, Cloudera >> - >> >> Imran Rashid - iras...@apache.org, Cloudera >> - >> >> Gyorgy Gal, ggal ,gal.gyo...@gmail.com, Cloudera >> - >> >> Wing Yew Poon, wyp...@cloudera.com, Cloudera >> - >> >> Xilang Yan, xilang....@gmail.com, Shopee >> - >> >> Jianzhen Wu, myjianz...@gmail.com, Shopee >> - >> >> Nagella Jagadeewara Rao, jnage...@visa.com, Visa >> - >> >> Pralab Kumar, pralk...@visa.com, Visa >> - >> >> Prasad Shrikant, shrikant....@gmail.com, Visa >> - >> >> Brahma Reddy Battula, bra...@apache.org, Visa >> >> >> >> - >> >> Invite existing PPMC members to opt-in or otherwise go emeritus >> - >> >> Jean-Baptiste Onofré, jbono...@apache.org, Talend (opted-in via >> Retirement DISCUSS thread [2]) >> >> >> >> - >> >> Invite existing Committers to opt-out or otherwise continue >> >> >> >> - >> >> Establish Roadmap via follow up DISCUSS thread >> - >> >> Known Improvements from Forks which will need proposals and >> discussion: >> - >> >> Adding HA for Livy >> - >> >> Updating security capabilities (eg. kerberos for jdbc, fixing >> bugs in encryption) >> - >> >> Expanding the support for kubernetes >> - >> >> Responding to CVEs in dependencies (eg. log4j, thrift) >> - >> >> Livy rest cluster - IS THIS SAME AS HA for Livy ABOVE? >> - >> >> Support multi Spark versions >> - >> >> Implemented a metrics system for Livy >> - >> >> Support customize batch/interactive session lifecycle event >> handler, default log event with log4j, very helpful for >> troubleshooting >> - >> >> Optimize log to track which session id the log message came >> from, also very helpful for troubleshooting >> - >> >> Support customize Spark config optimization rules, can be used >> to optimize config for users’ job >> - >> >> A set of command line tool which can be used to replace Spark’s >> spark-submit, pyspark, spark-sql but actually submit application in >> Livy >> - >> >> We are planning to implement a JDBC state store, and allow >> multi Livy Thrift sessions to share one backend Spark application >> in the >> next few months. >> - >> >> These items and others that are brought to community may need >> consolidation or multiple configurable options and will need to be >> part of >> the discussion >> - >> >> One-pager Livy Improvement Proposals (LIP) may make sense to >> drive these discussions and convergence >> - >> >> Feature Branch Strategy for large changes >> - >> >> Large features are hard to review we will need to define a >> strategy >> - >> >> Determine the Improvements to be delivered across first 3 Releases >> with Target Release Dates >> >> >> >> - >> >> Ensure CVE and Dependency management hygiene is in place >> >> >> The above approach will usher the community back to an active status with >> a Roadmap of 3 or more release plans and security hygiene in place. >> Development Practices >> >> The Livy project follows a review before commit philosophy. Every commit >> automatically runs through the unit tests and generates coverage reports >> presented as a pull request comment. Our experience with this process leads >> us to believe that it helps ease new contributors into the project. They >> get feedback quickly on common mistakes, lowering the burden on reviewers. >> Those same reviewers get to lead by example, showing the new contributors >> that we value feedback within our community even when changes are done by >> more experienced folks. Taken from the original Apache Livy Proposal [1], >> this should continue to be true. As mentioned, Livy is a mature project and >> as such RTC is the most appropriate for continued quality and awareness. >> >> >> >> 1. >> >> Original Apache Livy Proposal >> https://cwiki.apache.org/confluence/display/incubator/LivyProposal >> 2. >> >> Retirement DISCUSS thread >> https://lists.apache.org/thread/gcstsrhbp91c5mm55htqn1l3djv8m7o0 >> >> >> On Tue, Oct 18, 2022 at 5:05 PM Madhawa Gunasekara <madhaw...@gmail.com> >> wrote: >> >>> Hi Larry, >>> >>> I'm an IPMC Member. madhawa30 at gmail dot com is my preferred email >>> address. >>> apache id: madhawa >>> >>> Thanks, >>> Madhawa >>> >>> >>> On Tue, Oct 18, 2022 at 10:05 PM larry mccay <lmc...@apache.org> wrote: >>> >>> > Hi Madhawa - >>> > >>> > That's awesome! >>> > Are you already a member of IPMC? >>> > If not, are you an ASF member? >>> > If you are an ASF member you can request that you be added as an IPMC >>> > member. >>> > >>> > Can you provide your company affiliation for the proposal and preferred >>> > email? >>> > >>> > thanks! >>> > >>> > --larry >>> > >>> > On Tue, Oct 18, 2022 at 2:18 PM Madhawa Gunasekara < >>> madhaw...@gmail.com> >>> > wrote: >>> > >>> > > Hi Larry, >>> > > >>> > > I'm interested in working with Livy and would like to join as a >>> Mentor. >>> > > >>> > > Thanks, >>> > > Madhawa >>> > > >>> > > >>> > > On Tue, Oct 18, 2022 at 6:57 PM larry mccay <lmc...@apache.org> >>> wrote: >>> > > >>> > > > Sorry, I missed commenting on this: >>> > > > >>> > > > "There is also no concept as an emeritus PPMC member at the ASF." >>> > > > >>> > > > I assume that we can remove PPMC members that do not opt-in >>> explicitly >>> > at >>> > > > this point. >>> > > > They will have every opportunity to rejoin. >>> > > > >>> > > > On Tue, Oct 18, 2022 at 12:48 PM larry mccay <lmc...@apache.org> >>> > wrote: >>> > > > >>> > > > > I will ask in a separate thread, @Justin Mclean < >>> justinmcl...@me.com >>> > > >>> > > - >>> > > > > thanks. >>> > > > > Adding JB adds another company and we are certainly open to >>> anyone >>> > else >>> > > > > that would like to join as a mentor. >>> > > > > At the end of the day, the mentors are for instilling the Apache >>> Way >>> > > > > knowledge and steering toward graduation. >>> > > > > I feel that this diversity, while nice to have, is less important >>> > than >>> > > > > that of the PPMC and committers for the long term health of the >>> > > > community. >>> > > > > >>> > > > > We need to push this podling to graduation as quickly as possible >>> > since >>> > > > it >>> > > > > is rather mature and needs to get to the next level. >>> > > > > >>> > > > > Again, any potential Mentors that would like to join are more >>> than >>> > > > welcome. >>> > > > > >>> > > > > On Tue, Oct 18, 2022 at 12:38 PM Justin Mclean < >>> > > jus...@classsoftware.com >>> > > > > >>> > > > > wrote: >>> > > > > >>> > > > >> Hi, >>> > > > >> >>> > > > >> I’m sorry, but Imran Rashid can’t be a mentor for the project as >>> > they >>> > > > are >>> > > > >> not an IPMC member. Currently, both Sunil and Larry (as they >>> are ASF >>> > > > >> members) need to ask to join the IPMC and NOTICE sent to the ASF >>> > > board. >>> > > > I >>> > > > >> would also prefer that mentors come from different companies. >>> > > > >> >>> > > > >> There is also no concept as an emeritus PPMC member at the ASF. >>> > > > >> >>> > > > >> Kind Regards, >>> > > > >> Justin >>> > > > >> >>> > > > >> >>> > > > >> >>> > --------------------------------------------------------------------- >>> > > > >> To unsubscribe, e-mail: >>> general-unsubscr...@incubator.apache.org >>> > > > >> For additional commands, e-mail: >>> general-h...@incubator.apache.org >>> > > > >> >>> > > > >> >>> > > > >>> > > >>> > >>> >>
