Hey Micah, I took a look a the PR, and I think that's a good start.
Regarding 2. I like the idea of the state diagram, but it feels to me that there are a lot of states in there. Maybe we can take the Airflow AIP states as a start <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals>? This has Draft, Vote, Accepted, Completed, and Abandoned. For the vote, I would also lean towards a consensus approval vote, the same as code modifications. For Airflow, next to the PMC, committers also have binding votes. I don't have a strong opinion on that, but inclined to go with only binding for PMC to aim for velocity. Regarding 4. I think this is the tricky one. It is similar to releases where you have the scope creep and the release gets pushed back. With the releases, we're doing a three-month cadence, and I think that works pretty well (with some exceptions). I think something similar would be good for a new spec where we agree on some target date and if certain features are not near completion, we can roll them over to the next spec version. Regarding 3. I'm happy to take a look at this and come up with a draft. Let's see if we can refer to the ASF pages (or contribute this to comdev <https://community.apache.org/>). Thanks again Micah for taking a stab at this. Kind regards, Fokko Op do 25 jul 2024 om 09:06 schreef Micah Kornfield <emkornfi...@gmail.com>: > Sorry Owen for hijacking or misunderstanding the thread. I think there is > some overlap in the list > > To move the conversation along I put up a strawman proposal to cover > merging PRs [1]. I will fork a separate discussion thread to raise > visibility on it once some initial feedback is gathered. > > For 2 (specification changes) it seems like there is already a good > starting with the Iceberg improvement proposal process. I think there is > still some ambiguity here, as it isn't clear to me if some recent proposals > (I'm thinking mostly of a few additions proposed for the REST API) are > stuck because the proposers did not have bandwidth to complete them, we > aren't sure whether to hold a vote or the community in fact did not want to > move forward with them? Maybe something like adding a state diagram > similar to PEP's state diagram [2], could help? Also, per Owen's original > list perhaps a list of time minimums in each state? Also, I suppose > clarity on the type of vote needed to accept an improvement? Are there any > other aspects to discuss in a more focused thread? > > For 4, I'll start a discussion thread, but the thing that makes the most > sense to me is some sort of time based cadence? Or are there other > criteria the community has typically used? > > Thanks, > Micah > > [1] https://github.com/apache/iceberg/pull/10780 > [2] https://peps.python.org/_images/process_flow.svg > > On Tue, Jul 23, 2024 at 8:40 AM Jack Ye <yezhao...@gmail.com> wrote: > >> +1 on the list Micah provides, and ideally most of the matters in 3 and 5 >> could directly be deferred to ASF guidelines. >> >> -Jack >> >> On Tue, Jul 23, 2024 at 8:30 AM Ryan Blue <b...@databricks.com.invalid> >> wrote: >> >>> I agree with Micah as well. I also don't want to get bogged down in >>> bureaucracy and more rules inevitably lead to more bureaucracy. I also like >>> the prioritization he suggests. >>> >>> I also think it's important to clarify that this thread isn't trying to >>> rehash the previous threads in a new place. I think Owen's intent is to >>> gather a list of topics and then discuss each topic individually. That's in >>> line with suggestions like Micah's, which have been echoed by other PMC >>> members (and also ASF members). Let's gather topics, figure out whether we >>> need to add a bylaw and discuss each one individually. >>> >>> On Tue, Jul 23, 2024 at 7:45 AM Russell Spitzer < >>> russell.spit...@gmail.com> wrote: >>> >>>> Micah has a great list there for me. I'm similarly not as interested in >>>> the bureaucracy of the project and more interested in actually discussing >>>> how we operate from a technical perspective as the community grows. >>>> >>>> On Tue, Jul 23, 2024 at 1:01 AM Micah Kornfield <emkornfi...@gmail.com> >>>> wrote: >>>> >>>>> My 2 cents on this topic. I think we are getting bogged down in >>>>> relatively minor details/bureaucratic points. This is a reiteration of a >>>>> previous recommendation on the topic, but in the interest of making >>>>> progress here, I'd propose let's break this conversation down and focus on >>>>> incremental definitions/proposals. >>>>> >>>>> I've added a potential breakdown below. This is roughly ordered from >>>>> codifying day to day business, and extends to longer term concerns. I >>>>> would aim for minimalism on each of these and add additional guidance when >>>>> we need it. It is important to keep in mind that most decisions made on a >>>>> day to day basis are fairly easily reversible and more process/formalism >>>>> can always be added if some part of the process has bugs in it. >>>>> >>>>> 1. Requirements to merge a PR (1 committer approval + X amount of >>>>> time to allow others to review?) >>>>> 2. Requirements major projects/features including spec changes >>>>> 3. Define roles/responsibilities and provide guidance on how members >>>>> can grow into those roles. (Cover release manager, comitter, PMC member, >>>>> community member, with links as appropriate). >>>>> 4. Create guidelines on starting new revision to the specification >>>>> (i.e. when will the table V3 spec be considered closed?). >>>>> 5. People issues (if the community thinks this is still necessary). >>>>> >>>>> Thanks, >>>>> Micah >>>>> >>>>> On Mon, Jul 22, 2024 at 10:05 AM Jack Ye <yezhao...@gmail.com> wrote: >>>>> >>>>>> Just to follow up on the other topics, here are my comments, mainly >>>>>> to reconcile with what have been discussed in different threads, which >>>>>> could help formulating these multiple-choice questions: >>>>>> >>>>>> > What should the minimum voting period be? >>>>>> >>>>>> Do we decide a minimum voting period for all topics, or do we decide >>>>>> a different period for each topic? So far it seems like the ASF >>>>>> convention >>>>>> has different minimum periods defined, like 3 days for release [1], 7 >>>>>> days >>>>>> for new committers/PMC members [2]. >>>>>> >>>>>> > I'd like to include a couple sentences about the different hats at >>>>>> Apache and that votes should be for the benefit of the project and not >>>>>> our >>>>>> employers. >>>>>> > I'd like to propose that we include text to formally include censor >>>>>> and potential removal for disclosing sensitive information from the >>>>>> private >>>>>> list. >>>>>> >>>>>> Personally I am definitely +1 for this, but going back to the topic >>>>>> of Iceberg vs ASF bylaws/guidelines, I feel ideally this should also be a >>>>>> part of ASF general bylaws/guidelines that Iceberg simply just >>>>>> references. >>>>>> >>>>>> > Requirements for each topic >>>>>> >>>>>> I think this is also missing code modification and the improvement >>>>>> proposal votes. My impression so far after discussions in the initial >>>>>> bylaws doc [3] is that ASF definition of "code modification" is closer >>>>>> to a >>>>>> full design process and uses consensus approval. But in reality "merging >>>>>> PR" is a much more lightweight "vote" that typically requires just 1 >>>>>> committer approval of the PR. On the other hand, the Iceberg improvement >>>>>> proposal process is currently described to use the code modification >>>>>> consensus approval vote [4], but there are other options like 2/3 >>>>>> majority >>>>>> and majority vote that were proposed. >>>>>> >>>>>> > consensus, lazy consensus, lazy majority, lazy 2/3's >>>>>> >>>>>> In the initial bylaws doc [3] it was pointed out that the definition >>>>>> of "lazy consensus" is different in different documents. In general the >>>>>> definition is "no -1", but there is also the definition [5] of "at least >>>>>> 1 +1, no -1". I ended up giving it a different name of "minimum >>>>>> consensus", >>>>>> and actually this has been the model used for merging pull requests. We >>>>>> might want to clarify that part first before voting for these options. >>>>>> >>>>>> [1] https://www.apache.org/legal/release-policy.html#release-approval >>>>>> [2] https://community.apache.org/newcommitter.html#discussion >>>>>> [3] >>>>>> https://docs.google.com/document/d/1S3igb5NqSlYE3dq_qRsP3X2gwhe54fx-Sxq5hqyOe6I/edit >>>>>> [4] https://iceberg.apache.org/contribute/#how-are-proposals-adopted >>>>>> [5] https://orc.apache.org/develop/bylaws/ >>>>>> >>>>>> -Jack >>>>>> >>>>>> On Fri, Jul 19, 2024 at 2:29 PM Jack Ye <yezhao...@gmail.com> wrote: >>>>>> >>>>>>> > specifically the discussion of the standard roles >>>>>>> >>>>>>> Yes, there are also other places with different definitions. For >>>>>>> example the default project guideline [1] has additional description of >>>>>>> the >>>>>>> PMC member and chair responsibilities. There are a few other places like >>>>>>> ASF glossary [2] where these terms are defined, I cannot recall on the >>>>>>> top >>>>>>> of my head, but I can try to dig those up later. >>>>>>> >>>>>>> > putting together a committer requirements/guidelines doc >>>>>>> >>>>>>> +1. For some context, the committer guideline discussion came from >>>>>>> both some initial feedback on devlist, as well as comments in the roles >>>>>>> and >>>>>>> responsibilities section. >>>>>>> >>>>>>> Because I initially used the definition of Hadoop bylaws [3] for >>>>>>> committers, which writes: "The project’s Committers are responsible for >>>>>>> the >>>>>>> project’s technical management." There were then multiple people >>>>>>> pointing >>>>>>> out that committers are not necessarily technical. If we look into the >>>>>>> current definition in the link Owen provided: "A committer is a >>>>>>> developer >>>>>>> who has write access to the code repository and has a signed CLA on >>>>>>> file. >>>>>>> They have an apache.org mail address. Not needing to depend on >>>>>>> other people to make patches to the code or documentation, they are >>>>>>> actually making short-term decisions for the project." And in ASF >>>>>>> Glossarsy: "An individual who has the privilege to directly commit >>>>>>> changes >>>>>>> to an Apache codebase (commit access)." >>>>>>> >>>>>>> Indeed it does not have a requirement for committers to be >>>>>>> technical, but it is arguably still centered around the right to commit >>>>>>> code and merge patches. And that sparked all the discussions of what >>>>>>> exactly a committer means, what are the criteria, do we limit committer >>>>>>> rights to commit to just sub-project, what about people that do not >>>>>>> contribute code, etc. I don't know if there could be a better definition >>>>>>> though, maybe we could discuss that in a separate thread that is >>>>>>> visible to >>>>>>> more Apache members and people in other projects. >>>>>>> >>>>>>> I later raised the thread >>>>>>> https://lists.apache.org/thread/4fykr8316cw9lhvgq168tx5yj9zy7gc3 >>>>>>> where you can find more details about the discussions in the thread as >>>>>>> well >>>>>>> as in the linked Google doc for committer guideline. Topics like minimum >>>>>>> time requirement and recency requirements were discussed there. >>>>>>> Personally, >>>>>>> I included all those aspects in the draft because my main goal is to >>>>>>> have >>>>>>> an open discussion about these topics so that we can potentially reach >>>>>>> some >>>>>>> consensus, and eventually form a clear guideline. It would be nice if we >>>>>>> can do that as a part of this process and also moderated in the same >>>>>>> way. >>>>>>> >>>>>>> -Jack >>>>>>> >>>>>>> [1] >>>>>>> https://cwiki.apache.org/confluence/display/INCUBATOR/Default+Project+Guidelines >>>>>>> [2] https://www.apache.org/foundation/glossary >>>>>>> [3] https://hadoop.apache.org/bylaws.html >>>>>>> >>>>>>> >>>>>>> On Fri, Jul 19, 2024 at 11:54 AM Ryan Blue >>>>>>> <b...@databricks.com.invalid> wrote: >>>>>>> >>>>>>>> Another thing that has come up is putting together a committer >>>>>>>> requirements/guidelines doc. I think it would be great to have a >>>>>>>> discussion about how we want to do that. Specifically, I'm against >>>>>>>> adding >>>>>>>> requirements for new committers (such as time-based minimums or recency >>>>>>>> requirements) that exclude people from consideration. I think it would >>>>>>>> be >>>>>>>> helpful to clarify the path to becoming a committer, though. To me, >>>>>>>> it's >>>>>>>> about building trust and that trust is recognized by a PMC vote. >>>>>>>> >>>>>>>> Ryan >>>>>>>> >>>>>>>> On Fri, Jul 19, 2024 at 11:00 AM Owen O'Malley < >>>>>>>> owen.omal...@gmail.com> wrote: >>>>>>>> >>>>>>>>> I meant specifically the discussion of the standard roles (eg. >>>>>>>>> users, committers, pmc, pmc chair) that are well covered in >>>>>>>>> https://www.apache.org/foundation/how-it-works/#roles >>>>>>>>> >>>>>>>>> .. Owen >>>>>>>>> >>>>>>>>> On Fri, Jul 19, 2024 at 10:43 AM Jack Ye <yezhao...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Thank you Owen for moving this forward, we heard you were sick, >>>>>>>>>> hope you are fully recovered now! >>>>>>>>>> >>>>>>>>>> One point regarding "referring to the Apache documentation": I am >>>>>>>>>> totally for that, but during the initial investigation, I found out >>>>>>>>>> that >>>>>>>>>> the Apache documentations are scattered around, and also have >>>>>>>>>> conflicting >>>>>>>>>> information. >>>>>>>>>> >>>>>>>>>> For example, regarding a "vote for committer or PMC member": >>>>>>>>>> - this new committer doc [1] writes that "A positive result is >>>>>>>>>> achieved by Consensus Approval: at least 3 +1 votes and no vetoes." >>>>>>>>>> - the Apache voting process doc [2] writes that "Votes on >>>>>>>>>> procedural issues follow the common format of majority rule unless >>>>>>>>>> otherwise stated.", and when consulting a few Apache members, most >>>>>>>>>> of them >>>>>>>>>> consider voting for committer or PMC member a procedural issue. >>>>>>>>>> >>>>>>>>>> Similar situations were found for other topics like description >>>>>>>>>> of roles and responsibilities, code modification, etc. >>>>>>>>>> >>>>>>>>>> I think it is a great chance for ASF in general to consolidate >>>>>>>>>> these information, especially for matters that have common >>>>>>>>>> guidelines in >>>>>>>>>> ASF that should be adhered to by all projects. With that, we can >>>>>>>>>> figure out >>>>>>>>>> what to put in the Iceberg specific bylaws, either to directly refer >>>>>>>>>> to ASF >>>>>>>>>> official information, or to add additional information and >>>>>>>>>> guidelines. >>>>>>>>>> >>>>>>>>>> Regarding sub-projects, the main reason I proposed it at the >>>>>>>>>> beginning was to allow a proper definition of release manager >>>>>>>>>> responsibility, since each sub-project is released independently. It >>>>>>>>>> was >>>>>>>>>> not intended to be tied to committer responsibilities. >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Jack Ye >>>>>>>>>> >>>>>>>>>> [1] https://community.apache.org/newcommitter.html >>>>>>>>>> [2] https://www.apache.org/foundation/voting >>>>>>>>>> >>>>>>>>>> On Fri, Jul 19, 2024 at 10:22 AM Owen O'Malley < >>>>>>>>>> owen.omal...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Everyone is welcome to vote. The Iceberg PMC will have the only >>>>>>>>>>> binding votes. >>>>>>>>>>> >>>>>>>>>>> .. Owen >>>>>>>>>>> >>>>>>>>>>> On Jul 19, 2024, at 10:19, Wing Yew Poon >>>>>>>>>>> <wyp...@cloudera.com.invalid> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Owen, >>>>>>>>>>> Thanks for doing this. >>>>>>>>>>> Once you have the questions and choices, who gets to vote on >>>>>>>>>>> them? >>>>>>>>>>> - Wing Yew >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Jul 19, 2024 at 10:07 AM Owen O'Malley < >>>>>>>>>>> owen.omal...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> All, >>>>>>>>>>>> Sorry for the long pause on bylaws discussion. It was a >>>>>>>>>>>> result of wanting to avoid the long US holiday week (July 4th) and >>>>>>>>>>>> my >>>>>>>>>>>> procrastination, which was furthered by a side conversation that >>>>>>>>>>>> asked me >>>>>>>>>>>> to consider how to move forward in an Apache way. >>>>>>>>>>>> I'd like to thank Jack for moving this to this point. One >>>>>>>>>>>> concern that I had was there were lots of discussions and >>>>>>>>>>>> decisions that >>>>>>>>>>>> were being made off of our email lists, which isn't the way that >>>>>>>>>>>> Apache >>>>>>>>>>>> should work. >>>>>>>>>>>> For finishing this off, I'd like to come up with a set of >>>>>>>>>>>> questions that should be answered by multiple choice questions and >>>>>>>>>>>> then use >>>>>>>>>>>> single transferable vote (STV) to resolve them. STV just means >>>>>>>>>>>> that each >>>>>>>>>>>> person lists their choices in a ranked order with a formal way to >>>>>>>>>>>> resolve >>>>>>>>>>>> how the votes work. >>>>>>>>>>>> The questions that I have heard so far are: >>>>>>>>>>>> >>>>>>>>>>>> 1. Should the PMC chair be term-limited and if so, what is >>>>>>>>>>>> the period? *In my experience, this isn't necessary in most >>>>>>>>>>>> projects and is often ignored. In Hadoop, Chris Douglas was a >>>>>>>>>>>> great chair >>>>>>>>>>>> and held it for 5 years in spite of the 1 year limit.* >>>>>>>>>>>> 1. No term limit >>>>>>>>>>>> 2. 1 year >>>>>>>>>>>> 3. 2 year >>>>>>>>>>>> 2. What should the minimum voting period be?* I'd suggest 3 >>>>>>>>>>>> days is far better as long as it isn't abused by holding >>>>>>>>>>>> important votes >>>>>>>>>>>> over holiday weekends.* >>>>>>>>>>>> 1. 3 days (72 hours) >>>>>>>>>>>> 2. 7 days >>>>>>>>>>>> 3. Should we keep the section on roles or just reference >>>>>>>>>>>> the Apache documentation >>>>>>>>>>>> <https://www.apache.org/foundation/how-it-works/#roles>. *I'd >>>>>>>>>>>> suggest that we reference the Apache documentation.* >>>>>>>>>>>> 4. I'd like to include a couple sentences about the >>>>>>>>>>>> different hats at Apache and that votes should be for the >>>>>>>>>>>> benefit of the >>>>>>>>>>>> project and not our employers. >>>>>>>>>>>> 5. I'd like to propose that we include text to formally >>>>>>>>>>>> include censor and potential removal for disclosing sensitive >>>>>>>>>>>> information >>>>>>>>>>>> from the private list. >>>>>>>>>>>> 6. I'd like to propose branch committers. It has helped >>>>>>>>>>>> Hadoop a lot to enable people to work on development branches >>>>>>>>>>>> for large >>>>>>>>>>>> features before they are given general committership. It is >>>>>>>>>>>> better to have >>>>>>>>>>>> the branch work done at Apache and be visible than having large >>>>>>>>>>>> branches >>>>>>>>>>>> come in late in the project. >>>>>>>>>>>> 7. Requirements for each topic (each could be consensus, >>>>>>>>>>>> lazy consensus, lazy majority, lazy 2/3's) >>>>>>>>>>>> 1. Add committer >>>>>>>>>>>> 2. Remove committer >>>>>>>>>>>> 3. Add PMC >>>>>>>>>>>> 4. Remove PMC >>>>>>>>>>>> 5. Accept design proposal >>>>>>>>>>>> 6. Add subproject >>>>>>>>>>>> 7. Remove subproject >>>>>>>>>>>> 8. Release (can't be lazy consensus) >>>>>>>>>>>> 9. Modifying bylaws >>>>>>>>>>>> >>>>>>>>>>>> Thoughts? Missing questions? >>>>>>>>>>>> >>>>>>>>>>>> .. Owen >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Ryan Blue >>>>>>>> Databricks >>>>>>>> >>>>>>> >>> >>> -- >>> Ryan Blue >>> Databricks >>> >>
