Hi,
I feel a "soft consensus" :) that people see some benefits of
introducing (possibly optional) process of proposing new features.
I think that in order to proceed with this we need to agree on goals
that we want to achieve. Whether the process should or should not be
optional, which form it should have, and answers on all these other
questions could be answered after that.
So, I'll try to state some issues I see with our current approach,
please feel free to correct any of them, or add any other:
- due to the "soft consensus" approach, we actually delegate the final
responsibility of "feature acceptance" to reviewer(s) - these might or
might not be happy with that
- by splitting this into
first-consensus-then-implementation-then-review approach, we remove the
burden of responsibility of respective feature from reviewers - they can
focus only on the main purpose of the review - that is verifying the
quality of code
- as mentioned before, this brings better visibility to (core) features
- and last but not least makes it possible to prioritize work and
build more complex long-term goals
I think it is essential to have a consensus on whether or not these are
some points we want to target (that is, we see our current approach as
sub-optimal in these areas) or not.
Jan
On 12/17/19 7:08 PM, Pablo Estrada wrote:
It seems that lots of people see benefit in a more formalized BIP
process. I think that makes sense, though I'd like to give people the
freedom to choose the medium for their design discussions.
The projects I'm aware of usually do this through wiki-type mediums.
We have cwiki, though lots of people like working with Gdocs'
collaboration features. Are there other mediums that could be used for
this?
A possible implementation is: We could keep cwiki as the 'index' - so
anyone proposing a new BIP would have to add a new BIP entry in the
cwiki, but they'd be free to link to a Gdoc from there, or to develop
the proposal in the cwiki entry itself.
Thoughts?
Best
-P.
On Tue, Dec 17, 2019 at 9:14 AM Maximilian Michels <m...@apache.org
<mailto:m...@apache.org>> wrote:
The main benefit of BIPs I see is the visibility they create for the
project users and contributors.
Right now, we have a long unordnered list of design documents.
Some of
the documents are not even in that list. With BIPs, we would end
up with
an ordered list "BIP-1, BIP-2, .." which reflects important design
decisions over time.
Simply assigning an id, makes it a lot more formal. In my eyes,
the id
assignment would also require that you communicate the changes in
a way
that the community can accept the proposal, preferably via lazy
consensus. All in all, this could help communicate changes in Beam
better.
JIRA, on the other hand, contains concrete implementation steps
and all
kinds of other changes.
Cheers,
Max
On 16.12.19 21:41, Robert Bradshaw wrote:
> Additional process is a two-edged sword: it can help move stuff
> forward, to the correct decision, but it can also add significant
> overhead.
>
> I think there are many proposals for which the existing processes of
> deriving consensus (over email, possibly followed by a formal
vote or
> lazy consensus) are sufficient. However, sometimes they're not.
> Specifically, for long-term roadmaps, it would be useful to have
them
> in a standard place that can be tracked and understood (I don't
think
> we've been able to use JIRA effectively for this here). I also think
> there are some proposals that reach a certain level of
complexity that
> trying to address them by occasionally responding to email
threads as
> they come up is insufficient. For these latter, I think there is a
> need for commitment for a group of people in the community to commit
> to clearly defining and driving a solution to the problem via a more
> formal process. Often the one making the proposal has sufficient
> motivation, but sometimes what lacks is be (non-sporadic) investment
> by those trying to understand, evaluate, and incorporate the
proposal.
>
> So I'm (strongly) +1 for exploring a more formal process, but -1 on
> requiring it.
>
> On Sun, Dec 15, 2019 at 1:07 AM Jan Lukavský <je...@seznam.cz
<mailto:je...@seznam.cz>> wrote:
>>
>> Hi,
>>
>> thanks for reactions so far. I agree that there are many
questions that have to be clarified. I'd propose to split this
into two parts:
>>
>> a) first reach a consensus that we want this process in the
first place
>>
>> b) after that, we need to clarify all the details - that will
probably be somewhat iterative procedure
>>
>> I'm not sure if there is something more we need to clarify
before we can cast a vote on (a).
>>
>> Thoughts?
>>
>> Jan
>>
>> On 12/10/19 3:46 PM, Łukasz Gajowy wrote:
>>
>> +1 for formalizing the process, enhancing it and documenting
clearly.
>>
>> I noticed that Apache Airflow has a cool way of both creating
AIPs and keeping track of all of them. There is a "Create new AIP"
button on their Confluence. This way, no AIP gets lost and all are
kept in one place. Please keep in mind that this is also the
problem we want to solve in Beam and try to keep track of all the
documents we have so far*. It's certainly good to solve that
problem too, if possible.
>>
>> Also the AIP structure is something that I find nice - There's
place for all additional resources, JIRAs, discussion in comments
and state of the proposal. Even if we don't choose to use
Confluence, we definitely could use a similar template with all
that information for our google docs proposals or any other tool
we stick to.
>>
>> Thanks!
>>
>> *thank you, Ismael and Alexey, for all the reminders under the
proposals to add them to Confluence list! :)
>>
>> wt., 10 gru 2019 o 13:29 jincheng sun <sunjincheng...@gmail.com
<mailto:sunjincheng...@gmail.com>> napisał(a):
>>>
>>> Thanks for bring up this discussion Jan!
>>>
>>> +1 for cearly define BIP for beam.
>>>
>>> And I think would be nice to initialize a concept document for
BIP. Just a reminder: the document may contains:
>>>
>>> - How many kinds of improvement in beam.
>>> - What kind of improvement should to create a BIP.
>>> - What should be included in a BIP.
>>> - Who can create the BIP.
>>> - Who can participate in the discussion of BIP and who can
vote for BIP.
>>> - What are the possible limitations of BiP, such as whether it
is necessary to complete the dev of BIP in one release.
>>> - How to track a BIP.
>>>
>>> Here is a question: I found out a policy[1] in beam, but only
contains the poilcy of release , my question is does beam have
something called Bylaws? Similar as Flink[1].
>>>
>>> Anyway, I like your proposals Jan :)
>>>
>>> Best,
>>> Jincheng
>>> [1] https://beam.apache.org/community/policies/
>>> [2]
https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Approvals
>>>
>>>
>>> David Morávek <david.mora...@gmail.com
<mailto:david.mora...@gmail.com>> 于2019年12月10日周二 下午2:33写道：
>>>>
>>>> Hi Jan,
>>>>
>>>> I think this is more pretty much what we currently do, just a
little bit more transparent for the community. If the process is
standardized, it can open doors for bigger contributions from
people not familiar with the process. Also it's way easier to
track progress of BIPs, than documents linked from the mailing list.
>>>>
>>>> Big +1 ;)
>>>>
>>>> D.
>>>>
>>>> On Sun, Dec 8, 2019 at 12:42 PM Jan Lukavský <je...@seznam.cz
<mailto:je...@seznam.cz>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'd like to revive a discussion that was taken some year and
a half ago
>>>>> [1], which included a concept of "BIP" (Beam Improvement
Proposal) - an
>>>>> equivalent of "FLIP" (flink), "KIP" (kafka), "SPIP" (spark),
and so on.
>>>>>
>>>>> The discussion then ended without any (public) conclusion,
so I'd like
>>>>> to pick up from there. There were questions related to:
>>>>>
>>>>> a) how does the concept of BIP differ from simple plain JIRA?
>>>>>
>>>>> b) what does it bring to the community?
>>>>>
>>>>> I'd like to outline my point of view on both of these
aspects (they are
>>>>> related).
>>>>>
>>>>> BIP differs from JIRA by definition of a process:
>>>>>
>>>>> BIP -> vote -> consensus -> JIRA -> implementation
>>>>>
>>>>> This process (although it might seem a little unnecessary
formal) brings
>>>>> the following benefits:
>>>>>
>>>>> i) improves community's overall awareness of planned and
in-progress
>>>>> features
>>>>>
>>>>> ii) makes it possible to prioritize long-term goals
(create "roadmap"
>>>>> that was mentioned in the referred thread)
>>>>>
>>>>> iii) by casting explicit vote on each improvement
proposal diminishes
>>>>> the probability of wasted work - as opposed to our current
state, where
>>>>> it is hard to tell when there is a consensus and what
actions need to be
>>>>> done in order to reach one if there isn't
>>>>>
>>>>> iv) BIPs that eventually pass a vote can be regarded as
"to be
>>>>> included in some short term" and so new BIPs can build upon
them,
>>>>> without the risk of having to be redefined if their
dependency for
>>>>> whatever reason don't make it to the implementation
>>>>>
>>>>> Although this "process" might look rigid and corporate, it
actually
>>>>> brings better transparency and overall community health. This is
>>>>> especially important as the community grows and becomes more
and more
>>>>> distributed. There are many, many open questions in this
proposal that
>>>>> need to be clarified, my current intent is to grab a grasp
about how the
>>>>> community feels about this.
>>>>>
>>>>> Looking forward to any comments,
>>>>>
>>>>> Jan
>>>>>
>>>>> [1]
>>>>>
https://lists.apache.org/thread.html/4e1fffa2fde8e750c6d769bf4335853ad05b360b8bd248ad119cc185%40%3Cdev.beam.apache.org%3E
>>>>>
