Re: [DISCUSS] BIP reloaded

So how should we do the voting now? Ask for a vote on BIP-1, but what of open issues... could people add open issues, or should this be done before the vote? _/ _/ Alex Van Boxel On Mon, Feb 17, 2020 at 4:29 PM Jan Lukavský wrote: > Hi Alex, > > +1, I think that the current structure is

Re: [DISCUSS] BIP reloaded

OK, I've reordered and tweaked it a bit based on your suggestions, but I'm going to stop here. I'm spending far more time on this than the implementation. I did add an open issues section though that people can add issues too that can be discussed and voted on. Those that make sense? _/ _/ Alex

Re: [DISCUSS] BIP reloaded

I jumped into these wiki pages and figured out how Airflow did theirs using the Page Properties table on each BIP [1] and how these automatically update the index using the Page Properties Report [2]. I would consider creating BIPs for ongoing efforts to flesh out these table, to establish the

Re: [DISCUSS] BIP reloaded

a = motivation b => *added current state in Beam* c = alternatives d = implementation *(I prefer this to define before the alternatives)* e = *rest of document?* _/ _/ Alex Van Boxel On Sun, Feb 9, 2020 at 7:50 PM Jan Lukavský wrote: > It's absolutely fine. :-) I think that the scope and

Re: [DISCUSS] BIP reloaded

It's absolutely fine. :-) I think that the scope and quality of your document suits very well for the first BIP. What I would find generally useful is a general structure that would be something like:  a) definition of the problem  b) explanation why current Beam options don't fit well for

Re: [DISCUSS] BIP reloaded

I'm sorry, I stole the number 1 from you. Feel free to give suggestions on the form, so we can get a good template for further BIPs _/ _/ Alex Van Boxel On Sun, Feb 9, 2020 at 6:43 PM Jan Lukavský wrote: > Hi Alex, > > this is cool! Thanks for pushing this topic forward! > > Jan > On 2/9/20

Re: [DISCUSS] BIP reloaded

Hi Alex, this is cool! Thanks for pushing this topic forward! Jan On 2/9/20 6:36 PM, Alex Van Boxel wrote: BIP-1 is available here: https://cwiki.apache.org/confluence/display/BEAM/%5BBIP-1%5D+Beam+Schema+Options  _/ _/ Alex Van Boxel On Sat, Feb 1, 2020 at 9:11 PM Kenneth Knowles

Re: [DISCUSS] BIP reloaded

BIP-1 is available here: https://cwiki.apache.org/confluence/display/BEAM/%5BBIP-1%5D+Beam+Schema+Options _/ _/ Alex Van Boxel On Sat, Feb 1, 2020 at 9:11 PM Kenneth Knowles wrote: > Sounds great. If you scrape recent dev@ for proposals that are not yet > implemented, I think you will find

Re: [DISCUSS] BIP reloaded

Sounds great. If you scrape recent dev@ for proposals that are not yet implemented, I think you will find some, and you could ask them to add as a BIP if they are still interested. Kenn On Sat, Feb 1, 2020 at 1:11 AM Jan Lukavský wrote: > Hi Kenn, > > yes, I can do that. I think that there

Re: [DISCUSS] BIP reloaded

Hi Kenn, yes, I can do that. I think that there should be at least one first BIP, I can try to setup one. But (as opposed to my previous proposal) I'll try to setup a fresh one, not the one of [BEAM-8550], because that one already has a PR and rebasing the PR on master for such a long period

Re: [DISCUSS] BIP reloaded

These stages sound like a great starting point to me. Would you be the volunteer to set up a cwiki page for BIPs? Kenn On Mon, Jan 20, 2020 at 3:30 AM Jan Lukavský wrote: > I agree that we can take inspiration from other projects. Besides the > organizational part (what should be part of BIP,

Re: [DISCUSS] BIP reloaded

I agree that we can take inspiration from other projects. Besides the organizational part (what should be part of BIP, where to store it, how to edit it and how to make it available to the whole community) - for which the cwiki might be the best option - there are still open questions about

Re: [DISCUSS] BIP reloaded

Focusing this thread on the BIP process seems wise, without changing much else in the same thread. I don't think the BIP process has to do with exactly how design docs are written or archived, but the ability to *at a glance* understand: - what are the high level proposals - status of the

Re: [DISCUSS] BIP reloaded

I think that, besides ownership of a feature, a BIP (or whatever document or process) should contain the following:  * description of a problem that the improvement addresses  - this is currently often part of design doc  * description of multiple possible solutions (if multiple exist, which

Re: [DISCUSS] BIP reloaded

Maybe tweaking the current process a bit is enough. I like the Docs for having discussions but there no good as a *proper design document*, for the following reasons: I see design documents full of discussions and wonder: - Who will be the *main owner* and the *co-owners* (meaning people that

Re: [DISCUSS] BIP reloaded

It does seem that the community would find this useful. I agree with Robert that it has downsides and it is not appropriate all the time. We added https://beam.apache.org/roadmap/ a little while ago. I think that the granularity of a BIP is about the same as the granularity of what we would want

Re: [DISCUSS] BIP reloaded

Hi, I feel a "soft consensus" :) that people see some benefits of introducing (possibly optional) process of proposing new features. I think that in order to proceed with this we need to agree on goals that we want to achieve. Whether the process should or should not be optional, which form

Re: [DISCUSS] BIP reloaded

It seems that lots of people see benefit in a more formalized BIP process. I think that makes sense, though I'd like to give people the freedom to choose the medium for their design discussions. The projects I'm aware of usually do this through wiki-type mediums. We have cwiki, though lots of

Re: [DISCUSS] BIP reloaded

The main benefit of BIPs I see is the visibility they create for the project users and contributors. Right now, we have a long unordnered list of design documents. Some of the documents are not even in that list. With BIPs, we would end up with an ordered list "BIP-1, BIP-2, .." which

Re: [DISCUSS] BIP reloaded

Additional process is a two-edged sword: it can help move stuff forward, to the correct decision, but it can also add significant overhead. I think there are many proposals for which the existing processes of deriving consensus (over email, possibly followed by a formal vote or lazy consensus)

Re: [DISCUSS] BIP reloaded

Hi, thanks for reactions so far. I agree that there are many questions that have to be clarified. I'd propose to split this into two parts:  a) first reach a consensus that we want this process in the first place  b) after that, we need to clarify all the details - that will probably be

Re: [DISCUSS] BIP reloaded

+1 for formalizing the process, enhancing it and documenting clearly. I noticed that Apache Airflow has a cool way of both creating AIPs and keeping track of all of them. There is a "Create new AIP" button on

Re: [DISCUSS] BIP reloaded

Thanks for bring up this discussion Jan! +1 for cearly define BIP for beam. And I think would be nice to initialize a concept document for BIP. Just a reminder: the document may contains: - How many kinds of improvement in beam. - What kind of improvement should to create a BIP. - What should

Re: [DISCUSS] BIP reloaded

Hi Jan, I think this is more pretty much what we currently do, just a little bit more transparent for the community. If the process is standardized, it can open doors for bigger contributions from people not familiar with the process. Also it's way easier to track progress of BIPs, than documents