I think we actually tried similar thing once: - Issue: https://issues.apache.org/jira/browse/STORM-2016 - Wiki page: https://cwiki.apache.org/confluence/display/STORM/A.+Design+doc%3A+adding+jars+and+maven+artifacts+at+submission - Discussion thread: http://mail-archives.apache.org/mod_mbox/storm-dev/201608.mbox/%3ccaf5108i9+tjanz0lgrktmkvqel7f+53k9uyzxct6zhsu6oh...@mail.gmail.com%3E - Pull request: https://github.com/apache/storm/pull/1608
That was definitely new feature which involved huge code change, and close to core feature (though it was in 'external' directory), so ideal to have KIP-like thing. I'm +1 to have this procedure for 'core' and 'new feature', especially for 'breaking change'. For connectors, we should "lower" the workload instead of adding the workload. So -0 for that if we just apply procedure without putting efforts to lower the workload. We're maintaining 20 of connectors officially, and some of them are already outdated. (Who really uses ES 1.x nowadays?) Committer sponsor seems not work as expected: there's no balance between committer sponsors. We may want to consider bahir or alike approach, or even renew committer sponsors and drop supporting if it doesn't meet several committer sponsors. Code quality comes out when we maintain only we can manage it. We also may need to define 'beta' or 'unstable' for newly introduced thing, or even having incubating phase from other repo. 'storm-kafka-client' for example, it has been having critical or even blocker bugs dragging whole release, and we also allowed breaking backward compatibility between 1.0.x and 1.1.0. (Hope anyone doesn't take this as offensive. Newly introduced thing should have time to stabilize and that's natural.) In this case, we may need to allow more breaking change whatever it can help stabilizing the module. So I'd also like to see our efforts to lower workload. We've some possible improvements: introducing merge script, removing CHANGELOG (It's so easy to be out of control with handling multiple version lines.), or whatever we could automate or get rid some redundant procedures. Thanks, Jungtaek Lim (HeartSaVioR) 2017년 6월 10일 (토) 오후 2:14, Roshan Naik <ros...@hortonworks.com>님이 작성: > Seems reasonable to have a way to make it easy for people to follow > important things for people who are not glued to the mailing lists. > > If the desire is to keep it light weight and still achieve the above goal > .. the SIP can be just a short title and description with a link to the > JIRA which contains the design doc and other details and other discussion… > all in one place. Some overhead may come in the form of voting procedures > if we decide to emulate Kafka procedures. > > I suspect, SIPs for minor new settings/configs to some of the connectors > can lead to too many SIPs … and make it less “follow-able”. But I don’t > feel strongly for/against that. > > -roshan > > > On 6/9/17, 9:16 PM, "Harsha" <st...@harsha.io> wrote: > > Arun, > For big features we did follow design doc/review. Making it > formal makes everyone to follow a process. > Again this process is not for bug fixes as we stated its about New > Features/Config Changes/Public interface changes. I don't think it puts > any extra effort for anyone who is writing detailed JIRA but by making > it formal makes everyone to add these details in a centra process. Not > everyone will look at mailing list but its easier to follow a wiki > page. > We should atleast give it a try before we vote it out. > > Roshan, > Adding connector should require a SIP as well and changing any > public interfaces should be a KIP. Intention here is we've > central place where everyone can follow in detail whats the > public interface/new feature changes went in. We've changed > KafkaSpout quite a bit and there is current discussion thats > going to change it , having this documented in a central place > will make it easy to follow and recording them in release > notes > as well. > > Taylor, > We can't call it a too tedious process without even giving it a > try. This has been followed to a greater success at kafka and > also Flink started the process as well > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > . > If it actually proved to more of hindrance than helping the community > we > can move away from it. > > " Kafka has somewhat of a reputation for setting potentially too high a > bar. I'd rather not see that happen with this community." > Sure. But it also depends on the community. Just because some community > enforcing too high bar that doesn't mean we are trying to do it via > this > process. Again we always have option if we ever veer too far in the > wrong direction to bring up and improve or remove this process. > > We should also as a community strive to have better quality and I am > hoping this will give us a chance to not only let users know what are > changes coming in but also keep the dev list to have a chance and join > the discussion. > > -Harsha > > On Jun 9, 2017, 7:18 PM -0700, Arun Iyer <ai...@hortonworks.com>, > wrote: > I am for documenting and upfront design reviews, but maybe we should > keep it less formal and make it part of the JIRA to start with. > > Do we have any upcoming features for which we would like to see a > proposal? May be start with a couple of proposals > and see it works out before making it formal. > > > Thanks, > Arun > > > > 6/9/17, 6:49 PM, "P. Taylor Goetz" <ptgo...@gmail.com> wrote: > > -0 > > The KIP process feels kind of heavy. I'd rather start with a lighter > effort like improving JIRA submissions and pull requests (some pull > requests/JIRAs, even from committers and PMC members, are woefully > inadequate in terms of detail), and see how that works out. > > I share Bobby's concern that doing so might raise the bar for > contributions and potentially have a chilling effect. We don't want to > scare away contributors. Kafka has somewhat of a reputation for setting > potentially too high a bar. I'd rather not see that happen with this > community. > > I will say that I like the idea of proposals for big features, ideally > before any coding even begins -- so that others have a chance to > collaborate. But I'm hesitant to impose too much process, voting, etc. > That could scare people off. > > I think we should think carefully before going down this trail. > > -Taylor > > On Jun 9, 2017, at 8:57 PM, Priyank Shah <ps...@hortonworks.com> > wrote: > > +1 for SIPs including a new connector. The person writing SIP can > provide details about the external system for which connector is being > written to let others know why a certain design decision was made. This > will make it easy for reviewers. > > On 6/9/17, 5:24 PM, "Satish Duggana" <satish.dugg...@gmail.com> wrote: > > +1 for SIPs. It is so useful as mentioned by others in earlier mails. > It > would be very useful for new contributors and others who are looking > out > for a feature design and decisions taken etc. > > Whenever a minor feature is added to a connector it may not need a > separate > SIP but the existing README can be updated with details for users. It > can > be discussed and decided apropos whether a SIP is really required for > any > enhancement which is not really big. > > > On Sat, Jun 10, 2017 at 5:13 AM, Roshan Naik <ros...@hortonworks.com> > wrote: > > If I am looking at the Kafka site correctly, I see that Kafka has a > total > of 167 KIPs so far. > So I assume that minor new features would not be parrt of the SIP ? > > Unlike Kafka, since Storm has a number of connectors (that keep > growing), > I am speculating the SIP process might get somewhat unwieldy if it were > to > track little changes in each of the connectors. > > Also thinking that a SIP may not be needed to justify a new connector, > but > useful if we are replacing an old connector with a new one. > > -roshan > > > > On 6/9/17, 3:19 PM, "Harsha" <st...@harsha.io> wrote: > > Hi Bobby, > In general, a KIP is required for adding New features, > config > changes or backward-incompatible changes. Don't require > adding a KIP for bug-fixes. Devs who wants to add any > features will write up a wiki which has JIRA link, mailing > list discussion link and outline the Motivation, Public > interface changes and protocol changes etc ..a good example > here is > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 48+Delegation+token+support+for+Kafka. > They can start the discussion thread once its ready and once everyone > agrees its in a good shape, a Vote thread starts . Once there are > required votes are in one can start the PR process and get it merged > in. > Each release we can collect what features/fixes especially > to > public interfaces that went in and roll it out in release > notes. This will give a better idea for the users on what > changed and added from previous version. > We can only enforce this to new feature/config/backward > incompatible change. Having this go through the discussion > phase will give us the early feedback and potentially caught > any issues before the implementation. > Thanks, > Harsha > > On Fri, Jun 9, 2017 at 2:24 PM Bobby Evans <ev...@yahoo-inc.com.invalid > > wrote: > > Can you please explain how KIP currently works and how you would > like to see something similar in storm? > If we make the process more formal we will probably have less > people > contributing, but we will probably have better overall patches. It > is a balancing act and having never used KIP I would like to > understand it better before going all in on it. > - Bobby > > > On Friday, June 9, 2017, 4:09:38 PM CDT, Stig Døssing > <generalbas....@gmail.com> wrote: > > This sounds like a good idea. KIPs seem to work well for Kafka. > It's > easy > for discussions to get lost or just not seen on the mailing list. > > 2017-06-09 21:36 GMT+02:00 Harsha <st...@harsha.io>: > > Hi All, > We’ve seen good adoption of KIP approach in Kafka > community > and would like to see we adopt the similar approach for > storm > as well. > Its hard to keep track of proposed changes and mailing list > threads to > know what all changes that are coming into and what > design/backward > incompatible changes being approved. It will be good to have > this > documented and go through discussion then Vote phase to get them > approved before we merge the PRs. This will keep everyone > informed of > what changes happened even if they are not following the mailing > list > they can go to wiki to see the list of changes went into a > release. > Community overall will be well informed of the changes as well. > Would > like to hear your thoughts. > > Thanks, > Harsha > > > > > > > > > >
