Seems reasonable to have a way to make it easy for people to follow important
things for people who are not glued to the mailing lists.
If the desire is to keep it light weight and still achieve the above goal ..
the SIP can be just a short title and description with a link to the JIRA
which contains the design doc and other details and other discussion… all in
one place. Some overhead may come in the form of voting procedures if we
decide to emulate Kafka procedures.
I suspect, SIPs for minor new settings/configs to some of the connectors can
lead to too many SIPs … and make it less “follow-able”. But I don’t feel
strongly for/against that.
-roshan
On 6/9/17, 9:16 PM, "Harsha" <st...@harsha.io> wrote:
Arun,
For big features we did follow design doc/review. Making it
formal makes everyone to follow a process.
Again this process is not for bug fixes as we stated its about New
Features/Config Changes/Public interface changes. I don't think it puts
any extra effort for anyone who is writing detailed JIRA but by making
it formal makes everyone to add these details in a centra process. Not
everyone will look at mailing list but its easier to follow a wiki page.
We should atleast give it a try before we vote it out.
Roshan,
Adding connector should require a SIP as well and changing any
public interfaces should be a KIP. Intention here is we've
central place where everyone can follow in detail whats the
public interface/new feature changes went in. We've changed
KafkaSpout quite a bit and there is current discussion thats
going to change it , having this documented in a central place
will make it easy to follow and recording them in release notes
as well.
Taylor,
We can't call it a too tedious process without even giving it a
try. This has been followed to a greater success at kafka and
also Flink started the process as well
https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
.
If it actually proved to more of hindrance than helping the community we
can move away from it.
" Kafka has somewhat of a reputation for setting potentially too high a
bar. I'd rather not see that happen with this community."
Sure. But it also depends on the community. Just because some community
enforcing too high bar that doesn't mean we are trying to do it via this
process. Again we always have option if we ever veer too far in the
wrong direction to bring up and improve or remove this process.
We should also as a community strive to have better quality and I am
hoping this will give us a chance to not only let users know what are
changes coming in but also keep the dev list to have a chance and join
the discussion.
-Harsha
On Jun 9, 2017, 7:18 PM -0700, Arun Iyer <ai...@hortonworks.com>, wrote:
I am for documenting and upfront design reviews, but maybe we should
keep it less formal and make it part of the JIRA to start with.
Do we have any upcoming features for which we would like to see a
proposal? May be start with a couple of proposals
and see it works out before making it formal.
Thanks,
Arun
6/9/17, 6:49 PM, "P. Taylor Goetz" <ptgo...@gmail.com> wrote:
-0
The KIP process feels kind of heavy. I'd rather start with a lighter
effort like improving JIRA submissions and pull requests (some pull
requests/JIRAs, even from committers and PMC members, are woefully
inadequate in terms of detail), and see how that works out.
I share Bobby's concern that doing so might raise the bar for
contributions and potentially have a chilling effect. We don't want to
scare away contributors. Kafka has somewhat of a reputation for setting
potentially too high a bar. I'd rather not see that happen with this
community.
I will say that I like the idea of proposals for big features, ideally
before any coding even begins -- so that others have a chance to
collaborate. But I'm hesitant to impose too much process, voting, etc.
That could scare people off.
I think we should think carefully before going down this trail.
-Taylor
On Jun 9, 2017, at 8:57 PM, Priyank Shah <ps...@hortonworks.com> wrote:
+1 for SIPs including a new connector. The person writing SIP can
provide details about the external system for which connector is being
written to let others know why a certain design decision was made. This
will make it easy for reviewers.
On 6/9/17, 5:24 PM, "Satish Duggana" <satish.dugg...@gmail.com> wrote:
+1 for SIPs. It is so useful as mentioned by others in earlier mails. It
would be very useful for new contributors and others who are looking out
for a feature design and decisions taken etc.
Whenever a minor feature is added to a connector it may not need a
separate
SIP but the existing README can be updated with details for users. It
can
be discussed and decided apropos whether a SIP is really required for
any
enhancement which is not really big.
On Sat, Jun 10, 2017 at 5:13 AM, Roshan Naik <ros...@hortonworks.com>
wrote:
If I am looking at the Kafka site correctly, I see that Kafka has a
total
of 167 KIPs so far.
So I assume that minor new features would not be parrt of the SIP ?
Unlike Kafka, since Storm has a number of connectors (that keep
growing),
I am speculating the SIP process might get somewhat unwieldy if it were
to
track little changes in each of the connectors.
Also thinking that a SIP may not be needed to justify a new connector,
but
useful if we are replacing an old connector with a new one.
-roshan
On 6/9/17, 3:19 PM, "Harsha" <st...@harsha.io> wrote:
Hi Bobby,
In general, a KIP is required for adding New features,
config
changes or backward-incompatible changes. Don't require
adding a KIP for bug-fixes. Devs who wants to add any
features will write up a wiki which has JIRA link, mailing
list discussion link and outline the Motivation, Public
interface changes and protocol changes etc ..a good example
here is
https://cwiki.apache.org/confluence/display/KAFKA/KIP-
48+Delegation+token+support+for+Kafka.
They can start the discussion thread once its ready and once everyone
agrees its in a good shape, a Vote thread starts . Once there are
required votes are in one can start the PR process and get it merged
in.
Each release we can collect what features/fixes especially
to
public interfaces that went in and roll it out in release
notes. This will give a better idea for the users on what
changed and added from previous version.
We can only enforce this to new feature/config/backward
incompatible change. Having this go through the discussion
phase will give us the early feedback and potentially caught
any issues before the implementation.
Thanks,
Harsha
On Fri, Jun 9, 2017 at 2:24 PM Bobby Evans <ev...@yahoo-inc.com.invalid
wrote:
Can you please explain how KIP currently works and how you would
like to see something similar in storm?
If we make the process more formal we will probably have less
people
contributing, but we will probably have better overall patches. It
is a balancing act and having never used KIP I would like to
understand it better before going all in on it.
- Bobby
On Friday, June 9, 2017, 4:09:38 PM CDT, Stig Døssing
<generalbas....@gmail.com> wrote:
This sounds like a good idea. KIPs seem to work well for Kafka.
It's
easy
for discussions to get lost or just not seen on the mailing list.
2017-06-09 21:36 GMT+02:00 Harsha <st...@harsha.io>:
Hi All,
We’ve seen good adoption of KIP approach in Kafka
community
and would like to see we adopt the similar approach for
storm
as well.
Its hard to keep track of proposed changes and mailing list
threads to
know what all changes that are coming into and what
design/backward
incompatible changes being approved. It will be good to have
this
documented and go through discussion then Vote phase to get them
approved before we merge the PRs. This will keep everyone
informed of
what changes happened even if they are not following the mailing
list
they can go to wiki to see the list of changes went into a
release.
Community overall will be well informed of the changes as well.
Would
like to hear your thoughts.
Thanks,
Harsha
