Also, in the interest of getting releases out, we have 3 open RC cycles in 
flight.

Discussion energy might be better focused on that.

-Taylor

> On Jan 30, 2018, at 7:52 PM, P. Taylor Goetz <ptgo...@gmail.com> wrote:
> 
> 
> 
>> On Jan 30, 2018, at 7:31 PM, Harsha <st...@harsha.io> wrote:
>> 
>> Hi,
>>           In general connectors are independent of Storm run-time for most 
>> parts. I.e if the APIs are not changed (storm-core or trident haven't 
>> changed in years except the package re-name). You can take the latest 
>> connector and run in storm 1.0 or higher. So the users doesn't need to 
>> upgrade their storm cluster just to get a latest connector upgrade. Which 
>> they might be doing it but by making the release separate and stating the 
>> minimum supported storm version for the connectors will help the users.
>> This makes it easier for the connectors to be released  independently of the 
>> core/run-time and makes it easy for them to be fixed and released more 
>> often. But moving them to Bahir or other external project will make it 
>> detached from Storm itself that it might not see any co-ordination as 
>> reviewers from storm  will need to be aware of an external project. 
>> My proposal would be 
>> 1. Can we create a sub-project in git under Storm so we can move the 
>> connectors there and everything else related Storm applies there.
>> 2.  Can we keep maintaining storm connectors within same repo but different 
>> release module for it .
> 
> +1 That’s exactly my point. Just jettisoning connectors to Bahir without 
> commitments from the Storm community would be a mistake.
> 
> Releasing connectors independently can be handled easily at the Maven level. 
> No need for a separate repo initiaially.
> 
> 
>> 
>> This is a separate topic but can improve the release timelines if we have 
>> multiple release managers that are handling the maint release and also main 
>> release versions. Its good to have rotation of release managers from PMC so 
>> that everyone will understand the process and can spread the 
>> responsibilities. There are threads started before but don't think they are 
>> addressed or any action item is taken. We should start another thread to 
>> discuss this process as well.
> 
> Breaking up external modules into separately released versions would be a 
> great way to indoctrinate those new to the license grooming and release 
> process. Everyone could participate.
> 
> 
>> 
>> Thanks,
>> Harsha
> 
> -Taylor
> 
>> 
>>> On Tue, Jan 30, 2018, at 9:49 AM, Hugo Da Cruz Louro wrote:
>>> I think that the bahir approach makes sense for connectors that don’t 
>>> fall into the "first class support” category. I am in favor of moving 
>>> such lower adoption connectors and have the interested communities 
>>> support them with the most suitable release cycle. Connectors that are 
>>> idle, such as some examples that Jungtaek gave, we should consider 
>>> removing them altogether, especially if they are so outdated that they 
>>> may not even work.
>>> 
>>> Mainstream connectors such as storm-kafka-client should be kept in the 
>>> Storm repo. For example, Flink keeps flink-connector-kafka-0.x in the 
>>> Flink repo.
>>> 
>>> I am in agreement with Jungtaek when he says: "fixing critical bugs in 
>>> storm-kafka-client should trigger release, instead of waiting for Storm 
>>> core to have some fixes to be worth to release”. Storm’s release cadence 
>>> is currently not very high and one can argue that Storm entirely could 
>>> benefit from more frequent releases. If it is sto rm-kafka-client 
>>> triggering those releases, so be it. Moving forward I do not expect the 
>>> storm-kafka-client connector to be subject to so many changes that it 
>>> would warrant its own release cycle.
>>> 
>>> I also would like to highlight that although storm-kafka-client has been 
>>> the center of this discussion, as it was mentioned in this 
>>> thread<https://goo.gl/VY7QTG>, storm-kafka-client has had a much less 
>>> rocky road to stability compared to for example storm-kafka. Therefore 
>>> it’s worth evaluating if the challenges that we have faced with storm-
>>> kafka-client have been out of norm for such an important and complex 
>>> feature, and if they warrant significant changes in how we do things.
>>> 
>>> Thanks,
>>> Hugo
>>> 
>>> On Jan 29, 2018, at 9:18 PM, Jungtaek Lim 
>>> <kabh...@gmail.com<mailto:kabh...@gmail.com>> wrote:
>>> 
>>> Let me add a proof of my opinion: major patch of storm-eventhubs hasn't
>>> been getting even a comment over 4 months.
>>> https://github.com/apache/storm/pull/2322
>>> 
>>> I'd rather want to discuss regarding discontinue supporting officially if
>>> we no longer interest of, or we don't have resource to support, or any
>>> valid reasons. If we agree on discontinue supporting officially, we can
>>> move out to other repo. and let it self maintained. It may be able to get
>>> attention and have enough contributors so that we feel better to get to
>>> Storm core Repository again, or it can be silently forgotten. It shouldn't
>>> affect Storm core repository at any case.
>>> 
>>> 2018년 1월 30일 (화) 오후 2:03, Jungtaek Lim <kabh...@gmail.com>님이 작성:
>>> 
>>> If we worry about breaking somethings along with our
>>> users/consumers/distributors, picking one of less used/updated connector as
>>> experiment makes more sense to me. It's OK if we want to pick one of most
>>> active and widely used connector intentionally to accelerate experiment.
>>> 
>>> Decoupling connectors and moving to other repo. like Bahir will make it
>>> clear who are having interest of which connectors. storm-eventhubs for
>>> example, major code contributions were done from MS developers. Now they
>>> are gone, and I don't know even storm-eventhubs are compatible with recent
>>> Azure Eventhub. That's just a one of them. I've seen many connectors in
>>> same, or similar, or possible (say truck number 1) situation.
>>> 
>>> -Jungtaek Lim (HeartSaVioR)
>>> 
>>> 2018년 1월 30일 (화) 오후 1:30, P. Taylor Goetz <ptgo...@gmail.com>님이 작성:
>>> 
>>> 
>>> On Jan 29, 2018, at 8:03 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
>>> 
>>> 
>>> 
>>> - Do we ensure they're all maintained?
>>> -- Did we exclude inactive committers/PMCs for connector's committer
>>> 
>>> sponsors, and do they have enough committer sponsors after that?
>>> 
>>> 
>>> Good point. We’ve had some sponsors go silent recently. Maybe ping
>>> sponsors and ask if they wish to maintain sponsorship?
>>> 
>>> As a sponsor for a number of connectors, I’ll check on the ones I’ve
>>> sponsored.
>>> 
>>> - Do they all worth to keep maintaining in Storm main repository?
>>> 
>>> 
>>> Again, that’s a question of whether there is user/dev interest.
>>> 
>>> 
>>> -- Should we trigger release if we find and resolve critical/blocker issue
>>> from them? If not, why we allow to leave the thing which is in main
>>> repository as inconsistent state?
>>> 
>>> 
>>> Some are tied to fairly well established protocols, some target really
>>> volatile APIs. Bug reports and mailing list activity may not be a good
>>> status indicator.
>>> 
>>> Storm’s Kafka integration was the initial model for the “batteries
>>> included” impetus behind `external`. If we want to evolve how that works,
>>> why not start there, see what works/doesn’t work, and adapt.
>>> 
>>> I don’t want to shock our users/consumers/distributors.
>>> 
>>> 
>>> -Taylor
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 

Reply via email to