I think that the bahir approach makes sense for connectors that don’t fall into the "first class support” category. I am in favor of moving such lower adoption connectors and have the interested communities support them with the most suitable release cycle. Connectors that are idle, such as some examples that Jungtaek gave, we should consider removing them altogether, especially if they are so outdated that they may not even work.
Mainstream connectors such as storm-kafka-client should be kept in the Storm repo. For example, Flink keeps flink-connector-kafka-0.x in the Flink repo. I am in agreement with Jungtaek when he says: "fixing critical bugs in storm-kafka-client should trigger release, instead of waiting for Storm core to have some fixes to be worth to release”. Storm’s release cadence is currently not very high and one can argue that Storm entirely could benefit from more frequent releases. If it is sto rm-kafka-client triggering those releases, so be it. Moving forward I do not expect the storm-kafka-client connector to be subject to so many changes that it would warrant its own release cycle. I also would like to highlight that although storm-kafka-client has been the center of this discussion, as it was mentioned in this thread<https://goo.gl/VY7QTG>, storm-kafka-client has had a much less rocky road to stability compared to for example storm-kafka. Therefore it’s worth evaluating if the challenges that we have faced with storm-kafka-client have been out of norm for such an important and complex feature, and if they warrant significant changes in how we do things. Thanks, Hugo On Jan 29, 2018, at 9:18 PM, Jungtaek Lim <kabh...@gmail.com<mailto:kabh...@gmail.com>> wrote: Let me add a proof of my opinion: major patch of storm-eventhubs hasn't been getting even a comment over 4 months. https://github.com/apache/storm/pull/2322 I'd rather want to discuss regarding discontinue supporting officially if we no longer interest of, or we don't have resource to support, or any valid reasons. If we agree on discontinue supporting officially, we can move out to other repo. and let it self maintained. It may be able to get attention and have enough contributors so that we feel better to get to Storm core Repository again, or it can be silently forgotten. It shouldn't affect Storm core repository at any case. 2018년 1월 30일 (화) 오후 2:03, Jungtaek Lim <kabh...@gmail.com>님이 작성: If we worry about breaking somethings along with our users/consumers/distributors, picking one of less used/updated connector as experiment makes more sense to me. It's OK if we want to pick one of most active and widely used connector intentionally to accelerate experiment. Decoupling connectors and moving to other repo. like Bahir will make it clear who are having interest of which connectors. storm-eventhubs for example, major code contributions were done from MS developers. Now they are gone, and I don't know even storm-eventhubs are compatible with recent Azure Eventhub. That's just a one of them. I've seen many connectors in same, or similar, or possible (say truck number 1) situation. -Jungtaek Lim (HeartSaVioR) 2018년 1월 30일 (화) 오후 1:30, P. Taylor Goetz <ptgo...@gmail.com>님이 작성: On Jan 29, 2018, at 8:03 PM, Jungtaek Lim <kabh...@gmail.com> wrote: - Do we ensure they're all maintained? -- Did we exclude inactive committers/PMCs for connector's committer sponsors, and do they have enough committer sponsors after that? Good point. We’ve had some sponsors go silent recently. Maybe ping sponsors and ask if they wish to maintain sponsorship? As a sponsor for a number of connectors, I’ll check on the ones I’ve sponsored. - Do they all worth to keep maintaining in Storm main repository? Again, that’s a question of whether there is user/dev interest. -- Should we trigger release if we find and resolve critical/blocker issue from them? If not, why we allow to leave the thing which is in main repository as inconsistent state? Some are tied to fairly well established protocols, some target really volatile APIs. Bug reports and mailing list activity may not be a good status indicator. Storm’s Kafka integration was the initial model for the “batteries included” impetus behind `external`. If we want to evolve how that works, why not start there, see what works/doesn’t work, and adapt. I don’t want to shock our users/consumers/distributors. -Taylor
