Also, in the interest of getting releases out, we have 3 open RC cycles in flight.
Discussion energy might be better focused on that. -Taylor > On Jan 30, 2018, at 7:52 PM, P. Taylor Goetz <ptgo...@gmail.com> wrote: > > > >> On Jan 30, 2018, at 7:31 PM, Harsha <st...@harsha.io> wrote: >> >> Hi, >> In general connectors are independent of Storm run-time for most >> parts. I.e if the APIs are not changed (storm-core or trident haven't >> changed in years except the package re-name). You can take the latest >> connector and run in storm 1.0 or higher. So the users doesn't need to >> upgrade their storm cluster just to get a latest connector upgrade. Which >> they might be doing it but by making the release separate and stating the >> minimum supported storm version for the connectors will help the users. >> This makes it easier for the connectors to be released independently of the >> core/run-time and makes it easy for them to be fixed and released more >> often. But moving them to Bahir or other external project will make it >> detached from Storm itself that it might not see any co-ordination as >> reviewers from storm will need to be aware of an external project. >> My proposal would be >> 1. Can we create a sub-project in git under Storm so we can move the >> connectors there and everything else related Storm applies there. >> 2. Can we keep maintaining storm connectors within same repo but different >> release module for it . > > +1 That’s exactly my point. Just jettisoning connectors to Bahir without > commitments from the Storm community would be a mistake. > > Releasing connectors independently can be handled easily at the Maven level. > No need for a separate repo initiaially. > > >> >> This is a separate topic but can improve the release timelines if we have >> multiple release managers that are handling the maint release and also main >> release versions. Its good to have rotation of release managers from PMC so >> that everyone will understand the process and can spread the >> responsibilities. There are threads started before but don't think they are >> addressed or any action item is taken. We should start another thread to >> discuss this process as well. > > Breaking up external modules into separately released versions would be a > great way to indoctrinate those new to the license grooming and release > process. Everyone could participate. > > >> >> Thanks, >> Harsha > > -Taylor > >> >>> On Tue, Jan 30, 2018, at 9:49 AM, Hugo Da Cruz Louro wrote: >>> I think that the bahir approach makes sense for connectors that don’t >>> fall into the "first class support” category. I am in favor of moving >>> such lower adoption connectors and have the interested communities >>> support them with the most suitable release cycle. Connectors that are >>> idle, such as some examples that Jungtaek gave, we should consider >>> removing them altogether, especially if they are so outdated that they >>> may not even work. >>> >>> Mainstream connectors such as storm-kafka-client should be kept in the >>> Storm repo. For example, Flink keeps flink-connector-kafka-0.x in the >>> Flink repo. >>> >>> I am in agreement with Jungtaek when he says: "fixing critical bugs in >>> storm-kafka-client should trigger release, instead of waiting for Storm >>> core to have some fixes to be worth to release”. Storm’s release cadence >>> is currently not very high and one can argue that Storm entirely could >>> benefit from more frequent releases. If it is sto rm-kafka-client >>> triggering those releases, so be it. Moving forward I do not expect the >>> storm-kafka-client connector to be subject to so many changes that it >>> would warrant its own release cycle. >>> >>> I also would like to highlight that although storm-kafka-client has been >>> the center of this discussion, as it was mentioned in this >>> thread<https://goo.gl/VY7QTG>, storm-kafka-client has had a much less >>> rocky road to stability compared to for example storm-kafka. Therefore >>> it’s worth evaluating if the challenges that we have faced with storm- >>> kafka-client have been out of norm for such an important and complex >>> feature, and if they warrant significant changes in how we do things. >>> >>> Thanks, >>> Hugo >>> >>> On Jan 29, 2018, at 9:18 PM, Jungtaek Lim >>> <kabh...@gmail.com<mailto:kabh...@gmail.com>> wrote: >>> >>> Let me add a proof of my opinion: major patch of storm-eventhubs hasn't >>> been getting even a comment over 4 months. >>> https://github.com/apache/storm/pull/2322 >>> >>> I'd rather want to discuss regarding discontinue supporting officially if >>> we no longer interest of, or we don't have resource to support, or any >>> valid reasons. If we agree on discontinue supporting officially, we can >>> move out to other repo. and let it self maintained. It may be able to get >>> attention and have enough contributors so that we feel better to get to >>> Storm core Repository again, or it can be silently forgotten. It shouldn't >>> affect Storm core repository at any case. >>> >>> 2018년 1월 30일 (화) 오후 2:03, Jungtaek Lim <kabh...@gmail.com>님이 작성: >>> >>> If we worry about breaking somethings along with our >>> users/consumers/distributors, picking one of less used/updated connector as >>> experiment makes more sense to me. It's OK if we want to pick one of most >>> active and widely used connector intentionally to accelerate experiment. >>> >>> Decoupling connectors and moving to other repo. like Bahir will make it >>> clear who are having interest of which connectors. storm-eventhubs for >>> example, major code contributions were done from MS developers. Now they >>> are gone, and I don't know even storm-eventhubs are compatible with recent >>> Azure Eventhub. That's just a one of them. I've seen many connectors in >>> same, or similar, or possible (say truck number 1) situation. >>> >>> -Jungtaek Lim (HeartSaVioR) >>> >>> 2018년 1월 30일 (화) 오후 1:30, P. Taylor Goetz <ptgo...@gmail.com>님이 작성: >>> >>> >>> On Jan 29, 2018, at 8:03 PM, Jungtaek Lim <kabh...@gmail.com> wrote: >>> >>> >>> >>> - Do we ensure they're all maintained? >>> -- Did we exclude inactive committers/PMCs for connector's committer >>> >>> sponsors, and do they have enough committer sponsors after that? >>> >>> >>> Good point. We’ve had some sponsors go silent recently. Maybe ping >>> sponsors and ask if they wish to maintain sponsorship? >>> >>> As a sponsor for a number of connectors, I’ll check on the ones I’ve >>> sponsored. >>> >>> - Do they all worth to keep maintaining in Storm main repository? >>> >>> >>> Again, that’s a question of whether there is user/dev interest. >>> >>> >>> -- Should we trigger release if we find and resolve critical/blocker issue >>> from them? If not, why we allow to leave the thing which is in main >>> repository as inconsistent state? >>> >>> >>> Some are tied to fairly well established protocols, some target really >>> volatile APIs. Bug reports and mailing list activity may not be a good >>> status indicator. >>> >>> Storm’s Kafka integration was the initial model for the “batteries >>> included” impetus behind `external`. If we want to evolve how that works, >>> why not start there, see what works/doesn’t work, and adapt. >>> >>> I don’t want to shock our users/consumers/distributors. >>> >>> >>> -Taylor >>> >>> >>> >>> >>> >>> >>>