> On Jan 30, 2018, at 7:31 PM, Harsha <st...@harsha.io> wrote: > > Hi, > In general connectors are independent of Storm run-time for most > parts. I.e if the APIs are not changed (storm-core or trident haven't changed > in years except the package re-name). You can take the latest connector and > run in storm 1.0 or higher. So the users doesn't need to upgrade their storm > cluster just to get a latest connector upgrade. Which they might be doing it > but by making the release separate and stating the minimum supported storm > version for the connectors will help the users. > This makes it easier for the connectors to be released independently of the > core/run-time and makes it easy for them to be fixed and released more often. > But moving them to Bahir or other external project will make it detached from > Storm itself that it might not see any co-ordination as reviewers from storm > will need to be aware of an external project. > My proposal would be > 1. Can we create a sub-project in git under Storm so we can move the > connectors there and everything else related Storm applies there. > 2. Can we keep maintaining storm connectors within same repo but different > release module for it .
+1 That’s exactly my point. Just jettisoning connectors to Bahir without commitments from the Storm community would be a mistake. Releasing connectors independently can be handled easily at the Maven level. No need for a separate repo initiaially. > > This is a separate topic but can improve the release timelines if we have > multiple release managers that are handling the maint release and also main > release versions. Its good to have rotation of release managers from PMC so > that everyone will understand the process and can spread the > responsibilities. There are threads started before but don't think they are > addressed or any action item is taken. We should start another thread to > discuss this process as well. Breaking up external modules into separately released versions would be a great way to indoctrinate those new to the license grooming and release process. Everyone could participate. > > Thanks, > Harsha -Taylor > >> On Tue, Jan 30, 2018, at 9:49 AM, Hugo Da Cruz Louro wrote: >> I think that the bahir approach makes sense for connectors that don’t >> fall into the "first class support” category. I am in favor of moving >> such lower adoption connectors and have the interested communities >> support them with the most suitable release cycle. Connectors that are >> idle, such as some examples that Jungtaek gave, we should consider >> removing them altogether, especially if they are so outdated that they >> may not even work. >> >> Mainstream connectors such as storm-kafka-client should be kept in the >> Storm repo. For example, Flink keeps flink-connector-kafka-0.x in the >> Flink repo. >> >> I am in agreement with Jungtaek when he says: "fixing critical bugs in >> storm-kafka-client should trigger release, instead of waiting for Storm >> core to have some fixes to be worth to release”. Storm’s release cadence >> is currently not very high and one can argue that Storm entirely could >> benefit from more frequent releases. If it is sto rm-kafka-client >> triggering those releases, so be it. Moving forward I do not expect the >> storm-kafka-client connector to be subject to so many changes that it >> would warrant its own release cycle. >> >> I also would like to highlight that although storm-kafka-client has been >> the center of this discussion, as it was mentioned in this >> thread<https://goo.gl/VY7QTG>, storm-kafka-client has had a much less >> rocky road to stability compared to for example storm-kafka. Therefore >> it’s worth evaluating if the challenges that we have faced with storm- >> kafka-client have been out of norm for such an important and complex >> feature, and if they warrant significant changes in how we do things. >> >> Thanks, >> Hugo >> >> On Jan 29, 2018, at 9:18 PM, Jungtaek Lim >> <kabh...@gmail.com<mailto:kabh...@gmail.com>> wrote: >> >> Let me add a proof of my opinion: major patch of storm-eventhubs hasn't >> been getting even a comment over 4 months. >> https://github.com/apache/storm/pull/2322 >> >> I'd rather want to discuss regarding discontinue supporting officially if >> we no longer interest of, or we don't have resource to support, or any >> valid reasons. If we agree on discontinue supporting officially, we can >> move out to other repo. and let it self maintained. It may be able to get >> attention and have enough contributors so that we feel better to get to >> Storm core Repository again, or it can be silently forgotten. It shouldn't >> affect Storm core repository at any case. >> >> 2018년 1월 30일 (화) 오후 2:03, Jungtaek Lim <kabh...@gmail.com>님이 작성: >> >> If we worry about breaking somethings along with our >> users/consumers/distributors, picking one of less used/updated connector as >> experiment makes more sense to me. It's OK if we want to pick one of most >> active and widely used connector intentionally to accelerate experiment. >> >> Decoupling connectors and moving to other repo. like Bahir will make it >> clear who are having interest of which connectors. storm-eventhubs for >> example, major code contributions were done from MS developers. Now they >> are gone, and I don't know even storm-eventhubs are compatible with recent >> Azure Eventhub. That's just a one of them. I've seen many connectors in >> same, or similar, or possible (say truck number 1) situation. >> >> -Jungtaek Lim (HeartSaVioR) >> >> 2018년 1월 30일 (화) 오후 1:30, P. Taylor Goetz <ptgo...@gmail.com>님이 작성: >> >> >> On Jan 29, 2018, at 8:03 PM, Jungtaek Lim <kabh...@gmail.com> wrote: >> >> >> >> - Do we ensure they're all maintained? >> -- Did we exclude inactive committers/PMCs for connector's committer >> >> sponsors, and do they have enough committer sponsors after that? >> >> >> Good point. We’ve had some sponsors go silent recently. Maybe ping >> sponsors and ask if they wish to maintain sponsorship? >> >> As a sponsor for a number of connectors, I’ll check on the ones I’ve >> sponsored. >> >> - Do they all worth to keep maintaining in Storm main repository? >> >> >> Again, that’s a question of whether there is user/dev interest. >> >> >> -- Should we trigger release if we find and resolve critical/blocker issue >> from them? If not, why we allow to leave the thing which is in main >> repository as inconsistent state? >> >> >> Some are tied to fairly well established protocols, some target really >> volatile APIs. Bug reports and mailing list activity may not be a good >> status indicator. >> >> Storm’s Kafka integration was the initial model for the “batteries >> included” impetus behind `external`. If we want to evolve how that works, >> why not start there, see what works/doesn’t work, and adapt. >> >> I don’t want to shock our users/consumers/distributors. >> >> >> -Taylor >> >> >> >> >> >> >>
