If the storm-kafka-client is fairly stable with the changes we made in the 1.2 release, then would we just want to continue the current process ?
If we want to decouple, I think option 2 may be better than option 1 to start with. When you say storm-kafka-client-vX.X.X, is it going to be completely independent of the Storm version and will work across Storm 1.0.x, 1.x and 2.0 (future) storm releases? Thanks, Arun On 1/29/18, 12:27 PM, "P. Taylor Goetz" <ptgo...@gmail.com> wrote: >To give some background information, the Spark PMC decided to remove a number >of connectors from their repo (and thus releases). Some members of the >community wanted to see some sort of official community support for those >connectors, thus the Apache Bahir project was created. Flink also decided to >follow that model. > >I don’t feel we’ve reached the same conclusion (to remove connectors from our >distribution), or, perhaps, not yet. I think where we are right now is wanting >to decouple storm-kafka-client from Storm’s release cycle so updates can be >released more often. > >As I mentioned in the vote thread, there are two approaches we could take: > >1. Move storm-kafka-client to a new git repo under the purview of the Storm >PMC. >2. Leave storm-kafka-client in place, but decouple it from the main >build/release process so it can be released independently of storm proper. > >I lean toward option 2. One downside as Stig pointed out is that it would make >tagging awkward, but I don’t see that as much of a problem — we could simply >keep the top-level tag convention “vX.X.X” and add another along the lines of >“storm-kafka-client-vX.X.X”. Yes, it adds another tag/tag convention, but even >if we moved all the connectors to another repo and versioned them all >independently we would have to do the same in that repo. > >If we start with option 2, it would make it easier to rollback if for some >reason we later decided it was’t a good idea. It could also represent a “try >before you buy” option toward moving to option 1. > >The argument I see for option 1 would be a better IDE experience — i.e. you >would have two separate IDE projects that might make development/testing with >various versions easier. > >I’m open to either approach and would volunteer to do the work to make it >happen. > >-Taylor > >> On Jan 29, 2018, at 12:45 AM, Jungtaek Lim <kabh...@gmail.com> wrote: >> >> My idea is basically came from Apache Bahir. (http://bahir.apache.org/) It >> was for Apache Spark, but Flink decided to migrate their connectors to >> Bahir so it is also for Apache Flink. They're also maintaining some >> connectors (I'd say first class support) in their repositories, but not >> all. I think we could select some of connectors to support as first class, >> and move out others to Bahir or another storm repository (storm-connectors? >> storm-externals?). >> >> - Jungtaek Lim (HeartSavioR) >> >> 2018년 1월 29일 (월) 오후 2:30, Jungtaek Lim <kabh...@gmail.com>님이 작성: >> >>> Hi devs, >>> >>> This is initial post to separate out discussion topic from vote thread, >>> and continue discussing. >>> >>> Background of the topic: >>> 1. Releasing Storm requires huge bootstrapping, and normally takes several >>> months to release bugfix version. Note that it is not minor version... >>> Minor version is released per near a year. Connectors are maintained with >>> same release cadence, which makes connectors also long period to release, >>> whether it is (implicitly) beta or not. >>> 2. Most of the change for connectors are not related to Storm core. It >>> tends to be compatible with all release versions with same major version. >>> 3. (IMHO) We have too many connectors which we even can't maintain >>> actively. For example, ES connector couldn't support ES higher than 1.x. >>> 4. Connectors are having same release version for Storm core, hence newly >>> added connector will have at least 1.x version which no one would think it >>> is beta. >>> >>> Downside: >>> 1. Detached connectors can be easy to be forgotten. (easier than current) >>> 2. Connectors may have hard time if we bring backward incompatible change >>> to Storm core. We may remedy this with having supported version range for >>> specific connector version. >>> >>> Please put your opinion regarding topic. You're encouraged to copy your >>> previous post in vote thread which helps to centralize opinions in current >>> thread. >>> >>> Thanks, >>> Jungtaek Lim (HeartSaVioR) >>> >
