Thank you for your responses. @Luciano: I don't have a strong preference for one of the two options, but I would like to understand the implications of the two before we start setting up the infrastructure. Regarding the release cycle: For the Flink connectors, I would actually try to make the release cycle dependent on the connectors, not so much on Flink itself. In my experience, connectors could benefit from a more frequent release schedule. For example Kafka seems to release new versions quite frequently (recently), or at least the release cycle of Kafka and Flink is not aligned ;) So maybe it would make sense for bahir to release independent of the engine projects, on a monthly or 2-monthly schedule, with an independent versioning scheme.
@Ted: Flink has bugfix releases quite frequently, but major releases are at a okay level (3-4 months in between). Since 1.0.0 Flink provides interface stability, so there should not be an issue with independent connector releases. On Thu, Aug 11, 2016 at 6:13 PM, Ted Yu <[email protected]> wrote: > Having Flink connectors in the same repo seems to make more sense at the > moment. > > Certain artifacts can be shared between the two types of connectors. > > Flink seems to have more frequent releases recently. But Bahir doesn't have > to follow each Flink patch release. > > Just my two cents. > > On Thu, Aug 11, 2016 at 9:03 AM, Luciano Resende <[email protected]> > wrote: > > > On Thu, Aug 11, 2016 at 5:42 AM, Robert Metzger <[email protected]> > > wrote: > > > > > > > > > > > @Luciano: So the idea is to have separate repositories for each project > > > contributing connectors? > > > I'm wondering if it makes sense to keep the code in the same repository > > to > > > have some synergies (like the release scripts, CI, documentation, a > > common > > > parent pom with rat etc.). Otherwise, it would maybe make more sense to > > > create a Bahir-style project for Flink, to avoid maintaining completely > > > disjunct codebases in the same JIRA, ML, ... > > > > > > > > > > > But we most likely would have very different release schedules with the > > different set of extensions, where Spark extensions will tend to follow > > Spark release cycles, and Flink release cycles. As for the overhead, I > > believe release scripts might be the one piece that would be replicated, > > but I can volunteer the infrastructure overhead for now. All rest, such > as > > JIRA, ML, etc will be common. But, anyway, I don't want to make this an > > issue for Flink to bring up the extensions here, so if you have a strong > > preference on having all in the same repo, we could start with that. > > > > Thoughts ? > > >
