Thank you for your responses. I will get in touch with the Bahir community to see what they are thinking about this. Once we know a bit more about the details of such a collaboration, we can make a final decision here.
On Tue, Aug 9, 2016 at 3:47 PM, Till Rohrmann <trohrm...@apache.org> wrote: > I agree with Stephan that the main problem is maintenance overhead for the > Flink community. If we could maintain all connectors ourselves then there > would not be an immediate need to out source the connectors. Thus, the > solution should reduce the workload for the core project. > > Personally, I would favour option #2 if Apache Bahir is willing to add > Flink as another supported streaming platform. This would give the > streaming connectors a more prominent home than the personal repository of > a contributor. > > Furthermore, contributing the connectors to Apache Bahir entails that the > connector artifacts are deployed to a central repository (I assume). That > way one can more easily add them to one's project instead of having to > build and install the code yourself. > > I'm also not sure what happens to a public github repository which has not > been forked and is then deleted. I would assume that the content is then > lost. > > To create a "flink-connectors" repository independent of Flink sounds quite > similar to creating another Apache Bahir. I think it would be better to > join forces with the existing Bahir community if possible. > > On Tue, Aug 9, 2016 at 2:56 PM, Stephan Ewen <se...@apache.org> wrote: > > > Thanks Robert for bringing this up. > > > > I think that "flink-contrib" will not really solve the problem, because > the > > connectors would still have to be maintained by the core community, we > > would need to guarantee test stability. It will be to a large extend > > similar to adding them to "flink-streaming-connectors", except that we > may > > declare via the artifact name that the quality is not guaranteed to be > too > > high. > > > > Regarding moving the code to a separate repository: I think that also > only > > solves the problem, if that repository is organizationally different from > > the core clink repository. Different release cycles, different > maintainers. > > > > > > The quintessence is for me that the connectors need to be handled by a > > different organization than the core flink community. > > That would leave us with > > - The contributors' own repository > > - Apache Bahir > > - An organizationally different "flink-connectors" repository, possibly > > with different committers / PMC > > > > > > > > > > > > On Tue, Aug 9, 2016 at 1:41 PM, Tzu-Li (Gordon) Tai <tzuli...@gmail.com> > > wrote: > > > > > Good point. Discussion for each new connector is also an overhead we > > should > > > avoid. > > > > > > IMHO, option #2 doesn’t seem like a reasonable staging area. Say we > > decide > > > we’d like to move a connector from Bahir to Flink in the future, > there’ll > > > be two of the connector in separate Apache projects. Personally I think > > > option #2 is more like a way to go if some day we’d like to migrate > some > > of > > > our currently supported connectors in Flink to Bahir (perhaps an > > > alternative to the discussion in "Externalizing the Flink connectors”). > > > > > > Defining option #1 as a staging area seems nice; the contributor > notifies > > > the Flink community of the work by adding it to the 3rd party packages > > > page, and in the future we can contact the contributor to ask whether > > > they’d like to migrate the connector to Flink when we see more user > > demand > > > and committer support. > > > > > > With this approach, do you think we should assume that future new > > > connectors always go to option #1 first? If not, I would assume that > in > > > the future, Flink JIRAs for new connectors are only opened if we’d like > > to > > > add them directly to Flink (perhaps the contribution guideline can link > > to > > > Flink development Roadmaps like [1] as a reference to connectors that > the > > > community has already considered to support?). > > > > > > [1] > > > https://docs.google.com/document/d/1ExmtVpeVVT3TIhO1JoBpC5JKXm- > > > 778DAD7eqw5GANwE/edit# > > > <https://docs.google.com/document/d/1ExmtVpeVVT3TIhO1JoBpC5JKXm- > > > 778DAD7eqw5GANwE/edit> > > > > > > On August 9, 2016 at 6:10:21 PM, Robert Metzger (rmetz...@apache.org) > > > wrote: > > > > > > Hi Gordon, > > > thank you for your response. > > > > > > I agree with your observation that some "staging" area is helpful to > test > > > how many contributors / users are interested in a connector. But I > wonder > > > if #1 or #2 can also serve as a staging area: As soon as we see that > > there > > > is a lot of interest in a connector, we add it to the main > > > "flink-*-connectors" directory. > > > Having too many ways to contribute connectors might make the > discussions > > > for each new connector quite complicated. > > > > > > You are right, the intention of the "Externalizing the Flink > connectors” > > > discussion was more about the release intervals, test time etc. > > > > > > Regards, > > > Robert > > > > > >
