Re: [DISCUSS] Streaming connector contributions

Okay, I agree that this is something we should make very clear. I'll start a DISCUSS thread. On Wed, Aug 17, 2016 at 10:07 AM, Stephan Ewen wrote: > Cool to see that the Bahir people are so open and helpful! > > Concerning moving the connectors: I think we should set up a

Re: [DISCUSS] Streaming connector contributions

Cool to see that the Bahir people are so open and helpful! Concerning moving the connectors: I think we should set up a vote thread, or at least a discussion with the possibility to object. Removing someones code from the repository is a bit of a sensitive thing in my experience, so let's make

Re: [DISCUSS] Streaming connector contributions

Bahir is now creating a second repository "bahir-flink" for our connectors: https://issues.apache.org/jira/browse/INFRA-12440 If there are no objections here on the dev list, I would like to move the "flume" and "redis" streaming connector to Bahir. Once they are in, and the file structure at

Re: [DISCUSS] Streaming connector contributions

Hi, I just wanted to let you know that I've started a discussion at the Bahir dev list [1]. They seem to be very open to give some of our streaming connectors a new home. We are currently discussing some specifics there (whether to put the code into the same repository or into separate ones).

Re: [DISCUSS] Streaming connector contributions

Thank you for your responses. I will get in touch with the Bahir community to see what they are thinking about this. Once we know a bit more about the details of such a collaboration, we can make a final decision here. On Tue, Aug 9, 2016 at 3:47 PM, Till Rohrmann wrote: >

Re: [DISCUSS] Streaming connector contributions

I agree with Stephan that the main problem is maintenance overhead for the Flink community. If we could maintain all connectors ourselves then there would not be an immediate need to out source the connectors. Thus, the solution should reduce the workload for the core project. Personally, I would

Re: [DISCUSS] Streaming connector contributions

Thanks Robert for bringing this up. I think that "flink-contrib" will not really solve the problem, because the connectors would still have to be maintained by the core community, we would need to guarantee test stability. It will be to a large extend similar to adding them to

Re: [DISCUSS] Streaming connector contributions

Good point. Discussion for each new connector is also an overhead we should avoid. IMHO, option #2 doesn’t seem like a reasonable staging area. Say we decide we’d like to move a connector from Bahir to Flink in the future, there’ll be two of the connector in separate Apache projects. Personally I

Re: [DISCUSS] Streaming connector contributions

Hi Gordon, thank you for your response. I agree with your observation that some "staging" area is helpful to test how many contributors / users are interested in a connector. But I wonder if #1 or #2 can also serve as a staging area: As soon as we see that there is a lot of interest in a

Re: [DISCUSS] Streaming connector contributions

Hi Robert, Thank you for bringing the discussion to the mailing lists. #2 seems like a good option, if we can reach consensus with the Bahir community. However, should we also be considering “staging” (perhaps under “flink-contrib”?) a connector contribution until more committers can maintain

[DISCUSS] Streaming connector contributions

Hi, I would like to start a discussion regarding the streaming connectors in Flink. Currently, there are 12 connectors in the main repository, 4 more are pending as pull requests (rethinkdb, kafka 0.10, ActiveMQ, HBase), and there were some more discussed on the mailing list / JIRA / pull