IMHO we can do that. There should be a disclaimer that the third party software is not officially supported.
On Thu, Oct 8, 2015 at 2:25 PM, Matthias J. Sax <mj...@apache.org> wrote: > Should we add a new page at Flink project web page? > > On 10/08/2015 12:56 PM, Maximilian Michels wrote: >> +1 for your pragmatic approach, Vasia. A simple collection of third >> party software using Flink should be enough for now; of course, >> outside the Apache realm. >> >> On Thu, Oct 8, 2015 at 12:45 PM, Chiwan Park <chiwanp...@apache.org> wrote: >>> +1 for Vasia’s suggestion. From a long-term perspective, the site like >>> Spark Packages [1] would be helpful to manage external contribution. >>> >>> [1] http://spark-packages.org >>> >>>> On Oct 8, 2015, at 12:28 PM, Matthias J. Sax <mj...@apache.org> wrote: >>>> >>>> Thanks for the feedback. >>>> >>>> I think, the repository does not need to build on a single Flink >>>> release. From my point of view, there should be a single parent module >>>> that contains *independent modules* for each extension/library (there >>>> should be no "cross dependencies" between the modules and each module >>>> can specify the flink dependencies it needs by itself). This make is >>>> most flexible. And if a library works on an old release, it might just >>>> stay there as is. If a library changes (due to Flink changes), it might >>>> just be contained multiple times for different Flink releases. >>>> >>>> Each module should provide a short doc (README) that shows how to use an >>>> integrate it with Flink. Thus, the responsibility goes to the >>>> contributor to maintain the library. If it breaks and is not maintained >>>> any further, we can simple remove it. >>>> >>>> I agree, that the community might not be able to maintain those >>>> extension/libraries right now. I would put the responsibility (more or >>>> less completely) on the contributor and delete project that do not fix >>>> any more. >>>> >>>> @Vasia: a link to a library could be included in the README. If anybody >>>> only wants to share a library but not contribute code, the parent README >>>> could contain a list of additional links. >>>> >>>> >>>> -Matthias >>>> >>>> >>>> On 10/08/2015 12:15 PM, Vasiliki Kalavri wrote: >>>>> How about, for now, we simply create a page where we gather links/short >>>>> descriptions of all these contributions >>>>> and let the maintenance and dependency management to the tool/library >>>>> creators? >>>>> This way we will at least have these contributions in one place and link >>>>> to >>>>> them somewhere from the website. >>>>> >>>>> -Vasia. >>>>> >>>>> On 8 October 2015 at 12:06, Maximilian Michels <m...@apache.org> wrote: >>>>> >>>>>> Hi Matthias, >>>>>> >>>>>> Thanks for bringing up this idea. Actually, it has been discussed a >>>>>> couple of times on the mailing list whether we should have a central >>>>>> place for third-party extensions/contributions/libraries. This could >>>>>> either be something package-based or, like you proposed, another >>>>>> repository. >>>>>> >>>>>> An external place for contributions raises a couple of questions >>>>>> >>>>>> - Which version should the external contributions be based on? >>>>>> - How do we make sure, the extensions are continuously updated? >>>>>> (dedicated maintainers or automatic compatibility checks) >>>>>> - How do we easily plug-in the external modules into Flink? >>>>>> >>>>>> In the long term, we really need a solution for these questions. The >>>>>> code base of Flink is growing and more and more packages go to >>>>>> flink-contrib/flink-staging. I would find something packaged-based >>>>>> better than a repository. Quite frankly, momentarily, I think >>>>>> developing such a plugin system is out of scope for most Flink >>>>>> developers. At the current pace of Flink development, collecting these >>>>>> contributions externally without properly maintaining them, doesn't >>>>>> make much sense to me. >>>>>> >>>>>> Cheers, >>>>>> Max >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Oct 7, 2015 at 11:42 AM, Matthias J. Sax <mj...@apache.org> >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> many people are building quite exiting stuff on top of Flink. It is hard >>>>>>> to keep an good overview on what stuff is available and what not. What >>>>>>> do you think about starting a second git repository "flink-external" >>>>>>> that collects all those code? >>>>>>> >>>>>>> The ideas would be to collect stuff in a central point, such that people >>>>>>> can access it easily and get an overview what is already available (this >>>>>>> might also avoid duplicate development). It might also be a good point >>>>>>> to show common patterns. In order to collect as much as possible, the >>>>>>> contributing requirement (with respect to testing etc) could be lower >>>>>>> than for Flink itself. >>>>>>> >>>>>>> For example, I recently started a small flink-clojure module with a >>>>>>> simple word-count example to answer a question on SO. Including this in >>>>>>> Flink would not be appropriate. However, for a flink-external repro it >>>>>>> might be nice to have. >>>>>>> >>>>>>> What do you think about it? >>>>>>> >>>>>>> >>>>>>> -Matthias >>>>>>> >>>>>> >>>>> >>>> >>> >>> >>> >>> Regards, >>> Chiwan Park >>> >>> >>> >
