Re: Pulling Streaming out of staging and project restructure

> I created a JIRA for this discussion: > > https://issues.apache.org/jira/browse/FLINK-3205 > > > > > > On Fri, Oct 2, 2015 at 7:58 PM, fhueske <fhue...@gmail.com> wrote: > > > > > +1 > > > > > > > > > From: Henry Saputra >

Re: Pulling Streaming out of staging and project restructure

> > > If nobody disagrees, I'll open a PR for these changes. > > > I created a JIRA for this discussion: > > > https://issues.apache.org/jira/browse/FLINK-3205 > > > > > > > > > On Fri, Oct 2, 2015 at 7:58 PM, fhueske <fhue...@gmail.com> wr

Re: Pulling Streaming out of staging and project restructure

gt; > > > - move some modules from flink-staging to flink-batch-connectors > > > > > > > > > > > > If nobody disagrees, I'll open a PR for these changes. > > > > I created a JIRA for this discussion: > > > > https://iss

Re: Pulling Streaming out of staging and project restructure

ail.com> wrote: > > > +1 > > > > > > From: Henry Saputra > > Sent: Friday, October 2, 2015 19:34 > > To: dev@flink.apache.org > > Subject: Re: Pulling Streaming out of staging and project restructure > > > > > > +1 > > > >

Re: Pulling Streaming out of staging and project restructure

eske <fhue...@gmail.com> wrote: > +1 > > > From: Henry Saputra > Sent: Friday, October 2, 2015 19:34 > To: dev@flink.apache.org > Subject: Re: Pulling Streaming out of staging and project restructure > > > +1 > > On Friday, October 2, 2015, Matthias J. Sax <

Re: Pulling Streaming out of staging and project restructure

@matthias +1 for that approach On Fri, Oct 2, 2015 at 11:21 AM, Matthias J. Sax wrote: > It think, rename "flink-storm-compatibility-core" to just "flink-storm" > would be the cleanest solution. > > So in flink-contrib there would be two modules: > - flink-storm > -

Re: Pulling Streaming out of staging and project restructure

@Matthias: +1. On Fri, Oct 2, 2015 at 11:27 AM, Stephan Ewen wrote: > @matthias +1 for that approach > > On Fri, Oct 2, 2015 at 11:21 AM, Matthias J. Sax wrote: > > > It think, rename "flink-storm-compatibility-core" to just "flink-storm" > > would be the

Re: Pulling Streaming out of staging and project restructure

+1 On Fri, 2 Oct 2015 at 11:37 Márton Balassi wrote: > @Matthias: +1. > > On Fri, Oct 2, 2015 at 11:27 AM, Stephan Ewen wrote: > > > @matthias +1 for that approach > > > > On Fri, Oct 2, 2015 at 11:21 AM, Matthias J. Sax > wrote: >

Re: Pulling Streaming out of staging and project restructure

+1 Matthias, let's limit the overhead this has for the module maintainers. On Fri, Oct 2, 2015 at 12:17 AM, Matthias J. Sax wrote: > I will commit something to flink-storm-compatibility tomorrow that > contains some internal package restructuring. I think, renaming the > three

Re: Pulling Streaming out of staging and project restructure

It think, rename "flink-storm-compatibility-core" to just "flink-storm" would be the cleanest solution. So in flink-contrib there would be two modules: - flink-storm - flink-storm-examples Please let me know if you have any objection about it. -Matthias On 10/02/2015 10:45 AM, Matthias J.

Re: Pulling Streaming out of staging and project restructure

+1 for the new project structure. Getting rid of our code dump is a good thing. On Fri, Oct 2, 2015 at 10:25 AM, Maximilian Michels wrote: > +1 Matthias, let's limit the overhead this has for the module maintainers. > > On Fri, Oct 2, 2015 at 12:17 AM, Matthias J. Sax

Re: Pulling Streaming out of staging and project restructure

Sure. Will do that. -Matthias On 10/02/2015 10:35 AM, Stephan Ewen wrote: > @Matthias: How about getting rid of the storm-compatibility-parent and > making the core and examples projects directly projects in "contrib" > > On Fri, Oct 2, 2015 at 10:34 AM, Till Rohrmann

RE: Pulling Streaming out of staging and project restructure

+1 From: Henry Saputra Sent: Friday, October 2, 2015 19:34 To: dev@flink.apache.org Subject: Re: Pulling Streaming out of staging and project restructure +1 On Friday, October 2, 2015, Matthias J. Sax <mj...@apache.org> wrote: > It think, rename "flink-storm-compatibility

Re: Pulling Streaming out of staging and project restructure

Great to see streaming graduating. :) I like the outline, both getting rid of staging, having the examples together and generally flattening the structure are very reasonable to me. You have listed flink-streaming-examples under flink-streaming-connectors and left out some less prominent maven

Re: Pulling Streaming out of staging and project restructure

+1 for Robert's comments. On Thu, Oct 1, 2015 at 3:16 PM, Robert Metzger wrote: > Big +1 for graduating streaming out of staging. It is widely used, also in > production and we are spending a lot of effort into hardening it. > I also agree with the proposed new maven module

Re: Pulling Streaming out of staging and project restructure

If we remove flink-staging because projects tend to get stuck there, what mechanism prevents the same happening with flink-contrib? On 01.10.2015 15:19, Stephan Ewen wrote: +1 for Robert's comments. On Thu, Oct 1, 2015 at 3:16 PM, Robert Metzger wrote: Big +1 for

Re: Pulling Streaming out of staging and project restructure

I like it in general. But while we're at it, what is the purpose of the flink-tests project, or rather which tests belong there instead of the individual projects? Where would new projects reside in, that previously would have been put into flink-staging? Lastly, I'd like to merge

Re: Pulling Streaming out of staging and project restructure

+1 I wanted to suggest that we rename modules to fully accept streaming as first class, qualifying also "batch" as "batch" (e.g., flink-java --> flink-dataset-java, flink-streaming --> flink-datastream, etc). This would break maven dependencies (temporary hell :-) so it's not a decision to take

Re: Pulling Streaming out of staging and project restructure

Big +1 for graduating streaming out of staging. It is widely used, also in production and we are spending a lot of effort into hardening it. I also agree with the proposed new maven module structure. We have to carefully test the reworked structure for the scripts which are generating the hadoop1

Re: Pulling Streaming out of staging and project restructure

@Chesnay: Nothing prevents projects from getting stuck there. But at least we don't have two staging repositories (dist, staging). Also I would not say that there has been no graduation out of staging. Yarn was also there once, streaming and gelly are currently leaving it. So I would actually say

Re: Pulling Streaming out of staging and project restructure

+1 to Robert and practicality :-) As I said before, I do not feel strongly about this, I was torn myself. On Thu, Oct 1, 2015 at 3:28 PM, Chesnay Schepler wrote: > If we remove flink-staging because projects tend to get stuck there, what > mechanism prevents the same

Re: Pulling Streaming out of staging and project restructure

+1 For pulling out and the restructure. Enough good arguments have been brought forward and I agree with all of them. On Thu, 1 Oct 2015 at 17:47 Ufuk Celebi wrote: > > > On 01 Oct 2015, at 16:48, Robert Metzger wrote: > > > > @Chesnay: Nothing prevents

Re: Pulling Streaming out of staging and project restructure

+1 for the new Maven project structure +1 for removing the flink-testing-utils module +1 for moving flink-language-binding to flink-python On Thu, Oct 1, 2015 at 6:27 PM, Aljoscha Krettek wrote: > +1 For pulling out and the restructure. Enough good arguments have been >

Re: Pulling Streaming out of staging and project restructure

+1 I like the idea moving "staging" projects into appropriate modules. While we are at it, I would like to propose changing " flink-hadoop-compatibility" to "flink-hadoop". It is in my bucket list but would be nice if it is part of re-org. Supporting Hadoop in the connector implicitly means

Re: Pulling Streaming out of staging and project restructure

I will commit something to flink-storm-compatibility tomorrow that contains some internal package restructuring. I think, renaming the three modules in this commit would be a smart move as both changes result in merge conflicts when rebasing open PRs. Thus we can limit this pain to a single time.