[jira] [Created] (FLINK-2794) Exactly-once support

Maximilian Michels created FLINK-2794: - Summary: Exactly-once support Key: FLINK-2794 URL: https://issues.apache.org/jira/browse/FLINK-2794 Project: Flink Issue Type: Bug

Re: Pulling Streaming out of staging and project restructure

Great to see streaming graduating. :) I like the outline, both getting rid of staging, having the examples together and generally flattening the structure are very reasonable to me. You have listed flink-streaming-examples under flink-streaming-connectors and left out some less prominent maven

Re: Pulling Streaming out of staging and project restructure

+1 for Robert's comments. On Thu, Oct 1, 2015 at 3:16 PM, Robert Metzger wrote: > Big +1 for graduating streaming out of staging. It is widely used, also in > production and we are spending a lot of effort into hardening it. > I also agree with the proposed new maven module

Re: Pulling Streaming out of staging and project restructure

If we remove flink-staging because projects tend to get stuck there, what mechanism prevents the same happening with flink-contrib? On 01.10.2015 15:19, Stephan Ewen wrote: +1 for Robert's comments. On Thu, Oct 1, 2015 at 3:16 PM, Robert Metzger wrote: Big +1 for

[jira] [Created] (FLINK-2795) Print JobExecutionResult for interactively invoked jobs

Maximilian Michels created FLINK-2795: - Summary: Print JobExecutionResult for interactively invoked jobs Key: FLINK-2795 URL: https://issues.apache.org/jira/browse/FLINK-2795 Project: Flink

Pulling Streaming out of staging and project restructure

Hi all! We are making good headway with reworking the last parts of the Window API. After that, the streaming API should be good to be pulled out of staging. Since we are reorganizing the projects as part of that, I would shift a bit more to bring things a bit more up to date. In this

Re: Pulling Streaming out of staging and project restructure

I like it in general. But while we're at it, what is the purpose of the flink-tests project, or rather which tests belong there instead of the individual projects? Where would new projects reside in, that previously would have been put into flink-staging? Lastly, I'd like to merge

Re: Pulling Streaming out of staging and project restructure

+1 I wanted to suggest that we rename modules to fully accept streaming as first class, qualifying also "batch" as "batch" (e.g., flink-java --> flink-dataset-java, flink-streaming --> flink-datastream, etc). This would break maven dependencies (temporary hell :-) so it's not a decision to take

Re: Pulling Streaming out of staging and project restructure

Big +1 for graduating streaming out of staging. It is widely used, also in production and we are spending a lot of effort into hardening it. I also agree with the proposed new maven module structure. We have to carefully test the reworked structure for the scripts which are generating the hadoop1

[jira] [Created] (FLINK-2798) Prepare new web dashboard for executing in on YARN

Robert Metzger created FLINK-2798: - Summary: Prepare new web dashboard for executing in on YARN Key: FLINK-2798 URL: https://issues.apache.org/jira/browse/FLINK-2798 Project: Flink Issue

Re: Pulling Streaming out of staging and project restructure

@Chesnay: Nothing prevents projects from getting stuck there. But at least we don't have two staging repositories (dist, staging). Also I would not say that there has been no graduation out of staging. Yarn was also there once, streaming and gelly are currently leaving it. So I would actually say

Re: Pulling Streaming out of staging and project restructure

+1 to Robert and practicality :-) As I said before, I do not feel strongly about this, I was torn myself. On Thu, Oct 1, 2015 at 3:28 PM, Chesnay Schepler wrote: > If we remove flink-staging because projects tend to get stuck there, what > mechanism prevents the same

Re: Pulling Streaming out of staging and project restructure

+1 For pulling out and the restructure. Enough good arguments have been brought forward and I agree with all of them. On Thu, 1 Oct 2015 at 17:47 Ufuk Celebi wrote: > > > On 01 Oct 2015, at 16:48, Robert Metzger wrote: > > > > @Chesnay: Nothing prevents

Re: Pulling Streaming out of staging and project restructure

+1 for the new Maven project structure +1 for removing the flink-testing-utils module +1 for moving flink-language-binding to flink-python On Thu, Oct 1, 2015 at 6:27 PM, Aljoscha Krettek wrote: > +1 For pulling out and the restructure. Enough good arguments have been >

Hash-based aggregation

Hello, I would really like to see FLINK-2237 solved. I would implement this feature over the weekend, if the CompactingHashTable can be used to solve it (see my comment there). Could you please give me some advice on whether is this a viable approach, or you perhaps see some difficulties that I'm

Re: Pulling Streaming out of staging and project restructure

+1 I like the idea moving "staging" projects into appropriate modules. While we are at it, I would like to propose changing " flink-hadoop-compatibility" to "flink-hadoop". It is in my bucket list but would be nice if it is part of re-org. Supporting Hadoop in the connector implicitly means

Re: Pulling Streaming out of staging and project restructure

I will commit something to flink-storm-compatibility tomorrow that contains some internal package restructuring. I think, renaming the three modules in this commit would be a smart move as both changes result in merge conflicts when rebasing open PRs. Thus we can limit this pain to a single time.