RE: User Feedback

> From: Vasiliki Kalavri > Sent: February 9, 2016 10:54:51am PST > To: dev@flink.apache.org > Cc: Martin Junghanns > Subject: Re: User Feedback > > Hi Martin, > > thank you for the feedback. Let me try to answer some of your concerns. > > > On 9 February 2016 at 15:35, Martin Neumann

Re: User Feedback

Hi Martin, thank you for the feedback. Let me try to answer some of your concerns. On 9 February 2016 at 15:35, Martin Neumann wrote: > During this year's FOSDEM Martin Junghans and I set together and gathered > some feedback for the Flink project. It is based on our

Re: Case style anonymous functions not supported by Scala API

Not the DataSet but the JoinDataSet and the CoGroupDataSet do in the form of an apply function. ​ On Tue, Feb 9, 2016 at 11:09 AM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > Sure, it was just a draft. I agree that filter and mapPartition make sense, > but coGroup and join don't

Re: Case style anonymous functions not supported by Scala API

This looks like a good design to me :-) The only thing is that it is not complete. For example, the filter, mapPartition, coGroup and join functions are missing. Cheers, Till ​ On Tue, Feb 9, 2016 at 1:18 AM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > What do you think of

Re: Case style anonymous functions not supported by Scala API

Sure, it was just a draft. I agree that filter and mapPartition make sense, but coGroup and join don't look like they take a function. On Tue, Feb 9, 2016 at 10:08 AM, Till Rohrmann wrote: > This looks like a good design to me :-) The only thing is that it is not >

[jira] [Created] (FLINK-3376) Add an illustration of Event Time and Watermarks to the docsq

Stephan Ewen created FLINK-3376: --- Summary: Add an illustration of Event Time and Watermarks to the docsq Key: FLINK-3376 URL: https://issues.apache.org/jira/browse/FLINK-3376 Project: Flink

[jira] [Created] (FLINK-3375) Allow Watermark Generation in the Kafka Source

Stephan Ewen created FLINK-3375: --- Summary: Allow Watermark Generation in the Kafka Source Key: FLINK-3375 URL: https://issues.apache.org/jira/browse/FLINK-3375 Project: Flink Issue Type:

[jira] [Created] (FLINK-3374) CEPITCase testSimplePatternEventTime fails

Ufuk Celebi created FLINK-3374: -- Summary: CEPITCase testSimplePatternEventTime fails Key: FLINK-3374 URL: https://issues.apache.org/jira/browse/FLINK-3374 Project: Flink Issue Type: Test

[jira] [Created] (FLINK-3373) Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems

Jakob Sultan Ericsson created FLINK-3373: Summary: Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems Key: FLINK-3373 URL:

[jira] [Created] (FLINK-3378) Consolidate TestingCluster and FokableFlinkMiniCluster

Maximilian Michels created FLINK-3378: - Summary: Consolidate TestingCluster and FokableFlinkMiniCluster Key: FLINK-3378 URL: https://issues.apache.org/jira/browse/FLINK-3378 Project: Flink

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

I would also like to keep this RC. 0.10-SNAPSHOT will be equally stable. On Tue, Feb 9, 2016 at 2:57 PM, Stephan Ewen wrote: > I would be in favor of continuing the release candidate. > The change is more a feature improvement than a bug, and these should come > before release

User Feedback

During this year's FOSDEM Martin Junghans and I set together and gathered some feedback for the Flink project. It is based on our personal experience as well as the feedback and questions from People we taught the system. This is going to be a longer email therefore I have split things into

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

I would be in favor of continuing the release candidate. The change is more a feature improvement than a bug, and these should come before release candidates (otherwise we never get one done). The 1.0 release is also very close, so this improvement will be available anyways pretty soon. I

Re: Case style anonymous functions not supported by Scala API

I see, thanks for the tip! I'll work on it; meanwhile, I've added some functions and Scaladoc: https://github.com/radicalbit/flink/blob/1159-implicit/flink-scala/src/main/scala/org/apache/flink/api/scala/extensions/package.scala On Tue, Feb 9, 2016 at 12:01 PM, Till Rohrmann

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

Hey Nick, I agree that this can be problematic when running multiple jobs on YARN. Since there is a chance that this might be the last release 0.10 release, I would be OK to cancel the vote for your fix. Still, let's hear the opinion of others before doing this. What do you think? – Ufuk On

Re: Case style anonymous functions not supported by Scala API

I agree with you, but I acknowledge that there may be concerns regarding the stability of the API. Perhaps the rationale behind the proposal of Stephan and Till is to provide it as an extension to test how the developers feel about it. It would be ideal to have a larger feedback from the

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

Thanks Ufuk, +1 for a new RC 2016-02-09 13:49 GMT+01:00 Ufuk Celebi : > Hey Nick, > > I agree that this can be problematic when running multiple jobs on > YARN. Since there is a chance that this might be the last release 0.10 > release, I would be OK to cancel the vote for your

Re: Case style anonymous functions not supported by Scala API

Thanks for bringing this up Stefano, it would a very welcome addition indeed. I like the approach of having extensions through implicits as well. IMHO though this should be the default behavior, without the need to add another import. On Tue, Feb 9, 2016 at 1:29 PM, Stefano Baghino <

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

Hi Ufuk, This is nice to have but not a blocker. So unless we find blocker for the current RC I prefer to continue evaluate and VOTE current RC. - Henry On Tuesday, February 9, 2016, Ufuk Celebi wrote: > Hey Nick, > > I agree that this can be problematic when running

[jira] [Created] (FLINK-3379) Refactor TimestampExtractor

Stephan Ewen created FLINK-3379: --- Summary: Refactor TimestampExtractor Key: FLINK-3379 URL: https://issues.apache.org/jira/browse/FLINK-3379 Project: Flink Issue Type: Improvement

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

+1 (binding) Re-ran the tests from the previous RC with the new RC - Built the code (Linux x64, Scala 2.10, Hadoop 2.3.0) - Ran all tests - Checked that there are no cross-version dependencies - Checked that LICENSE and NOTICE are up to date - Check planVisualizer util - ran examples

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

Given the initial feedback, I would like to keep the RC going as well. The upcoming 1.0 release will include the fix and if users complain about it for 0.10 we can do another release in the future. — +1 for this release. Forwarding my tests from the last vote. Additionally, I’ve verified

Breaking changes in the Streaming API

Hi everyone! There are two remaining issues right now pending for 1.0 that will cause breaking API changes in the Streaming API. 1) [FLINK-3379] The Timestamp Extractor needs to be changed. That really seems necessary, based on the user feedback, because a lot of people mentioned that they are

[jira] [Created] (FLINK-3380) Unstable Test: JobSubmissionFailsITCase

Klou created FLINK-3380: --- Summary: Unstable Test: JobSubmissionFailsITCase Key: FLINK-3380 URL: https://issues.apache.org/jira/browse/FLINK-3380 Project: Flink Issue Type: Bug Reporter:

[jira] [Created] (FLINK-3382) Improve clarity of object reuse in ReusingMutableToRegularIteratorWrapper

Greg Hogan created FLINK-3382: - Summary: Improve clarity of object reuse in ReusingMutableToRegularIteratorWrapper Key: FLINK-3382 URL: https://issues.apache.org/jira/browse/FLINK-3382 Project: Flink

Re: Case style anonymous functions not supported by Scala API

Hi Till, I do agree with your point, so much so so that at the time being I'd suggest to keeping these additions as optional, up to the end-user to opt-in. Adding them by default would effectively be an addition to the DataSet API (despite being separated at a source file level). I think your

Re: Case style anonymous functions not supported by Scala API

I think that if we make the class part of the default ".api.scala" package object, we effectively add these methods to DataSet.scala, because they will be always be available on the data set. If we want to retain the liberty to not commit to this change now, then we should probably ask users to

Re: User Feedback

I can elaborate in the project(...) method: ".returns()" is there to supply TypeInformation in cases where the system cannot determine it. In the case of "project()", the system can perfectly determine the output type info from the input and the projection. For just getting a typed result, I

Re: Case style anonymous functions not supported by Scala API

What we could do is to add the implicit class to the package object of org.apache.flink.api.scala. Since one always has to import this package in order to have the proper TypeInformations, you wouldn’t have to import the extension explicitly. The thing with the API is that you don’t want to break

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

Hi Nick, Thanks for bringing up the issue. I believe some people (including me) will try to run Apache Flink on YARN =) If more 0.10.x releases needed before 1.0 definitely would love to include this in. - Henry On Tue, Feb 9, 2016 at 9:33 AM, Nick Dimiduk wrote: > Agree

[jira] [Created] (FLINK-3383) Separate Maven deployment from CI testing

Maximilian Michels created FLINK-3383: - Summary: Separate Maven deployment from CI testing Key: FLINK-3383 URL: https://issues.apache.org/jira/browse/FLINK-3383 Project: Flink Issue

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

Agree it's not a blocker. I can manage my own build with this patch if necessary. Just thought I might not be the only one running streaming on yarn. On Tuesday, February 9, 2016, Henry Saputra wrote: > Hi Ufuk, > > This is nice to have but not a blocker. > > So unless