Re: Case style anonymous functions not supported by Scala API

What do you think of something like this? https://github.com/radicalbit/flink/commit/21a889a437875c88921c93e87d88a378c6b4299e In this way, several extensions can be collected in this package object and picked altogether or a-là-carte (e.g. import org.apache.flink.api.scala.extensions.AcceptPartia

Re: Option to disable chaining?

Disabling operator chaining is also the only way to see metrics from each step right now. On Monday, February 8, 2016, Stephan Ewen wrote: > There is no fundamental reason to not implement this for batch as well. > > In Streaming, users seem to want more control about threads and resources > (gi

Re: Option to disable chaining?

There is no fundamental reason to not implement this for batch as well. In Streaming, users seem to want more control about threads and resources (given that these are often continuous pipelines), while in batch that was not requested so far. But I see that a non-chained function is more safe wit

Re: Option to disable chaining?

Is there a reason to not also implement this for batch processing? This would allow object reuse to be truly disabled. On Mon, Feb 8, 2016 at 2:02 PM, Stephan Ewen wrote: > It may be useful for example when you have two MapFunctions and each does > something CPU intensive, or communicates with a

Re: [VOTE] Release Apache Flink 0.10.2 (RC2)

Perhaps too late for the RC, but I've backported FLINK-3293 to this branch via FLINK-3372. Would be nice for those wanting to monitory yarn application submissions. On Mon, Feb 8, 2016 at 9:37 AM, Ufuk Celebi wrote: > Dear Flink community, > > Please vote on releasing the following candidate as

Re: Option to disable chaining?

It may be useful for example when you have two MapFunctions and each does something CPU intensive, or communicates with an external service. Without chaining, you will have two threads and an elastic channel between the functions to buffer some records, which may help in such a case. On Mon, Feb

Re: Option to disable chaining?

When is this useful in streaming? On Mon, Feb 8, 2016 at 1:46 PM, Nick Dimiduk wrote: > > https://ci.apache.org/projects/flink/flink-docs-release-0.10/api/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.html#disableOperatorChaining() > > On Mon, Feb 8, 2016 at 10:34 AM

Re: Option to disable chaining?

https://ci.apache.org/projects/flink/flink-docs-release-0.10/api/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.html#disableOperatorChaining() On Mon, Feb 8, 2016 at 10:34 AM, Greg Hogan wrote: > Is it possible to force operator chaining to be disabled? Similar to how

Option to disable chaining?

Is it possible to force operator chaining to be disabled? Similar to how object reuse can be enabled or disabled? Greg

[jira] [Created] (FLINK-3372) Setting custom YARN application name is ignored

Nick Dimiduk created FLINK-3372: --- Summary: Setting custom YARN application name is ignored Key: FLINK-3372 URL: https://issues.apache.org/jira/browse/FLINK-3372 Project: Flink Issue Type: Bug

[VOTE] Release Apache Flink 0.10.2 (RC2)

Dear Flink community, Please vote on releasing the following candidate as Apache Flink version 0.10.2. Please note that this vote has a slightly shorter voting period of 48 hours. Only a single change has been made since the last release candidate. Since the community has already done extensive t

[RESULT] [VOTE] Release Apache Flink 0.10.2 (RC1)

This vote is cancelled in favor of RC2. On Mon, Feb 8, 2016 at 3:18 PM, Ufuk Celebi wrote: > >> On 08 Feb 2016, at 15:13, Robert Metzger wrote: >> >> -1 (binding) >> >> the quickstart for 0.10.2 has the Flink version set to 0.10-SNAPSHOT, not >> to 0.10.2. >> https://repository.apache.org/conten

[jira] [Created] (FLINK-3371) Move TriggerCotext and TriggerResult to their own classes

Stephan Ewen created FLINK-3371: --- Summary: Move TriggerCotext and TriggerResult to their own classes Key: FLINK-3371 URL: https://issues.apache.org/jira/browse/FLINK-3371 Project: Flink Issue T

[jira] [Created] (FLINK-3370) Add an aligned version of the window operator

Stephan Ewen created FLINK-3370: --- Summary: Add an aligned version of the window operator Key: FLINK-3370 URL: https://issues.apache.org/jira/browse/FLINK-3370 Project: Flink Issue Type: Improve

[jira] [Created] (FLINK-3369) RemoteTransportException should be instance of CancelTaskException

Ufuk Celebi created FLINK-3369: -- Summary: RemoteTransportException should be instance of CancelTaskException Key: FLINK-3369 URL: https://issues.apache.org/jira/browse/FLINK-3369 Project: Flink

[jira] [Created] (FLINK-3368) Kafka 0.8 consumer fails to recover from broker shutdowns

Robert Metzger created FLINK-3368: - Summary: Kafka 0.8 consumer fails to recover from broker shutdowns Key: FLINK-3368 URL: https://issues.apache.org/jira/browse/FLINK-3368 Project: Flink Iss

Re: [VOTE] Release Apache Flink 0.10.2 (RC1)

> On 08 Feb 2016, at 15:13, Robert Metzger wrote: > > -1 (binding) > > the quickstart for 0.10.2 has the Flink version set to 0.10-SNAPSHOT, not > to 0.10.2. > https://repository.apache.org/content/repositories/orgapacheflink-1060/org/apache/flink/flink-quickstart-java/0.10.2/flink-quickstart-j

Re: [VOTE] Release Apache Flink 0.10.2 (RC1)

-1 (binding) the quickstart for 0.10.2 has the Flink version set to 0.10-SNAPSHOT, not to 0.10.2. https://repository.apache.org/content/repositories/orgapacheflink-1060/org/apache/flink/flink-quickstart-java/0.10.2/flink-quickstart-java-0.10.2.jar The same applies to the 0.10.2-hadoop1 version: h

Re: Case style anonymous functions not supported by Scala API

I like the idea to support partial functions with Flink’s Scala API. However, I think that breaking the API and making it inconsistent with respect to the Java API is not the best option. I would rather be in favour of the first proposal where we add a new method xxxWith via implicit conversions.

Re: Remove final flag from ResultPartitionWriter class

> On 08 Feb 2016, at 13:57, Chesnay Schepler wrote: > > Hello all, > > The final flag on the > org.apache.flink.runtime.io.network.api.writer.ResultPartitionWriter class is > causing issues for me. > > The flag requires me to run a test I'm working on with a > @RunWith(PowerMockRunner.class

Remove final flag from ResultPartitionWriter class

Hello all, The final flag on the org.apache.flink.runtime.io.network.api.writer.ResultPartitionWriter class is causing issues for me. The flag requires me to run a test I'm working on with a @RunWith(PowerMockRunner.class) annotation so that i can use @PrepareForTest({ResultPartitionWriter.

Re: [VOTE] Release Apache Flink 0.10.2 (RC1)

+1 (binding) - build against Scala 2.11 - start-local (CLI, Webclient, Dashboard, log files, examples with build-in and external data) - start-cluster, local (2 JVMs, CLI + Webclient, Dashboard, log files, examples with build-in and external data) 2016-02-07 20:21 GMT+01:00 Ufuk Celebi : > +1 (b

Re: [ANNOUNCE] Please annotate public interfaces!

I created FLINK-3366 for renaming @Experimental to @PublicEvolving and FLINK-3367 to annotate all remaining API classes with @PublicEvolving. 2016-02-08 9:57 GMT+01:00 Maximilian Michels : > Never thought about "Experimental" meaning unstable but I agree it > sounds better to use the term "Evolvi

[jira] [Created] (FLINK-3367) Annotate all user-facing API classes with @Public or @PublicEvolving

Fabian Hueske created FLINK-3367: Summary: Annotate all user-facing API classes with @Public or @PublicEvolving Key: FLINK-3367 URL: https://issues.apache.org/jira/browse/FLINK-3367 Project: Flink

[jira] [Created] (FLINK-3366) Rename @Experimental annotation to @PublicEvolving

Fabian Hueske created FLINK-3366: Summary: Rename @Experimental annotation to @PublicEvolving Key: FLINK-3366 URL: https://issues.apache.org/jira/browse/FLINK-3366 Project: Flink Issue Type:

Re: Tips for testing

Great, thank you very much! On Monday, February 8, 2016, Maximilian Michels wrote: > Hi Stefano, > > Thanks for the pull request. > > You may use the following to only run tests from flink-runtime: > > mvn verify -pl flink-runtime > > Alternatively, change to the flink-runtime directory and run

[jira] [Created] (FLINK-3365) BlobLibraryCacheManager does not shutdown Timer thread

Stephan Ewen created FLINK-3365: --- Summary: BlobLibraryCacheManager does not shutdown Timer thread Key: FLINK-3365 URL: https://issues.apache.org/jira/browse/FLINK-3365 Project: Flink Issue Type

[jira] [Created] (FLINK-3364) Don't initialize SavepointStore in JobManager constructor

Ufuk Celebi created FLINK-3364: -- Summary: Don't initialize SavepointStore in JobManager constructor Key: FLINK-3364 URL: https://issues.apache.org/jira/browse/FLINK-3364 Project: Flink Issue Typ

Re: Release Flink 1.0.0

Okay, I didn't check the individual PRs closely. I agree that we should not merge big core changes if we are not certain about their stability. On Mon, Feb 8, 2016 at 12:04 PM, Fabian Hueske wrote: > Regarding FLINK-2237 (hash-based combiner), I think we need a bit more > time. > It is a fairly

Re: Release Flink 1.0.0

Regarding FLINK-2237 (hash-based combiner), I think we need a bit more time. It is a fairly large contribution and touches/adds core functionality. I started reviewing the PR and will suggest a few changes in the next days. 2016-02-08 11:48 GMT+01:00 Robert Metzger : > There are still some 8 ope

[jira] [Created] (FLINK-3363) JobManager does not shut down dedicated executor properly

Stephan Ewen created FLINK-3363: --- Summary: JobManager does not shut down dedicated executor properly Key: FLINK-3363 URL: https://issues.apache.org/jira/browse/FLINK-3363 Project: Flink Issue T

Re: Release Flink 1.0.0

There are still some 8 open blockers for the 1.0 release: https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20priority%20%3D%20Blocker%20ORDER%20BY%20key%20DESC I also think that there are some pull requests which are almost ready to merge

Re: Tips for testing

Hi Stefano, Thanks for the pull request. You may use the following to only run tests from flink-runtime: > mvn verify -pl flink-runtime Alternatively, change to the flink-runtime directory and run "mvn verify" from there. This assumes Flink has been built and installed locally before. If not, M

Re: [ANNOUNCE] Please annotate public interfaces!

Never thought about "Experimental" meaning unstable but I agree it sounds better to use the term "Evolving". On Sun, Feb 7, 2016 at 12:45 AM, Fabian Hueske wrote: > I agree, Experimental rather suggests unstable behavior than potentially > changing interfaces. > +1 for renaming to @PublicEvolving

Re: Tips for testing

Hi Maximilian, thanks for the tips. I've already opened a PR (here ). Is there a way from the command line to do what can be done with IntelliJ? I mean build only what's needed and run a specific test? I've tried `mvn test` but it skips the build phase an

Re: Tips for testing

Hi Stefano, 1) Please open a pull request. If the String depends on the locale, this looks like a bug. 2) "mvn clean install" is the way to go for the complete check. If you only want to run certain tests, selecting and running them from IntelliJ works pretty well. In addition, it is nice to push