[jira] [Created] (FLINK-5861) TaskManager's components support updating JobManagerGateway

Biao Liu created FLINK-5861: --- Summary: TaskManager's components support updating JobManagerGateway Key: FLINK-5861 URL: https://issues.apache.org/jira/browse/FLINK-5861 Project: Flink Issue Type:

Re: [DISCUSS] Table API / SQL indicators for event and processing time

Hi all, As I said in another thread, the main difference between stream and table is that a stream is an ordered list while a table is an unordered set. Without considering the out-of-order problem in practice, whether event-time or processing-time can be just taken as a monotonically increasing

[jira] [Created] (FLINK-5860) Replace all the file creating from java.io.tmpdir with TemporaryFolder

shijinkui created FLINK-5860: Summary: Replace all the file creating from java.io.tmpdir with TemporaryFolder Key: FLINK-5860 URL: https://issues.apache.org/jira/browse/FLINK-5860 Project: Flink

[jira] [Created] (FLINK-5859) support partition pruning on Table API & SQL

godfrey he created FLINK-5859: - Summary: support partition pruning on Table API & SQL Key: FLINK-5859 URL: https://issues.apache.org/jira/browse/FLINK-5859 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-5858) Support multiple sinks in same execution DAG

godfrey he created FLINK-5858: - Summary: Support multiple sinks in same execution DAG Key: FLINK-5858 URL: https://issues.apache.org/jira/browse/FLINK-5858 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-5856) Need return redundant containers to yarn for yarn mode

shuai.xu created FLINK-5856: --- Summary: Need return redundant containers to yarn for yarn mode Key: FLINK-5856 URL: https://issues.apache.org/jira/browse/FLINK-5856 Project: Flink Issue Type: Bug

Re: Contribute to Flink

much thanks, Timo, I am starting to feel great at the Flink community! cheers, Jin On Mon, Feb 20, 2017 at 10:00 PM, Timo Walther wrote: > Welcome to the Flink community, Jin! > > I gave you contributor permissions, so you can assign issues to yourself. > > Regards, > Timo >

Re: [DISCUSS] Flink ML roadmap

Hi all, Thanks Stavros for pushing forward the discussion which I feel really relevant. Since I'm approaching actively the community just right now and I haven't enough experience and such visibility around the Flink community, I'd limit myself to share an opinion as a Flink user. I'm using

[jira] [Created] (FLINK-5855) Unprotected access to pendingFilesPerCheckpoint in BucketingSink

Ted Yu created FLINK-5855: - Summary: Unprotected access to pendingFilesPerCheckpoint in BucketingSink Key: FLINK-5855 URL: https://issues.apache.org/jira/browse/FLINK-5855 Project: Flink Issue

Re: Question about Async IO

You can also issue multiple calls in one "invoke()" call (have multiple Futures) and then chain these futures and return only something once all Futures are complete. On Mon, Feb 20, 2017 at 4:01 PM, Till Rohrmann wrote: > In order to output 0 elements you have to pass an

Re: [DISCUSS] Table API / SQL indicators for event and processing time

Hi everybody, When Timo wrote to the Calcite mailing list, Julian Hyde replied and gave good advice and explained why a system attribute for event-time would be a problem [1]. I thought about this and agree with Julian. Here is a document to describe the problem, constraints in Flink and a

[jira] [Created] (FLINK-5854) Introduce some Flink-specific base Exception types

Stephan Ewen created FLINK-5854: --- Summary: Introduce some Flink-specific base Exception types Key: FLINK-5854 URL: https://issues.apache.org/jira/browse/FLINK-5854 Project: Flink Issue Type:

Re: [DISCUSS] Flink ML roadmap

I think Flink ML could be a success. Many use cases out there could benefit from such algorithms especially online ones. I agree examples should be created showing how it could be used. I was not aware of the project re-structuring issues. GPUs is really important nowdays but it is still not the

[jira] [Created] (FLINK-5853) Add consecutive event based filters

Dawid Wysakowicz created FLINK-5853: --- Summary: Add consecutive event based filters Key: FLINK-5853 URL: https://issues.apache.org/jira/browse/FLINK-5853 Project: Flink Issue Type: New

[jira] [Created] (FLINK-5852) Move JSON generation code into static methods

Chesnay Schepler created FLINK-5852: --- Summary: Move JSON generation code into static methods Key: FLINK-5852 URL: https://issues.apache.org/jira/browse/FLINK-5852 Project: Flink Issue

[jira] [Created] (FLINK-5851) Renaming AsyncCollector into ResultPromise/ResultFuture

Till Rohrmann created FLINK-5851: Summary: Renaming AsyncCollector into ResultPromise/ResultFuture Key: FLINK-5851 URL: https://issues.apache.org/jira/browse/FLINK-5851 Project: Flink Issue

Re: Question about Async IO

In order to output 0 elements you have to pass an empty collection to the `collect` method. You're right that our online documentation is lacking the fact that you're only supposed to call `collect` once. It's actually documented in the JavaDocs of this method. We should change this. You're also

Re: Question about Async IO

Hi Till, Thanks, for the explanation! How do I express if I don't want to collect any elements in the async collector? Like 0 output from a flatmap. Also it doesn't seem to be specified anywhere that the AsyncCollector is "completed", it is just a collector. You should be able to collect

Re: KeyGroupRangeAssignment ?

Hi, Thank you for clarifications (I am working with KeyedStream so a custom partitioner does not help). So I should set maxParallelism>=parallelism and change my keys (from input.keyBy(0)) such that key group assignment works as expected), but I can’t modify these keys in order to make it

Re: Question about Async IO

Hi Gyula, the assumption is that the AsyncCollector is either completed by the user or, if you have a timeout defined, that it will be completed with a timeout exception. This means that if you have no timeout defined, then you have to make sure that the collector is completed. Otherwise you will

Re: Contribute to Flink

Welcome to the Flink community, Jin! I gave you contributor permissions, so you can assign issues to yourself. Regards, Timo Am 20/02/17 um 14:47 schrieb Jin Mingjian: Hi, Flink dev community, I'd like to contribute to Flink. Particularly, I am interested in kinds of optimization works in

Question about Async IO

Hi, I was looking at the AsyncFunction interface and try to wrap my head around the implementation and the assumptions and I have some questions, maybe somebody could help me out :) What happens if the user does not collect any data or set a future to do so in the invoke method? Also what

Contribute to Flink

Hi, Flink dev community, I'd like to contribute to Flink. Particularly, I am interested in kinds of optimization works in Flink. To be familiar with the process of contribution, I pick up some starter issue as the entrance contribution(s) such as [FLINK-5692](

Re: [DISCUSS] Flink ML roadmap

Hello guys, My couple of cents. All Flink presentations, articles, etc. articulate that Flink is for ETL, data ingestion. CEP is a maximum. If you visit http://flink.apache.org/usecases.html, you'll there aren't any explicit ML or Graphs there. It's also stated that Flink is suitable when "Data

Re: [DISCUSS] Flink ML roadmap

Hello guys, May be we will be able to focus our forces on some E2E scenario or show case for Flink as also ML supporting engine, and in such a way actualize the roadmap? This means: we can take some real life/production problem, like Fraud detection in some area, and try to solve this problem

Re: [DISCUSS] Flink ML roadmap

Hello all, thank you for opening this discussion Stavros, note that it's almost exactly 1 year since I last opened such a topic (linked by Gabor) and the comments there are still relevant. I think Gabor described the current state quite well, development in the libraries is hard without

Re: [DISCUSS] Flink ML roadmap

Hi Stavros, Thanks for bringing this up. There have been past [1] and recent [2, 3] discussions about the Flink libraries, because there are some stalling PRs and overloaded committers. (Actually, Till is the only committer shepherd of the both the CEP and ML library, and AFAIK he has a ton

Re: KeyGroupRangeAssignment ?

Hi Ovidiu, the way Flink works is to assign key group ranges to operators. For each element you calculate a hash value and based on that you assign it to a key group. Thus, in your example, you have either a key group with more than 1 key or multiple key groups with 1 or more keys assigned to an

Re: [Discuss] Organizing Documentation for Configuration Options

Unfortunately, I didn't have time to work on this yet and it looks like I won't have time this week. I've created the following issue to track the implementation with two sub tasks: https://issues.apache.org/jira/browse/FLINK-5779 If someone else wants to take this up in the meantime, feel free

Re: Interested to Contribute to Flink

Hi Karthik, great to hear that :-) Best you first take a look at the how to contribute guide [1] which will help you to get started. [1] http://flink.apache.org/how-to-contribute.html Cheers, Till On Fri, Feb 17, 2017 at 11:01 PM, Karthik Ramakrishnan < karthik.ramakrishnan...@gmail.com>

[DISCUSS] Flink ML roadmap

(Resending with the appropriate topic) Hi, I would like to start a discussion about next steps for Flink ML. Currently there is a lot of work going on but needs a push forward. Some topics to discuss: a) How several features should be planned and get aligned with Flink releases. b) Priorities

[jira] [Created] (FLINK-5850) implement OAuth 2.0 check in Web Backend API

Fabian Wollert created FLINK-5850: - Summary: implement OAuth 2.0 check in Web Backend API Key: FLINK-5850 URL: https://issues.apache.org/jira/browse/FLINK-5850 Project: Flink Issue Type:

[jira] [Created] (FLINK-5849) Kafka Consumer checkpointed state may contain undefined offsets

Tzu-Li (Gordon) Tai created FLINK-5849: -- Summary: Kafka Consumer checkpointed state may contain undefined offsets Key: FLINK-5849 URL: https://issues.apache.org/jira/browse/FLINK-5849 Project:

[jira] [Created] (FLINK-5848) make Flink Web Backend a little bit more restful

Fabian Wollert created FLINK-5848: - Summary: make Flink Web Backend a little bit more restful Key: FLINK-5848 URL: https://issues.apache.org/jira/browse/FLINK-5848 Project: Flink Issue Type:

[jira] [Created] (FLINK-5847) Not able to deploy Flink 1.3-SNAPSHOT on Openshit Paas

xinyang gao created FLINK-5847: -- Summary: Not able to deploy Flink 1.3-SNAPSHOT on Openshit Paas Key: FLINK-5847 URL: https://issues.apache.org/jira/browse/FLINK-5847 Project: Flink Issue Type:

[jira] [Created] (FLINK-5846) CEP: make the operators backwards compatible.

Kostas Kloudas created FLINK-5846: - Summary: CEP: make the operators backwards compatible. Key: FLINK-5846 URL: https://issues.apache.org/jira/browse/FLINK-5846 Project: Flink Issue Type:

[jira] [Created] (FLINK-5845) CEP: unify key and non-keyed operators

Kostas Kloudas created FLINK-5845: - Summary: CEP: unify key and non-keyed operators Key: FLINK-5845 URL: https://issues.apache.org/jira/browse/FLINK-5845 Project: Flink Issue Type: Bug