[jira] [Created] (FLINK-9052) Flink complains when scala.Option is used inside POJO

2018-03-21 Thread Truong Duc Kien (JIRA)
Truong Duc Kien created FLINK-9052: -- Summary: Flink complains when scala.Option is used inside POJO Key: FLINK-9052 URL: https://issues.apache.org/jira/browse/FLINK-9052 Project: Flink Issue

[jira] [Created] (FLINK-9051) Invert flip6 profile

2018-03-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9051: --- Summary: Invert flip6 profile Key: FLINK-9051 URL: https://issues.apache.org/jira/browse/FLINK-9051 Project: Flink Issue Type: Improvement Co

[jira] [Created] (FLINK-9050) Expose operator IO counter metrics

2018-03-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9050: --- Summary: Expose operator IO counter metrics Key: FLINK-9050 URL: https://issues.apache.org/jira/browse/FLINK-9050 Project: Flink Issue Type: New Featur

[jira] [Created] (FLINK-9049) Create unified interfaces to configure and instatiate TableSink

2018-03-21 Thread Shuyi Chen (JIRA)
Shuyi Chen created FLINK-9049: - Summary: Create unified interfaces to configure and instatiate TableSink Key: FLINK-9049 URL: https://issues.apache.org/jira/browse/FLINK-9049 Project: Flink Issu

[jira] [Created] (FLINK-9048) LocalFlinkMiniClusterITCase#testLocalFlinkMiniClusterWithMultipleTaskManagers sometimes fails

2018-03-21 Thread Ted Yu (JIRA)
Ted Yu created FLINK-9048: - Summary: LocalFlinkMiniClusterITCase#testLocalFlinkMiniClusterWithMultipleTaskManagers sometimes fails Key: FLINK-9048 URL: https://issues.apache.org/jira/browse/FLINK-9048 Projec

[jira] [Created] (FLINK-9047) SlotPool can fail to release slots

2018-03-21 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-9047: Summary: SlotPool can fail to release slots Key: FLINK-9047 URL: https://issues.apache.org/jira/browse/FLINK-9047 Project: Flink Issue Type: Bug Co

Re: [DISCUSS] Flink security improvements

2018-03-21 Thread Eron Wright
Please accept my apologies also. On Mon, Mar 19, 2018 at 2:46 AM, Till Rohrmann wrote: > Hi Shuyi, > > sorry for the unresponsiveness on your proposal. Currently, the community > is strongly focused on fixing and testing Flink 1.5 so that we can release > it soon. My gut feeling is that the comm

Re: FLIP-6 / Kubernetes

2018-03-21 Thread Eron Wright
Thanks Stephan for the information, it's exciting. This is an aspect for which the FLIP-6 design doc is lacking detail; somewhere we should work through how this will work (e.g. from a job recovery and upgrade perspective). If there's a doc or an open issue, please share, thanks! On Wed, Mar 2

Re: Flip 6 mesos support

2018-03-21 Thread Till Rohrmann
The resources consumed by the JobMaster can be specified by `jobmanager.heap.mb`. Cheers, Till On Wed, Mar 21, 2018 at 3:20 PM, Renjie Liu wrote: > Hi, Till: > > In fact, I want to ask the resources consume by job manager > > Till Rohrmann 于 2018年3月21日周三 下午8:17写道: > > > As many as the applicat

[jira] [Created] (FLINK-9046) Improve error messages when receiving unexpected messages.

2018-03-21 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-9046: - Summary: Improve error messages when receiving unexpected messages. Key: FLINK-9046 URL: https://issues.apache.org/jira/browse/FLINK-9046 Project: Flink Is

Re: Flip 6 mesos support

2018-03-21 Thread Renjie Liu
Hi, Till: In fact, I want to ask the resources consume by job manager Till Rohrmann 于 2018年3月21日周三 下午8:17写道: > As many as the application needs to run. If you start a job with > parallelism 10 then it will ask for 10 slots (assuming slot sharing). > > On Wed, Mar 21, 2018 at 12:04 PM, Renjie Li

[jira] [Created] (FLINK-9045) LocalEnvironment with web UI does not work with flip-6

2018-03-21 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-9045: --- Summary: LocalEnvironment with web UI does not work with flip-6 Key: FLINK-9045 URL: https://issues.apache.org/jira/browse/FLINK-9045 Project: Flink Issue Type

[jira] [Created] (FLINK-9044) Fork a new stream in a stream

2018-03-21 Thread Herbert Koelman (JIRA)
Herbert Koelman created FLINK-9044: -- Summary: Fork a new stream in a stream Key: FLINK-9044 URL: https://issues.apache.org/jira/browse/FLINK-9044 Project: Flink Issue Type: Wish

Re: Request for contributor permissions

2018-03-21 Thread Till Rohrmann
Welcome to the community Qi Yu. I've given you contributor permissions. Cheers, Till On Wed, Mar 21, 2018 at 12:13 PM, Qi Yu wrote: > Hi devs, > I'm very interested in Flink and willing to contribute to it. Cloud > anybody give me the contributor permissions? My jira username is “yuqi”. > Thank

[jira] [Created] (FLINK-9043) Flink recover from checkpoint like Spark Streaming

2018-03-21 Thread godfrey johnson (JIRA)
godfrey johnson created FLINK-9043: -- Summary: Flink recover from checkpoint like Spark Streaming Key: FLINK-9043 URL: https://issues.apache.org/jira/browse/FLINK-9043 Project: Flink Issue T

Re: FLIP-6 / Kubernetes

2018-03-21 Thread Stephan Ewen
Hi Christophe! I could imagine 1.5.1 or 1.6. It is important to point out that you can use Flink with Kubernetes quite well already. That addition is mainly intended to make it more idiomatic for Kubernetes users. Best, Stephan On Wed, Mar 21, 2018 at 12:03 PM, Christophe Jolif wrote: > Hi

Re: Flip 6 mesos support

2018-03-21 Thread Till Rohrmann
As many as the application needs to run. If you start a job with parallelism 10 then it will ask for 10 slots (assuming slot sharing). On Wed, Mar 21, 2018 at 12:04 PM, Renjie Liu wrote: > So how many slots a job manager may consume? > > On Wed, Mar 21, 2018 at 6:50 PM Till Rohrmann > wrote: >

Request for contributor permissions

2018-03-21 Thread Qi Yu
Hi devs, I'm very interested in Flink and willing to contribute to it. Cloud anybody give me the contributor permissions? My jira username is “yuqi”. Thanks~ Regards, Qi Yu

Re: Flip 6 mesos support

2018-03-21 Thread Renjie Liu
So how many slots a job manager may consume? On Wed, Mar 21, 2018 at 6:50 PM Till Rohrmann wrote: > At the moment this is not possible. In order to do this, you will have to > use the per-job mode and run each job on a dedicated Flink cluster. > > On Wed, Mar 21, 2018 at 11:33 AM, Renjie Liu >

Re: FLIP-6 / Kubernetes

2018-03-21 Thread Christophe Jolif
Hi Stephan, > This feature did not make it for 1.5, but should come very soon after. Are you talking about a 1.5.1 here? What is the envisioned timeline? Thanks! On Tue, Mar 20, 2018 at 9:30 PM, Stephan Ewen wrote: > @Eron That is definitely the way we want to suggest as the way to use k8s >

[jira] [Created] (FLINK-9042) Port ResumeCheckpointManuallyITCase to flip6

2018-03-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9042: --- Summary: Port ResumeCheckpointManuallyITCase to flip6 Key: FLINK-9042 URL: https://issues.apache.org/jira/browse/FLINK-9042 Project: Flink Issue Type:

Re: Flip 6 mesos support

2018-03-21 Thread Till Rohrmann
At the moment this is not possible. In order to do this, you will have to use the per-job mode and run each job on a dedicated Flink cluster. On Wed, Mar 21, 2018 at 11:33 AM, Renjie Liu wrote: > For example, we have 2 jobs. > For job 1, I want to start job manger with 1 CPU and 100M memory. Job

[jira] [Created] (FLINK-9041) Refactor StreamTaskTest to not use scala and akka

2018-03-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9041: --- Summary: Refactor StreamTaskTest to not use scala and akka Key: FLINK-9041 URL: https://issues.apache.org/jira/browse/FLINK-9041 Project: Flink Issue T

[jira] [Created] (FLINK-9040) JobVertex#setMaxParallelism does not valid argument

2018-03-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9040: --- Summary: JobVertex#setMaxParallelism does not valid argument Key: FLINK-9040 URL: https://issues.apache.org/jira/browse/FLINK-9040 Project: Flink Issue

Re: Flip 6 mesos support

2018-03-21 Thread Renjie Liu
For example, we have 2 jobs. For job 1, I want to start job manger with 1 CPU and 100M memory. Job 1 need s10 slots, and I want to deploy these 10 slots in 2 task managers, each with 5 cores and 1G memory. For job 2, I want to start job manager with 2 CPU and 200M memory. Job 2 needs 100 slots and

Re: Flip 6 mesos support

2018-03-21 Thread Till Rohrmann
Hi Renjie, what do you mean with specifying different JM and TM resources for different jobs exactly? Cheers, Till On Wed, Mar 21, 2018 at 10:55 AM, Renjie Liu wrote: > Hi, Till: > > How to specify job manager and task manager resources for different jobs in > session mode? > > On Sun, Mar 18,

[jira] [Created] (FLINK-9039) Broken link to Hadoop Setup Guide in docs

2018-03-21 Thread Florian Schmidt (JIRA)
Florian Schmidt created FLINK-9039: -- Summary: Broken link to Hadoop Setup Guide in docs Key: FLINK-9039 URL: https://issues.apache.org/jira/browse/FLINK-9039 Project: Flink Issue Type: Bug

Re: Flip 6 mesos support

2018-03-21 Thread Renjie Liu
Hi, Till: How to specify job manager and task manager resources for different jobs in session mode? On Sun, Mar 18, 2018 at 1:10 AM Till Rohrmann wrote: > Hi Shuyi, > > best if you look at the other e2e tests in the flink-end-to-end-tests > module. For example the Kafka e2e test under > flink/f

[jira] [Created] (FLINK-9038) YARNSessionCapacitySchedulerITCase fails on travis

2018-03-21 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9038: --- Summary: YARNSessionCapacitySchedulerITCase fails on travis Key: FLINK-9038 URL: https://issues.apache.org/jira/browse/FLINK-9038 Project: Flink Issue

Re: Is Hadoop 3.0 integration planned?

2018-03-21 Thread Stephan Ewen
That is definitely a good thing to have, would like to have a discussion about how to approach that after 1.5 is released. On Wed, Mar 21, 2018 at 5:39 AM, Jayant Ameta wrote: > > Jayant Ameta >

[jira] [Created] (FLINK-9037) Test flake Kafka09ITCase#testCancelingEmptyTopic

2018-03-21 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-9037: --- Summary: Test flake Kafka09ITCase#testCancelingEmptyTopic Key: FLINK-9037 URL: https://issues.apache.org/jira/browse/FLINK-9037 Project: Flink Issue Type: Bug