[jira] [Created] (FLINK-9878) IO worker threads BLOCKED on SSL Session Cache while CMS full gc

2018-07-17 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-9878: -- Summary: IO worker threads BLOCKED on SSL Session Cache while CMS full gc Key: FLINK-9878 URL: https://issues.apache.org/jira/browse/FLINK-9878 Project: Flink

[jira] [Created] (FLINK-9876) ExecutionGraphCoLocationRestartTest testConstraintsAfterRestart sometimes fails with timeout

2018-07-17 Thread Florian Schmidt (JIRA)
Florian Schmidt created FLINK-9876: -- Summary: ExecutionGraphCoLocationRestartTest testConstraintsAfterRestart sometimes fails with timeout Key: FLINK-9876 URL: https://issues.apache.org/jira/browse/FLINK-9876

Re: [DISCUSS] Improve broadcast serialization

2018-07-17 Thread Piotr Nowojski
Hi Generally speaking this would be a nice optimisation, however it might be tricky to implement. The thing to keep in mind is that currently interface allow to interleave broadcasting and normal sending, because of that at any given time some serialisers can have more data then others. For

[jira] [Created] (FLINK-9874) set_conf_ssl in E2E tests fails on macOS

2018-07-17 Thread Florian Schmidt (JIRA)
Florian Schmidt created FLINK-9874: -- Summary: set_conf_ssl in E2E tests fails on macOS Key: FLINK-9874 URL: https://issues.apache.org/jira/browse/FLINK-9874 Project: Flink Issue Type: Bug

[ANNOUNCE] Weekly community update #29

2018-07-17 Thread Till Rohrmann
Dear community, this is the weekly community update thread #29. Please post any news and updates you want to share with the community to this thread. # Feature freeze Flink 1.6 The Flink community has cut off the release branch for Flink 1.6 [1]. From now on, the community will concentrate on

Re: Flink CLI properties with HA

2018-07-17 Thread Till Rohrmann
Hi Sampath, technically the client does not need to know the `high-availability.storageDir` to submit a job. However, due to how we construct the ZooKeeperHaServices it is still needed. The reason behind this is that we use the same services for the server and the client. Thus, the implementation

[jira] [Created] (FLINK-9877) Add separate docs page for different join types in DataStream API

2018-07-17 Thread Florian Schmidt (JIRA)
Florian Schmidt created FLINK-9877: -- Summary: Add separate docs page for different join types in DataStream API Key: FLINK-9877 URL: https://issues.apache.org/jira/browse/FLINK-9877 Project: Flink

[jira] [Created] (FLINK-9875) Add concurrent creation of execution job vertex

2018-07-17 Thread JIRA
陈梓立 created FLINK-9875: -- Summary: Add concurrent creation of execution job vertex Key: FLINK-9875 URL: https://issues.apache.org/jira/browse/FLINK-9875 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-9880) Incorrect argument order calling BucketerContext#update

2018-07-17 Thread Ted Yu (JIRA)
Ted Yu created FLINK-9880: - Summary: Incorrect argument order calling BucketerContext#update Key: FLINK-9880 URL: https://issues.apache.org/jira/browse/FLINK-9880 Project: Flink Issue Type: Bug

What is the way to prove performance improvement?

2018-07-17 Thread 陈梓立
Hi all, Recently I pull 3 PRs about performance improvements[1][2][3]. Unit tests will verify their correctness, and in the real scenario, we have benchmark report to confirm that they do help for performance. I wonder what is the formal way to verify a performance improvement. Is it to give out

[jira] [Created] (FLINK-9881) Typo in a function name in table.scala

2018-07-17 Thread Ashwin Sinha (JIRA)
Ashwin Sinha created FLINK-9881: --- Summary: Typo in a function name in table.scala Key: FLINK-9881 URL: https://issues.apache.org/jira/browse/FLINK-9881 Project: Flink Issue Type: Improvement

回复:[DISCUSS] Improve broadcast serialization

2018-07-17 Thread Zhijiang(wangzhijiang999)
Hi Piotr, Thanks for your replies and professional suggestions! My initial thought is just as you said in first suggestion. The current RecordWriter will emit StreamRecord to some subpartition via ChannelSelector or broadcast events/watermark to all subpartitions directly. If the

[jira] [Created] (FLINK-9882) A function access can be private in client

2018-07-17 Thread Riley Li (JIRA)
Riley Li created FLINK-9882: --- Summary: A function access can be private in client Key: FLINK-9882 URL: https://issues.apache.org/jira/browse/FLINK-9882 Project: Flink Issue Type: Improvement

Flink 1.5 job distribution over cluster nodes

2018-07-17 Thread Shachar Carmeli
Hi, We have 4 jobs with parallelism 3 that are running over 3 task manager with 4 slots per each , each task manager runs on a different VM , On Flink 1.3.2 the jobs were evenly distributed per node each job took one task slot of each task manager . After upgrading to flink 1.5 , each job is

[jira] [Created] (FLINK-9870) Support field mapping and time attributes for table sinks

2018-07-17 Thread Timo Walther (JIRA)
Timo Walther created FLINK-9870: --- Summary: Support field mapping and time attributes for table sinks Key: FLINK-9870 URL: https://issues.apache.org/jira/browse/FLINK-9870 Project: Flink Issue

Re: Flink CLI properties with HA

2018-07-17 Thread Sampath Bhat
Hi vino Should the flink CLI have access to the path mentioned in *high-availability.storageDir*? If my flink cluster is on set of machines and i submit my job from flink CLI from another independent machine by giving necessary details will the CLI try to access *high-availability.storageDir

Re: Flink CLI properties with HA

2018-07-17 Thread vino yang
Hi Sampath, It seems Flink CLI for standalone would not access *high-availability.storageDir.* What's the exception stack trace in your environment? Thanks, vino. 2018-07-17 15:08 GMT+08:00 Sampath Bhat : > Hi vino > > Should the flink CLI have access to the path mentioned in >

Re: Flink WindowedStream - Need assistance

2018-07-17 Thread Timo Walther
Hi Titus, have you looked into ProcessFunction? ProcessFunction[1] gives you access to the two important streaming primitives "time" and "state". So in your case you can decide flexibly what you want to put into state and when you want to set and fire a timer (for clean-up) per key.

[jira] [Created] (FLINK-9879) Find sane defaults for (advanced) SSL session parameters

2018-07-17 Thread Nico Kruber (JIRA)
Nico Kruber created FLINK-9879: -- Summary: Find sane defaults for (advanced) SSL session parameters Key: FLINK-9879 URL: https://issues.apache.org/jira/browse/FLINK-9879 Project: Flink Issue

[ANNOUNCE] Program for Flink Forward Berlin 2018 has been announced

2018-07-17 Thread Fabian Hueske
Hi everyone, I'd like to announce the program for Flink Forward Berlin 2018. The program committee [1] assembled a program of about 50 talks on use cases, operations, ecosystem, tech deep dive, and research topics. The conference will host speakers from Airbnb, Amazon, Google, ING, Lyft,

[jira] [Created] (FLINK-9871) Use Description class for ConfigOptions with rich formatting

2018-07-17 Thread Dawid Wysakowicz (JIRA)
Dawid Wysakowicz created FLINK-9871: --- Summary: Use Description class for ConfigOptions with rich formatting Key: FLINK-9871 URL: https://issues.apache.org/jira/browse/FLINK-9871 Project: Flink

Re: Flink WindowedStream - Need assistance

2018-07-17 Thread Titus Rakkesh
Friends, any assistance regarding this? On Mon, Jul 16, 2018 at 3:44 PM, Titus Rakkesh wrote: > Dear All, > > We have 2 independent streams which will receive elements in different > frequency, > > DataStream> splittedActivationTuple; > > DataStream> unionReloadsStream; > > We have a

[jira] [Created] (FLINK-9872) SavepointITCase#testSavepointForJobWithIteration does not properly cancell jobs

2018-07-17 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9872: --- Summary: SavepointITCase#testSavepointForJobWithIteration does not properly cancell jobs Key: FLINK-9872 URL: https://issues.apache.org/jira/browse/FLINK-9872

[jira] [Created] (FLINK-9873) Log actual state when aborting checkpoint due to task not running

2018-07-17 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-9873: --- Summary: Log actual state when aborting checkpoint due to task not running Key: FLINK-9873 URL: https://issues.apache.org/jira/browse/FLINK-9873 Project: Flink