Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Yangze Guo
+1 (non-binding) It will make debugging more convenient and hence improve usability. Best, Yangze Guo On Fri, Feb 21, 2020 at 2:50 PM Yang Wang wrote: > > Hi Jark, > > The GC log is not enabled by default. If the users want, he/she should > configure it by the following config option. >

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Yang Wang
Hi Jark, The GC log is not enabled by default. If the users want, he/she should configure it by the following config option. env.java.opts.jobmanager: -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/opt/flink/log/jobmanager-gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=2

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Hequn Cheng
Congratulations Jingsong! Well deserved. Best, Hequn On Fri, Feb 21, 2020 at 2:42 PM Yang Wang wrote: > Congratulations!Jingsong. Well deserved. > > > Best, > Yang > > Zhijiang 于2020年2月21日周五 下午1:18写道: > >> Congrats Jingsong! Welcome on board! >> >> Best, >> Zhijiang >> >>

Re: [VOTE] FLIP-104: Add More Metrics to Jobmanager

2020-02-20 Thread Jark Wu
Thanks Yadong, I think we can use different color to distinguish the memory usage (from green to red?). Besides, I think we should add an unit on the "Garbage Collection" -> "Time", it's hard to know what the value mean. Would be better to display the value like "10ms", "5ns". Best, Jark On

Re: [VOTE] FLIP-98: Better Back Pressure Detection

2020-02-20 Thread Yang Wang
+1 non-binding It is really straightforward. Best, Yang Jark Wu 于2020年2月21日周五 下午1:52写道: > +1 (binding) > > On Fri, 21 Feb 2020 at 11:51, Kurt Young wrote: > > > +1 (binding) > > > > > > > > On Fri, Feb 21, 2020 at 1:09 AM Zhijiang > .invalid> > > wrote: > > > > > +1 (binding). > > > It

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Yang Wang
Congratulations!Jingsong. Well deserved. Best, Yang Zhijiang 于2020年2月21日周五 下午1:18写道: > Congrats Jingsong! Welcome on board! > > Best, > Zhijiang > > -- > From:Zhenghua Gao > Send Time:2020 Feb. 21 (Fri.) 12:49 > To:godfrey he >

Re: [Discuss] Update the pull request description template.

2020-02-20 Thread Yang Wang
I second xintong's suggestion. When i open a PR, i also check the item list in the template. It help to know whether i should test the PR in a real cluster(Yarn/K8s/Mesos). Or i should be more careful when touching the per-record code paths.If we have some dependencies changes, i will need to

Re: [VOTE] FLIP-101: Add Pending Slots Detail

2020-02-20 Thread Kurt Young
I agree with Jark, even if we have pending slots now, a dedicated tab seems to be too much. Best, Kurt On Fri, Feb 21, 2020 at 2:12 PM Jark Wu wrote: > Thanks Yadong, > > I think a pending slot view will be helpful. But will it be verbose when > there is no pending slot, but a "pending slot"

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Jark Wu
+1 (binding) A minor question: I didn't find the GC log in the POC web. Is it only generated when some configuration is enabled? Best, Jark On Fri, 21 Feb 2020 at 11:40, Kurt Young wrote: > +1 (binding) > > > On Fri, Feb 21, 2020 at 11:25 AM lining jing > wrote: > > > +1 (non-binding) > > It

Re: [VOTE] FLIP-101: Add Pending Slots Detail

2020-02-20 Thread Jark Wu
Thanks Yadong, I think a pending slot view will be helpful. But will it be verbose when there is no pending slot, but a "pending slot" in the tab? What do you think to show the pending slot page when click the "?" on the vertex status? Best, Jark On Thu, 20 Feb 2020 at 17:50, Yadong Xie wrote:

Re: [VOTE] FLIP-100: Add Attempt Information

2020-02-20 Thread Jark Wu
Thanks Yadong, I think one of the purpose to find the failed attempts is that we want to look into the logs in the failed node (node may change across attempts). However, from the POC, I didn't find a log link on each attempts. Maybe we can add such feature? Best, Jark On Fri, 21 Feb 2020 at

Re: [DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-20 Thread Yang Wang
Hi Canbing, Thanks a lot for sharing your thoughts to improve the Flink on K8s native integration. Frankly speaking, your discussion title confuses me and i am wondering whether you want to refactor the whole design. However, after i dive into the details and the provided document, i realize

Re: [VOTE] FLIP-99: Make Max Exception Configurable

2020-02-20 Thread Jark Wu
+1 (binding) Best, Jark On Thu, 20 Feb 2020 at 23:17, Congxian Qiu wrote: > +1 to this feature. this is helpful when troubleshooting. > > Best, > Congxian > > > Yadong Xie 于2020年2月20日周四 下午5:39写道: > > > Hi all > > > > I want to start the vote for FLIP-99, which proposes to make the max > >

Re: [VOTE] FLIP-98: Better Back Pressure Detection

2020-02-20 Thread Jark Wu
+1 (binding) On Fri, 21 Feb 2020 at 11:51, Kurt Young wrote: > +1 (binding) > > > > On Fri, Feb 21, 2020 at 1:09 AM Zhijiang .invalid> > wrote: > > > +1 (binding). > > It seems more clearly and directly to highlight the back pressured vertex > > in topology, which can raise the attention of

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Zhijiang
Congrats Jingsong! Welcome on board! Best, Zhijiang -- From:Zhenghua Gao Send Time:2020 Feb. 21 (Fri.) 12:49 To:godfrey he Cc:dev ; user Subject:Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer Congrats Jingsong! Best

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Zhenghua Gao
Congrats Jingsong! *Best Regards,* *Zhenghua Gao* On Fri, Feb 21, 2020 at 11:59 AM godfrey he wrote: > Congrats Jingsong! Well deserved. > > Best, > godfrey > > Jeff Zhang 于2020年2月21日周五 上午11:49写道: > >> Congratulations!Jingsong. You deserve it >> >> wenlong.lwl 于2020年2月21日周五 上午11:43写道: >>

Re: [DISCUSS] Support "create [temporary] table" and "create [temporary] view" statements

2020-02-20 Thread Jark Wu
Hi Caizhi, Thanks for raising this. I also think it is a bug in SQL CLI that CREATE VIEW creates a temporary view. We should fix that. If we want to fix it, we have to support create permanent view on TableEnvironment first which is not supported yet. Best, Jark On Fri, 21 Feb 2020 at 11:40,

[jira] [Created] (FLINK-16195) append constant field to unique key set on project when deriving unique keys in FlinkRelMdUniqueKeys

2020-02-20 Thread godfrey he (Jira)
godfrey he created FLINK-16195: -- Summary: append constant field to unique key set on project when deriving unique keys in FlinkRelMdUniqueKeys Key: FLINK-16195 URL: https://issues.apache.org/jira/browse/FLINK-16195

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

2020-02-20 Thread Xintong Song
> > 1. Should the managed memory be part of direct memory? > The answer is no. Managed memory is currently allocated by accessing to private field of Unsafe. It is not accounted for in JVM's direct memory limit and corresponding metrics. To that end, it is equivalent to native memory. > 2.

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread godfrey he
Congrats Jingsong! Well deserved. Best, godfrey Jeff Zhang 于2020年2月21日周五 上午11:49写道: > Congratulations!Jingsong. You deserve it > > wenlong.lwl 于2020年2月21日周五 上午11:43写道: > >> Congrats Jingsong! >> >> On Fri, 21 Feb 2020 at 11:41, Dian Fu wrote: >> >> > Congrats Jingsong! >> > >> > > 在

Re: [VOTE] FLIP-98: Better Back Pressure Detection

2020-02-20 Thread Kurt Young
+1 (binding) On Fri, Feb 21, 2020 at 1:09 AM Zhijiang wrote: > +1 (binding). > It seems more clearly and directly to highlight the back pressured vertex > in topology, which can raise the attention of users. > > Best, > Zhijiang > > >

Re: [VOTE] FLIP-100: Add Attempt Information

2020-02-20 Thread Kurt Young
Hi Yadong, Thanks for the proposal, it's a useful feature, especially for batch jobs. But according to the examples you gave, I can't tell whether i got required information from that. Can you replace the demo job to a more complex batch job and then we can see some differences of start/stop time

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Jeff Zhang
Congratulations!Jingsong. You deserve it wenlong.lwl 于2020年2月21日周五 上午11:43写道: > Congrats Jingsong! > > On Fri, 21 Feb 2020 at 11:41, Dian Fu wrote: > > > Congrats Jingsong! > > > > > 在 2020年2月21日,上午11:39,Jark Wu 写道: > > > > > > Congratulations Jingsong! Well deserved. > > > > > > Best, > > >

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Xingbo Huang
Congratulations Jingsong! Well deserved. Best, Xingbo wenlong.lwl 于2020年2月21日周五 上午11:43写道: > Congrats Jingsong! > > On Fri, 21 Feb 2020 at 11:41, Dian Fu wrote: > > > Congrats Jingsong! > > > > > 在 2020年2月21日,上午11:39,Jark Wu 写道: > > > > > > Congratulations Jingsong! Well deserved. > > > > >

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

2020-02-20 Thread Kurt Young
Some questions related to "managed memory": 1. Should the managed memory be part of direct memory? 2. Should the shuffle memory also be part of the managed memory? Best, Kurt On Fri, Feb 21, 2020 at 10:41 AM Xintong Song wrote: > Thanks for driving this FLIP, Yadong. > > +1 (non-binding) for

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread wenlong.lwl
Congrats Jingsong! On Fri, 21 Feb 2020 at 11:41, Dian Fu wrote: > Congrats Jingsong! > > > 在 2020年2月21日,上午11:39,Jark Wu 写道: > > > > Congratulations Jingsong! Well deserved. > > > > Best, > > Jark > > > > On Fri, 21 Feb 2020 at 11:32, zoudan wrote: > > > >> Congratulations! Jingsong > >> > >>

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Dian Fu
Congrats Jingsong! > 在 2020年2月21日,上午11:39,Jark Wu 写道: > > Congratulations Jingsong! Well deserved. > > Best, > Jark > > On Fri, 21 Feb 2020 at 11:32, zoudan wrote: > >> Congratulations! Jingsong >> >> >> Best, >> Dan Zou >>

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Kurt Young
+1 (binding) On Fri, Feb 21, 2020 at 11:25 AM lining jing wrote: > +1 (non-binding) > It lists all log files, the user could see the GC log. > > Xintong Song 于2020年2月21日周五 上午10:44写道: > > > +1 (non-binding) > > > > I like the ideas of having a list of all log files, and make them > >

Re: [DISCUSS] Support "create [temporary] table" and "create [temporary] view" statements

2020-02-20 Thread Jingsong Li
Hi Caizhi, thanks for starting this discussion. There is a FLIP-71 [1] to describe the whole story of view. Sql-cli now implements a wrong way, and a separate way, which should be deprecated and unified to TableEnvironment. > Shall we make it clear and support create table / create temporary

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Jark Wu
Congratulations Jingsong! Well deserved. Best, Jark On Fri, 21 Feb 2020 at 11:32, zoudan wrote: > Congratulations! Jingsong > > > Best, > Dan Zou >

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread zoudan
Congratulations! Jingsong Best, Dan Zou

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Rui Li
Congrats Jingsong! On Fri, Feb 21, 2020 at 11:05 AM Kurt Young wrote: > Hi everyone, > > I'm very happy to announce that Jingsong Lee accepted the offer of the > Flink PMC to > become a committer of the Flink project. > > Jingsong Lee has been an active community member for more than a year

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread lining jing
+1 (non-binding) It lists all log files, the user could see the GC log. Xintong Song 于2020年2月21日周五 上午10:44写道: > +1 (non-binding) > > I like the ideas of having a list of all log files, and make them > downloadable. > > Thank you~ > > Xintong Song > > > > On Fri, Feb 21, 2020 at 10:24 AM Yang

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Leonard Xu
Congratulations!! Jingsong. Best, Leonard Xu > 在 2020年2月21日,11:18,tison 写道: > > Congrats Jingsong, well deserved! > > Best, > tison. > > > Xintong Song mailto:tonysong...@gmail.com>> > 于2020年2月21日周五 上午11:09写道: > Congratulations, Jingsong. Well deserved~! > > Thank you~ > Xintong Song >

[DISCUSS] Support "create [temporary] table" and "create [temporary] view" statements

2020-02-20 Thread Caizhi Weng
Hi dear Flink developers, I'm quite confused by the "create view" statement in SQL client. >From traditional database we expect that a view behaves just like a table except that it will be computed when used. We also expect that a view should be created under a database just like a table. But

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread tison
Congrats Jingsong, well deserved! Best, tison. Xintong Song 于2020年2月21日周五 上午11:09写道: > Congratulations, Jingsong. Well deserved~! > > > Thank you~ > > Xintong Song > > > > On Fri, Feb 21, 2020 at 11:05 AM Kurt Young wrote: > >> Hi everyone, >> >> I'm very happy to announce that Jingsong Lee

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Benchao Li
Congrats Jingsong, well deserved! Kurt Young 于2020年2月21日周五 上午11:05写道: > Hi everyone, > > I'm very happy to announce that Jingsong Lee accepted the offer of the > Flink PMC to > become a committer of the Flink project. > > Jingsong Lee has been an active community member for more than a year

Re: [ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Xintong Song
Congratulations, Jingsong. Well deserved~! Thank you~ Xintong Song On Fri, Feb 21, 2020 at 11:05 AM Kurt Young wrote: > Hi everyone, > > I'm very happy to announce that Jingsong Lee accepted the offer of the > Flink PMC to > become a committer of the Flink project. > > Jingsong Lee has

[ANNOUNCE] Jingsong Lee becomes a Flink committer

2020-02-20 Thread Kurt Young
Hi everyone, I'm very happy to announce that Jingsong Lee accepted the offer of the Flink PMC to become a committer of the Flink project. Jingsong Lee has been an active community member for more than a year now. He is mainly focus on Flink SQL, played an essential role during blink planner

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Xintong Song
+1 (non-binding) I like the ideas of having a list of all log files, and make them downloadable. Thank you~ Xintong Song On Fri, Feb 21, 2020 at 10:24 AM Yang Wang wrote: > +1 non-binding > > The log files list are shown in the new design. I like the idea. Since the > user defined

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

2020-02-20 Thread Xintong Song
Thanks for driving this FLIP, Yadong. +1 (non-binding) for the FLIP in general. I think this really helps our users to understand and use the new FLIP-49 memory configuration. I have a few minor comments. - There's a frame "Other" in the frame "Non-Heap", besides "JVM Overhead" and "JVM

Re: [VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Yang Wang
+1 non-binding The log files list are shown in the new design. I like the idea. Since the user defined logs(e.g. my-log.file) could be found here now. Best, Yang Yadong Xie 于2020年2月20日周四 下午5:56写道: > Hi all > > I want to start the vote for FLIP-103, which proposes to provide better > user log

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-20 Thread Yang Wang
Thanks Stephan for kicking off the 1.11 release and Zhijiang and Piotr for the volunteering. A short release cycle is really good. Since we will not push many features simultaneously in every release. +1 for feature freeze date for end of April. Best, Yang Hequn Cheng 于2020年2月20日周四 下午8:27写道:

Re: [DISCUSS] Improvements on FLIP Process

2020-02-20 Thread Xintong Song
+1 for David's proposal. I think it happens a lot that we implement the FLIPs differently from the original proposals, based on more understandings during the implementation. Some times the differences are even not minors, which of course needs consensus and maybe a vote on the mailing list.

Re: [ANNOUNCE] Apache Flink Python API(PyFlink) 1.9.2 released

2020-02-20 Thread Xingbo Huang
Hi Chad, Beam Portability module solves the management of the Python runtime environment and the communication between the Python runtime environment Python VM and the Java runtime environment JVM in PyFlink. For details, you can refer to FLIP-58[1] and the Blog[2]. Hope to resolve your doubts.

[DISCUSS] FLINK-16194: Refactor the Kubernetes architecture design

2020-02-20 Thread felixzheng zheng
Hi everyone, I would like to kick off a discussion on refactoring the existing Kubernetes resources construction architecture design. I created a design document [1] that clarifies our motivation to do this and some improvement proposals for the new design. Briefly, we would like to 1.

[jira] [Created] (FLINK-16194) Refactor the Kubernetes resouces construction architecture

2020-02-20 Thread Canbin Zheng (Jira)
Canbin Zheng created FLINK-16194: Summary: Refactor the Kubernetes resouces construction architecture Key: FLINK-16194 URL: https://issues.apache.org/jira/browse/FLINK-16194 Project: Flink

Re: [DISCUSS] AssignerWithPeriodicWatermarks with max delay

2020-02-20 Thread Eduardo Winpenny Tejedor
Hi Till, Thanks for setting me on the right path! I'll finish reading the FLIP and Jira item and I'll contact Jiangjie Qin on how to best help. Regards, Eduardo On Wed, 19 Feb 2020, 14:44 Till Rohrmann, wrote: > Hi Eduardo, > > great to hear that you want to contribute to Flink. > > I'm not

[jira] [Created] (FLINK-16193) Improve error messaging when a key is assigned to the wrong key group range

2020-02-20 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-16193: Summary: Improve error messaging when a key is assigned to the wrong key group range Key: FLINK-16193 URL: https://issues.apache.org/jira/browse/FLINK-16193 Project:

[jira] [Created] (FLINK-16192) Remove "Legacy State" and Checkpoint Compatibility with Flink 1.2

2020-02-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-16192: Summary: Remove "Legacy State" and Checkpoint Compatibility with Flink 1.2 Key: FLINK-16192 URL: https://issues.apache.org/jira/browse/FLINK-16192 Project: Flink

Re: [DISCUSS] Improvements on FLIP Process

2020-02-20 Thread David Anderson
I'd like to bring up another problem that I see with the current FLIP process, which relates to what happens after a FLIP has been implemented. A few months ago I read through all of the then existing FLIPs. My reason for doing so is that on a great many topics, the only "documentation" we have

Re: [VOTE] FLIP-98: Better Back Pressure Detection

2020-02-20 Thread Zhijiang
+1 (binding). It seems more clearly and directly to highlight the back pressured vertex in topology, which can raise the attention of users. Best, Zhijiang -- From:Congxian Qiu Send Time:2020 Feb. 20 (Thu.) 21:49

Re: [DISCUSS] Drop connectors for Elasticsearch 2.x and 5.x

2020-02-20 Thread Robert Metzger
Thank you all. I have proposed a PR to upgrade the ES5 connector: https://github.com/apache/flink/pull/11162 On Wed, Feb 19, 2020 at 3:02 PM Till Rohrmann wrote: > +1 for removing ES2 and updating ES5 in 1.11. > > +1 for revisiting the removal of ES5 for 1.12. > > Cheers, > Till > > On Tue,

[jira] [Created] (FLINK-16191) Improve error message on Windows when RocksDB Paths are too long

2020-02-20 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-16191: Summary: Improve error message on Windows when RocksDB Paths are too long Key: FLINK-16191 URL: https://issues.apache.org/jira/browse/FLINK-16191 Project: Flink

Re: [VOTE] FLIP-99: Make Max Exception Configurable

2020-02-20 Thread Congxian Qiu
+1 to this feature. this is helpful when troubleshooting. Best, Congxian Yadong Xie 于2020年2月20日周四 下午5:39写道: > Hi all > > I want to start the vote for FLIP-99, which proposes to make the max > exception configurable in web UI. > > To help everyone better understand the proposal, we spent some

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

2020-02-20 Thread David Anderson
We stopped using these windows scripts in our training sessions a couple of years ago, and I can't remember seeing anyone use them at all since then. While our community includes many developers using Windows for development, in my experience pretty much everyone moves to some sort of linux-based

[jira] [Created] (FLINK-16190) Migrate existing java e2e tests away from FlinkDistribution

2020-02-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-16190: Summary: Migrate existing java e2e tests away from FlinkDistribution Key: FLINK-16190 URL: https://issues.apache.org/jira/browse/FLINK-16190 Project: Flink

[jira] [Created] (FLINK-16189) Move reponsibility fo copying distribution into FlinkResource

2020-02-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-16189: Summary: Move reponsibility fo copying distribution into FlinkResource Key: FLINK-16189 URL: https://issues.apache.org/jira/browse/FLINK-16189 Project: Flink

Re: [VOTE] FLIP-98: Better Back Pressure Detection

2020-02-20 Thread Congxian Qiu
+1 for this improvement, It's very helpful to know whether a task is backpressured or not. Best, Congxian Yadong Xie 于2020年2月20日周四 下午5:34写道: > Hi all > > I want to start the vote for FLIP-98, which proposes to provide better > backpressure detection experience in web UI. > > To help everyone

Re: [ANNOUNCE] Apache Flink Python API(PyFlink) 1.9.2 released

2020-02-20 Thread Xingbo Huang
Thanks a lot for the release. Great Work, Jincheng! Also thanks to participants who contribute to this release. Best, Xingbo Till Rohrmann 于2020年2月18日周二 下午11:40写道: > Thanks for updating the 1.9.2 release wrt Flink's Python API Jincheng! > > Cheers, > Till > > On Thu, Feb 13, 2020 at 12:25 PM

Re: [ANNOUNCE] Apache Flink-shaded 10.0 released

2020-02-20 Thread Xingbo Huang
Thanks a lot Chesnay and all contributes to this release. Best, Xingbo Hequn Cheng 于2020年2月20日周四 下午8:17写道: > Thanks a lot for the release Chesnay! > Also thanks to everyone who contributes to this release! > > Best, Hequn > > On Thu, Feb 20, 2020 at 11:11 AM Yu Li wrote: > > > Thanks Chesnay

[jira] [Created] (FLINK-16188) Make AutoClosableProcess constructor private

2020-02-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-16188: Summary: Make AutoClosableProcess constructor private Key: FLINK-16188 URL: https://issues.apache.org/jira/browse/FLINK-16188 Project: Flink Issue

[jira] [Created] (FLINK-16187) Support Python ML Pipeline API

2020-02-20 Thread Hequn Cheng (Jira)
Hequn Cheng created FLINK-16187: --- Summary: Support Python ML Pipeline API Key: FLINK-16187 URL: https://issues.apache.org/jira/browse/FLINK-16187 Project: Flink Issue Type: New Feature

Re: [VOTE] Support Python ML Pipeline API

2020-02-20 Thread Hequn Cheng
Thanks all for the votes. So far, we have - 4 binding +1 votes (Jincheng, Dian, Zhijiang and Rong) - 1 non-binding +1 votes (Yuan) - 0 -1 votes I will move the FLIP to accepted! Best, Hequn On Sat, Feb 15, 2020 at 12:45 AM Rong Rong wrote: > +1 (binding) > > On Fri, Feb 14, 2020 at

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-20 Thread Hequn Cheng
Thanks a lot for kicking off the 1.11 release and thanks Piotr and Zhijiang for volunteering as the release managers! Also +1 for the *anticipated feature freeze date* around end of April. A shorter release cycle would be nice. Best, Hequn On Thu, Feb 20, 2020 at 6:08 PM lining jing wrote: >

Re: [ANNOUNCE] Apache Flink-shaded 10.0 released

2020-02-20 Thread Hequn Cheng
Thanks a lot for the release Chesnay! Also thanks to everyone who contributes to this release! Best, Hequn On Thu, Feb 20, 2020 at 11:11 AM Yu Li wrote: > Thanks Chesnay and all participants for making the release possible! > > Best Regards, > Yu > > > On Thu, 20 Feb 2020 at 09:50, Zhu Zhu

Re: [DISCUSS] Improvements on FLIP Process

2020-02-20 Thread Hequn Cheng
Hi everyone, I appreciate all the feedback! Let me try to explain more. As there are a lot of people show concerns about the google doc. Let’s make an assumption first that the comment on the google doc is not allowed! Let’s make an assumption first that the comment on the google doc is not

[jira] [Created] (FLINK-16186) Speed up ElasticsearchITCase#testInvalidElasticsearchCluster

2020-02-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-16186: Summary: Speed up ElasticsearchITCase#testInvalidElasticsearchCluster Key: FLINK-16186 URL: https://issues.apache.org/jira/browse/FLINK-16186 Project: Flink

[DISCUSS] Drop Savepoint Compatibility with Flink 1.2

2020-02-20 Thread Stephan Ewen
Hi all! For some cleanup and simplifications, it would be helpful to drop Savepoint compatibility with Flink version 1.2. That version was released almost three years ago. I would expect that no one uses that old version any more in a way that they actively want to upgrade directly to 1.11.

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

2020-02-20 Thread Yadong Xie
Hi all Thanks to everyone involved in the discussion I have started the voting thread for all the sub FLIPs (including FLIP-98, FLIP-99, FLIP-100, FLIP-101, FLIP-102, FLIP-103, and FLIP-104) Please take some time to vote on the FLIP you are interested in, thanks lining jing 于2020年2月19日周三

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-20 Thread lining jing
Thanks for kicking off. +1 for the *anticipated feature freeze date* end of April. Zhu Zhu 于2020年2月20日周四 下午6:04写道: > Thanks Piotr and Zhijiang for volunteering as the release managers! > > +1 for aiming with the feature freeze date for end of April. > > Thanks, > Zhu Zhu > > Piotr Nowojski

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

2020-02-20 Thread Stephan Ewen
I think this question should go on the user mailing list, not on dev. You can also use Twitter to collect some data points. On Thu, Feb 20, 2020 at 11:07 AM Chesnay Schepler wrote: > Realistically you can't really use the windows scripts in production > since you can only start a standalone

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-20 Thread Jingsong Li
Thanks Stephan for the kicking off. Thanks Piotr and Zhijiang for volunteering. +1 for aiming with the feature freeze date for end of April. We should return to the quick release model. (3 months) Best, Jingsong Lee On Thu, Feb 20, 2020 at 6:04 PM Zhu Zhu wrote: > Thanks Piotr and Zhijiang

Re: [DISCUSS] Extend (or maintain) "shell" script support for Windows

2020-02-20 Thread Chesnay Schepler
Realistically you can't really use the windows scripts in production since you can only start a standalone cluster consisting of 1 JM and 1 TM. The .bat scripts are already limited to a significant degree. On 20/02/2020 03:48, Xintong Song wrote: I'm also in favor of dropping the bat scripts.

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-20 Thread Zhu Zhu
Thanks Piotr and Zhijiang for volunteering as the release managers! +1 for aiming with the feature freeze date for end of April. Thanks, Zhu Zhu Piotr Nowojski 于2020年2月20日周四 下午4:26写道: > Hi, > > Thank you for giving me and Zhijiang opportunity and entrusting us with > the release :) > > +1 for

[VOTE] FLIP-104: Add More Metrics to Jobmanager

2020-02-20 Thread Yadong Xie
Hi all I want to start the vote for FLIP-104, which proposes to add more metrics to job manager. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web: http://101.132.122.69:8081/#/job-manager/config POC web:

[jira] [Created] (FLINK-16185) Support JSON functions provides by calcite for blink planner

2020-02-20 Thread Zili Chen (Jira)
Zili Chen created FLINK-16185: - Summary: Support JSON functions provides by calcite for blink planner Key: FLINK-16185 URL: https://issues.apache.org/jira/browse/FLINK-16185 Project: Flink

[VOTE] FLIP-103: Better TM/JM Log Display

2020-02-20 Thread Yadong Xie
Hi all I want to start the vote for FLIP-103, which proposes to provide better user log display UI in task manager and job manager. To help everyone better understand the proposal, we spent some efforts on making an online POC Task manager: previous web:

[VOTE] FLIP-102: Add More Metrics to TaskManager

2020-02-20 Thread Yadong Xie
Hi all I want to start the vote for FLIP-102, which proposes to add more metrics to the task manager in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web:

[VOTE] FLIP-101: Add Pending Slots Detail

2020-02-20 Thread Yadong Xie
Hi all I want to start the vote for FLIP-101, which proposes to add pending slots information to help users check which vertex/subtask is blocked. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web:

[VOTE] FLIP-100: Add Attempt Information

2020-02-20 Thread Yadong Xie
Hi all I want to start the vote for FLIP-100, which proposes to add attempt information inside subtask and timeline in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC Timeline Attempt (click the vertex timeline to see the differences):

[VOTE] FLIP-99: Make Max Exception Configurable

2020-02-20 Thread Yadong Xie
Hi all I want to start the vote for FLIP-99, which proposes to make the max exception configurable in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web: http://101.132.122.69:8081/#/job/543e9dc0cb2cca4433116007f0931d1a/exceptions

Re: [Discussion] Job generation / submission hooks & Atlas integration

2020-02-20 Thread Gyula Fóra
Hi all! Thank you for the patience! We have created a small design document for the change proposal detailing the minimal required changes in Flink for the initial version of the Atlas integration. You can find the document here:

[VOTE] FLIP-98: Better Back Pressure Detection

2020-02-20 Thread Yadong Xie
Hi all I want to start the vote for FLIP-98, which proposes to provide better backpressure detection experience in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web:

[jira] [Created] (FLINK-16184) Submit job with fixed/configurable JobID

2020-02-20 Thread lining (Jira)
lining created FLINK-16184: -- Summary: Submit job with fixed/configurable JobID Key: FLINK-16184 URL: https://issues.apache.org/jira/browse/FLINK-16184 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-16183) Make identifier parsing in Table API more lenient

2020-02-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-16183: Summary: Make identifier parsing in Table API more lenient Key: FLINK-16183 URL: https://issues.apache.org/jira/browse/FLINK-16183 Project: Flink

Re: [DISCUSS] Kicking off the 1.11 release cycle

2020-02-20 Thread Piotr Nowojski
Hi, Thank you for giving me and Zhijiang opportunity and entrusting us with the release :) +1 for aiming with the feature freeze for and of April. I think all things consider that’s a good timing. Piotrek > On 20 Feb 2020, at 04:22, Yu Li wrote: > > Thanks Stephan for the kicking off! And

[jira] [Created] (FLINK-16182) Remove invalid check in input type inference logic

2020-02-20 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-16182: Summary: Remove invalid check in input type inference logic Key: FLINK-16182 URL: https://issues.apache.org/jira/browse/FLINK-16182 Project: Flink