Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

> > >>> up > > >>>> whether to change the underlying architecture of Flink's Web > Dashboard > > >>> from > > >>>> Angular1 to something else. This email thread should be solely to > > discuss > > >>>> the

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

ttps://issues.apache.org/jira/browse/FLINK-10706> the discussion > > came > > >>> up > > >>>> whether to change the underlying architecture of Flink's Web > Dashboard > > >>> from > > >>>> Angular1 to something else. This

Re: [DISCUSS] Change underlying Frontend Architecture for Flink Web Dashboard

s already made towards >> the >> > improvements of Flink Web UI. >> > >> > Till, Robert, and Fabian, >> > Please take a look at the upgrading plan from Angular 1.x to 7.0 >> proposed >> > by Yadong. Any feedback is appreciated. >

[DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

Hi all Flink Web UI is the main platform for most users to monitor their jobs and clusters. We have reconstructed Flink web in 1.9.0 version, but there are still some shortcomings. This discussion thread aims to provide a better experience for Flink UI users. Here is the design doc I drafted:

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

> names in job graph visualization? For some complex jobs, especially SQL > jobs, the task names are quite long which makes the job graph hard to read. > > Best, > Paul Lam > > > 在 2019年9月27日，10:13，Yadong Xie 写道： > > > > Hi all > > > > Flink W

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation > > On Fri, Sep 27, 2019 at 3:57 PM Paul Lam wrote: > > > Filed a jira to track this[1]. Thanks a lot. > > > > [1] https://issues.apache.org/jira/browse/FLINK-14242 < > > https://issues.apache.or

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

Congratulations Jark! Danny Chan 于2019年11月11日周一 上午10:41写道： > Congrats Jark! Well deserved ! > > Best, > Danny Chan > 在 2019年11月11日 +0800 AM9:59，dev@flink.apache.org，写道： > > > > Congrats Jark! >

Re: [ANNOUNCE] Apache Flink 1.9.1 released

Hi Thomas Weise We have an issue related to this problem https://issues.apache.org/jira/browse/FLINK-14147, but I think you could open another JIRA The watermark data comes from metric REST API, which is too heavy for high parallelism jobs, the API almost sure will break when the parallelism is

Confluence permission for FLIP creation

Hi all I'm proposing to improve the Flink user experience in the Web UI and I have created a draft proposal doc. Could anyone grant the permission to convert the doc to a FLIP in Confluence, my id is vthink...@gmail.com Best

Re: Confluence permission for FLIP creation

Thanks a lot! Fabian Hueske 于2019年9月25日周三 下午3:55写道： > Hi, > > I gave you the permissions for the wiki. > > Thanks, Fabian > > Am Mi., 25. Sept. 2019 um 05:04 Uhr schrieb Yadong Xie < > vthink...@gmail.com > >: > > > Hi all > > > > I'm propos

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

when having a NFS, it > > should be enough to know where the file is located. However, this > > assumption does not hold in the general case. > > > > Cheers, > > Till > > > > On Mon, Sep 30, 2019 at 11:43 AM Yadong Xie wrote: > > > > > Hi Xintong Song &g

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

nion. > > Cheers, > Till > > On Fri, Feb 7, 2020 at 9:58 AM Yadong Xie wrote: > > > Hi Till > > > > FLIP-75 has been open since September, and the design doc has been > iterated > > over 3 versions and more than 20 patches. > > I had a tr

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

not sure > whether I would call this non-heap though) and the mapped memory. > > 3. Displaying the memory configurations in three nested boxes does not look > so nice to me. I'm not sure how else one could display it, though. > > 4. What does JVM limit mean in Non-heap.JVM-Overhead? &

Re: [VOTE] FLIP-100: Add Attempt Information

> > > Hi Yadong, > > > > Thanks for the updating. LGTM now. > > > > +1 (non-binding) > > > > Yadong Xie 于2020年2月25日周二 下午4:41写道： > > > > > Hi Kurt > > > > > > There will be no differences between batch jobs and stream jobs

Re: [VOTE] FLIP-100: Add Attempt Information

out changing the SubtasksTimeInfo response type into an array of > SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a > flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the previous > attempts. > > Cheers, > Till > > On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie

Re: [DISCUSS] Disable "Squash and merge" button for Flink repository on GitHub

Hi Jark There is a conversation about this here: https://github.community/t5/How-to-use-Git-and-GitHub/Authorship-of-merge-commits-made-by-Github-Apps-changed/td-p/48797 I think GitHub will fix it soon, it is a bug, not a feature :). Jingsong Li 于2020年3月5日周四 下午8:32写道： > Thanks for deep

Re: [VOTE] FLIP-100: Add Attempt Information

rmation for all attempts including > >> previous ones will be returned > >> > >> That being said, FLIP-100 looks good to me. From my side there is not > >> anything > >> else to discuss. > >> > >> @Kurt and @Jark: Can you look into the impr

[VOTE] FLIP-100[NEW]: Add Attempt Information

Hi all I want to start the vote for FLIP-100, which proposes to add attempt information inside subtask and timeline in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC Timeline Attempt (click the vertex timeline to see the differences):

Re: [DISCUSS] Disable "Squash and merge" button for Flink repository on GitHub

e misses the "This closes #" line to track back > > commits to PRs/reviews. > > > > The button goes against best practice, it should go away. > > > > Best, > > Stephan > > > > > > On Thu, Mar 5, 2020 at 1:51 PM Yadong Xie wrote: > &

Re: [VOTE] FLIP-100: Add Attempt Information

Hi all The rest API part has been updated with Gary and Till's suggestions here is the link: https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information Yadong Xie 于2020年3月3日周二 下午9:14写道： > Hi Chesnay > > most discussions in this vote are about the more fea

Re: Flink dev blog

Hi all maybe we can use markdown & GitHub to make the submission easy to review I have set up a similar blog for Flink-china blog before(deprecated), glad to offer help if needed here is the link: https://github.com/flink-china/doc Seth Wiesman 于2020年3月3日周二 下午10:51写道： > For lack of a better

Re: [VOTE] FLIP-100: Add Attempt Information

n changes to the > proposal. > > My conclusion is that this proposal needs more discussions. > > > On 20/02/2020 10:46, Yadong Xie wrote: > > Hi all > > > > I want to start the vote for FLIP-100, which proposes to add attempt > > information inside subtask and timeline in

Re: [VOTE] FLIP-99: Make Max Exception Configurable

+1 (binding) > > Best, > Gary > > On Thu, Feb 20, 2020 at 10:39 AM Yadong Xie wrote: > > > Hi all > > > > I want to start the vote for FLIP-99, which proposes to make the max > > exception configurable in web UI. > > > > To help every

Re: [VOTE] FLIP-98: Better Back Pressure Detection

ate +1 (binding). > > Cheers, > Till > > On Tue, Feb 25, 2020 at 4:55 AM Yadong Xie wrote: > > > Thanks all for the votes. > > > > So far, we have > > > > - 3 binding +1 votes (Kurt, Jark, zhijiang) > > - 4 non-binding +1 votes

Re: [VOTE] FLIP-103: Better TM/JM Log Display

： > late +1 (binding) > > Cheers, > Till > > On Tue, Feb 25, 2020 at 4:46 AM Yadong Xie wrote: > > > Thanks all for the votes. > > > > So far, we have > > > > - 4 binding +1 votes (Kurt, Jark, jincheng, Zhu Zhu) > > - 5 non-binding +1 votes (Yang

Re: [VOTE] FLIP-100: Add Attempt Information

+Attempt+Information Till Rohrmann 于2020年2月27日周四 下午9:49写道： > I think your proposal is better because then we don't have to bump the REST > API version. > > Cheers, > Till > > On Thu, Feb 27, 2020 at 8:02 AM Yadong Xie wrote: > > > Hi Till > > > > We

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

onfigurations. Therefore, I would > > > suggest to remove the "Other" frame, but add another frame inside "JVM > > > Overhead", besides "Configuration", with "JVM limit" as the title and > > > "non-heap max metric minus metaspace configu

Re: [VOTE] FLIP-98: Better Back Pressure Detection

---------- > > > > > From:Congxian Qiu > > > > > Send Time:2020 Feb. 20 (Thu.) 21:49 > > > > > To:dev@flink.apache.org > > > > > Subject:Re: [VOTE] FLIP-98: Better Back Pressure Detection > > > > > &

Re: [VOTE] FLIP-104: Add More Metrics to Jobmanager

ou~ > > > > > > Xintong Song > > > > > > > > > > > > On Fri, Feb 21, 2020 at 2:47 PM Jark Wu wrote: > > > > > > > Thanks Yadong, > > > > > > > > I think we can use different color to distinguis

Re: [VOTE] FLIP-100: Add Attempt Information

> > to the examples you gave, I can't tell whether i got required > > information > > > > from that. > > > > Can you replace the demo job to a more complex batch job and then we > > can > > > > see some > > > > differences of start/stop

Re: [VOTE] FLIP-100: Add Attempt Information

l whether i got required information > from that. > Can you replace the demo job to a more complex batch job and then we can > see some > differences of start/stop time of different tasks and attempts? > > Best, > Kurt > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie

Re: [VOTE] FLIP-104: Add More Metrics to Jobmanager

fferent color to distinguish the memory usage (from > > green to red?). > > Besides, I think we should add an unit on the "Garbage Collection" -> > > "Time", it's hard to know what the value mean. > > Would be better to display the value like "10ms&q

Re: [VOTE] FLIP-100: Add Attempt Information

tten the arrays? One could toggle the > historic attempts on and off with a query parameter. > * I think 'attempt-history' would be a better name instead of > 'attempts-time-info'. > > Let me know what you think. > > Best, > Gary > > On Thu, Feb 20, 2020 at 10:46 AM Yadong

Re: [VOTE] FLIP-103: Better TM/JM Log Display

t; > > > +1 (non-binding) > > > > I like the ideas of having a list of all log files, and make them > > downloadable. > > > > Thank you~ > > > > Xintong Song > > > > > > > > On Fri, Feb 21, 2020 at 10:24 AM Yang Wang < > >

Re: [VOTE] FLIP-101: Add Pending Slots Detail

ong, > > > > > > I think a pending slot view will be helpful. But will it be verbose > when > > > there is no pending slot, but a "pending slot" in the tab? > > > What do you think to show the pending slot page when click the "?" on > the &g

Re: [ANNOUNCE] Yu Li became a Flink committer

Well deserved! Yangze Guo 于2020年1月23日周四 下午5:06写道： > Congratulations! > > Best, > Yangze Guo > > > On Thu, Jan 23, 2020 at 4:59 PM Stephan Ewen wrote: > > > > Hi all! > > > > We are announcing that Yu Li has joined the rank of Flink committers. > > > > Yu joined already in late December, but

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

; > the document. > > I wonder if we can move this FLIP to the VOTing stage? > > > > On Wed, Jan 22, 2020 at 6:27 PM Till Rohrmann > > wrote: > > > > > Thanks for the update Yadong. Big +1 for the proposed improvements for > > > Flink's web UI. I think t

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

start this > practice. We would need a community discussion first, though. > > Cheers, > Till > > On Mon, Feb 3, 2020 at 10:28 AM Yadong Xie wrote: > > > Hi Till > > I didn’t find how to create of sub flip at cwiki.apache.org > > do you mean to create 9 more FLIPS instea

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

om/document/d/1Fak632iOroOLZFADqwZWu2SS-LUQqCLHnm8Vs3XM5to/edit?usp=sharing> - FLIP-105: Better Jobmanager Log Display <https://docs.google.com/document/d/1ayXaZflelaymQuF3l6UOuGEg6zzbSGGewoDq-9SBOPY/edit?usp=sharing> Yadong Xie 于2020年2月9日周日 下午7:24写道： > Hi Till > I got y

[VOTE] FLIP-75: Flink Web UI Improvement Proposal

Hi all, I would like to start the vote for FLIP-75 which proposes to provide a better experience for Flink users in web UI. In order to make the vote easier, the FLIP-75 was split into several top-level sub FLIPs: - FLIP-98: Better Back Pressure Detection

[VOTE] FLIP-102: Add More Metrics to TaskManager

Hi all I want to start the vote for FLIP-102, which proposes to add more metrics to the task manager in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web:

[VOTE] FLIP-103: Better TM/JM Log Display

Hi all I want to start the vote for FLIP-103, which proposes to provide better user log display UI in task manager and job manager. To help everyone better understand the proposal, we spent some efforts on making an online POC Task manager: previous web:

[VOTE] FLIP-104: Add More Metrics to Jobmanager

Hi all I want to start the vote for FLIP-104, which proposes to add more metrics to job manager. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web: http://101.132.122.69:8081/#/job-manager/config POC web:

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

would allow > > user to see GC issues (long/frequent pauses, lots of CPU time spent in > the > > GC) would be quite useful for analysing performance/stability issues, > > without a need of connecting profilers in a distributed environment? > > > > Piotrek >

Re: [VOTE] FLIP-75: Flink Web UI Improvement Proposal

wever I agree this > probably > > could be expanded/improved in the future. > > > > Piotrek > > > > > On 14 Feb 2020, at 03:28, Yadong Xie wrote: > > > > > > Hi all, > > > > > > I would like to start the vote for FLIP-75 wh

[VOTE] FLIP-98: Better Back Pressure Detection

Hi all I want to start the vote for FLIP-98, which proposes to provide better backpressure detection experience in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web:

[VOTE] FLIP-99: Make Max Exception Configurable

Hi all I want to start the vote for FLIP-99, which proposes to make the max exception configurable in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web: http://101.132.122.69:8081/#/job/543e9dc0cb2cca4433116007f0931d1a/exceptions

[VOTE] FLIP-100: Add Attempt Information

Hi all I want to start the vote for FLIP-100, which proposes to add attempt information inside subtask and timeline in web UI. To help everyone better understand the proposal, we spent some efforts on making an online POC Timeline Attempt (click the vertex timeline to see the differences):

[VOTE] FLIP-101: Add Pending Slots Detail

Hi all I want to start the vote for FLIP-101, which proposes to add pending slots information to help users check which vertex/subtask is blocked. To help everyone better understand the proposal, we spent some efforts on making an online POC previous web:

Re: [DISCUSS] FLIP-75: Flink Web UI Improvement Proposal

>- Add TaskManagerResourceInfo in TaskManagerDetailsInfo to show >TaskManager Resource FLINK-14435 ><https://issues.apache.org/jira/browse/FLINK-14435>. > > I will continue to update the rest part of the backend design in the doc, > let's keep discuss here

Re: [VOTE] FLIP-100[NEW]: Add Attempt Information

ng wrote: > > > +1 > > > > Best, > > Kurt > > > > > > On Wed, Mar 4, 2020 at 8:19 PM Gary Yao wrote: > > > > > +1 (binding) > > > > > > Best, > > > Gary > > > > > > On Wed, Mar 4, 2020 at 1:18 PM Ya

Re: [VOTE] FLIP-103: Better TM/JM Log Display

> Gary might be able to help with this effort. > > Cheers, > Till > > On Fri, Feb 28, 2020 at 11:08 AM Yadong Xie wrote: > > > Hi Till > > > > In addition to the front end, the FLIP-103 would also include some rest > API > > changes. > > Coul

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

t means. I fear that most > users won't understand what JVM limit actually means. > > Cheers, > Till > > On Wed, Feb 26, 2020 at 11:15 AM Yadong Xie wrote: > > > Hi Till > > > > Thanks a lot for your response > > > > > 2. I'm not entirely sure

[VOTE] NEW FLIP-102: Add More Metrics to TaskManager

Hi all I want to start a new vote for FLIP-102, which proposes to add more metrics to the task manager in web UI. The new FLIP-102 was revisited and adapted following the old ML discussion

Re: [VOTE] FLIP-102: Add More Metrics to TaskManager

Hi all there have been lots of discussions since the vote started and many suggestions have been made Matthias and I had updated the FLIP-102 following the suggestions and discussions I want to

Re: [VOTE] NEW FLIP-102: Add More Metrics to TaskManager

; manager > > > > > metrics. This FLIP should definitely help them understand the > memory > > > > > footprints and tune the configurations for task managers. > > > > > > > > > > The design part of this proposal looks really good

Re: [VOTE] FLIP-104: Add More Metrics to Jobmanager

red size of network memory and framework/task off-heap > > > memory. > > > >- Heap memory: We might observe that the memory usage keeps > growing > > > >until GC is triggered, thus eventually the utilization might > > > fluctuates at > > > >

[VOTE] NEW FLIP-104: Add More Metrics to JobManager

Hi all I want to start a new vote for FLIP-104, which proposes to add more metrics to the job manager in web UI. The new FLIP-104 was revisited and adapted following the old ML discussion

Re: [VOTE] NEW FLIP-104: Add More Metrics to JobManager

gt; > > > > On Tue, Oct 27, 2020 at 10:29 AM Xintong Song > > > > > wrote: > > > > > > > > > Thanks for reviving this FLIP, Yandong. > > > > > > > > > > The proposed changes look good to me. > > > > > +1 for a

[jira] [Created] (FLINK-10858) Create Angular Scaffold in Flink Dashboard

Yadong Xie created FLINK-10858: -- Summary: Create Angular Scaffold in Flink Dashboard Key: FLINK-10858 URL: https://issues.apache.org/jira/browse/FLINK-10858 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10870) Reconstruct Flink Dashboard Overview Module to Angular 7.0

Yadong Xie created FLINK-10870: -- Summary: Reconstruct Flink Dashboard Overview Module to Angular 7.0 Key: FLINK-10870 URL: https://issues.apache.org/jira/browse/FLINK-10870 Project: Flink Issue

[jira] [Created] (FLINK-14127) Better BackPressure Detection in WebUI

Yadong Xie created FLINK-14127: -- Summary: Better BackPressure Detection in WebUI Key: FLINK-14127 URL: https://issues.apache.org/jira/browse/FLINK-14127 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-14125) Display memory and CPU usage in the overview page

Yadong Xie created FLINK-14125: -- Summary: Display memory and CPU usage in the overview page Key: FLINK-14125 URL: https://issues.apache.org/jira/browse/FLINK-14125 Project: Flink Issue Type

[jira] [Created] (FLINK-14137) Show Attempt History in Vertex SubTask

Yadong Xie created FLINK-14137: -- Summary: Show Attempt History in Vertex SubTask Key: FLINK-14137 URL: https://issues.apache.org/jira/browse/FLINK-14137 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-14136) Operator Topology and Metrics Inside Vertex

Yadong Xie created FLINK-14136: -- Summary: Operator Topology and Metrics Inside Vertex Key: FLINK-14136 URL: https://issues.apache.org/jira/browse/FLINK-14136 Project: Flink Issue Type

[jira] [Created] (FLINK-14138) Show Pending Slots in Job Detail

Yadong Xie created FLINK-14138: -- Summary: Show Pending Slots in Job Detail Key: FLINK-14138 URL: https://issues.apache.org/jira/browse/FLINK-14138 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-14143) Failed Attempt does not display in the timeline

Yadong Xie created FLINK-14143: -- Summary: Failed Attempt does not display in the timeline Key: FLINK-14143 URL: https://issues.apache.org/jira/browse/FLINK-14143 Project: Flink Issue Type

[jira] [Created] (FLINK-14141) Locate Jobs/SubTask/Vertex Running on TaskManager

Yadong Xie created FLINK-14141: -- Summary: Locate Jobs/SubTask/Vertex Running on TaskManager Key: FLINK-14141 URL: https://issues.apache.org/jira/browse/FLINK-14141 Project: Flink Issue Type

[jira] [Created] (FLINK-14142) Add more metrics to task manager list

Yadong Xie created FLINK-14142: -- Summary: Add more metrics to task manager list Key: FLINK-14142 URL: https://issues.apache.org/jira/browse/FLINK-14142 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-14144) Add Metric for job manager

Yadong Xie created FLINK-14144: -- Summary: Add Metric for job manager Key: FLINK-14144 URL: https://issues.apache.org/jira/browse/FLINK-14144 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-14471) Hide error message when metric api failed

Yadong Xie created FLINK-14471: -- Summary: Hide error message when metric api failed Key: FLINK-14471 URL: https://issues.apache.org/jira/browse/FLINK-14471 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14393) add an option to enable/disable cancel job in web ui

Yadong Xie created FLINK-14393: -- Summary: add an option to enable/disable cancel job in web ui Key: FLINK-14393 URL: https://issues.apache.org/jira/browse/FLINK-14393 Project: Flink Issue Type

[jira] [Created] (FLINK-16335) vertex timeline should display all status other than the last one

Yadong Xie created FLINK-16335: -- Summary: vertex timeline should display all status other than the last one Key: FLINK-16335 URL: https://issues.apache.org/jira/browse/FLINK-16335 Project: Flink

[jira] [Created] (FLINK-17130) Web UI: Enable listing JM Logs and displaying Logs by Filename

Yadong Xie created FLINK-17130: -- Summary: Web UI: Enable listing JM Logs and displaying Logs by Filename Key: FLINK-17130 URL: https://issues.apache.org/jira/browse/FLINK-17130 Project: Flink

[jira] [Created] (FLINK-17347) upgrade to latest angular version

Yadong Xie created FLINK-17347: -- Summary: upgrade to latest angular version Key: FLINK-17347 URL: https://issues.apache.org/jira/browse/FLINK-17347 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-17608) Add TM log and stdout page back

Yadong Xie created FLINK-17608: -- Summary: Add TM log and stdout page back Key: FLINK-17608 URL: https://issues.apache.org/jira/browse/FLINK-17608 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-16741) add log list and read log by name for taskmanager in the web

Yadong Xie created FLINK-16741: -- Summary: add log list and read log by name for taskmanager in the web Key: FLINK-16741 URL: https://issues.apache.org/jira/browse/FLINK-16741 Project: Flink

[jira] [Created] (FLINK-19764) Add More Metrics to TaskManager in Web UI

Yadong Xie created FLINK-19764: -- Summary: Add More Metrics to TaskManager in Web UI Key: FLINK-19764 URL: https://issues.apache.org/jira/browse/FLINK-19764 Project: Flink Issue Type

[jira] [Created] (FLINK-19812) Add More Metrics to Jobmanager UI

Yadong Xie created FLINK-19812: -- Summary: Add More Metrics to Jobmanager UI Key: FLINK-19812 URL: https://issues.apache.org/jira/browse/FLINK-19812 Project: Flink Issue Type: Improvement