Thanks Rui for driving this! +1 for this idea.
Best, Zakelly On Mon, Apr 8, 2024 at 7:17 PM Ahmed Hamdy <hamdy10...@gmail.com> wrote: > Acknowledged, +1 to start a vote. > Best Regards > Ahmed Hamdy > > > On Mon, 8 Apr 2024 at 12:04, Rui Fan <1996fan...@gmail.com> wrote: > > > Sorry, it's a typo. It should be FLINK-32558[1]. > > > > [1] https://issues.apache.org/jira/browse/FLINK-32558 > > > > Best, > > Rui > > > > On Mon, Apr 8, 2024 at 6:44 PM Ahmed Hamdy <hamdy10...@gmail.com> wrote: > > > > > Hi Rui, > > > Thanks for the proposal. > > > Is the deprecation Jira mentioned (FLINK-32258) correct? > > > Best Regards > > > Ahmed Hamdy > > > > > > > > > On Sun, 7 Apr 2024 at 03:37, Rui Fan <1996fan...@gmail.com> wrote: > > > > > > > If there are no extra comments, I will start voting in three days, > > thank > > > > you~ > > > > > > > > Best, > > > > Rui > > > > > > > > On Thu, Mar 28, 2024 at 4:46 PM Muhammet Orazov > > > > <mor+fl...@morazow.com.invalid> wrote: > > > > > > > > > Hey Rui, > > > > > > > > > > Thanks for the detailed explanation and updating the FLIP! > > > > > > > > > > It is much clearer definitely, thanks for the proposal. > > > > > > > > > > Best, > > > > > Muhammet > > > > > > > > > > On 2024-03-28 07:37, Rui Fan wrote: > > > > > > Hi Muhammet, > > > > > > > > > > > > Thanks for your reply! > > > > > > > > > > > >> The execution mode is also used for the DataStream API [1], > > > > > >> would that also affect/hide the DataStream execution mode > > > > > >> if we remove it from the WebUI? > > > > > > > > > > > > Sorry, I didn't describe it clearly in FLIP-441[2], I have > updated > > > it. > > > > > > Let me clarify the Execution Mode here: > > > > > > > > > > > > 1. Flink 1.19 website[3] also mentions the Execution mode, but it > > > > > > actually matches the JobType[4] in the Flink code. Both of them > > > > > > have 2 types: STREAMING and BATCH. > > > > > > 2. execution.runtime-mode can be set to 3 types: STREAMING, > > > > > > BATCH and AUTOMATIC. But the jobType will be inferred as > > > > > > STREAMING or BATCH when execution.runtime-mode is set > > > > > > to AUTOMATIC. > > > > > > 3. The ExecutionMode I describe is: code link[5] , as we can > > > > > > see, ExecutionMode has 4 enums: PIPELINED, > > > > > > PIPELINED_FORCED, BATCH and BATCH_FORCED. > > > > > > And we can see a flink streaming job from Flink WebUI, > > > > > > the Execution mode is PIPELINE instead of STREAMING. > > > > > > I attached a screenshot to the FLIP doc[2], you can see it there. > > > > > > 4. What this proposal wants to do is to remove the ExecutionMode > > > > > > with four enumerations on Flink WebUI and introduce the > > > > > > JobType with two enumerations (STREAMING or BATCH). > > > > > > STREAMING or BATCH is clearer and more accurate for users. > > > > > > > > > > > > Please let me know if it's not clear or anything is wrong, > thanks a > > > > > > lot! > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/datastream/execution_mode/ > > > > > > [2] https://cwiki.apache.org/confluence/x/agrPEQ > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/datastream/execution_mode/ > > > > > > [4] > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/flink/blob/f31c128bfc457b64dd7734f71123b74faa2958ba/flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/JobType.java#L22 > > > > > > [5] > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/flink/blob/f31c128bfc457b64dd7734f71123b74faa2958ba/flink-core/src/main/java/org/apache/flink/api/common/ExecutionMode.java#L54 > > > > > > > > > > > > Best, > > > > > > Rui > > > > > > > > > > > > On Thu, Mar 28, 2024 at 1:33 AM Venkatakrishnan Sowrirajan > > > > > > <vsowr...@asu.edu> > > > > > > wrote: > > > > > > > > > > > >> Rui, > > > > > >> > > > > > >> I assume the current proposal would also handle the case of > mixed > > > mode > > > > > >> (BATCH + STREAMING within the same app) in the future, right? > > > > > >> > > > > > >> Regards > > > > > >> Venkat > > > > > >> > > > > > >> On Wed, Mar 27, 2024 at 10:15 AM Venkatakrishnan Sowrirajan < > > > > > >> vsowr...@asu.edu> wrote: > > > > > >> > > > > > >>> This will be a very useful addition to Flink UI. Thanks Rui for > > > > > >>> starting > > > > > >>> a FLIP for this improvement. > > > > > >>> > > > > > >>> Regards > > > > > >>> Venkata krishnan > > > > > >>> > > > > > >>> > > > > > >>> On Wed, Mar 27, 2024 at 4:49 AM Muhammet Orazov > > > > > >>> <mor+fl...@morazow.com.invalid> wrote: > > > > > >>> > > > > > >>>> Hello Rui, > > > > > >>>> > > > > > >>>> Thanks for the proposal! It looks good! > > > > > >>>> > > > > > >>>> I have minor clarification from my side: > > > > > >>>> > > > > > >>>> The execution mode is also used for the DataStream API [1], > > > > > >>>> would that also affect/hide the DataStream execution mode > > > > > >>>> if we remove it from the WebUI? > > > > > >>>> > > > > > >>>> Best, > > > > > >>>> Muhammet > > > > > >>>> > > > > > >>>> [1]: > > > > > >>>> > > > > > >>>> > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/dev/datastream/execution_mode/__;!!IKRxdwAv5BmarQ!eFyqVJyje_8hu1vMSUwKGBsj8vqsFDisEvJ5AxPV0LduhhHWF3rPKYEEE-09biA0unFbfMy5AVQZMgBv1AOa5oTHmcYlkUE$ > > > > > >>>> > > > > > >>>> > > > > > >>>> On 2024-03-27 06:23, Rui Fan wrote: > > > > > >>>> > Hi flink developers, > > > > > >>>> > > > > > > >>>> > I'd like to start a discussion to discuss FLIP-441: > > > > > >>>> > Show the JobType and remove Execution Mode on Flink > WebUI[1]. > > > > > >>>> > > > > > > >>>> > Currently, the jobType has 2 types in Flink: STREAMING and > > > BATCH. > > > > > >>>> > They work on completely different principles, such as: > > > scheduler, > > > > > >>>> > shuffle, join, etc. These differences lead to different > > > > > troubleshooting > > > > > >>>> > processes, so when users are maintaining a job or > > > troubleshooting, > > > > > >>>> > it's needed to know whether the current job is a STREAMING > or > > > > > >>>> > BATCH job. Unfortunately, Flink WebUI doesn't expose it to > the > > > > > >>>> > users so far. > > > > > >>>> > > > > > > >>>> > Also, Execution Mode is related to DataSet api, it has been > > > marked > > > > > >>>> > as @Deprecated in FLINK-32258 (1.18), but it's still shown > in > > > > Flink > > > > > >>>> > WebUI. > > > > > >>>> > > > > > > >>>> > Looking forward to hearing more thoughts about it! Thank > you~ > > > > > >>>> > > > > > > >>>> > [1] > > > > > >>>> > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/x/agrPEQ__;!!IKRxdwAv5BmarQ!eFyqVJyje_8hu1vMSUwKGBsj8vqsFDisEvJ5AxPV0LduhhHWF3rPKYEEE-09biA0unFbfMy5AVQZMgBv1AOa5oTHayPyFj8$ > > > > > >>>> > [2] > > > > > >>>> > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://issues.apache.org/jira/browse/FLINK-32558__;!!IKRxdwAv5BmarQ!eFyqVJyje_8hu1vMSUwKGBsj8vqsFDisEvJ5AxPV0LduhhHWF3rPKYEEE-09biA0unFbfMy5AVQZMgBv1AOa5oTHftYeOLE$ > > > > > >>>> > > > > > > >>>> > Best, > > > > > >>>> > Rui > > > > > >>>> > > > > > >>> > > > > > > > > > > > > > > >
