[
https://issues.apache.org/jira/browse/FLINK-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TisonKun closed FLINK-12176.
----------------------------
Resolution: Won't Fix
After digging out of this topic, I found that there is an inconsistency between
per job mode and session mode. That is, per job mode always extracts the job
graph and submits it to Dispatcher(recover from SingleJobSubmittedJobGraphStore
actually); will session mode execute the whole user program.
This means that with current implementation if a batch job with multiple parts,
e.g., call {{collect}} or {{print}}, it acts quite different between session
mode and per job mode.
I regard this as part of topics we should discuss on the client api thread(a
unified view of submission in per job and session, at least program execution
inconsistency sounds depressive). Thus close this issue as won't fix with
current status.
> Unify JobGraph creation in CliFrontend
> --------------------------------------
>
> Key: FLINK-12176
> URL: https://issues.apache.org/jira/browse/FLINK-12176
> Project: Flink
> Issue Type: Improvement
> Components: Command Line Client
> Affects Versions: 1.9.0
> Reporter: TisonKun
> Assignee: TisonKun
> Priority: Major
> Attachments: patch.diff
>
>
> Currently, we create {{JobGraph}} by the following process
> * if the cluster start in job mode, we create {{JobGraph}} by
> {{PackagedProgramUtils#createJobGraph}} and deploy a job cluster
> * if the cluster start in session mode, we create {{JobGraph}} and submit it
> by {{CliFrontend#executeProgram}}, which internally the same as above but
> using {{ContextEnvironment}} instead of {{OptimizerPlanEnvironment}}.
> {{ContextEnvironment}} not only create the job graph but also submit it.
> However, the processes of {{JobGraph}} creation in job mode and session mode
> are similar. That means, we can unify the process by always create
> {{JobGraph}} by {{PackagedProgramUtils#createJobGraph}}. And,
> * in job mode, deploy job cluster with the {{JobGraph}}
> * in session mode, submit the {{JobGraph}} to the session cluster
> From a higher view, it is helpful for a common view of job submission in both
> job and session mode and give opportunities to refactor legacy client codes.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
