ASF GitHub Bot commented on FLINK-6489:

GitHub user StephanEwen opened a pull request:


    [FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManager local mode from 
the Shell Scripts

    ## What is the purpose of the change
    The JobManager local mode is problematic:
      - The JobManager does not start all components properly, making some 
operations fail/timeout unexpectedly
      - It introduces code complexity in maintaining the JobManager entry point 
that also spawns an embedded TaskManager
    The JobManager local mode is also unnecessary. The `start-cluster.sh` 
script sets up a proper local cluster without additional configuration.
    This pull request removes the local mode from all UNIX scripts and changes 
the Windows `start-local.bat` to a  `start-cluster.bat` that also starts two 
separate processes for JobManager and TaskManager.
    ## Brief change log
      - Changes the `start-local.bat` to a  `start-cluster.bat` which spawns 
two processes (using `start` command). The two processes show up as separate 
windows and can be stopped separately.
      - Removes the `start-local-sh` and `stop-local.sh` scripts.
      - Removes the passing of the *local* parameter between the shell scripts 
and always passes *cluster* as the execution mode parameter to the JobManager.
    The next step would be to remove the execution mode from the JobManagers 
command line argument parsing and setup logic.
    ## Verifying this change
      - Building Flink, starting a cluster via the shell scripts (Linux, MacOS, 
Windows) and checking the WebUI that the cluster is properly up. No actual job 
execution needed.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (yes / **no)**
      - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (yes / **no)**
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / 
**no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    ## Documentation
      - Does this pull request introduce a new feature? (yes / **no)**
      - If yes, how is the feature documented? (not applicable / **docs** / 
JavaDocs / not documented)
    Docs are updated to not refer to `start-local.xy` any more.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink remove_local_mode

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5528
commit 99d564928f8299d2ad868ed5c23a9c88d8899a47
Author: Stephan Ewen <sewen@...>
Date:   2018-02-16T20:14:41Z

    [FLINK-6489] [shell scripts] Remove local mode from Windows start-local.bat
    Instead, this uses 'start' to start JobManager and TaskManager background 

commit f142deebd951a498f2247b2ce243c15b19a81e84
Author: Stephan Ewen <sewen@...>
Date:   2018-02-16T20:15:49Z

    [FLINK-8696] [shell scripts] Remove JobManager local mode from shell scripts

commit 11e87c4e2e879afb753354660ce3e33fb38a60d8
Author: Stephan Ewen <sewen@...>
Date:   2018-02-19T18:13:12Z

    [FLINK-6489] [FLINK-8696] [docs] Update docs to use 'start-cluster.sh' 
instead of 'start-local.sh'
    (and likewise for 'start-cluster.bat' vs. 'start-local.bat')


> Rework 'start-local.bat' to 'start-local-cluster.bat'
> -----------------------------------------------------
>                 Key: FLINK-6489
>                 URL: https://issues.apache.org/jira/browse/FLINK-6489
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Startup Shell Scripts
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Major
> To get rid of the JobManager local mode, we need to make sure that the 
> {{start-local.bat}} script for windows also starts a separate JobManager and 
> TaskManager process, rather than only the JobManager (and relying on that one 
> to spawn a TaskManager)

This message was sent by Atlassian JIRA

Reply via email to