GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/5528
[FLINK-6489] [FLINK-8696] [shell scripts] Remove JobManager local mode from
the Shell Scripts
## What is the purpose of the change
The JobManager local mode is problematic:
- The JobManager does not start all components properly, making some
operations fail/timeout unexpectedly
- It introduces code complexity in maintaining the JobManager entry point
that also spawns an embedded TaskManager
The JobManager local mode is also unnecessary. The `start-cluster.sh`
script sets up a proper local cluster without additional configuration.
This pull request removes the local mode from all UNIX scripts and changes
the Windows `start-local.bat` to a `start-cluster.bat` that also starts two
separate processes for JobManager and TaskManager.
## Brief change log
- Changes the `start-local.bat` to a `start-cluster.bat` which spawns
two processes (using `start` command). The two processes show up as separate
windows and can be stopped separately.
- Removes the `start-local-sh` and `stop-local.sh` scripts.
- Removes the passing of the *local* parameter between the shell scripts
and always passes *cluster* as the execution mode parameter to the JobManager.
The next step would be to remove the execution mode from the JobManagers
command line argument parsing and setup logic.
## Verifying this change
- Building Flink, starting a cluster via the shell scripts (Linux, MacOS,
Windows) and checking the WebUI that the cluster is properly up. No actual job
execution needed.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no)**
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no)**
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes /
**no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no)**
- If yes, how is the feature documented? (not applicable / **docs** /
JavaDocs / not documented)
Docs are updated to not refer to `start-local.xy` any more.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink remove_local_mode
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5528.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5528
----
commit 99d564928f8299d2ad868ed5c23a9c88d8899a47
Author: Stephan Ewen <sewen@...>
Date: 2018-02-16T20:14:41Z
[FLINK-6489] [shell scripts] Remove local mode from Windows start-local.bat
Instead, this uses 'start' to start JobManager and TaskManager background
processes.
commit f142deebd951a498f2247b2ce243c15b19a81e84
Author: Stephan Ewen <sewen@...>
Date: 2018-02-16T20:15:49Z
[FLINK-8696] [shell scripts] Remove JobManager local mode from shell scripts
commit 11e87c4e2e879afb753354660ce3e33fb38a60d8
Author: Stephan Ewen <sewen@...>
Date: 2018-02-19T18:13:12Z
[FLINK-6489] [FLINK-8696] [docs] Update docs to use 'start-cluster.sh'
instead of 'start-local.sh'
(and likewise for 'start-cluster.bat' vs. 'start-local.bat')
----
---
