GitHub user egorklimov opened a pull request:
https://github.com/apache/zeppelin/pull/3229
[ZEPPELIN-3869] Close interpreters in multithreaded mode
### What is this PR for?
This PR provides multithreading realization of interpreters closing.
Benchmark based on [JMH](http://openjdk.java.net/projects/code-tools/jmh/)
- `ZeppelinServerBenchmarkTest`:
* Creates `InterpreterSettingManager` and `InterpreterFactory`;
* Runs 6 interpreters (`python`, `sh`, `md`, `groovy`, `angular`, `spark`)
for each user in isolated mode;
* Measures InterpreterSettingManager closing.
Result with usersCnt - 7, 5, 3, 1 (42, 30, 18, 6 interpreter processes):
```
Benchmark (usersCnt) Mode
Cnt Score Error Units
ZeppelinServerBenchmarkTest.testDefaultServerShutdown 7 avgt
10 16,690 ± 0,191 s/op
ZeppelinServerBenchmarkTest.testDefaultServerShutdown 5 avgt
10 11,532 ± 0,064 s/op
ZeppelinServerBenchmarkTest.testDefaultServerShutdown 3 avgt
10 6,942 ± 0,020 s/op
ZeppelinServerBenchmarkTest.testDefaultServerShutdown 1 avgt
10 2,348 ± 0,018 s/op
ZeppelinServerBenchmarkTest.testServerShutdownInParallel 7 avgt
10 3,612 ± 0,317 s/op
ZeppelinServerBenchmarkTest.testServerShutdownInParallel 5 avgt
10 2,635 ± 0,075 s/op
ZeppelinServerBenchmarkTest.testServerShutdownInParallel 3 avgt
10 2,473 ± 0,072 s/op
ZeppelinServerBenchmarkTest.testServerShutdownInParallel 1 avgt
10 2,378 ± 0,029 s/op
```
[benchmark.zip](https://github.com/TinkoffCreditSystems/zeppelin/files/2591694/benchmark.zip)
contains:
* `Benchmark.patch` - patch with `ZeppelinServerBenchmarkTest`;
* `benchmark.csv` - benchmark result;
* `benchmark-result.json` - results (you could visualize it using [JMH
Visualizer](http://jmh.morethan.io/))
### How to run benchmark
1. Apply patch from `benchmark.zip`
2. Add [IntelliJ IDEA JMH
Plugin](https://plugins.jetbrains.com/plugin/7529-jmh-plugin)
3. Build Zeppelin
4. Run ZeppelinServerBenchmarkTest in IntelliJ IDEA
### What type of PR is it?
Refactoring
### What is the Jira issue?
[[ZEPPELIN-3869]](https://issues.apache.org/jira/browse/ZEPPELIN-3869)
### How should this be tested?
* CI pass
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/TinkoffCreditSystems/zeppelin ZP-71
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/3229.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3229
----
commit becfdf15f6cc80669fe70ae4443a21c807946827
Author: George Klimov <klim.electronicmail@...>
Date: 2018-10-28T13:55:36Z
Parallel close
commit efacb77176f8a2f33d55120ebe8194844f4a7e3c
Author: George Klimov <klim.electronicmail@...>
Date: 2018-11-08T09:21:36Z
Fix code style issues
commit 380f539f14c3d76870cac06ef3c95bd1a11798ed
Author: George Klimov <klim.electronicmail@...>
Date: 2018-11-08T14:33:47Z
Refactor code
Remove sync
Bring back sync on close
commit 4bab30f76f302b30227d62377cd735bf1c723896
Author: George Klimov <klim.electronicmail@...>
Date: 2018-11-14T09:52:03Z
Add exceptions
commit e7abdd305df86595c2c11f4743e7d9cbb0558206
Author: George Klimov <klim.electronicmail@...>
Date: 2018-11-14T10:39:56Z
Fix issues
commit 1cad17a0e2d029ecf08f9d64add8fc2fc7b077d0
Author: George Klimov <klim.electronicmail@...>
Date: 2018-11-14T11:13:22Z
Remove watchdog check
----
---
