GitHub user Leemoonsoo opened a pull request:
https://github.com/apache/zeppelin/pull/2017
[ZEPPELIN-1746] Flaky test: PySparkInterpreterTest
### What is this PR for?
PySparkInterpreter (and PySparkInterpreterMatplotlibTest,
SparkInterpreterTest, SparkSqlInterpreterTest) test does not terminate
SparkInterpreter after test end.
In the beginning, keeping SparkInterpreter alive was designed to prevent
create/destroy SparkInterpreter multiple times in various tests, because it
takes some time.
However, that is somehow misbehaving and getting problems like
1. starting another SparkInterpreter before one SparkInterpreter terminates
2. Does not finish PySparkInterpreter process explicitly cause py4j client
somehow creates a lot of connection attempt, that consumes all capacity of
systems port open, and make next SparkInterpreter start fail.
This PR terminates SparkInterpreter after test class finishes, to prevent
above behavior.
### What type of PR is it?
Bug Fix
### Todos
* [x] - Terminate SparkInterpreter after each test class
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1746
### How should this be tested?
CI becomes green
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Leemoonsoo/zeppelin ZEPPELIN-1746
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2017.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2017
----
commit e6ba4e1414b6cedaddbcfbd251e721a9d8e91cc4
Author: Lee moon soo <[email protected]>
Date: 2017-02-13T17:48:59Z
Try start and terminate spark context after each test class
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
