GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/4720
[SPARK-5751] [SQL] Revamped HiveThriftServer2Suite for robustness
`HiveThriftServer2Suite` has been notorious for its flakiness for a while.
This was mostly due to spawning and communicate with external server processes.
This PR revamped this test suite for better robustness:
1. Fixes a racing condition occurred while using `tail -f` to check log file
It's possible that the line we are looking for has already been printed
into the log file before we start the `tail -f` process. This PR uses `tail -n
+0 -f` to ensure all lines are checked.
2. Retries up to 3 times if the server fails to start
In most of the cases, the server fails to start because of port
conflict. This PR no longer asks the system to choose an available TCP port,
but uses a random port first, and retries up to 3 times if the server fails to
start.
3. A server instance is reused among all test cases within a single suite
The original `HiveThriftServer2Suite` is splitted into two test suites,
`HiveThriftBinaryServerSuite` and `HiveThriftHttpServerSuite`. Each suite
starts a `HiveThriftServer2` instance and reuses it for all of its test cases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liancheng/spark revamp-thrift-server-tests
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4720.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4720
----
commit 6f14eb1b0725545856d126516d9f97b8264141b6
Author: Cheng Lian <[email protected]>
Date: 2015-02-22T15:52:15Z
Revamped HiveThriftServer2Suite for robustness
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
