GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/19343
[SPARK-22121][CORE] Correct database location for namenode HA.
## What changes were proposed in this pull request?
If hdfs HA is turned on after a hive database is already created, the db
location may still reference just one namenode, instead of the
nameservice, if users do not properly follow all upgrade instructions.
After this change, spark detects the misconfiguration and tries to
auto-adjust for it, since this is the behavior from hive as well.
## How was this patch tested?
Added unit tests. Also deployed on a cluster with hdfs ha, with the
database location set to only one instance, and then I failed over the namenode
so the other instance was the active one. After this change, things worked
without a problem.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark SPARK-22121
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19343.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19343
----
commit c2e125eacb48971ee72dd61859a95ca8ae6a9fc8
Author: Imran Rashid <[email protected]>
Date: 2017-09-26T00:55:58Z
[SPARK-22121][CORE] Correct database location for namenode HA.
If hdfs HA is turned on after a hive database is already created, the db
location may still reference just one namenode, instead of the
nameservice, if users do not properly follow all upgrade instructions.
After this change, spark detects the misconfiguration and tries to
auto-adjust for it, since this is the behavior from hive as well.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
