Repository: oozie Updated Branches: refs/heads/master 7c4c8ecfd -> 10e8ecc20
OOZIE-1865 Oozie servers can't talk to each other with Oozie HA and Kerberos (rkanter) Project: http://git-wip-us.apache.org/repos/asf/oozie/repo Commit: http://git-wip-us.apache.org/repos/asf/oozie/commit/10e8ecc2 Tree: http://git-wip-us.apache.org/repos/asf/oozie/tree/10e8ecc2 Diff: http://git-wip-us.apache.org/repos/asf/oozie/diff/10e8ecc2 Branch: refs/heads/master Commit: 10e8ecc20c98345d9741c26c74f3e668ee08d785 Parents: 7c4c8ec Author: Robert Kanter <[email protected]> Authored: Sat Jul 5 11:40:16 2014 -0700 Committer: Robert Kanter <[email protected]> Committed: Sat Jul 5 11:40:16 2014 -0700 ---------------------------------------------------------------------- docs/src/site/twiki/AG_Install.twiki | 38 +++++++++++++++++++++++++------ release-log.txt | 1 + 2 files changed, 32 insertions(+), 7 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/oozie/blob/10e8ecc2/docs/src/site/twiki/AG_Install.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/AG_Install.twiki b/docs/src/site/twiki/AG_Install.twiki index 0d22fe9..8b0d110 100644 --- a/docs/src/site/twiki/AG_Install.twiki +++ b/docs/src/site/twiki/AG_Install.twiki @@ -798,7 +798,7 @@ their own HA, they should have their own namespace. The default value is shown </property> </verbatim> -5. Change the value of OOZIE_BASE_URL in oozie-env.sh to point to the loadbalancer or virtual IP, for example: +5. Change the value of =OOZIE_BASE_URL= in oozie-env.sh to point to the loadbalancer or virtual IP, for example: <verbatim> export OOZIE_BASE_URL="http://my.loadbalancer.hostname:11000/oozie"; @@ -811,20 +811,22 @@ default is =${OOZIE_HTTP_HOSTNAME}= (i.e. the hostname). export OOZIE_INSTANCE_ID="${OOZIE_HTTP_HOSTNAME}" </verbatim> -7. Start the ZooKeeper servers. +7. (Optional) If using a secure cluster, see [[AG_Install#Security][Security]] below on configuring Kerberos with Oozie HA. -8. Start the Oozie servers. +8. Start the ZooKeeper servers. + +9. Start the Oozie servers. Note: If one of the Oozie servers becomes unavailable, querying Oozie for the logs from a job in the Web UI, REST API, or client may be missing information until that server comes back up. ---++++ Security -Oozie HA works with the existing Oozie security framework and settings. For log streaming to work properly in a secure -setup =oozie.authentication.type= must be set properly on each server (though this is already required if using security in the -first place). +Oozie HA works with the existing Oozie security framework and settings. +See the [[AG_Install#Oozie_User_Authentication_Configuration][Oozie User Authentication Configuration]] section for details. +Below are some additional steps and information specific to Oozie HA: -(Optional) To prevent unauthorized users or programs from interacting with or reading the znodes used by Oozie in ZooKeeper, +1. (Optional) To prevent unauthorized users or programs from interacting with or reading the znodes used by Oozie in ZooKeeper, you can tell Oozie to use Kerberos-backed ACLs. To enforce this for all of the Oozie-related znodes, simply add the following property to oozie-site.xml in all Oozie servers and set it to =true=. The default is =false=. @@ -852,6 +854,28 @@ kerberos.removeHostFromPrincipal=true kerberos.removeRealmFromPrincipal=true </verbatim> +2. Until Hadoop 2.5.0 and later, there is a known limitation where each Oozie server can only use one HTTP principal. However, +for Oozie HA, we need to use two HTTP principals: =HTTP/oozie-server-host@realm= and =HTTP/load-balancer-host@realm=. This +allows access to each Oozie server directly and through the load balancer. While users should always go through the load balancer, +certain features (e.g. log streaming) require the Oozie servers to talk to each other directly; it can also be helpful for an +administrator to talk directly to an Oozie server. So, if using a Hadoop version prior to 2.5.0, you will have to choose which +HTTP principal to use as you cannot use both; it is recommended to choose =HTTP/load-balancer-host@realm= so users can connect +through the load balancer. This will prevent Oozie servers from talking to each other directly, which will effectively disable +log streaming. + +For Hadoop 2.5.0 and later: + +2a. When creating the keytab used by Oozie, make sure to include Oozie's principal and the two HTTP principals mentioned above. + +2b. Set =oozie.authentication.kerberos.principal= to * (that is, an asterisks) so it will use both HTTP principals. + +For earlier versions of Hadoop: + +2a. When creating the keytab used by Oozie, make sure to include Oozie's principal and the load balancer HTTP principal + +2b. Set =oozie.authentication.kerberos.principal= to =HTTP/load-balancer-host@realm=. + + ---++++ JobId sequence Oozie in HA mode, uses ZK to generate job id sequence. Job Ids are of following format. <Id sequence>-<yyMMddHHmmss(server start time)>-<system_id>-<W/C/B> http://git-wip-us.apache.org/repos/asf/oozie/blob/10e8ecc2/release-log.txt ---------------------------------------------------------------------- diff --git a/release-log.txt b/release-log.txt index b700474..eee06a4 100644 --- a/release-log.txt +++ b/release-log.txt @@ -1,5 +1,6 @@ -- Oozie 4.1.0 release (trunk - unreleased) +OOZIE-1865 Oozie servers can't talk to each other with Oozie HA and Kerberos (rkanter) OOZIE-1821 Oozie java action fails due to AlreadyBeingCreatedException (abhishek.agarwal via rkanter) OOZIE-1532 Purging should remove completed children job for long running coordinator jobs (bzhang) OOZIE-1909 log prefix information missing in JavaActionExecutor.check (ryota)
