Repository: incubator-hawq-docs Updated Branches: refs/heads/master f6d40aaca -> f7d9536ae
HAWQ-1479 - doc ranger kerberos integration (closes #128) Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/f7d9536a Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/f7d9536a Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/f7d9536a Branch: refs/heads/master Commit: f7d9536ae242dd2b08a59fc7370ca6df09d3fd23 Parents: f6d40aa Author: Lisa Owen <[email protected]> Authored: Mon Jul 17 11:34:07 2017 -0700 Committer: David Yozie <[email protected]> Committed: Mon Jul 17 11:34:07 2017 -0700 ---------------------------------------------------------------------- .../source/subnavs/apache-hawq-nav.erb | 3 + .../ranger-integration-config.html.md.erb | 85 +++++--- markdown/ranger/ranger-kerberos.html.md.erb | 207 +++++++++++++++++++ markdown/ranger/ranger-overview.html.md.erb | 2 +- 4 files changed, 266 insertions(+), 31 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/f7d9536a/book/master_middleman/source/subnavs/apache-hawq-nav.erb ---------------------------------------------------------------------- diff --git a/book/master_middleman/source/subnavs/apache-hawq-nav.erb b/book/master_middleman/source/subnavs/apache-hawq-nav.erb index 85eb424..f473dab 100644 --- a/book/master_middleman/source/subnavs/apache-hawq-nav.erb +++ b/book/master_middleman/source/subnavs/apache-hawq-nav.erb @@ -198,6 +198,9 @@ <li> <a href="/docs/userguide/2.2.0.0-incubating/ranger/ranger-ha.html">High Availability and HAWQ Ranger</a> </li> + <li> + <a href="/docs/userguide/2.2.0.0-incubating/ranger/ranger-kerberos.html">HAWQ Ranger Kerberos Integration</a> + </li> </ul> </li> <li> http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/f7d9536a/markdown/ranger/ranger-integration-config.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/ranger/ranger-integration-config.html.md.erb b/markdown/ranger/ranger-integration-config.html.md.erb index 7c91d6e..b535042 100644 --- a/markdown/ranger/ranger-integration-config.html.md.erb +++ b/markdown/ranger/ranger-integration-config.html.md.erb @@ -62,44 +62,65 @@ To use HAWQ Ranger integration, install a compatible Hadoop distribution and Apa ``` bash root@ranger-admin-node$ chown -R ranger:ranger hawq ``` -5. The `enable-ranger-plugin.sh` script configures Ranger connectivity to your HAWQ cluster. The command has the syntax: +5. Configure Ranger connectivity to your HAWQ cluster with the `enable-ranger-plugin.sh` script. The command has the syntax: ``` pre - enable-ranger-plugin.sh -r <ranger_admin_node>:<ranger_port> -u <ranger_user> -p <ranger_password> -h <hawq_master>:<hawq_port> -w <hawq_user> -q <hawq_password> - ``` - - Log in to the HAWQ master node as the `gpadmin` user and execute the `enable-ranger-plugin.sh` script. Ensure that \<hawq_master\> identifies the fully qualified domain name of the HAWQ master node. For example: - - ``` bash - gpadmin@master$ cd /usr/local/hawq/ranger/bin - gpadmin@master$ ./enable-ranger-plugin.sh -r ranger_host:6080 -u admin -p admin -h hawq_master:5432 -w gpadmin -q gpadmin - - RANGER URL = ranger_host:6080 - RANGER User = admin - RANGER Password = [*****] - HAWQ HOST = hawq_master - HAWQ PORT = 5432 - HAWQ User = gpadmin - HAWQ Password = [*******] - HAWQ service definition was not found in Ranger Admin, creating it by uploading /usr/local/hawq_<version>/ranger/etc/ranger-servicedef-hawq.json - HAWQ service instance was not found in Ranger Admin, creating it. - Updated POLICY_MGR_URL to http://ranger_host:6080 in /usr/local/hawq_<version>/ranger/etc/rps.properties - Updated default value of JAVA_HOME to /usr/jdk64/jdk1.8.0_77 in /usr/local/hawq_<version>/ranger/etc/rps.properties - ``` - - **Note**: You can also enter the short form of the command: `./enable-ranger-plugin.sh -r` and the script will prompt you for entries. - + enable-ranger-plugin.sh -r <ranger_admin_node>:<ranger_port> -u <ranger_user> -p <ranger_password> [-h <hawq_master>:<hawq_port> -t <lookup_authentication_type> -s <hawq_kerberos_service_name>] -w <hawq_user> -q <hawq_password> + ``` + + **Note**: You can also enter the short form of the command: `./enable-ranger-plugin.sh -r` and the script will prompt you for entries. + + 1. Log in to the HAWQ master node as the `gpadmin` user, set up your HAWQ environment, and navigate to the HAWQ Ranger Plug-in `bin` directory: + + ``` bash + $ ssh gpadmin@master + gpadmin@master$ . /usr/local/greenplum_path.sh + gpadmin@master$ cd /usr/local/hawq/ranger/bin + ``` + + 2. Execute the `enable-ranger-plugin.sh` script. You will provide different options for the script depending upon whether or not you have enabled Kerberos in your HAWQ cluster. + + ***If Kerberos is not enabled for Ranger***: + + ``` bash + gpadmin@master$ ./enable-ranger-plugin.sh -r ranger_host:6080 -u admin -p admin -h hawq_master:5432 -w gpadmin -q gpadmin + + RANGER URL = ranger_host:6080 + RANGER User = admin + RANGER Password = [*****] + HAWQ HOST = hawq_master + HAWQ PORT = 5432 + HAWQ User = gpadmin + HAWQ Password = [*******] + HAWQ Lookup Authentication Type = simple + HAWQ Kerberos Service Name = postgres + HAWQ service definition was not found in Ranger Admin, creating it by uploading /usr/local/hawq_<version>/ranger/etc/ranger-servicedef-hawq.json + HAWQ service instance was not found in Ranger Admin, creating it. + Updated POLICY_MGR_URL to http://ranger_host:6080 in /usr/local/hawq_<version>/ranger/etc/rps.properties + Updated default value of JAVA_HOME to /usr/jdk64/jdk1.8.0_77 in /usr/local/hawq_<version>/ranger/etc/rps.properties + ``` + + ***If Kerberos is enabled for Ranger***: + + Provide the `-s` option, specifying the \<hawq\_kerberos\_service\_name\>; default is `postgres`. Also provide the `-t` option, specifying a \<lookup\_authentication\_type\> of `kerberos`. For example: + + ``` bash + gpadmin@master$ ./enable-ranger-plugin.sh -r ranger_host:6080 -u admin -p admin -h hawq_master:5432 -s postgres -t kerberos -w gpadmin -q gpadmin + ``` + + See [Configure HAWQ for Kerberized Ranger](ranger-kerberos.html#rps2ra_kerb_cfg) for additional configuration required when Kerberos is enabled for Ranger. + When the script completes, the default HAWQ service definition is registered in the Ranger Admin UI. This service definition is named `hawq`. 6. Locate the `pg_hba.conf` file in the master directory of the HAWQ master node. To display the HAWQ master directory: - + ``` bash gpadmin@master$ hawq config --show hawq_master_directory GUC : hawq_master_directory Value : /data/hawq/master ``` - Edit the `pg_hba.conf` file on the HAWQ master node to configure HAWQ access for \<hawq_user\> on the \<ranger-admin-node\>. For example, you would add an entry similar to the following for the example `enable-ranger-plugin.sh` call above: +7. Edit the `pg_hba.conf` file to configure HAWQ access for \<hawq_user\> on the \<ranger-admin-node\>. For example, you would add an entry similar to the following for the example `enable-ranger-plugin.sh` call above: ``` bash host all gpadmin ranger_host/32 trust @@ -109,11 +130,15 @@ To use HAWQ Ranger integration, install a compatible Hadoop distribution and Apa ``` bash gpadmin@master$ hawq stop cluster --reload - ``` + ``` + + **Note:** If you have enabled Kerberos user authentication for HAWQ, you may not need to create a `pg_hba.conf` entry, or your entry will differ. See [Configure Ranger for Kerberized HAWQ](ranger-kerberos.html#ra2hawq_kerb_cfg) for additional configuration information. -7. After HAWQ reloads the configuration, use the fully-qualified domain name to log into the Ambari server. Click the **Ranger** link to display the Ranger Summary page, then select **Quick Links > Ranger Admin UI**. +9. Log in to the Ambari UI. + +10. Click the **Ranger** service in the left pane to display the Ranger Summary page, then select **Quick Links > Ranger Admin UI**. -8. Log in to the Ranger Access Manager. Click the **Edit** button for the **HAWQ** service. Ensure that the Active Status is set to Enabled, and click **Test Connection**. You should receive a message that Ranger connected successfully. If the connection fails, verify the `hawq` service Config Properties, as well as your `pg_hba.conf` entries, and re-test the connection. +11. Log in to the Ranger Access Manager. Navigate to the **HAWQ** service and click the **Edit** button for the `hawq` service definition. Ensure that the Active Status is set to Enabled, and click **Test Connection**. You should receive a message that Ranger connected successfully. If the connection fails, verify the `hawq` service Config Properties, as well as your `pg_hba.conf` entries, and re-test the connection. ## <a id="enable"></a>Step 2: Configure HAWQ to Use Ranger Policy Management http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/f7d9536a/markdown/ranger/ranger-kerberos.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/ranger/ranger-kerberos.html.md.erb b/markdown/ranger/ranger-kerberos.html.md.erb new file mode 100644 index 0000000..c0b4f40 --- /dev/null +++ b/markdown/ranger/ranger-kerberos.html.md.erb @@ -0,0 +1,207 @@ +--- +title: HAWQ Ranger Kerberos Integration +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +When you have enabled Ranger Authorization for HAWQ, your HAWQ installation includes the Ranger Administrative UI and HAWQ Ranger Plug-in Service. + +Specific HAWQ Ranger configuration is required when Kerberos authentication is enabled for HAWQ or for Ranger. You must configure Kerberos support for: + +- HAWQ resource lookup by the Ranger Administration host during HAWQ policy definition +- HAWQ Ranger Plug-in Service communication with the Ranger Administration host for policy refresh + +Use the following procedures to configure Kerberos support for your Ranger-authorized HAWQ cluster. + +## Prerequisites <a id="kerb_ranger_prereq"></a> + +Before you configure Kerberos for your Ranger-authorized HAWQ cluster, ensure that you have: + +- Installed Java 1.7.0\_17 or later on all nodes in your cluster. Java 1.7.0_17 is required to use Kerberos-authenticated JDBC on Red Hat Enterprise Linux 6.x or 7.x. +- (Non-OpenJDK Java installations) Installed the Java Cryptography Extension (JCE) on all nodes in your cluster. + - If you manage your cluster with Ambari, you installed the JCE on each node before you enabled Kerberos with the Ambari **Kerberos Security Wizard**. + - If you manage your cluster from the command line, you must manually install the extension on these systems. +- Noted the host name or IP address of your Ranger Administration host (\<ranger-admin-node\>) and HAWQ master (\<master\>) nodes. +- Identified an existing Kerberos Key Distribution Center (KDC) or set up your KDC as described in [Install and Configure a Kerberos KDC Server](../clientaccess/kerberos.html#task_setup_kdc). + - Note the host name or IP address of your KDC (\<kdc-server\>). + - Note the name of the Kerberos \<realm\> in which your cluster resides. +- Enabled Ranger Authorization for HAWQ. See [Configuring HAWQ to use Ranger Policy Management](ranger-integration-config.html). + + +## Configure Ranger for Kerberized HAWQ<a id="ra2hawq_kerb_cfg"></a> + +When you define HAWQ Ranger authorization policies, the Ranger Administration Host uses JDBC to connect to HAWQ during policy definition to look up policy resource names. When Kerberos user authentication is enabled for HAWQ, you must configure this connection for Kerberos. + +To configure Ranger access to a HAWQ cluster enabled with Kerberos user authentication, you must: + +- Identify an existing HAWQ administrative role or create a new HAWQ administrative role for Ranger lookup of HAWQ resources +- Create a Kerberos principal for the lookup role +- Update the Ranger HAWQ service definition + +### Procedure <a id="kerb_ra2hawq_proc"></a> + +Perform the following procedure to enable the Ranger Administration Host to look up resources in your kerberized HAWQ cluster. You will perform operations on the HAWQ \<master\>, \<ranger-admin-node\>, and \<kdc-server\> nodes. + +1. Log in to the HAWQ master node and set up your environment: + + ``` shell + $ ssh gpadmin@<master> + gpadmin@master$ . /usr/local/hawq/greenplum_path.sh + ``` + +2. Identify an existing HAWQ administrative role or create a new HAWQ administrative role for Ranger resource lookup. For example, to create a new administrative role: + + ``` shell + gpadmin@master$ psql -c 'CREATE ROLE "rangerlookup_hawq" with LOGIN SUPERUSER;' + ``` + + You may choose a different name for the Ranger lookup role. + +3. Log in to the KDC server system and generate a principal for the HAWQ `rangerlookup_hawq` role. Substitute your Kerberos \<realm\>. For example: + + ``` shell + $ ssh root@<kdc-server> + root@kdc-server$ kadmin.local -q "addprinc -pw changeme [email protected]" + ``` + + You do not need to generate a keytab file for the `rangerlookup_hawq` principal because you will provide the password in the HAWQ service definition of the Ranger Admin UI. + +4. Validate the `rangerlookup_hawq` role and principal by generating a ticket and attempting to connect to HAWQ. For example: + + ``` shell + $ ssh gpadmin@<hawq-node> + gpadmin@hawq-node$ kinit rangerlookup_hawq + Password for [email protected] + gpadmin@hawq-node$ psql -d postgres -p 5432 -U rangerlookup_hawq -h master + ``` + +5. Start the Ranger Admin UI in a supported web browser. The default URL is \<ranger-admin-node\>:6080. + +6. Locate the HAWQ service definition and click the **Edit** button. + +7. Update the applicable **Config Properties** fields: + + **HAWQ User Name*** : Enter the HAWQ Ranger lookup role you identified or created in Step 2 above. + **HAWQ User Password*** : Enter the password you assigned to the HAWQ Ranger lookup role principal. + **HAWQ Kerberos Service Name** : Enter `postgres` or your custom HAWQ service name. + **HAWQ Authentication Type** : Select **Kerberos** from the drop-down list. + +8. Click the **Test Connection** button. If you connect successfully, Ranger displays a dialog with the message: + + ``` pre + Connected Successfully. + ``` + + You successfully configured Ranger to use Kerberos authentication when looking up HAWQ resource names. + +9. **Save** your changes to permanently configure Kerberos authentication for HAWQ Ranger resource lookup. + + +## Configure HAWQ for Kerberized Ranger<a id="rps2ra_kerb_cfg"></a> + +When Ranger Authorization is enabled for HAWQ, the Ranger Plug-in Service fetches policies from the Ranger Administration host. If Kerberos is enabled for Ranger, you must specifically configure the Ranger Plug-in Service for this communication. + +This procedure includes: + +- Adding an administrative Ranger user named `postgres` +- Updating the HAWQ Ranger Plug-in Service configuration to utilize Kerberos + + +### Procedure <a id="kerb_rps2ra_proc"></a> + +Perform the following steps to enable Ranger Plug-in Service policy refresh from a kerberized Ranger: + +1. Create a `postgres` Ranger administrative user. + + 1. Open the Ranger Admin UI. The default URL is \<ranger-admin-node\>:6080. + 2. Navigate to **Settings > Users/Groups**. + 3. Click the **Add New User** button. + 4. Fill out the required fields in the **User Detail** dialog: + + **User Name*** : Enter `postgres`. + **New Password*** : Enter the user's password. + **Password Confirm*** : Enter the password again. + **First Name*** : Enter the first name of the user. + **Select Role*** : Select the **Admin** role. + +2. Update the HAWQ Ranger Plug-in Service configuration to use Kerberos authentication when communicating with Ranger. + + 2. Edit the HAWQ Ranger Plug-in Service `rps.properties` configuration file and set the `RPS_AUTH` property to specify Kerberos authentication. + + ``` shell + gpadmin@master$ vi $GPHOME/ranger/etc/rps.properties + ``` + + ``` pre + RPS_AUTH=kerberos + ``` + + 3. `rps.properties` also includes `RPS_PRINCIPAL` and `RPS_KEYTAB` properties. If these properties are unset, the Ranger Plug-in Service uses the cached Kerberos credentials that are available for the HAWQ runtime user account (typically `gpadmin`). Set these properties if the default values are not appropriate for your environment. + +3. Stop the Ranger Plug-in Service: + + ``` shell + gpadmin@master$ $GPHOME/ranger/bin/rps.sh stop + ``` + +4. Generate a ticket for the `postgres@<realm>` principal: + + ``` shell + gpadmin@master$ kinit -kt /etc/security/keytabs/hawq.service.keytab [email protected] + ``` + +5. Restart the HAWQ Ranger Plug-in Service: + + ``` shell + gpadmin@master$ $GPHOME/ranger/bin/rps.sh start + ``` + +6. If you have enabled HAWQ Ranger High Availability, synchronize the HAWQ Ranger Plug-in Service configuration files from the HAWQ master node to the standby master node as described in the [Register a Standby Ranger Plug-in Service](ranger-integration-config.html#rps_ha_cfg_standbyrps) procedure, including restarting the standby master node. + +7. Verify the configuration. + + 1. Examine the HAWQ Ranger Plug-in Service log file: + + ```shell + gpadmin@master$ cd $GPHOME/ranger/plugin-service/logs + gpadmin@master$ vi rps.log + ``` + + Locate the most recent entry that identifies the authentication type used: + + ``` pre + 2017-06-26 18:21:45,518 INFO [org.apache.ranger.audit.queue.AuditBatchQueue0] org.apache.ranger.audit.queue.AuditBatchQueue: Running queue rps.async.batch as user [email protected] (auth:KERBEROS) + ``` + + This entry should specify `auth:KERBEROS`. + + 2. Create a new HAWQ policy in the Ranger Admin UI and **Save**. + + 3. Examine the HAWQ Ranger Plug-in Service log file again. Locate the most recent entry that identifies a new policy version: + + ``` pre + 2017-06-26 18:52:02,183 INFO [Thread-5] org.apache.ranger.plugin.util.PolicyRefresher: PolicyRefresher(serviceName=hawq): found updated version. lastKnownVersion=11; newVersion=12 + ``` + + If you configured Kerberos correctly, you will see a `newVersion=<n>` entry in this log. If you did *not* configure this communication correctly, you will encounter error messages similar to the following: + + ``` pre + 2017-06-26 18:24:17,050 ERROR [Thread-6] org.apache.ranger.plugin.util.PolicyRefresher: PolicyRefresher(serviceName=hawq): failed to refresh policies. Will continue to use last known version of policies (10) + ``` \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/f7d9536a/markdown/ranger/ranger-overview.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/ranger/ranger-overview.html.md.erb b/markdown/ranger/ranger-overview.html.md.erb index 87563d7..604f343 100644 --- a/markdown/ranger/ranger-overview.html.md.erb +++ b/markdown/ranger/ranger-overview.html.md.erb @@ -41,7 +41,7 @@ The Ranger plug-in service caches Ranger policies locally on each HAWQ node to a In this release, HAWQ integration with Ranger has several limitations: -- Neither Kerberos authentication nor SSL encryption is supported between a HAWQ node and the Ranger plug-in service, or between the plug-in service and the Ranger Policy Manager. +- SSL encryption is not supported between a HAWQ node and the Ranger plug-in service, or between the plug-in service and the Ranger Policy Manager. - Some authorization checks for superuser-restricted authorization events are handled by HAWQ natively, even when Ranger integration is enabled. See [HAWQ-Native Authorization](../clientaccess/hawq-access-checks.html#alwaysnative).
