chihsuan commented on code in PR #490: URL: https://github.com/apache/ozone-site/pull/490#discussion_r3594664520
########## static/compose/knox-ozone/Dockerfile: ########## @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Optional image that bakes the Ozone topology into the official Knox gateway +# image. The default docker-compose.yaml mounts the topology instead. +FROM apache/knox:latest Review Comment: nit: should we consider pinning to a verified version (e.g. `apache/knox:2.1.0`) instead of `latest`, so the documented steps don't silently break on a future Knox major release? Same for the `KNOX_IMAGE_VERSION:-latest` default in `docker-compose.yaml`. ########## docs/05-administrator-guide/02-configuration/03-security/04-knox.md: ########## @@ -1,9 +1,251 @@ --- -draft: true sidebar_label: Apache Knox --- # Configuring Apache Knox -**TODO:** File a subtask under [HDDS-9859](https://issues.apache.org/jira/browse/HDDS-9859) and complete this page or section. -**TODO:** Uncomment link to this page in src/pages/index.js +[Apache Knox](https://knox.apache.org/) is a reverse proxy gateway for Hadoop ecosystem REST APIs and web UIs. Knox 2.0.0 and later can proxy Ozone services, giving users a single HTTPS entry point while hiding internal cluster hostnames and ports. + +Ozone integration was added in [KNOX-2833](https://issues.apache.org/jira/browse/KNOX-2833) (OM, SCM, and Recon UIs) and extended in [KNOX-2914](https://issues.apache.org/jira/browse/KNOX-2914) (HttpFS). Knox does **not** proxy the S3 Gateway today; use a load balancer or direct S3G access instead. + +## Supported Ozone services + +| Knox service role | Ozone component | Default HTTP port | Knox path prefix | +|---|---|---|---| +| `OZONE` | Ozone Manager UI | 9874 | `/ozone/` | +| `OZONE-SCM` | Storage Container Manager UI | 9876 | `/ozone-scm/` | +| `OZONE-RECON` | Recon UI | 9888 | `/ozone-recon/` | +| `OZONE-HTTPFS` | HttpFS Gateway | 14000 | `/httpfs/` | + +## Prerequisites + +- Apache Knox **2.0.0 or later** installed and running. +- Ozone HTTP endpoints reachable from the Knox host (`ozone.om.http-address`, `ozone.scm.http-address`, `ozone.recon.http-address`, and HttpFS on port 14000 by default). +- For production: configure TLS on Knox and Kerberos SPNEGO on Ozone HTTP services. See [HTTPS / SPNEGO](./https) and [Kerberos](./kerberos). + +## Topology configuration + +Knox routes traffic based on **topology** XML files in `{GATEWAY_HOME}/conf/topologies/`. Create a topology (for example `ozone.xml`) that declares authentication providers and Ozone backend URLs. + +### Example topology + +The following example uses Knox demo LDAP for authentication. Replace hostnames and ports with your cluster values. + +```xml +<?xml version="1.0" encoding="utf-8"?> +<topology> + <gateway> + <provider> + <role>authentication</role> + <name>ShiroProvider</name> + <enabled>true</enabled> + <param> + <name>sessionTimeout</name> + <value>30</value> + </param> + <param> + <name>main.ldapRealm</name> + <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value> + </param> + <param> + <name>main.ldapContextFactory</name> + <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value> + </param> + <param> + <name>main.ldapRealm.contextFactory</name> + <value>$ldapContextFactory</value> + </param> + <param> + <name>main.ldapRealm.userDnTemplate</name> + <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value> + </param> + <param> + <name>main.ldapRealm.contextFactory.url</name> + <value>ldap://knox-ldap.example.com:33389</value> + </param> + <param> + <name>main.ldapRealm.contextFactory.authenticationMechanism</name> + <value>simple</value> + </param> + <param> + <name>urls./**</name> + <value>authcBasic</value> + </param> + </provider> + + <provider> + <role>identity-assertion</role> + <name>Default</name> + <enabled>true</enabled> + </provider> + + <provider> + <role>hostmap</role> + <name>static</name> + <enabled>true</enabled> + <param> + <name>om1.example.com</name> + <value>om1.internal.example.com</value> + </param> + </provider> + </gateway> + + <service> + <role>OZONE</role> + <url>http://om1.internal.example.com:9874</url> + </service> + + <service> + <role>OZONE-SCM</role> + <url>http://scm1.internal.example.com:9876</url> + </service> + + <service> + <role>OZONE-RECON</role> + <url>http://recon.internal.example.com:9888</url> + </service> + + <service> + <role>OZONE-HTTPFS</role> + <url>http://httpfs.internal.example.com:14000/webhdfs/</url> + </service> +</topology> +``` + +Place the file in `{GATEWAY_HOME}/conf/topologies/ozone.xml`. Knox automatically redeploys the topology when the file changes. + +### High availability + +When multiple Ozone Managers or SCMs run in HA, add a `<service>` entry for each instance. Knox uses a `?host={host}` query parameter on OM and SCM UI links so the gateway routes to the correct backend ([KNOX-2930](https://github.com/apache/knox/pull/767)). Configure the `hostmap` provider to rewrite internal hostnames embedded in UI responses. + +### Secure clusters + +For Kerberized Ozone clusters: + +1. Enable SPNEGO on Ozone HTTP services as described in [HTTPS / SPNEGO](./https). +2. Configure Knox authentication with your enterprise identity provider (LDAP, SAML, or Kerberos) instead of demo LDAP. +3. Terminate TLS at Knox and use HTTPS between Knox and Ozone backends where possible. + +## Ozone-side configuration + +### HttpFS proxy user + +When Knox proxies HttpFS, configure a proxy user on the HttpFS service so Knox can impersonate end users. Add the following to `httpfs-site.xml` (or the equivalent `CORE-SITE.XML_` environment variables in Docker): + +```xml +<property> + <name>httpfs.proxyuser.knoxuser.hosts</name> + <value>*</value> +</property> +<property> + <name>httpfs.proxyuser.knoxuser.groups</name> + <value>*</value> +</property> +``` + +Replace `knoxuser` with the Unix or service account Knox uses to connect to HttpFS. Restrict `hosts` and `groups` in production. See [HttpFS Gateway](../../../user-guide/client-interfaces/httpfs#proxy-user-configuration) for details. + +### HTTP addresses + +Ensure each proxied service publishes the HTTP address Knox will use: + +| Property | Example | +|---|---| +| `ozone.om.http-address` | `om1.example.com:9874` | +| `ozone.scm.http-address` | `scm1.example.com:9876` | +| `ozone.recon.http-address` | `recon.example.com:9888` | + +## Accessing Ozone through Knox + +After Knox and the topology are deployed, users reach Ozone through the Knox gateway URL. + +### Knox homepage + +Open the Knox homepage to see Ozone service cards: + +```text +https://<knox-host>:8443/gateway/homepage/home +``` + +Authenticate with the credentials configured in the topology (for example, demo LDAP user `guest` with password `guest-password`). + +### Web UIs + +Direct links follow this pattern. For OM and SCM, the `host` query parameter must be the full backend URL (not just the hostname): + +```text +https://<knox-host>:8443/gateway/ozone/ozone/index.html?host=http://<om-host>:9874 +https://<knox-host>:8443/gateway/ozone/ozone-scm/index.html?host=http://<scm-host>:9876 +https://<knox-host>:8443/gateway/ozone/ozone-recon/index.html +``` + +Replace `ozone` in the path with your topology name if different. In HA deployments, use the URL of the specific OM or SCM instance you want to reach. + +### HttpFS REST API + +Access HttpFS through Knox using the `/httpfs/` path prefix: + +```bash +curl -ik -u guest:guest-password \ + "https://localhost:8443/gateway/ozone/httpfs/v1/?op=LISTSTATUS&user.name=hadoop"; Review Comment: nit: This section (general usage, above the Docker quickstart) uses `https://localhost:8443` while the surrounding examples use `<knox-host>`. Would it make sense to use `<knox-host>` here too? ########## docs/05-administrator-guide/02-configuration/03-security/04-knox.md: ########## @@ -1,9 +1,251 @@ --- -draft: true sidebar_label: Apache Knox --- # Configuring Apache Knox -**TODO:** File a subtask under [HDDS-9859](https://issues.apache.org/jira/browse/HDDS-9859) and complete this page or section. -**TODO:** Uncomment link to this page in src/pages/index.js +[Apache Knox](https://knox.apache.org/) is a reverse proxy gateway for Hadoop ecosystem REST APIs and web UIs. Knox 2.0.0 and later can proxy Ozone services, giving users a single HTTPS entry point while hiding internal cluster hostnames and ports. Review Comment: `OZONE-HTTPFS` is not available in `Knox 2.0.0`. It was added by [KNOX-2914](https://github.com/apache/knox/pull/765) (https://github.com/apache/knox/pull/765) after the 2.0.0 release, and the service definition first appears in the v2.1.0[ tag](https://github.com/apache/knox/tree/v2.1.0/gateway-service-definitions/src/main/resources/services/ozone-httpfs). Could we require Knox 2.1.0 here, or clarify that 2.0.0 supports the Ozone UIs while HttpFS proxying requires 2.1.0? ########## static/compose/knox-ozone/docker-compose.yaml: ########## @@ -0,0 +1,119 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +x-image: + &image + image: ${OZONE_IMAGE:-apache/ozone}:${OZONE_IMAGE_VERSION:-2.1.1}${OZONE_IMAGE_FLAVOR:-} + +x-common-config: + &common-config + OZONE-SITE.XML_hdds.datanode.dir: /data/hdds + OZONE-SITE.XML_ozone.metadata.dirs: /data/metadata + OZONE-SITE.XML_ozone.om.address: om + OZONE-SITE.XML_ozone.om.http-address: om:9874 + OZONE-SITE.XML_ozone.recon.address: recon:9891 + OZONE-SITE.XML_ozone.recon.db.dir: /data/metadata/recon + OZONE-SITE.XML_ozone.recon.http-address: recon:9888 + OZONE-SITE.XML_ozone.replication: "1" + OZONE-SITE.XML_ozone.scm.block.client.address: scm + OZONE-SITE.XML_ozone.scm.client.address: scm + OZONE-SITE.XML_ozone.scm.datanode.id.dir: /data/metadata + OZONE-SITE.XML_ozone.scm.http-address: scm:9876 + OZONE-SITE.XML_ozone.scm.names: scm + OZONE-SITE.XML_hdds.scm.safemode.min.datanode: "1" + OZONE-SITE.XML_hdds.scm.safemode.healthy.pipeline.pct: "0" + no_proxy: om,recon,scm,s3g,httpfs,knox-ldap,knox-gateway,localhost,127.0.0.1 + +version: "3" Review Comment: nit: Docker Compose v2 treats the top-level version field as obsolete and ignores it with a warning. We can probably remove this. https://docs.docker.com/reference/compose-file/version-and-name/ ########## static/compose/knox-ozone/docker-compose.yaml: ########## @@ -0,0 +1,119 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to you under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +x-image: + &image + image: ${OZONE_IMAGE:-apache/ozone}:${OZONE_IMAGE_VERSION:-2.1.1}${OZONE_IMAGE_FLAVOR:-} + +x-common-config: + &common-config + OZONE-SITE.XML_hdds.datanode.dir: /data/hdds + OZONE-SITE.XML_ozone.metadata.dirs: /data/metadata + OZONE-SITE.XML_ozone.om.address: om + OZONE-SITE.XML_ozone.om.http-address: om:9874 + OZONE-SITE.XML_ozone.recon.address: recon:9891 + OZONE-SITE.XML_ozone.recon.db.dir: /data/metadata/recon + OZONE-SITE.XML_ozone.recon.http-address: recon:9888 + OZONE-SITE.XML_ozone.replication: "1" + OZONE-SITE.XML_ozone.scm.block.client.address: scm + OZONE-SITE.XML_ozone.scm.client.address: scm + OZONE-SITE.XML_ozone.scm.datanode.id.dir: /data/metadata + OZONE-SITE.XML_ozone.scm.http-address: scm:9876 + OZONE-SITE.XML_ozone.scm.names: scm + OZONE-SITE.XML_hdds.scm.safemode.min.datanode: "1" + OZONE-SITE.XML_hdds.scm.safemode.healthy.pipeline.pct: "0" + no_proxy: om,recon,scm,s3g,httpfs,knox-ldap,knox-gateway,localhost,127.0.0.1 + +version: "3" +services: + datanode: + <<: *image + ports: + - 9864 + command: [ ozone, datanode ] + environment: + <<: *common-config + + om: + <<: *image + ports: + - 9874:9874 + environment: + <<: *common-config + CORE-SITE.XML_hadoop.proxyuser.hadoop.hosts: "*" + CORE-SITE.XML_hadoop.proxyuser.hadoop.groups: "*" + ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION + WAITFOR: scm:9876 + command: [ ozone, om ] + + scm: + <<: *image + ports: + - 9876:9876 + environment: + <<: *common-config + ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION + command: [ ozone, scm ] + + recon: + <<: *image + ports: + - 9888:9888 + environment: + <<: *common-config + command: [ ozone, recon ] + + s3g: + <<: *image + ports: + - 9878:9878 + environment: + <<: *common-config + command: [ ozone, s3g ] + + httpfs: + <<: *image + ports: + - 14000:14000 + environment: + <<: *common-config + CORE-SITE.XML_fs.defaultFS: ofs://om + CORE-SITE.XML_httpfs.proxyuser.knoxuser.hosts: "*" + CORE-SITE.XML_httpfs.proxyuser.knoxuser.groups: "*" Review Comment: These variables are converted into properties in `core-site.xml`, but Ozone HttpFS loads its server settings from `httpfs-default.xml` and `httpfs-site.xml`. https://github.com/apache/ozone/blob/1a9439452d9aae2ffd9ed4d20e8ac1f3a9f98864/hadoop-ozone/httpfsgateway/src/main/java/org/apache/ozone/lib/server/Server.java#L49-L52 `HttpFSAuthenticationFilter#getProxyuserConfiguration()` reads `httpfs.proxyuser.*` from that HttpFS configuration, so these properties will not take effect here. Should these use the `HTTPFS-SITE.XML_` prefix instead, or should the compose stack mount an `httpfs-site.xml`? ########## docs/05-administrator-guide/02-configuration/03-security/04-knox.md: ########## @@ -1,9 +1,251 @@ --- -draft: true sidebar_label: Apache Knox --- # Configuring Apache Knox -**TODO:** File a subtask under [HDDS-9859](https://issues.apache.org/jira/browse/HDDS-9859) and complete this page or section. -**TODO:** Uncomment link to this page in src/pages/index.js +[Apache Knox](https://knox.apache.org/) is a reverse proxy gateway for Hadoop ecosystem REST APIs and web UIs. Knox 2.0.0 and later can proxy Ozone services, giving users a single HTTPS entry point while hiding internal cluster hostnames and ports. + +Ozone integration was added in [KNOX-2833](https://issues.apache.org/jira/browse/KNOX-2833) (OM, SCM, and Recon UIs) and extended in [KNOX-2914](https://issues.apache.org/jira/browse/KNOX-2914) (HttpFS). Knox does **not** proxy the S3 Gateway today; use a load balancer or direct S3G access instead. + +## Supported Ozone services + +| Knox service role | Ozone component | Default HTTP port | Knox path prefix | +|---|---|---|---| +| `OZONE` | Ozone Manager UI | 9874 | `/ozone/` | +| `OZONE-SCM` | Storage Container Manager UI | 9876 | `/ozone-scm/` | +| `OZONE-RECON` | Recon UI | 9888 | `/ozone-recon/` | +| `OZONE-HTTPFS` | HttpFS Gateway | 14000 | `/httpfs/` | + +## Prerequisites + +- Apache Knox **2.0.0 or later** installed and running. +- Ozone HTTP endpoints reachable from the Knox host (`ozone.om.http-address`, `ozone.scm.http-address`, `ozone.recon.http-address`, and HttpFS on port 14000 by default). +- For production: configure TLS on Knox and Kerberos SPNEGO on Ozone HTTP services. See [HTTPS / SPNEGO](./https) and [Kerberos](./kerberos). + +## Topology configuration + +Knox routes traffic based on **topology** XML files in `{GATEWAY_HOME}/conf/topologies/`. Create a topology (for example `ozone.xml`) that declares authentication providers and Ozone backend URLs. + +### Example topology + +The following example uses Knox demo LDAP for authentication. Replace hostnames and ports with your cluster values. + +```xml +<?xml version="1.0" encoding="utf-8"?> +<topology> + <gateway> + <provider> + <role>authentication</role> + <name>ShiroProvider</name> + <enabled>true</enabled> + <param> + <name>sessionTimeout</name> + <value>30</value> + </param> + <param> + <name>main.ldapRealm</name> + <value>org.apache.knox.gateway.shirorealm.KnoxLdapRealm</value> + </param> + <param> + <name>main.ldapContextFactory</name> + <value>org.apache.knox.gateway.shirorealm.KnoxLdapContextFactory</value> + </param> + <param> + <name>main.ldapRealm.contextFactory</name> + <value>$ldapContextFactory</value> + </param> + <param> + <name>main.ldapRealm.userDnTemplate</name> + <value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value> + </param> + <param> + <name>main.ldapRealm.contextFactory.url</name> + <value>ldap://knox-ldap.example.com:33389</value> + </param> + <param> + <name>main.ldapRealm.contextFactory.authenticationMechanism</name> + <value>simple</value> + </param> + <param> + <name>urls./**</name> + <value>authcBasic</value> + </param> + </provider> + + <provider> + <role>identity-assertion</role> + <name>Default</name> + <enabled>true</enabled> + </provider> + + <provider> + <role>hostmap</role> + <name>static</name> + <enabled>true</enabled> + <param> + <name>om1.example.com</name> + <value>om1.internal.example.com</value> + </param> + </provider> + </gateway> + + <service> + <role>OZONE</role> + <url>http://om1.internal.example.com:9874</url> + </service> + + <service> + <role>OZONE-SCM</role> + <url>http://scm1.internal.example.com:9876</url> + </service> + + <service> + <role>OZONE-RECON</role> + <url>http://recon.internal.example.com:9888</url> + </service> + + <service> + <role>OZONE-HTTPFS</role> + <url>http://httpfs.internal.example.com:14000/webhdfs/</url> + </service> +</topology> +``` + +Place the file in `{GATEWAY_HOME}/conf/topologies/ozone.xml`. Knox automatically redeploys the topology when the file changes. + +### High availability + +When multiple Ozone Managers or SCMs run in HA, add a `<service>` entry for each instance. Knox uses a `?host={host}` query parameter on OM and SCM UI links so the gateway routes to the correct backend ([KNOX-2930](https://github.com/apache/knox/pull/767)). Configure the `hostmap` provider to rewrite internal hostnames embedded in UI responses. + +### Secure clusters + +For Kerberized Ozone clusters: + +1. Enable SPNEGO on Ozone HTTP services as described in [HTTPS / SPNEGO](./https). +2. Configure Knox authentication with your enterprise identity provider (LDAP, SAML, or Kerberos) instead of demo LDAP. +3. Terminate TLS at Knox and use HTTPS between Knox and Ozone backends where possible. + +## Ozone-side configuration + +### HttpFS proxy user + +When Knox proxies HttpFS, configure a proxy user on the HttpFS service so Knox can impersonate end users. Add the following to `httpfs-site.xml` (or the equivalent `CORE-SITE.XML_` environment variables in Docker): + +```xml +<property> + <name>httpfs.proxyuser.knoxuser.hosts</name> + <value>*</value> +</property> +<property> + <name>httpfs.proxyuser.knoxuser.groups</name> + <value>*</value> +</property> +``` + +Replace `knoxuser` with the Unix or service account Knox uses to connect to HttpFS. Restrict `hosts` and `groups` in production. See [HttpFS Gateway](../../../user-guide/client-interfaces/httpfs#proxy-user-configuration) for details. + +### HTTP addresses + +Ensure each proxied service publishes the HTTP address Knox will use: + +| Property | Example | +|---|---| +| `ozone.om.http-address` | `om1.example.com:9874` | +| `ozone.scm.http-address` | `scm1.example.com:9876` | +| `ozone.recon.http-address` | `recon.example.com:9888` | + +## Accessing Ozone through Knox + +After Knox and the topology are deployed, users reach Ozone through the Knox gateway URL. + +### Knox homepage + +Open the Knox homepage to see Ozone service cards: + +```text +https://<knox-host>:8443/gateway/homepage/home +``` + +Authenticate with the credentials configured in the topology (for example, demo LDAP user `guest` with password `guest-password`). + +### Web UIs + +Direct links follow this pattern. For OM and SCM, the `host` query parameter must be the full backend URL (not just the hostname): + +```text +https://<knox-host>:8443/gateway/ozone/ozone/index.html?host=http://<om-host>:9874 +https://<knox-host>:8443/gateway/ozone/ozone-scm/index.html?host=http://<scm-host>:9876 +https://<knox-host>:8443/gateway/ozone/ozone-recon/index.html +``` + +Replace `ozone` in the path with your topology name if different. In HA deployments, use the URL of the specific OM or SCM instance you want to reach. + +### HttpFS REST API + +Access HttpFS through Knox using the `/httpfs/` path prefix: + +```bash +curl -ik -u guest:guest-password \ + "https://localhost:8443/gateway/ozone/httpfs/v1/?op=LISTSTATUS&user.name=hadoop"; +``` + +The `user.name` parameter specifies the Ozone user to impersonate. Knox authenticates the caller; HttpFS performs the operation as the proxied user. Review Comment: I tested the compose stack locally, and the `HttpFS` access log shows Knox rewrites `user.name` to the authenticated Knox user regardless of what the client sends: ``` GET /webhdfs/v1/?op=LISTSTATUS&user.name=guest ``` This is the request HttpFS received, even though the client sent `user.name=hadoop`. So "The user.name parameter specifies the Ozone user to impersonate" isn't accurate in this setup. ```suggestion The `user.name` parameter specifies the Ozone user to impersonate. Knox authenticates access to the gateway, while this quickstart’s HttpFS backend uses the `user.name` query parameter through pseudo authentication. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
