[
https://issues.apache.org/jira/browse/HDDS-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-15244:
-----------------------------------
Description:
The /logLevel HTTP endpoint, which allows users to view and modify log levels
at runtime, has a bug that makes it operationaly
broken in secure environments.
Technical Root Cause:
1. Missing Authentication: In HttpServer2.java, the logLevel servlet is
registered via addDefaultServlets() using addServlet. This internal call sets
requireAuth = false. Consequently, the internal SpnegoFilter is never applied
to the /logLevel path, even when Kerberos/SPNEGO is enabled for the
cluster.
2. Authorization Failure: The LogLevel.Servlet implementation calls
HttpServer2.hasAdministratorAccess(). In a secure cluster, this method requires
a
non-null request.getRemoteUser(). Because the authentication filter was
skipped, the remote user is always null, leading to a permanent 403 Forbidden
response for all users, including administrators.
Impact:
* Operational: Administrators cannot use the ozone insight tool or the web UI
to troubleshoot production issues in secure clusters by increasing log
verbosity.
Steps to Reproduce (Secure Cluster):
1. Start a secure Ozone cluster with Kerberos enabled.
2. Attempt to access http://<om-host>:9874/logLevel via a browser or curl
--negotiate.
3. The server will return 403 Forbidden with the message: "Unauthenticated
users are not authorized to access this page."
Proposed Fix:
* Modify HttpServer2.addDefaultApps() (or addDefaultServlets) to ensure the
logLevel servlet (and potentially others like stacks and conf) are registered
with requireAuth = true.
* Add a specific integration test in TestHttpServer2 or a new TestLogLevel to
verify that authentication is correctly challenged and authorization is
respected in both secure and non-secure modes.
Additional Context:
This issue is acknowledged in the documentation (Observability.md), which
states that the /logLevel endpoint is "not yet supported in secure environment."
This bug report aims to formalize the requirement to fix this inconsistency.
was:
The /logLevel HTTP endpoint, which allows users to view and modify log levels
at runtime, currently suffers from a design flaw that makes it operationaly
broken in secure environments and a security risk in unsecure environments.
Technical Root Cause:
1. Missing Authentication: In HttpServer2.java, the logLevel servlet is
registered via addDefaultServlets() using addServlet. This internal call sets
requireAuth = false. Consequently, the internal SpnegoFilter is never
applied to the /logLevel path, even when Kerberos/SPNEGO is enabled for the
cluster.
2. Authorization Failure: The LogLevel.Servlet implementation calls
HttpServer2.hasAdministratorAccess(). In a secure cluster, this method requires
a
non-null request.getRemoteUser(). Because the authentication filter was
skipped, the remote user is always null, leading to a permanent 403 Forbidden
response for all users, including administrators.
3. Unprotected Access in Non-Secure Mode: If authorization is disabled
(common in non-secure clusters), the endpoint remains completely open. Any user
can change log levels for any class without providing credentials.
Impact:
* Operational: Administrators cannot use the ozone insight tool or the web
UI to troubleshoot production issues in secure clusters by increasing log
verbosity.
* Security: In non-secure clusters, an attacker could potentially suppress
audit logs or flood the system by setting log levels to DEBUG/TRACE for
high-throughput classes.
Steps to Reproduce (Secure Cluster):
1. Start a secure Ozone cluster with Kerberos enabled.
2. Attempt to access http://<om-host>:9874/logLevel via a browser or curl
--negotiate.
3. The server will return 403 Forbidden with the message: "Unauthenticated
users are not authorized to access this page."
Proposed Fix:
* Modify HttpServer2.addDefaultApps() (or addDefaultServlets) to ensure the
logLevel servlet (and potentially others like stacks and conf) are registered
with requireAuth = true.
* Add a specific integration test in TestHttpServer2 or a new TestLogLevel
to verify that authentication is correctly challenged and authorization is
respected in both secure and non-secure modes.
Additional Context:
This issue is acknowledged in the documentation (Observability.md), which
states that the /logLevel endpoint is "not yet supported in secure environment."
This bug report aims to formalize the requirement to fix this inconsistency.
> LogLevel HTTP endpoint is inaccessible in secure clusters
> ---------------------------------------------------------
>
> Key: HDDS-15244
> URL: https://issues.apache.org/jira/browse/HDDS-15244
> Project: Apache Ozone
> Issue Type: Bug
> Components: Security
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> The /logLevel HTTP endpoint, which allows users to view and modify log levels
> at runtime, has a bug that makes it operationaly
> broken in secure environments.
> Technical Root Cause:
> 1. Missing Authentication: In HttpServer2.java, the logLevel servlet is
> registered via addDefaultServlets() using addServlet. This internal call sets
> requireAuth = false. Consequently, the internal SpnegoFilter is never applied
> to the /logLevel path, even when Kerberos/SPNEGO is enabled for the
> cluster.
> 2. Authorization Failure: The LogLevel.Servlet implementation calls
> HttpServer2.hasAdministratorAccess(). In a secure cluster, this method
> requires a
> non-null request.getRemoteUser(). Because the authentication filter was
> skipped, the remote user is always null, leading to a permanent 403 Forbidden
> response for all users, including administrators.
> Impact:
> * Operational: Administrators cannot use the ozone insight tool or the web
> UI to troubleshoot production issues in secure clusters by increasing log
> verbosity.
> Steps to Reproduce (Secure Cluster):
> 1. Start a secure Ozone cluster with Kerberos enabled.
> 2. Attempt to access http://<om-host>:9874/logLevel via a browser or curl
> --negotiate.
> 3. The server will return 403 Forbidden with the message: "Unauthenticated
> users are not authorized to access this page."
> Proposed Fix:
> * Modify HttpServer2.addDefaultApps() (or addDefaultServlets) to ensure the
> logLevel servlet (and potentially others like stacks and conf) are registered
> with requireAuth = true.
> * Add a specific integration test in TestHttpServer2 or a new TestLogLevel
> to verify that authentication is correctly challenged and authorization is
> respected in both secure and non-secure modes.
> Additional Context:
> This issue is acknowledged in the documentation (Observability.md), which
> states that the /logLevel endpoint is "not yet supported in secure
> environment."
> This bug report aims to formalize the requirement to fix this inconsistency.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
