Siyao Meng created HDDS-17679:
---------------------------------
Summary: SCM issues container tokens under the read-only
administrator check, although a container token is not limited to read
operations
Key: HDDS-17679
URL: https://issues.apache.org/jira/browse/HDDS-17679
Project: Apache Ozone
Issue Type: Bug
Reporter: Siyao Meng
Attachments: H-1-require-full-admin-for-container-token.patch
h3. Mechanism
{{SCMClientProtocolServer.getContainerToken}} authorizes the caller with
{{checkAdminAccess(remoteUser, true)}}, the read form of the SCM admin check.
{{StorageContainerManager.checkAdminAccess}} passes that form for a principal
in {{ozone.administrators}} and also for a principal that is only in
{{ozone.readonly.administrators}} or {{ozone.readonly.administrators.groups}}.
The method then calls {{ContainerTokenGenerator.generateToken(user,
containerID)}} for any container ID, with no further check.
A container token ({{ContainerTokenIdentifier}}, {{ContainerTokenSecretProto}})
carries the owner, the container ID, an expiry and the signing key id. It
carries no access mode, unlike the block token ({{OzoneBlockTokenIdentifier}}),
so it is accepted for every container command that requires a container token,
not only the read commands. The SCM operations that change a container's state
use the full admin form of the check and refuse the same read-only principal.
HDDS-4729 (1.2.0) added the token behind the full admin check. HDDS-8456
(1.4.0) classified every SCM client operation as read or full admin and put
{{getContainerToken}} on the read side; the JIRA and the pull request do not
discuss what the token authorizes.
h3. Trigger
# A secure cluster ({{ozone.security.enabled=true}},
{{ozone.authorization.enabled}} at its default of {{true}}) with
{{hdds.container.token.enabled=true}}. With container tokens disabled (the
default) the generator is the no-op one and no command requires a container
token, so the classification has no effect.
# A principal in {{ozone.readonly.administrators}} or its groups, and not in
{{ozone.administrators}}, asks SCM for a container token through
{{getContainerToken}}. SCM issues it.
Present from the 1.4.0 release tag onwards (HDDS-8456), by reading the code at
each release tag up to 2.2.2 and on master at 7294fc2da1.
h3. Impact
* A principal the configuration defines as read-only holds, for any container
ID, a credential that is not limited to read operations, while the SCM
operations that change a container's state refuse that principal.
* The SCM audit log records each issuance ({{GET_CONTAINER_TOKEN}} with the
user and the container ID). A token already issued stays valid until its expiry
({{hdds.block.token.expiry.time}}, default 1d), also after the principal leaves
the read-only list, which is a reconfigurable key.
h3. Reproduction
The regression test in the attached patch covers it and fails on unmodified
source: {{TestSCMClientProtocolServer.testGetContainerTokenRequiresFullAdmin}}
({{hadoop-hdds/server-scm}}). The SCM starter user, a full admin, obtains a
token, and a principal that is only in {{ozone.readonly.administrators}} must
be refused with {{AccessControlException}}. Without the change the second
assertion fails with {{Expecting code to raise a throwable}}. The test uses the
SCM setup of that class, where container tokens are not enabled, so the
generator is the no-op one; the authorization decision is the same code path.
The issue has no standalone reproduction.
h3. Patch
[^H-1-require-full-admin-for-container-token.patch], against
ec5ed5f7ea0e4b406aee53aae4906616819cba2b. It also applies to master at
7294fc2da1.
{{SCMClientProtocolServer.getContainerToken}} calls
{{checkAdminAccess(remoteUser, false)}}, so only a full SCM administrator
obtains a container token, as before HDDS-8456 and as for the SCM operations
that change a container's state. The token contents, the generator, the audit
action and the protocol are unchanged, and the tokens SCM embeds in its own
commands are not affected.
What changes for operators: with container tokens enabled, a principal that is
only a read-only administrator is now refused by the client commands that fetch
a container token through
{{ContainerOperationClient.getEncodedContainerToken}}, which are
{{ContainerStateVerifier}} and {{ChunkKeyHandler}} under {{ozone debug}} and
{{DNRPCLoadGenerator}} under {{ozone freon}}. With container tokens disabled
nothing changes, since {{getEncodedContainerToken}} does not call SCM then. The
alternative that keeps those commands available to read-only administrators is
an access mode in {{ContainerTokenSecretProto}}, mirroring the block token,
with read-only tokens issued under the read check and enforced where the token
is checked. That needs a protocol field, a change at every place that issues a
container token, and a decision for tokens without the field during a rolling
upgrade, so it is left for a follow-up if the project wants it.
With the patch {{TestSCMClientProtocolServer}} passes (9 tests) and checkstyle
on {{hadoop-hdds/server-scm}} is clean.
Found by code review of the datanode container state machine apply and block
commit paths, as part of the TLA+ verification effort under HDDS-15926, on
commit ec5ed5f7ea0e4b406aee53aae4906616819cba2b. Checked against HDDS issues
and apache/ozone pull requests for duplicates before filing. Related but not
duplicates: HDDS-4729 (introduced the container token behind the full admin
check), HDDS-8456 (classified {{getContainerToken}} as a read operation),
HDDS-15620 (authorization of {{ozone admin}} status commands for read-only
administrators), HDDS-17440 (block token verification). The attached patch is a
proposal for review. Generated with Specula (Claude Fable 5.1).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
