[
https://issues.apache.org/jira/browse/HDDS-17195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17195:
------------------------------
Description:
Ozone carries its own copy of the Hadoop RPC client
({{org.apache.hadoop.ipc_.Client}}). In secure mode, when the protocol has
{{@KerberosInfo}}, {{Connection.setupConnection}} binds the client socket to
the local address of the host in the caller's Kerberos principal before it
connects.
Hadoop lets operators turn that off with
{{ipc.client.bind.wildcard.addr=true}}. Ozone's copy reads the flag into
{{bindToWildCardAddress}} and computes {{bindTo}}, but then binds to
{{localAddr}} anyway:
{code:java}
InetAddress bindTo = bindToWildCardAddress ? null : localAddr;
LOG.debug("Binding {} to {}", principal, (bindToWildCardAddress) ? "0.0.0.0" :
localAddr);
this.socket.bind(new InetSocketAddress(localAddr, 0));
{code}
Upstream Hadoop honors the flag through
{{NetUtils.bindToLocalAddress(localAddr, bindToWildCardAddress)}}. Ozone cannot
call that method here: it is missing in Hadoop 2, and the Ozone filesystem jar
for Hadoop 2 uses the Hadoop 2 {{NetUtils}} at runtime.
h3. Impact on IPv6
Service-to-service RPC in a secure cluster is affected: OM to SCM, datanode to
SCM, and SCM to SCM. On a dual-stack host, the principal's host name resolves
to the IPv4 address first by default. When the target service is configured
with an IPv6 literal, the socket is bound to an IPv4 address and then connects
to an IPv6 address, which fails. The reverse combination fails the same way.
Upstream's escape hatch is {{ipc.client.bind.wildcard.addr=true}}, which Ozone
currently ignores. IPv6-only clusters are not affected. No Hadoop release fixes
this, because the code is Ozone's own copy.
h3. Fix
Bind the socket to the existing {{bindTo}} variable, which is {{null}}
(wildcard) when the flag is set. With the default ({{false}}), behavior is
unchanged.
Parent epic: HDDS-15763.
was:
Ozone carries its own copy of the Hadoop RPC client
({{org.apache.hadoop.ipc_.Client}}). In secure mode, when the protocol has
{{@KerberosInfo}}, {{Connection.setupConnection}} binds the client socket to
the local address of the host in the caller's Kerberos principal before it
connects.
Hadoop lets operators turn that off with
{{ipc.client.bind.wildcard.addr=true}}. Ozone's copy reads the flag into
{{bindToWildCardAddress}} and computes {{bindTo}}, but then binds to
{{localAddr}} anyway:
{code:java}
InetAddress bindTo = bindToWildCardAddress ? null : localAddr;
LOG.debug("Binding {} to {}", principal, (bindToWildCardAddress) ? "0.0.0.0" :
localAddr);
this.socket.bind(new InetSocketAddress(localAddr, 0));
{code}
Upstream Hadoop honors the flag through
{{NetUtils.bindToLocalAddress(localAddr, bindToWildCardAddress)}}. That method
is already in Hadoop 3.4.3.
h3. Impact on IPv6
Service-to-service RPC in a secure cluster is affected: OM to SCM, datanode to
SCM, and SCM to SCM. On a dual-stack host, the principal's host name resolves
to the IPv4 address first by default. When the target service is configured
with an IPv6 literal, the socket is bound to an IPv4 address and then connects
to an IPv6 address, which fails. The reverse combination fails the same way.
Upstream's escape hatch is {{ipc.client.bind.wildcard.addr=true}}, which Ozone
currently ignores. IPv6-only clusters are not affected. No Hadoop release fixes
this, because the code is Ozone's own copy.
h3. Fix
Use {{NetUtils.bindToLocalAddress}} as upstream does. With the default
({{false}}), behavior is unchanged.
Parent epic: HDDS-15763.
> Ozone RPC client ignores ipc.client.bind.wildcard.addr when binding to the
> Kerberos principal host
> --------------------------------------------------------------------------------------------------
>
> Key: HDDS-17195
> URL: https://issues.apache.org/jira/browse/HDDS-17195
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Attachments: HDDS-17195.001.patch
>
>
> Ozone carries its own copy of the Hadoop RPC client
> ({{org.apache.hadoop.ipc_.Client}}). In secure mode, when the protocol has
> {{@KerberosInfo}}, {{Connection.setupConnection}} binds the client socket to
> the local address of the host in the caller's Kerberos principal before it
> connects.
> Hadoop lets operators turn that off with
> {{ipc.client.bind.wildcard.addr=true}}. Ozone's copy reads the flag into
> {{bindToWildCardAddress}} and computes {{bindTo}}, but then binds to
> {{localAddr}} anyway:
> {code:java}
> InetAddress bindTo = bindToWildCardAddress ? null : localAddr;
> LOG.debug("Binding {} to {}", principal, (bindToWildCardAddress) ? "0.0.0.0"
> : localAddr);
> this.socket.bind(new InetSocketAddress(localAddr, 0));
> {code}
> Upstream Hadoop honors the flag through
> {{NetUtils.bindToLocalAddress(localAddr, bindToWildCardAddress)}}. Ozone
> cannot call that method here: it is missing in Hadoop 2, and the Ozone
> filesystem jar for Hadoop 2 uses the Hadoop 2 {{NetUtils}} at runtime.
> h3. Impact on IPv6
> Service-to-service RPC in a secure cluster is affected: OM to SCM, datanode
> to SCM, and SCM to SCM. On a dual-stack host, the principal's host name
> resolves to the IPv4 address first by default. When the target service is
> configured with an IPv6 literal, the socket is bound to an IPv4 address and
> then connects to an IPv6 address, which fails. The reverse combination fails
> the same way.
> Upstream's escape hatch is {{ipc.client.bind.wildcard.addr=true}}, which
> Ozone currently ignores. IPv6-only clusters are not affected. No Hadoop
> release fixes this, because the code is Ozone's own copy.
> h3. Fix
> Bind the socket to the existing {{bindTo}} variable, which is {{null}}
> (wildcard) when the flag is set. With the default ({{false}}), behavior is
> unchanged.
> Parent epic: HDDS-15763.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
