[
https://issues.apache.org/jira/browse/HDDS-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-9582:
-----------------------------------
Description:
OM transport factory creation logic:
{code:title=https://github.com/apache/ozone/blob/0b377f7620b4795031a517a7d5a48c5995210d35/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OmTransportFactory.java#L48-L66}
// if configured transport class is different than the default
// OmTransportFactory (Hadoop3OmTransportFactory), then
// check service loader for transport class and instantiate it
if (conf
.get(OZONE_OM_TRANSPORT_CLASS,
OZONE_OM_TRANSPORT_CLASS_DEFAULT) !=
OZONE_OM_TRANSPORT_CLASS_DEFAULT) {
ServiceLoader<OmTransportFactory> transportFactoryServiceLoader =
ServiceLoader.load(OmTransportFactory.class);
Iterator<OmTransportFactory> iterator =
transportFactoryServiceLoader.iterator();
if (iterator.hasNext()) {
return iterator.next();
}
}
return OmTransportFactory.class.getClassLoader()
.loadClass(OZONE_OM_TRANSPORT_CLASS_DEFAULT)
.asSubclass(OmTransportFactory.class)
.newInstance();
{code}
There are several problems:
# Based on the code above, {{ServiceLoader}} is intended to be invoked only if
non-default transport is configured. This is wrong, since we need to use
specific transport for Hadoop 2 for compatibility, regardless of the OM
transport setting. This is not caught by existing acceptance tests because we
don't test Hadoop 2.7 with security enabled.
# Configuring some specific transport in {{ozone-site.xml}} is not enough for
it to be applied, the class has to be set in {{META-INF/services}}, too.
# Comparison of actual vs. default setting uses reference equality instead of
{{equals(...)}}. Thus {{ServiceLoader}} is always invoked if
{{ozone.om.transport.class}} has any value in {{ozone-default.xml}}.
# Default setting does not match in XML and Java constant (this is being
changed in HDDS-9446, though).
I think we should prefer implementations in this order:
* found via service loader
* set in configuration
* default
was:
OM transport factory creation logic:
{code:title=https://github.com/apache/ozone/blob/0b377f7620b4795031a517a7d5a48c5995210d35/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OmTransportFactory.java#L48-L66}
// if configured transport class is different than the default
// OmTransportFactory (Hadoop3OmTransportFactory), then
// check service loader for transport class and instantiate it
if (conf
.get(OZONE_OM_TRANSPORT_CLASS,
OZONE_OM_TRANSPORT_CLASS_DEFAULT) !=
OZONE_OM_TRANSPORT_CLASS_DEFAULT) {
ServiceLoader<OmTransportFactory> transportFactoryServiceLoader =
ServiceLoader.load(OmTransportFactory.class);
Iterator<OmTransportFactory> iterator =
transportFactoryServiceLoader.iterator();
if (iterator.hasNext()) {
return iterator.next();
}
}
return OmTransportFactory.class.getClassLoader()
.loadClass(OZONE_OM_TRANSPORT_CLASS_DEFAULT)
.asSubclass(OmTransportFactory.class)
.newInstance();
{code}
There are several problems:
# Based on the code above, {{ServiceLoader}} is intended to be invoked only if
non-default transport is configured. This is wrong, since we need to use
specific transport for Hadoop 2 for compatibility, regardless of the OM
transport setting. This is not caught by existing acceptance tests because we
don't test Hadoop 2.7 with security enabled.
# Configuring some specific transport in {{ozone-site.xml}} is not enough for
it to be applied, the class has to be set in {{META-INF/services}}, too.
# Comparison of actual vs. default setting uses reference equality instead of
{{equals(...)}}. Thus {{ServiceLoader}} is always invoked if
{{ozone.om.transport.class}} has any value in {{ozone-default.xml}}.
# Default setting does not match in XML and Java constant (this is being
changed in HDDS-9446, though).
> OM transport factory configuration mismatch
> -------------------------------------------
>
> Key: HDDS-9582
> URL: https://issues.apache.org/jira/browse/HDDS-9582
> Project: Apache Ozone
> Issue Type: Bug
> Affects Versions: 1.4.0
> Reporter: Attila Doroszlai
> Priority: Critical
>
> OM transport factory creation logic:
> {code:title=https://github.com/apache/ozone/blob/0b377f7620b4795031a517a7d5a48c5995210d35/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/OmTransportFactory.java#L48-L66}
> // if configured transport class is different than the default
> // OmTransportFactory (Hadoop3OmTransportFactory), then
> // check service loader for transport class and instantiate it
> if (conf
> .get(OZONE_OM_TRANSPORT_CLASS,
> OZONE_OM_TRANSPORT_CLASS_DEFAULT) !=
> OZONE_OM_TRANSPORT_CLASS_DEFAULT) {
> ServiceLoader<OmTransportFactory> transportFactoryServiceLoader =
> ServiceLoader.load(OmTransportFactory.class);
> Iterator<OmTransportFactory> iterator =
> transportFactoryServiceLoader.iterator();
> if (iterator.hasNext()) {
> return iterator.next();
> }
> }
> return OmTransportFactory.class.getClassLoader()
> .loadClass(OZONE_OM_TRANSPORT_CLASS_DEFAULT)
> .asSubclass(OmTransportFactory.class)
> .newInstance();
> {code}
> There are several problems:
> # Based on the code above, {{ServiceLoader}} is intended to be invoked only
> if non-default transport is configured. This is wrong, since we need to use
> specific transport for Hadoop 2 for compatibility, regardless of the OM
> transport setting. This is not caught by existing acceptance tests because
> we don't test Hadoop 2.7 with security enabled.
> # Configuring some specific transport in {{ozone-site.xml}} is not enough for
> it to be applied, the class has to be set in {{META-INF/services}}, too.
> # Comparison of actual vs. default setting uses reference equality instead of
> {{equals(...)}}. Thus {{ServiceLoader}} is always invoked if
> {{ozone.om.transport.class}} has any value in {{ozone-default.xml}}.
> # Default setting does not match in XML and Java constant (this is being
> changed in HDDS-9446, though).
> I think we should prefer implementations in this order:
> * found via service loader
> * set in configuration
> * default
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
