[
https://issues.apache.org/jira/browse/HADOOP-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16713397#comment-16713397
]
Thomas Marquardt commented on HADOOP-15954:
-------------------------------------------
I have a few ideas about how we should design this feature:
1) There should be a public interface named CustomIdentityTransformer in the
org.apache.hadoop.fs.azurebfs.extensions package.
a) This class is already implemented in patch 004, but the name should be
changed from CustomIdentityTransformerAdaptee to CustomIdentityTransformer.
b) transformUserName and transformGroupName should be combined into a
single function named transformOwner if possible. Note that these methods are
invoked to transform the input of a FileSystem.setOwner call.
c) All implementations of identity transformer will implement this
interface, including the default implementation.
2) The AbfsConfiguration.getIdentityTransformer method should do the following:
a) If "*fs.azure.identity.transformer.type*" is not set, return immediately
without doing any additional work.
b) If "*fs.azure.identity.transformer.type*" is set:
i) Load the class.
ii) Create a new instance.
iii) Call initialize and pass in the configuration and account name.
iv) Return the new initialized instance of CustomIdentityTransformer.
3) Remove the configuration "*fs.azure.account.identity.type*". It is
superfluous.
4) Remove the enum TransformerType. It is superfluous since the
"*fs.azure.identity.transformer.type*" will define the type of identity
transformer used.
5) Remove the class
org.apache.hadoop.fs.azurebfs.oauth.CustomIdentityTransformerAdapter.
a) The Adapter Design model is not appropriate here. The Adapter Design
model is used when you have a public interface, but there are external
component that implement a different interface. In our case there is no need
to have an adapter, because the external component can implement our new public
interface directly.
6) Provide a default implementation named DefaultIdentityTransformer in the
org.apache.hadoop.fs.azurebfs.oauth2 package.
a) This class is partly implemented, but:
i) The name should be changed from StandardSPIdentityTransformer to
DefaultIdentityTransformer.
ii) DefaultIdentityTransformer should implement
CustomIdentityTransformer. The IdentityTransformer code should be moved into
DefaultIdentityTransformer.
7) Configuration used by DefaultIdentityTransformer should be prefixed with
“fs.azure.default.identity.transformer”:
a) “*fs.azure.default.identity.transformer.enable.short.name*” – user
principal names (UPNs) have the format “\{alias}@\{domain}”. If true, only
\{alias} is included when a UPN would otherwise appear in the output of APIs
like getFileStatus, getOwner, getAclStatus, etc.
b) “*fs.azure.default.identity.transformer.domain.name*” – If the domain
name is specified and “fs.azure.default.identity.transformer.enable.short.name”
is true, then the \{alias} part of a UPN can be specified as input to APIs like
setOwner and setAcl and it will be transformed to a UPN by appending @ and the
domain specified by this configuration property.
c)
“*fs.azure.default.identity.transformer.service.principal.substitution.list*” –
a comma separated list of names to be replaced with the service principal ID
specified by “*fs.azure.default.identity.transformer.service.principal.id*”.
This substitution occurs when setOwner, setAcl, modifyAclEntries, or
removeAclEntries are invoked with identities contained in the substitution list.
d) “*fs.azure.default.identity.transformer.service.principal.id*” – an
Azure Active Directory object ID used as the replacement for names contained in
the list specified by
“*fs.azure.default.identity.transformer.service.principal.substitution.list*”.
e) The patch 004 properties defined in ConfigurationKeys should be replaced
with the above. Please include descriptions.
8) Remove the class org.apache.hadoop.fs.azurebfs.oauth.IdentityTransformer.
9) Remove DEFAULT_ENABLE_SHORT_NAME from FileSystemConfigurations. It can be
moved to DefaultIdentityTransformer if it is needed.
10) I don’t see any reason for IdentityTransformerException to exist, since it
does not have any implementation. If it truly cannot be removed, I would
rename it to *IdentityTransformerInitializationException*.
11) Update all the places in AzureBlobFileSystemStore where the
CustomIdentityTransformer instance is invoked to check for null, and only
perform the transformation if non-null.
12) By default, there will be no transformation. A customer can set
fs.azure.identity.transformer.type to the default or a custom implementation as
needed.
13) We should have tests for the implementation of DefaultIdentityTransformer.
> ABFS: Enable owner and group conversion for MSI and login user using OAuth
> --------------------------------------------------------------------------
>
> Key: HADOOP-15954
> URL: https://issues.apache.org/jira/browse/HADOOP-15954
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.2.0
> Reporter: junhua gu
> Assignee: Da Zhou
> Priority: Major
> Attachments: HADOOP-15954-001.patch, HADOOP-15954-002.patch,
> HADOOP-15954-003.patch, HADOOP-15954-004.patch
>
>
> Add support for overwriting owner and group in set/get operations to be the
> service principal id when OAuth is used. Add support for upn short name
> format.
>
> Add Standard Transformer for SharedKey / Service
> Add interface provides an extensible model for customizing the acquisition of
> Identity Transformer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
