[
https://issues.apache.org/jira/browse/HADOOP-16068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16763049#comment-16763049
]
Steve Loughran commented on HADOOP-16068:
-----------------------------------------
HADOOP-16068 patch 003
* when DTs are on, custom DT is asked for canonical service name.
* Classic binding mechanism (used somewhere in HD/I) works with canonical
service name set by FileSystem default mechanism (uses IPAddr of azure a/c)
* if a token returned already sets a Kind, don't overwrite it with a local
instance (allows for independent unmarshalling, renewal, cancel)
* test of the binding process using a stub token manager
* Integration tests verify issue for jobs, save & load, no regression of
current code
* adds dtutils support for abfs and abfss URLs
* Addresses checkstyle issues of patch 002
The big piece of work is those integration tests; based on the S3A DT test
suites & verifying that DTs are created during job submission and the hadoop
dtutils command. The {{delegationTokenManager}} instance of the filesystem has
to be made accessible to the tests, which is done via a package-private
accessor; a method in AbstractAbfsIntegrationTest then provides that value to
any subclass which wants it, irrespective of which package they are in.
oh, and log4j for test runs turns off ShellBasedUnixGroupsMapping logging at
warning, as it keeps complaining that the newly created user,
"[email protected]" doesn't have a user mapping.
Testing: ABFS Ireland. I'm getting complaints from some tests about not running
with namespaces, but the new test works; both in IDE-runs and in parallel test
runs (i.e both JUnit JVM lifecycles)
> ABFS Auth and DT plugins to be bound to specific URI of the FS
> --------------------------------------------------------------
>
> Key: HADOOP-16068
> URL: https://issues.apache.org/jira/browse/HADOOP-16068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-16068-001.patch, HADOOP-16068-002.patch,
> HADOOP-16068-003.patch
>
>
> followup from HADOOP-15692: pass in the URI & conf of the owner FS to bind
> the plugins to the specific FS instance. Without that you can't have per FS
> auth
> +add a stub DT plugin for testing, verify that DTs are collected.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
