[
https://issues.apache.org/jira/browse/HADOOP-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908920#comment-17908920
]
Steve Loughran commented on HADOOP-19378:
-----------------------------------------
[~srahman] over to you. I guess I hadn't run the terasort tests (only kicked
off with -Dscale).
you'll need to include all CSE-KMS settings in the marshalled secrets, the goal
being that work submitted into a cluster can collect on the client all auth and
encryption details, rather have them in the cluster configuration.
I'd have expect this to have surfaced first in
{{ITestSessionDelegationInFilesystem.testDelegatedFileSystem().testDelegatedFileSystem}},
though it didn't
* make sure that test is removing all config options related to cse; it must
pick up all it needs from that delegation token
* once it is good, worry about the terasort. That sticks all its logs deep
under {{target/test/data}} and not something you can debug
> S3A: terasort test fail with CSE-kMS enabled and london region
> --------------------------------------------------------------
>
> Key: HADOOP-19378
> URL: https://issues.apache.org/jira/browse/HADOOP-19378
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
> Affects Versions: 3.4.2
> Environment: test bucket with cse-kms both bucket and kms in
> eu-west-2 region
> Reporter: Steve Loughran
> Assignee: Syed Shameerur Rahman
> Priority: Major
>
> terasort test runs failing; job logs show the process yarn spawns is trying
> to use the default region provider list
> {code}
> Unable to load region from any of the providers in the chain
> software.amazon.awssdk.regions.providers.DefaultAwsRegionProviderChain
> {code}
> these are spawned processes; all login info and encryption secrets are passed
> in the marshalled delegation token {{AbstractS3ATokenIdentifier}}, with the
> encryption stuff in {{EncryptionSecrets}}
> This is probably going to need to be extended with information about the KMS
> region, with this info extracted during instantiation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
