[jira] [Commented] (HADOOP-16565) Fix "com.amazonaws.SdkClientException: Unable to find a region via the region provider chain."

2019-09-16 Thread Steve Loughran (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930396#comment-16930396
 ] 

Steve Loughran commented on HADOOP-16565:
-

Maybe the last SDK update changed the requirements here I didn't notice because 
I'd set things up with an explicit declaration?

> Fix "com.amazonaws.SdkClientException: Unable to find a region via the region 
> provider chain."
> --
>
> Key: HADOOP-16565
> URL: https://issues.apache.org/jira/browse/HADOOP-16565
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> The error found during testing in the following tests:
> {noformat}
> [ERROR]   ITestS3ATemporaryCredentials.testInvalidSTSBinding:257 ? SdkClient 
> Unable to f...
> [ERROR]   ITestS3ATemporaryCredentials.testSTS:130 ? SdkClient Unable to find 
> a region v...
> [ERROR]   
> ITestS3ATemporaryCredentials.testSessionRequestExceptionTranslation:441->lambda$testSessionRequestExceptionTranslation$5:442
>  ? SdkClient
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenExpiry:222 ? SdkClient 
> Unable to ...
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenPropagation:193 ? 
> SdkClient Unabl...
> [ERROR]   ITestDelegatedMRJob.testJobSubmissionCollectsTokens:286 ? SdkClient 
> Unable to ...
> [ERROR]   ITestSessionDelegationInFileystem.testAddTokensFromFileSystem:235 ? 
> SdkClient ...
> [ERROR]   
> ITestSessionDelegationInFileystem.testCanRetrieveTokenFromCurrentUserCreds:260->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDTCredentialProviderFromCurrentUserCreds:278->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegatedFileSystem:308->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegationBindingMismatch1:432->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testFileSystemBoundToCreator:681 
> ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testGetDTfromFileSystem:212 ? 
> SdkClient Unab...
> [ERROR]   
> ITestSessionDelegationInFileystem.testHDFSFetchDTCommand:606->lambda$testHDFSFetchDTCommand$3:607
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testYarnCredentialPickup:576 ? 
> SdkClient Una...
> [ERROR]   ITestSessionDelegationTokens.testCreateAndUseDT:176 ? SdkClient 
> Unable to find...
> [ERROR]   ITestSessionDelegationTokens.testSaveLoadTokens:121 ? SdkClient 
> Unable to find...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16565) Fix "com.amazonaws.SdkClientException: Unable to find a region via the region provider chain."

2019-09-13 Thread Steve Loughran (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929313#comment-16929313
 ] 

Steve Loughran commented on HADOOP-16565:
-

maybe you always need to set a region, and I'd not noticed. In which case we 
should tune the defaults and document the error message

> Fix "com.amazonaws.SdkClientException: Unable to find a region via the region 
> provider chain."
> --
>
> Key: HADOOP-16565
> URL: https://issues.apache.org/jira/browse/HADOOP-16565
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> The error found during testing in the following tests:
> {noformat}
> [ERROR]   ITestS3ATemporaryCredentials.testInvalidSTSBinding:257 ? SdkClient 
> Unable to f...
> [ERROR]   ITestS3ATemporaryCredentials.testSTS:130 ? SdkClient Unable to find 
> a region v...
> [ERROR]   
> ITestS3ATemporaryCredentials.testSessionRequestExceptionTranslation:441->lambda$testSessionRequestExceptionTranslation$5:442
>  ? SdkClient
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenExpiry:222 ? SdkClient 
> Unable to ...
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenPropagation:193 ? 
> SdkClient Unabl...
> [ERROR]   ITestDelegatedMRJob.testJobSubmissionCollectsTokens:286 ? SdkClient 
> Unable to ...
> [ERROR]   ITestSessionDelegationInFileystem.testAddTokensFromFileSystem:235 ? 
> SdkClient ...
> [ERROR]   
> ITestSessionDelegationInFileystem.testCanRetrieveTokenFromCurrentUserCreds:260->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDTCredentialProviderFromCurrentUserCreds:278->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegatedFileSystem:308->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegationBindingMismatch1:432->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testFileSystemBoundToCreator:681 
> ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testGetDTfromFileSystem:212 ? 
> SdkClient Unab...
> [ERROR]   
> ITestSessionDelegationInFileystem.testHDFSFetchDTCommand:606->lambda$testHDFSFetchDTCommand$3:607
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testYarnCredentialPickup:576 ? 
> SdkClient Una...
> [ERROR]   ITestSessionDelegationTokens.testCreateAndUseDT:176 ? SdkClient 
> Unable to find...
> [ERROR]   ITestSessionDelegationTokens.testSaveLoadTokens:121 ? SdkClient 
> Unable to find...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16565) Fix "com.amazonaws.SdkClientException: Unable to find a region via the region provider chain."

2019-09-13 Thread Gabor Bota (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929284#comment-16929284
 ] 

Gabor Bota commented on HADOOP-16565:
-

It worked with the following STS config:

{noformat}

test.sts.endpoint
Specific endpoint to use for STS requests.
sts.amazonaws.com



sts.london.endpoint
sts.eu-west-2.amazonaws.com


sts.london.region
eu-west-2



sts.ireland.endpoint
sts.eu-west-1.amazonaws.com



fs.s3a.assumed.role.sts.endpoint
${sts.london.endpoint}


fs.s3a.assumed.role.sts.endpoint.region
${sts.london.region}

{noformat}

> Fix "com.amazonaws.SdkClientException: Unable to find a region via the region 
> provider chain."
> --
>
> Key: HADOOP-16565
> URL: https://issues.apache.org/jira/browse/HADOOP-16565
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> The error found during testing in the following tests:
> {noformat}
> [ERROR]   ITestS3ATemporaryCredentials.testInvalidSTSBinding:257 ? SdkClient 
> Unable to f...
> [ERROR]   ITestS3ATemporaryCredentials.testSTS:130 ? SdkClient Unable to find 
> a region v...
> [ERROR]   
> ITestS3ATemporaryCredentials.testSessionRequestExceptionTranslation:441->lambda$testSessionRequestExceptionTranslation$5:442
>  ? SdkClient
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenExpiry:222 ? SdkClient 
> Unable to ...
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenPropagation:193 ? 
> SdkClient Unabl...
> [ERROR]   ITestDelegatedMRJob.testJobSubmissionCollectsTokens:286 ? SdkClient 
> Unable to ...
> [ERROR]   ITestSessionDelegationInFileystem.testAddTokensFromFileSystem:235 ? 
> SdkClient ...
> [ERROR]   
> ITestSessionDelegationInFileystem.testCanRetrieveTokenFromCurrentUserCreds:260->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDTCredentialProviderFromCurrentUserCreds:278->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegatedFileSystem:308->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegationBindingMismatch1:432->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testFileSystemBoundToCreator:681 
> ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testGetDTfromFileSystem:212 ? 
> SdkClient Unab...
> [ERROR]   
> ITestSessionDelegationInFileystem.testHDFSFetchDTCommand:606->lambda$testHDFSFetchDTCommand$3:607
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testYarnCredentialPickup:576 ? 
> SdkClient Una...
> [ERROR]   ITestSessionDelegationTokens.testCreateAndUseDT:176 ? SdkClient 
> Unable to find...
> [ERROR]   ITestSessionDelegationTokens.testSaveLoadTokens:121 ? SdkClient 
> Unable to find...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16565) Fix "com.amazonaws.SdkClientException: Unable to find a region via the region provider chain."

2019-09-13 Thread Gabor Bota (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929242#comment-16929242
 ] 

Gabor Bota commented on HADOOP-16565:
-

I don't have any STS endpoint besides
{noformat}

test.sts.endpoint
Specific endpoint to use for STS requests.
sts.amazonaws.com

{noformat}
so I haven't changed a single setting and I was not getting this error but now 
I'm getting it. 


> Fix "com.amazonaws.SdkClientException: Unable to find a region via the region 
> provider chain."
> --
>
> Key: HADOOP-16565
> URL: https://issues.apache.org/jira/browse/HADOOP-16565
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> The error found during testing in the following tests:
> {noformat}
> [ERROR]   ITestS3ATemporaryCredentials.testInvalidSTSBinding:257 ? SdkClient 
> Unable to f...
> [ERROR]   ITestS3ATemporaryCredentials.testSTS:130 ? SdkClient Unable to find 
> a region v...
> [ERROR]   
> ITestS3ATemporaryCredentials.testSessionRequestExceptionTranslation:441->lambda$testSessionRequestExceptionTranslation$5:442
>  ? SdkClient
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenExpiry:222 ? SdkClient 
> Unable to ...
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenPropagation:193 ? 
> SdkClient Unabl...
> [ERROR]   ITestDelegatedMRJob.testJobSubmissionCollectsTokens:286 ? SdkClient 
> Unable to ...
> [ERROR]   ITestSessionDelegationInFileystem.testAddTokensFromFileSystem:235 ? 
> SdkClient ...
> [ERROR]   
> ITestSessionDelegationInFileystem.testCanRetrieveTokenFromCurrentUserCreds:260->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDTCredentialProviderFromCurrentUserCreds:278->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegatedFileSystem:308->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegationBindingMismatch1:432->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testFileSystemBoundToCreator:681 
> ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testGetDTfromFileSystem:212 ? 
> SdkClient Unab...
> [ERROR]   
> ITestSessionDelegationInFileystem.testHDFSFetchDTCommand:606->lambda$testHDFSFetchDTCommand$3:607
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testYarnCredentialPickup:576 ? 
> SdkClient Una...
> [ERROR]   ITestSessionDelegationTokens.testCreateAndUseDT:176 ? SdkClient 
> Unable to find...
> [ERROR]   ITestSessionDelegationTokens.testSaveLoadTokens:121 ? SdkClient 
> Unable to find...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16565) Fix "com.amazonaws.SdkClientException: Unable to find a region via the region provider chain."

2019-09-12 Thread Steve Loughran (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928709#comment-16928709
 ] 

Steve Loughran commented on HADOOP-16565:
-

Once you have identified the cause, add a new entry in the troubleshooting file 
for others

> Fix "com.amazonaws.SdkClientException: Unable to find a region via the region 
> provider chain."
> --
>
> Key: HADOOP-16565
> URL: https://issues.apache.org/jira/browse/HADOOP-16565
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.3.0
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> The error found during testing in the following tests:
> {noformat}
> [ERROR]   ITestS3ATemporaryCredentials.testInvalidSTSBinding:257 ? SdkClient 
> Unable to f...
> [ERROR]   ITestS3ATemporaryCredentials.testSTS:130 ? SdkClient Unable to find 
> a region v...
> [ERROR]   
> ITestS3ATemporaryCredentials.testSessionRequestExceptionTranslation:441->lambda$testSessionRequestExceptionTranslation$5:442
>  ? SdkClient
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenExpiry:222 ? SdkClient 
> Unable to ...
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenPropagation:193 ? 
> SdkClient Unabl...
> [ERROR]   ITestDelegatedMRJob.testJobSubmissionCollectsTokens:286 ? SdkClient 
> Unable to ...
> [ERROR]   ITestSessionDelegationInFileystem.testAddTokensFromFileSystem:235 ? 
> SdkClient ...
> [ERROR]   
> ITestSessionDelegationInFileystem.testCanRetrieveTokenFromCurrentUserCreds:260->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDTCredentialProviderFromCurrentUserCreds:278->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegatedFileSystem:308->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegationBindingMismatch1:432->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testFileSystemBoundToCreator:681 
> ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testGetDTfromFileSystem:212 ? 
> SdkClient Unab...
> [ERROR]   
> ITestSessionDelegationInFileystem.testHDFSFetchDTCommand:606->lambda$testHDFSFetchDTCommand$3:607
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testYarnCredentialPickup:576 ? 
> SdkClient Una...
> [ERROR]   ITestSessionDelegationTokens.testCreateAndUseDT:176 ? SdkClient 
> Unable to find...
> [ERROR]   ITestSessionDelegationTokens.testSaveLoadTokens:121 ? SdkClient 
> Unable to find...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org



[jira] [Commented] (HADOOP-16565) Fix "com.amazonaws.SdkClientException: Unable to find a region via the region provider chain."

2019-09-12 Thread Steve Loughran (Jira)


[ 
https://issues.apache.org/jira/browse/HADOOP-16565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928706#comment-16928706
 ] 

Steve Loughran commented on HADOOP-16565:
-

looks like I deliberately choose london for STS'' 
   sts.london.endpoint
   sts.eu-west-2.amazonaws.com
 
 
   sts.london.region
   eu-west-2
 

 
   sts.ireland.endpoint
   sts.eu-west-1.amazonaws.com
 

 
   fs.s3a.assumed.role.sts.endpoint
   ${sts.london.endpoint}
 
 
   fs.s3a.assumed.role.sts.endpoint.region
   ${sts.london.region}
 
 

> Fix "com.amazonaws.SdkClientException: Unable to find a region via the region 
> provider chain."
> --
>
> Key: HADOOP-16565
> URL: https://issues.apache.org/jira/browse/HADOOP-16565
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Gabor Bota
>Assignee: Gabor Bota
>Priority: Major
>
> The error found during testing in the following tests:
> {noformat}
> [ERROR]   ITestS3ATemporaryCredentials.testInvalidSTSBinding:257 ? SdkClient 
> Unable to f...
> [ERROR]   ITestS3ATemporaryCredentials.testSTS:130 ? SdkClient Unable to find 
> a region v...
> [ERROR]   
> ITestS3ATemporaryCredentials.testSessionRequestExceptionTranslation:441->lambda$testSessionRequestExceptionTranslation$5:442
>  ? SdkClient
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenExpiry:222 ? SdkClient 
> Unable to ...
> [ERROR]   ITestS3ATemporaryCredentials.testSessionTokenPropagation:193 ? 
> SdkClient Unabl...
> [ERROR]   ITestDelegatedMRJob.testJobSubmissionCollectsTokens:286 ? SdkClient 
> Unable to ...
> [ERROR]   ITestSessionDelegationInFileystem.testAddTokensFromFileSystem:235 ? 
> SdkClient ...
> [ERROR]   
> ITestSessionDelegationInFileystem.testCanRetrieveTokenFromCurrentUserCreds:260->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDTCredentialProviderFromCurrentUserCreds:278->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegatedFileSystem:308->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   
> ITestSessionDelegationInFileystem.testDelegationBindingMismatch1:432->createDelegationTokens:292->AbstractDelegationIT.mkTokens:88
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testFileSystemBoundToCreator:681 
> ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testGetDTfromFileSystem:212 ? 
> SdkClient Unab...
> [ERROR]   
> ITestSessionDelegationInFileystem.testHDFSFetchDTCommand:606->lambda$testHDFSFetchDTCommand$3:607
>  ? SdkClient
> [ERROR]   ITestSessionDelegationInFileystem.testYarnCredentialPickup:576 ? 
> SdkClient Una...
> [ERROR]   ITestSessionDelegationTokens.testCreateAndUseDT:176 ? SdkClient 
> Unable to find...
> [ERROR]   ITestSessionDelegationTokens.testSaveLoadTokens:121 ? SdkClient 
> Unable to find...
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org