[
https://issues.apache.org/jira/browse/HADOOP-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527493#comment-16527493
]
Steve Loughran commented on HADOOP-15573:
-----------------------------------------
{code}
bin/hadoop s3guard $ARN set-capacity -read 10 $IRL
2018-06-29 12:31:45,179 [main] DEBUG s3guard.S3GuardTool
(S3GuardTool.java:run(1458)) - Executing command set-capacity
Read capacity set to 10
2018-06-29 12:31:45,207 [main] DEBUG s3guard.S3GuardTool
(S3GuardTool.java:initS3AFileSystem(301)) - updated bucket store option
org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore
2018-06-29 12:31:45,974 [main] DEBUG s3a.S3AFileSystem
(S3AFileSystem.java:initialize(237)) - Initializing S3AFileSystem for
hwdev-steve-ireland-new
2018-06-29 12:31:45,976 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1001)) - Propagating entries under
fs.s3a.bucket.hwdev-steve-ireland-new.
2018-06-29 12:31:46,055 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1022)) - Updating fs.s3a.endpoint from
[core-site.xml]
2018-06-29 12:31:46,055 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1022)) - Updating
fs.s3a.committer.magic.enabled from [core-site.xml]
2018-06-29 12:31:46,056 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1022)) - Updating
fs.s3a.metadatastore.impl from [S3AUtils]
2018-06-29 12:31:46,260 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:createAWSCredentialProvider(628)) - Credential provider class is
org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
2018-06-29 12:31:46,261 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:createAWSCredentialProvider(628)) - Credential provider class is
org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
2018-06-29 12:31:46,269 [main] DEBUG auth.AssumedRoleCredentialProvider
(AssumedRoleCredentialProvider.java:<init>(117)) -
AssumedRoleCredentialProvider{role='arn:aws:iam::980678866538:role/stevel-s3-restricted',
session'stevel', duration=1800}
2018-06-29 12:31:46,271 [main] DEBUG auth.AssumedRoleCredentialProvider
(AssumedRoleCredentialProvider.java:<init>(130)) - Credentials to obtain role
credentials: AWSCredentialProviderList: SimpleAWSCredentialsProvider
2018-06-29 12:31:46,827 [main] DEBUG s3a.AWSCredentialProviderList
(AWSCredentialProviderList.java:getCredentials(122)) - Using credentials from
SimpleAWSCredentialsProvider
2018-06-29 12:31:47,566 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:createAWSCredentialProviderSet(572)) - For URI
s3a://hwdev-steve-ireland-new//, using credentials AWSCredentialProviderList:
AssumedRoleCredentialProvider{role='arn:aws:iam::980678866538:role/stevel-s3-restricted',
session'stevel', duration=1800}
2018-06-29 12:31:47,567 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.connection.maximum is 15
2018-06-29 12:31:47,567 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.attempts.maximum is 20
2018-06-29 12:31:47,567 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.connection.establish.timeout
is 5000
2018-06-29 12:31:47,568 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.connection.timeout is 5000
2018-06-29 12:31:47,568 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.socket.send.buffer is 65536
2018-06-29 12:31:47,568 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.socket.recv.buffer is 32678
2018-06-29 12:31:47,570 [main] DEBUG s3a.S3AFileSystem
(DefaultS3ClientFactory.java:initUserAgent(183)) - Using User-Agent: Hadoop
3.2.0-SNAPSHOT
2018-06-29 12:31:47,637 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.paging.maximum is 5000
2018-06-29 12:31:47,640 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:longBytesOption(887)) - Value of fs.s3a.block.size is 33554432
2018-06-29 12:31:47,640 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:longBytesOption(887)) - Value of fs.s3a.readahead.range is 524288
2018-06-29 12:31:47,640 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.max.total.tasks is 5
2018-06-29 12:31:47,641 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:longOption(865)) - Value of fs.s3a.threads.keepalivetime is 60
2018-06-29 12:31:47,656 [main] DEBUG s3a.AWSCredentialProviderList
(AWSCredentialProviderList.java:getCredentials(122)) - Using credentials from
AssumedRoleCredentialProvider{role='arn:aws:iam::980678866538:role/stevel-s3-restricted',
session'stevel', duration=1800}
2018-06-29 12:31:47,836 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:getEncryptionAlgorithm(1253)) - Data is unencrypted
2018-06-29 12:31:47,837 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:getEncryptionAlgorithm(1256)) - Using SSE-C with empty key
2018-06-29 12:31:47,837 [main] DEBUG s3a.S3AFileSystem
(S3AFileSystem.java:initialize(313)) - Input fadvise policy = normal
2018-06-29 12:31:47,837 [main] DEBUG s3a.S3AFileSystem
(S3AFileSystem.java:initialize(317)) - Filesystem support for magic committers
is enabled
2018-06-29 12:31:47,843 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.fast.upload.active.blocks is 4
2018-06-29 12:31:47,843 [main] DEBUG s3a.S3AFileSystem
(S3AFileSystem.java:initialize(333)) - Using S3ABlockOutputStream with buffer =
disk; block=8388608; queue limit=4
2018-06-29 12:31:47,846 [main] DEBUG s3guard.S3Guard
(S3Guard.java:getMetadataStoreClass(125)) - Metastore option source
fs.s3a.bucket.hwdev-steve-ireland-new.metadatastore.impl via [S3AUtils]
2018-06-29 12:31:47,847 [main] DEBUG s3guard.S3Guard
(S3Guard.java:getMetadataStore(97)) - Using NullMetadataStore metadata store
for s3a filesystem
2018-06-29 12:31:47,847 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:longOption(865)) - Value of fs.s3a.multipart.purge.age is 3600000
2018-06-29 12:31:47,924 [main] DEBUG s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:initialize(277)) - Inferring DynamoDB region from
S3 bucket: eu-west-1
2018-06-29 12:31:47,924 [main] DEBUG s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:createDynamoDB(255)) - Creating DynamoDB client
class
org.apache.hadoop.fs.s3a.s3guard.DynamoDBClientFactory$DefaultDynamoDBClientFactory
with S3 region eu-west-1
2018-06-29 12:31:47,924 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:createAWSCredentialProvider(628)) - Credential provider class is
org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider
2018-06-29 12:31:47,924 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:createAWSCredentialProvider(628)) - Credential provider class is
org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
2018-06-29 12:31:47,925 [main] DEBUG auth.AssumedRoleCredentialProvider
(AssumedRoleCredentialProvider.java:<init>(117)) -
AssumedRoleCredentialProvider{role='arn:aws:iam::980678866538:role/stevel-s3-restricted',
session'stevel', duration=1800}
2018-06-29 12:31:47,925 [main] DEBUG auth.AssumedRoleCredentialProvider
(AssumedRoleCredentialProvider.java:<init>(130)) - Credentials to obtain role
credentials: AWSCredentialProviderList: SimpleAWSCredentialsProvider
2018-06-29 12:31:47,927 [main] DEBUG s3a.AWSCredentialProviderList
(AWSCredentialProviderList.java:getCredentials(122)) - Using credentials from
SimpleAWSCredentialsProvider
2018-06-29 12:31:48,378 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:createAWSCredentialProviderSet(572)) - For URI (null URI), using
credentials AWSCredentialProviderList:
AssumedRoleCredentialProvider{role='arn:aws:iam::980678866538:role/stevel-s3-restricted',
session'stevel', duration=1800}
2018-06-29 12:31:48,379 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.connection.maximum is 15
2018-06-29 12:31:48,379 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.attempts.maximum is 20
2018-06-29 12:31:48,379 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.connection.establish.timeout
is 5000
2018-06-29 12:31:48,379 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.connection.timeout is 5000
2018-06-29 12:31:48,380 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.socket.send.buffer is 65536
2018-06-29 12:31:48,380 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:intOption(844)) - Value of fs.s3a.socket.recv.buffer is 32678
2018-06-29 12:31:48,380 [main] DEBUG s3a.S3AFileSystem
(DefaultS3ClientFactory.java:initUserAgent(183)) - Using User-Agent: Hadoop
3.2.0-SNAPSHOT
2018-06-29 12:31:48,382 [main] DEBUG s3guard.DynamoDBClientFactory
(DynamoDBClientFactory.java:createDynamoDBClient(84)) - Creating DynamoDB
client in region eu-west-1
2018-06-29 12:31:48,472 [main] DEBUG s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:initTable(893)) - Binding to table
hwdev-steve-ireland-new
2018-06-29 12:31:48,519 [main] DEBUG s3a.AWSCredentialProviderList
(AWSCredentialProviderList.java:getCredentials(122)) - Using credentials from
AssumedRoleCredentialProvider{role='arn:aws:iam::980678866538:role/stevel-s3-restricted',
session'stevel', duration=1800}
2018-06-29 12:31:48,867 [main] DEBUG s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:initTable(895)) - Table state:
{AttributeDefinitions: [{AttributeName: child,AttributeType: S},
{AttributeName: parent,AttributeType: S}],TableName:
hwdev-steve-ireland-new,KeySchema: [{AttributeName: parent,KeyType: HASH},
{AttributeName: child,KeyType: RANGE}],TableStatus: ACTIVE,CreationDateTime:
Fri Jun 29 12:20:03 BST 2018,ProvisionedThroughput: {NumberOfDecreasesToday:
0,ReadCapacityUnits: 500,WriteCapacityUnits: 100},TableSizeBytes: 0,ItemCount:
0,TableArn:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new,TableId:
99dfc220-a142-4816-9b3b-bfed91de2d30,}
2018-06-29 12:31:48,917 [main] DEBUG s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:initTable(920)) - Using existing DynamoDB table
hwdev-steve-ireland-new in region eu-west-1 created Fri Jun 29 12:20:13 BST 2018
2018-06-29 12:31:48,918 [main] INFO s3guard.S3GuardTool
(S3GuardTool.java:initMetadataStore(270)) - Metadata store
DynamoDBMetadataStore{region=eu-west-1, tableName=hwdev-steve-ireland-new} is
initialized.
2018-06-29 12:31:48,954 [main] INFO s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:updateParameters(1200)) - Current table capacity is
read: 500, write: 100
2018-06-29 12:31:48,954 [main] INFO s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:updateParameters(1202)) - Changing capacity of
table to read: 10, write: 100
2018-06-29 12:31:49,041 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 1
2018-06-29 12:31:49,546 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #1
2018-06-29 12:31:49,588 [main] INFO s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:retryEvent(1251)) - Retrying ProvisionTable on
hwdev-steve-ireland-new: org.apache.hadoop.fs.s3a.AWSServiceIOException:
ProvisionTable on hwdev-steve-ireland-new:
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: KHFS1RER89LU8OAC6967B5U8S7VV4KQNSO5AEMVJF66Q9ASUAAJG): User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: KHFS1RER89LU8OAC6967B5U8S7VV4KQNSO5AEMVJF66Q9ASUAAJG)
2018-06-29 12:31:49,589 [main] DEBUG s3guard.DynamoDBMetadataStore
(DynamoDBMetadataStore.java:retryEvent(1252)) - Retrying ProvisionTable on
hwdev-steve-ireland-new
org.apache.hadoop.fs.s3a.AWSServiceIOException: ProvisionTable on
hwdev-steve-ireland-new:
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: KHFS1RER89LU8OAC6967B5U8S7VV4KQNSO5AEMVJF66Q9ASUAAJG): User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: KHFS1RER89LU8OAC6967B5U8S7VV4KQNSO5AEMVJF66Q9ASUAAJG)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateDynamoDBException(S3AUtils.java:389)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:181)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:260)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:317)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:256)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:188)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:210)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.provisionTable(DynamoDBMetadataStore.java:1080)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.provisionTableBlocking(DynamoDBMetadataStore.java:1095)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.updateParameters(DynamoDBMetadataStore.java:1204)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$SetCapacity.run(S3GuardTool.java:498)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:350)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:1489)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.main(S3GuardTool.java:1498)
Caused by: com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException:
User: arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: KHFS1RER89LU8OAC6967B5U8S7VV4KQNSO5AEMVJF66Q9ASUAAJG)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:2925)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:2901)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeUpdateTable(AmazonDynamoDBClient.java:2757)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.updateTable(AmazonDynamoDBClient.java:2733)
at
com.amazonaws.services.dynamodbv2.document.Table.updateTable(Table.java:371)
at
com.amazonaws.services.dynamodbv2.document.Table.updateTable(Table.java:463)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$provisionTable$7(DynamoDBMetadataStore.java:1083)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$2(Invoker.java:190)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
... 13 more
2018-06-29 12:31:49,591 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 2
2018-06-29 12:31:50,093 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #2
2018-06-29 12:31:50,136 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 3
2018-06-29 12:31:50,638 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #3
2018-06-29 12:31:50,684 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 4
2018-06-29 12:31:51,184 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #4
2018-06-29 12:31:51,228 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 5
2018-06-29 12:31:51,730 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #5
2018-06-29 12:31:51,774 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 6
2018-06-29 12:31:52,276 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #6
2018-06-29 12:31:52,319 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 7
2018-06-29 12:31:52,822 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #7
2018-06-29 12:31:52,865 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 8
2018-06-29 12:31:53,367 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #8
2018-06-29 12:31:53,411 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 9
2018-06-29 12:31:53,915 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #9
2018-06-29 12:31:53,959 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 10
2018-06-29 12:31:54,461 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #10
2018-06-29 12:31:54,505 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 11
2018-06-29 12:31:55,009 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #11
2018-06-29 12:31:55,052 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 12
2018-06-29 12:31:55,556 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #12
2018-06-29 12:31:55,605 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 13
2018-06-29 12:31:56,109 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #13
2018-06-29 12:31:56,153 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 14
2018-06-29 12:31:56,658 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #14
2018-06-29 12:31:56,701 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 15
2018-06-29 12:31:57,202 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #15
2018-06-29 12:31:57,245 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 16
2018-06-29 12:31:57,750 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #16
2018-06-29 12:31:57,792 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 17
2018-06-29 12:31:58,298 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #17
2018-06-29 12:31:58,342 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 18
2018-06-29 12:31:58,847 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #18
2018-06-29 12:31:58,889 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 19
2018-06-29 12:31:59,394 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #19
2018-06-29 12:31:59,438 [main] DEBUG s3a.S3AStorageStatistics
(S3AStorageStatistics.java:incrementCounter(63)) - ignored_errors += 1 -> 20
2018-06-29 12:31:59,943 [main] DEBUG s3a.Invoker
(Invoker.java:retryUntranslated(314)) - retry #20
org.apache.hadoop.fs.s3a.AWSServiceIOException: ProvisionTable on
hwdev-steve-ireland-new:
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: 2I6CMPEON2V56JKK5O56O8U8CJVV4KQNSO5AEMVJF66Q9ASUAAJG): User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: 2I6CMPEON2V56JKK5O56O8U8CJVV4KQNSO5AEMVJF66Q9ASUAAJG)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateDynamoDBException(S3AUtils.java:389)
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:181)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:260)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:317)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:256)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:188)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:210)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.provisionTable(DynamoDBMetadataStore.java:1080)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.provisionTableBlocking(DynamoDBMetadataStore.java:1095)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.updateParameters(DynamoDBMetadataStore.java:1204)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$SetCapacity.run(S3GuardTool.java:498)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:350)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:1489)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.main(S3GuardTool.java:1498)
Caused by: com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException:
User: arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: 2I6CMPEON2V56JKK5O56O8U8CJVV4KQNSO5AEMVJF66Q9ASUAAJG)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:2925)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:2901)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeUpdateTable(AmazonDynamoDBClient.java:2757)
at
com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.updateTable(AmazonDynamoDBClient.java:2733)
at
com.amazonaws.services.dynamodbv2.document.Table.updateTable(Table.java:371)
at
com.amazonaws.services.dynamodbv2.document.Table.updateTable(Table.java:463)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$provisionTable$7(DynamoDBMetadataStore.java:1083)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$2(Invoker.java:190)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
... 13 more
2018-06-29 12:31:59,988 [main] INFO util.ExitUtil
(ExitUtil.java:terminate(210)) - Exiting with status -1:
org.apache.hadoop.fs.s3a.AWSServiceIOException: ProvisionTable on
hwdev-steve-ireland-new:
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: 2I6CMPEON2V56JKK5O56O8U8CJVV4KQNSO5AEMVJF66Q9ASUAAJG): User:
arn:aws:sts::980678866538:assumed-role/stevel-s3-restricted/stevel is not
authorized to perform: dynamodb:UpdateTable on resource:
arn:aws:dynamodb:eu-west-1:980678866538:table/hwdev-steve-ireland-new (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code: AccessDeniedException; Request
ID: 2I6CMPEON2V56JKK5O56O8U8CJVV4KQNSO5AEMVJF66Q9ASUAAJG)
2018-06-29 12:31:59,990 [pool-2-thread-1] DEBUG s3a.S3AFileSystem
(S3AFileSystem.java:close(2465)) - Filesystem s3a://hwdev-steve-ireland-new is
closed
{code}
> s3guard set-capacity to not retry on an access denied exception
> ---------------------------------------------------------------
>
> Key: HADOOP-15573
> URL: https://issues.apache.org/jira/browse/HADOOP-15573
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Steve Loughran
> Priority: Minor
>
> when you call {{hadoop s3guard set-capacity}} with restricted access, you are
> (correctly) blocked by AWS, but the client keeps retrying. It should fail
> fast on a 400/AccessDenied
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
