indrekj commented on issue #6442: Druid 0.13.0-incubating release notes URL: https://github.com/apache/incubator-druid/issues/6442#issuecomment-475591329 @dclim We're using s3 as deep storage and we had an incident when upgrading druid. Our s3 access policy is: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "S3:*", "Resource": [ "arn:aws:s3:::retracted-druid-dev/acceptance/*", "arn:aws:s3:::retracted-druid-dev/acceptance-indexer-logs/*" ], "Condition": {} }, { "Effect": "Allow", "Action": [ "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::retracted-druid-dev" ], "Condition": {} } ] } ``` Peons however started failing with: ``` Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: DA6A50FF8662CAC3; S3 Extended Request ID: UvfZA/YEDbKKlgCUS6u4Tk0e9DTyZkLJqY2iPgYyYOCy08gnLPTkbN1HPy7kyaDd0tkwABIve7A=) ... com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1150) ~[aws-java-sdk-s3-1.11.199.jar:?] at org.apache.druid.storage.s3.ServerSideEncryptingAmazonS3.getBucketAcl(ServerSideEncryptingAmazonS3.java:70) ~[?:?] at org.apache.druid.storage.s3.S3Utils.grantFullControlToBucketOwner(S3Utils.java:199) ~[?:?] ``` It didn't even retry. The peons were shutdown and the data was lost. We fixed it by adding `s3:GetBucketAcl` to the policy. This probably should also be mentioned in the release notes.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
