dclim opened a new issue #7875: Make required S3 permissions consistent for pushing segments and task logs URL: https://github.com/apache/incubator-druid/issues/7875 ### Description The code for pushing task logs and segments to S3 deep storage is mostly shared, but the segment code path has some additional logic to possibly get the bucket owner (by fetching the bucket ACL) and set the ACL of the segment object to give the bucket owner full access. See: https://github.com/apache/incubator-druid/blob/master/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentPusher.java#L147 and compare with: https://github.com/apache/incubator-druid/blob/master/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3TaskLogs.java#L138 The additional ACL permissions required for pushing segments can lead to scenarios where the task logs succeed in being written to S3 but the segment PUT request fails with an Access Denied. There is no reason that the ACL on the segment should be set but not on the task log, and making this consistent will avoid future debugging headache. Additionally - consider having a fallback where if the putObject request fails after setting the ACL, it retries without setting the ACL, even if `druid.storage.disableAcl=false`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
