Hi Peter, I've dealt with the cross-account delegation issues in the past (with no relation to Flink) and got into the same ownership problems (accounts can't access data, account A 'loses' access to it's own data).
My 2-cents are that: - The account that produces the data (A) should be the ONLY OWNER of that data. - The policy to access the data should be managed in ONE place only, the producing account (A). - If you wish to expose access to your data to other accounts (B, C, D), the best approach would be to: - In account A - Create a policy that defines the access you wish to expose. For example: read access to specific bucket & path: { > "Version": "2012-10-17", > "Statement": [ > { > "Effect": "Allow", > "Action": [ > "s3:GetObject", > "s3:ListBucket" > ], > "Resource": [ > "arn:aws:s3:::bucket-name", > "arn:aws:s3:::bucket-name/*" > ] > } > ] > } > > - In account A - Create a role and define which accounts you allow to AssumeRole (this let's you control if ALL or specific users of the other account should access the data): { "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam::account-B:root", "arn:aws:iam::account-C:root", "arn:aws:iam::account-D:root" ] }, "Action": "sts:AssumeRole" } ] } - In account A - attach the policy to the role. - In other accounts - THEY control which users have access to the data by allowing AssumeRole permissions to the role above from account A. This could be unrestricted (by *) or restricted to a specific role.: { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "arn:aws:iam::<account-A-id>:role/external-access-role" } ] } Now when a user AssumeRole to that external-access-role role, it will be granted the specified access without playing around with ownership configurations. Hope this helps, Rafi On Fri, May 22, 2020 at 11:39 PM Peter Groesbeck <peter.groesb...@gmail.com> wrote: > Hi, > > I'm using Flink StreamingFileSink running in one AWS account (A) to > another (B). I'm also leveraging a SecurityConfiguration in the CFN to > assume a role in account B so that when I write there the files are owned > by account B which then in turn allows account B to delegate to other AWS > accounts (C and D). The reason these files must be owned by the other > account is because AWS doesn't support cross account delegation: > > https://docs.aws.amazon.com/AmazonS3/latest/dev/example-walkthroughs-managing-access-example4.html > > SecurityConfiguration: > Type: AWS::EMR::SecurityConfiguration > Properties: > Name: String > SecurityConfiguration: > AuthorizationConfiguration: > EmrFsConfiguration: > RoleMappings: > - Role: arn:aws:iam::<B-account>:role/EMR_EC2_DefaultRole > IdentifierType: Prefix > Identifiers: > - s3://my-bucket/prefix/ > - Role: arn:aws:iam::<B-account>:role/EMR_DefaultRole > IdentifierType: Prefix > Identifiers: > - s3://my-bucket/prefix/ > > > I've referenced this in my Cluster block as well: > > ReleaseLabel: !Ref ReleaseLabel > SecurityConfiguration: !Ref SecurityConfiguration > ScaleDownBehavior: TERMINATE_AT_TASK_COMPLETION > > For some reason the files are still owned by account A. It seems like > Flink is using the old Hadoop FS implementation instead of EMRFS which > should (I believe) grant the proper ownership so that bucket permissions > can apply to the written objects and in turn delegate read permissinos to > accounts C, D ect. > > Any help would be greatly appreciated. > > Thanks, > Peter >