MonkeyCanCode commented on issue #144: URL: https://github.com/apache/polaris/issues/144#issuecomment-2303630952
@sfc-gh-adlee So what I refers to as working was https://github.com/apache/polaris/blob/main/docker-compose-jupyter.yml However, I did found one issue there with the demo code via spark and had raise https://github.com/apache/polaris/pull/176 to address this problem. However, you shouldn't get NPE for that. If you use the fixed version of the demo code in the above PR, that should overcome the problem (tested tonight with a fresh env). Then back to your question regarding AWS ARN and S3 access. Here are what you will needed on AWS side: 1. S3 bucket 2. IAM policy that defined access to the S3 bucket 3. IAM role that is associated with the IAM policy created in item 2 with self trust relation when using same AWS account (aka self-assuming (self-trusting) IAM role) 4. IAM user that is associated with IAM role created in item 3 Here is the reasoning, when Polaris is accessing S3, it is assuming assume role instead of using `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` directly. It is using the provided `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for user created in item 4 to invoke assumeRole which then use AWS Security Token Service (AWS STS) to generate temporary token to assume that role created in item 3. Here is the code that is doing this: https://github.com/apache/polaris/blob/e89ff19dc18c3cd5a357461222a52b91e682f5d0/polaris-core/src/main/java/io/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@polaris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
