pingtimeout opened a new issue, #3243: URL: https://github.com/apache/polaris/issues/3243
### Describe the bug While running benchmarks with much longer namespace names, I got the following error while creating a table: ``` Failed to get subscoped credentials: Packed policy consumes 118% of allotted space, please use smaller policy. ``` The path that caused this issue is composed by 14 nested namespaces, each with a 32 characters name. The problem is located in https://github.com/apache/polaris/blob/31c7784630889d8fb6f0156610b8e59a4021f1c3/polaris-core/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java#L175. It results in the following STS policy: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::my-very-long-s3-bucket/catalog/f5a4d86558ed1f7fddec42ce11d8ee3a/e3fa7fd44796b949fce728cb334f3c15/f031eb9dc709150a3e1e9a76e9af550a/5598924229adee97260cda483d70674c/0857633d48470f538d1fc4cdc789c753/c4c06b9b673a710f7ce865690ff8797b/9594c7f9e8a1cd28054ff71b933fdc3b/0018c223e978d13aeec87488bc333c70/d4c484ca41745b69286372286f3f30aa/362f8d6579892af8a9c5f38b4e664b07/f28f3a8b846271c68bdbc09992d88d28/915107073e394e3471173ec4633137fa/5eac9dac6dd84afcb35f4805af9d8b34/e1a24630cbf9ced9dffcc123b70b2e43/ab4ffa55f688360e0c12aef543c18351/*" }, { "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::my-very-long-s3-bucket", "Condition": { "StringLike": { "s3:prefix": "catalog/f5a4d86558ed1f7fddec42ce11d8ee3a/e3fa7fd44796b949fce728cb334f3c15/f031eb9dc709150a3e1e9a76e9af550a/5598924229adee97260cda483d70674c/0857633d48470f538d1fc4cdc789c753/c4c06b9b673a710f7ce865690ff8797b/9594c7f9e8a1cd28054ff71b933fdc3b/0018c223e978d13aeec87488bc333c70/d4c484ca41745b69286372286f3f30aa/362f8d6579892af8a9c5f38b4e664b07/f28f3a8b846271c68bdbc09992d88d28/915107073e394e3471173ec4633137fa/5eac9dac6dd84afcb35f4805af9d8b34/e1a24630cbf9ced9dffcc123b70b2e43/ab4ffa55f688360e0c12aef543c18351/*" } } }, { "Effect": "Allow", "Action": "s3:GetBucketLocation", "Resource": "arn:aws:s3:::my-very-long-s3-bucket" }, { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:GetObjectVersion" ], "Resource": "arn:aws:s3:::my-very-long-s3-bucket/catalog/f5a4d86558ed1f7fddec42ce11d8ee3a/e3fa7fd44796b949fce728cb334f3c15/f031eb9dc709150a3e1e9a76e9af550a/5598924229adee97260cda483d70674c/0857633d48470f538d1fc4cdc789c753/c4c06b9b673a710f7ce865690ff8797b/9594c7f9e8a1cd28054ff71b933fdc3b/0018c223e978d13aeec87488bc333c70/d4c484ca41745b69286372286f3f30aa/362f8d6579892af8a9c5f38b4e664b07/f28f3a8b846271c68bdbc09992d88d28/915107073e394e3471173ec4633137fa/5eac9dac6dd84afcb35f4805af9d8b34/e1a24630cbf9ced9dffcc123b70b2e43/ab4ffa55f688360e0c12aef543c18351/*" } ] } ``` I don't think we can eliminate the STS limitation at all. But I think Polaris could be improved to better deal with this error. For example: * The returned HTTP code (400) is correct (Bad Request) but the associated error message should be clearer for the user. The current one may not be understandable for everyone. * During namespace creation, no check is performed on the total length of the path. It might be worth adding so that the failure happens earlier. * Parts of the STS policy could be merged together (AFAICT): the `s3:PutObject` and `s3:DeleteObject` statements could be merged with the `s3:GetObject` and `s3:GetObjectVersion` statements. This would allow Polaris to support longer paths, while not removing the maximum path limitation entirely though. ### To Reproduce * Create a deeply nested namespace structure with long length (e.g. `f5a4d86558ed1f7fddec42ce11d8ee3a/e3fa7fd44796b949fce728cb334f3c15/f031eb9dc709150a3e1e9a76e9af550a/5598924229adee97260cda483d70674c/0857633d48470f538d1fc4cdc789c753/c4c06b9b673a710f7ce865690ff8797b/9594c7f9e8a1cd28054ff71b933fdc3b/0018c223e978d13aeec87488bc333c70/d4c484ca41745b69286372286f3f30aa/362f8d6579892af8a9c5f38b4e664b07/f28f3a8b846271c68bdbc09992d88d28/915107073e394e3471173ec4633137fa/5eac9dac6dd84afcb35f4805af9d8b34/e1a24630cbf9ced9dffcc123b70b2e43/ab4ffa55f688360e0c12aef543c18351`) * Create a table in the last namespace ### Actual Behavior _No response_ ### Expected Behavior _No response_ ### Additional context _No response_ ### System information _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
