stats-dev commented on code in PR #2948:
URL: https://github.com/apache/iceberg-python/pull/2948#discussion_r2730096972
##########
pyiceberg/io/fsspec.py:
##########
@@ -205,7 +207,16 @@ def _s3(properties: Properties) -> AbstractFileSystem:
else:
anon = False
- fs = S3FileSystem(anon=anon, client_kwargs=client_kwargs,
config_kwargs=config_kwargs)
+ s3_fs_kwargs = {
+ "anon": anon,
+ "client_kwargs": client_kwargs,
+ "config_kwargs": config_kwargs,
+ }
+
+ if profile_name := get_first_property_value(properties, S3_PROFILE_NAME,
AWS_PROFILE_NAME):
+ s3_fs_kwargs["profile"] = profile_name
Review Comment:
Thanks for updating your feedback!
I performed local verification using a script without mocks against real my
AWS profiles. I made two IAM Users with different permission
profile(s3-only-allowed vs glue-only-allowed) and tested.
First, I verified this locally using real AWS credentials.
- S3 (fsspec): using `s3.profile-name`=s3-only-allowed successfully wrote to
a test S3 bucket. Switching to a different profile without S3 permissions
resulted in `AccessDenied`, confirming the profile is actually used.
- Glue: Glue catalog calls (ex, list_namespaces) succeeded only when using
the glue-only-allowed profile. Using a profile without Glue permissions
resulted in an authorization failure.
So, using distinct profiles (ex, s3-allowed vs glue-allowed) correctly
isolates permissions — the client succeeded only for the service allowed by the
respective profile.
Second, I confirmed that providing a non-existent profile correctly raises
botocore.exceptions.ProfileNotFound from both Glue and S3 clients.
Third, I checked that specific properties like s3.profile-name and
glue.profile-name correctly override client.profile-name in a real environment..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
