kevinjqliu commented on code in PR #2948:
URL: https://github.com/apache/iceberg-python/pull/2948#discussion_r2725691803
##########
pyiceberg/io/fsspec.py:
##########
@@ -205,7 +207,13 @@ def _s3(properties: Properties) -> AbstractFileSystem:
else:
anon = False
- fs = S3FileSystem(anon=anon, client_kwargs=client_kwargs,
config_kwargs=config_kwargs)
+ session = None
+ if profile_name := get_first_property_value(properties, S3_PROFILE_NAME,
AWS_PROFILE_NAME):
+ from aiobotocore.session import AioSession
+
+ session = AioSession(profile=profile_name)
Review Comment:
Passing in the `AioSession` here will override the internal session object
From the docs:
https://s3fs.readthedocs.io/en/latest/api.html#s3fs.core.S3FileSystem
"""
session (aiobotocore AioSession object to be used for all connections.) –
**This session will be used inplace of creating a new session inside
S3FileSystem**. For example: aiobotocore.session.AioSession(profile=’test_user’)
"""
I think we can pass in profile name as kwarg to `S3FileSystem`
The kwarg will be passed into the internal AioSession object
https://github.com/fsspec/s3fs/blob/56402cd2565c5fa2aa84020c716560b3db27e8cd/s3fs/core.py#L563-L565
WDYT?
##########
pyiceberg/catalog/glue.py:
##########
Review Comment:
could you add this new config to the docs
https://py.iceberg.apache.org/configuration/#s3
https://py.iceberg.apache.org/configuration/#glue-catalog
##########
pyiceberg/catalog/glue.py:
##########
@@ -329,7 +329,7 @@ def __init__(self, name: str, client:
Optional["GlueClient"] = None, **propertie
retry_mode_prop_value = get_first_property_value(properties,
GLUE_RETRY_MODE)
session = boto3.Session(
- profile_name=properties.get(GLUE_PROFILE_NAME),
+ profile_name=properties.get(GLUE_PROFILE_NAME,
properties.get(AWS_PROFILE_NAME)),
Review Comment:
```suggestion
profile_name=get_first_property_value(properties,
GLUE_PROFILE_NAME, AWS_PROFILE_NAME),
```
we have a helper function for this 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
