gemini-code-assist[bot] commented on code in PR #38751:
URL: https://github.com/apache/beam/pull/38751#discussion_r3328917599
##########
sdks/python/apache_beam/io/gcp/gcsfilesystem.py:
##########
@@ -43,13 +52,32 @@ class GCSFileSystem(FileSystem):
"""A GCS ``FileSystem`` implementation for accessing files on GCS.
"""
- CHUNK_SIZE = gcsio.MAX_BATCH_OPERATION_SIZE # Chuck size in batch operations
GCS_PREFIX = 'gs://'
def __init__(self, pipeline_options):
super().__init__(pipeline_options)
self._pipeline_options = pipeline_options
+ @staticmethod
+ def _get_gcsio_module():
+ """Return the ``gcsio`` module, raising ImportError if it is unavailable.
+
+ ``gcsio`` is imported lazily (see the module-level import) so that this
+ filesystem can be looked up without the gcp extra installed. The dependency
+ is only required when the filesystem is actually used.
+ """
+ if gcsio is None:
+ raise ImportError(
+ 'Could not import apache_beam.io.gcp.gcsio. This usually means the '
+ 'gcp dependencies are not installed. Install them with: '
+ 'pip install apache-beam[gcp]')
+ return gcsio
+
+ @property
+ def CHUNK_SIZE(self):
+ """Chunk size in batch operations."""
+ return self._get_gcsio_module().MAX_BATCH_OPERATION_SIZE
Review Comment:
Changing `CHUNK_SIZE` from a class attribute to an instance `@property` is a
breaking change for any external code or subclasses that access
`GCSFileSystem.CHUNK_SIZE` directly on the class (which is a common pattern for
capitalized constants).
If class-level access needs to be preserved, you can implement a simple
class property descriptor to support both class and instance-level access:
```python
class classproperty(object):
def __init__(self, fget):
self.fget = fget
def __get__(self, instance, owner):
return self.fget(owner)
```
And then decorate `CHUNK_SIZE` with `@classproperty` (using `cls` instead of
`self`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
