MansiSingh17 opened a new pull request, #37674:
URL: https://github.com/apache/beam/pull/37674
Make GCS filesystem lookup lazy to match S3 behavior (fixes #37445)
FileSystems.get_filesystem() previously raised a ValueError for gs:// paths
when GCP dependencies were not installed, while s3:// paths returned a
filesystem object and deferred dependency validation until usage time. This PR
aligns GCS behavior with S3.
Changes:
- Removed top-level from apache_beam.io.gcp import gcsio import, replaced
with try/except ImportError for backward compatibility with existing mocks
- Made _gcsIO() method lazy load gcsio at call time
- Converted CHUNK_SIZE from a class-level variable to a lazy @property so it
doesn't require GCP deps at import time
- Fixed report_lineage to use lazy import of gcsio
- Added test_get_filesystem_does_not_require_gcp_extra to verify the fix
Testing:
pytest -q sdks/python/apache_beam/io/gcp/gcsfilesystem_test.py
18 passed, 1 failed (test_lineage fails locally due to missing
google.api_core — was already skipped on master before this change)
fixes #37445
Update CHANGES.md with noteworthy changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
