Mathieu Leduc-Hamel created BEAM-13250:
------------------------------------------
Summary: GCSFileSystem is now facilitating injection of the gcs url
Key: BEAM-13250
URL: https://issues.apache.org/jira/browse/BEAM-13250
Project: Beam
Issue Type: Improvement
Components: io-py-gcp
Affects Versions: 2.33.0
Reporter: Mathieu Leduc-Hamel
Fix For: Not applicable
Inside `apache_beam.io.gcp.gcsfilesystem.GCSFileSystem` is directly
instantiating for every operations an instance of
`apache_beam.io.gcp.gcsio.GcsIO` without allowing anyone to make use of the
flexibility of instantiating with a custom client storage.
As you can see inside `GcsIO` there's an optional parameter called
`storage_client` and that would give us the possibility to overwrite the
storage client by example depending of the run level and potentially make use
of a Gcs emulator locally when testing the infrastructure.
Right now the only way we have to overwrite the gcs url is by monkey patching
the default storage which is not most clean way of doing it.
Potential solution:
If `GCSFileSystem` was transformed by example by adding a single class method
to generate every one of the `GcsIO` that would give us the possibility to
create a custom `GCSFileSystem` class, inheriting from it, without duplicating
all of it's code and to inject our desired url depending of the run level.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
