[
https://issues.apache.org/jira/browse/BEAM-13250?focusedWorklogId=764780&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764780
]
ASF GitHub Bot logged work on BEAM-13250:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/May/22 17:41
Start Date: 01/May/22 17:41
Worklog Time Spent: 10m
Work Description: BigJerBD commented on PR #17455:
URL: https://github.com/apache/beam/pull/17455#issuecomment-1114294629
This could be done to simplify some tests that patches on the module level
```python
# in sdks/python/apache_beam/io/gcp/gcsfilesystem_test.py
@mock.patch('apache_beam.io.gcp.gcsfilesystem.gcsio') # this could be
removed
def test_create(self, mock_gcsio):
# Prepare mocks.
gcsio_mock = mock.MagicMock()
gcsfilesystem.gcsio.GcsIO = lambda: gcsio_mock # this would become:
self.fs.get_gcsio = lambda: gcsio_mock
...
```
Would it be enough?
it might looks irrelevant but at the very least sdk users using those
classes directly do not have to patch modules to override some gcsio.GcsIO
properties if considered necessary
Issue Time Tracking
-------------------
Worklog Id: (was: 764780)
Time Spent: 3h 20m (was: 3h 10m)
> GCSFileSystem is not facilitating injection of the gcs url
> ----------------------------------------------------------
>
> Key: BEAM-13250
> URL: https://issues.apache.org/jira/browse/BEAM-13250
> Project: Beam
> Issue Type: Improvement
> Components: io-py-gcp
> Affects Versions: 2.33.0
> Reporter: Mathieu Leduc-Hamel
> Priority: P2
> Fix For: Not applicable
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Inside `apache_beam.io.gcp.gcsfilesystem.GCSFileSystem` is directly
> instantiating for every operations an instance of
> `apache_beam.io.gcp.gcsio.GcsIO` without allowing anyone to make use of the
> flexibility of instantiating with a custom client storage.
> As you can see inside `GcsIO` there's an optional parameter called
> `storage_client` and that would give us the possibility to overwrite the
> storage client by example depending of the run level and potentially make use
> of a Gcs emulator locally when testing the infrastructure.
> Right now the only way we have to overwrite the gcs url is by monkey patching
> the default storage which is not most clean way of doing it.
>
> Potential solution:
> If `GCSFileSystem` was transformed by example by adding a single class method
> to generate every one of thse `GcsIO`, that would give us the possibility to
> create a custom `GCSFileSystem` class, inheriting from it, without
> duplicating all of it's code and to inject our desired url depending of the
> run level.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
