Re: GCS API call testing

2021-05-26 Thread Chamikara Jayalath
I think you need some support from the underlying GCS or HTTP client libraries to do this. All calls to GCS are done through gcsio library: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/gcsio.py Object insert requests to GCS are created here:

Re: GCS API call testing

2021-05-24 Thread Miguel Hernández Sandoval
Sorry, forgot to include the ticket [1] https://issues.apache.org/jira/browse/BEAM-11984 On Mon, May 24, 2021 at 7:54 AM Miguel Hernández Sandoval < rogelio.hernan...@wizeline.com> wrote: > Hi team, > I've been working on this ticket[1] where some API requests to GCS are > counted and kept as

GCS API call testing

2021-05-24 Thread Miguel Hernández Sandoval
Hi team, I've been working on this ticket[1] where some API requests to GCS are counted and kept as process-wide metrics. I was hoping you could give me some advice on how to test this kind of feature to test it using a real response from GCS instead of mocked responses and how to test it using a