Jayadeep Jayaraman created FLINK-32878:
------------------------------------------
Summary: Add entropy to gcs path for better scalability
Key: FLINK-32878
URL: https://issues.apache.org/jira/browse/FLINK-32878
Project: Flink
Issue Type: Improvement
Components: Connectors / FileSystem
Affects Versions: 1.17.1, 1.16.2, 1.15.4
Reporter: Jayadeep Jayaraman
Currently GCS is used as a backend for both checkpointing and sink. In both
these cases the file names are sequential which causes hotspotting in GCS and
results in HTTP 5XX status code.
As per [GCS best
practices]([https://cloud.google.com/storage/docs/request-rate)] it is
advisable to spread out the object creation with random names close to the
beginning of the file name.
There is similar work done already for S3 as part of FLINK-9061
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
