[
https://issues.apache.org/jira/browse/FLINK-32878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser reopened FLINK-32878:
------------------------------------
> Add entropy to gcs path for better scalability
> ----------------------------------------------
>
> Key: FLINK-32878
> URL: https://issues.apache.org/jira/browse/FLINK-32878
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
> Affects Versions: 1.15.4, 1.16.2, 1.17.1
> Reporter: Jayadeep Jayaraman
> Assignee: Cheena
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.19.0
>
>
> Currently GCS is used as a backend for both checkpointing and sink. In both
> these cases the file names are sequential which causes hotspotting in GCS and
> results in HTTP 5XX status code.
>
> As per [GCS best
> practices]([https://cloud.google.com/storage/docs/request-rate)] it is
> advisable to spread out the object creation with random names close to the
> beginning of the file name.
>
> There is similar work done already for S3 as part of FLINK-9061
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
