[
https://issues.apache.org/jira/browse/FLINK-24159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17410920#comment-17410920
]
Feifan Wang commented on FLINK-24159:
-------------------------------------
Hi [~trohrmann], can you help confirm this problem ?
> document of entropy injection may mislead users
> -----------------------------------------------
>
> Key: FLINK-24159
> URL: https://issues.apache.org/jira/browse/FLINK-24159
> Project: Flink
> Issue Type: Improvement
> Components: Documentation, Runtime / Checkpointing
> Reporter: Feifan Wang
> Priority: Major
>
> FLINK-9061 incroduce entropy inject to s3 path for better scalability, but in
> document of
> [entropy-injection-for-s3-file-systems|https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/filesystems/s3/#entropy-injection-for-s3-file-systems]
> use a example with checkpoint directory
> "{color:#FF0000}s3://my-bucket/checkpoints/_entropy_/dashboard-job/{color}",
> with this configuration every checkpoint key will still start with constant
> checkpoints/ prefix which actually reduces scalability.
> Thanks to dmtolpeko for describing this issue in his blog (
> [flink-and-s3-entropy-injection-for-checkpoints
> |http://cloudsqale.com/2021/01/02/flink-and-s3-entropy-injection-for-checkpoints/]).
> h3. Proposal
> alter the checkpoint directory in document of
> [entropy-injection-for-s3-file-systems|https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/filesystems/s3/#entropy-injection-for-s3-file-systems]
> to
> "{color:#FF0000}s3://my-bucket/_entropy_/checkpoints/dashboard-job/{color}"
> (make entropy key at start of keys).
>
> If this proposal is appropriate, I am glad to submit a PR to modify the
> document here. Any other ideas for this ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
