Feifan Wang created FLINK-24159:
-----------------------------------
Summary: document of entropy injection may mislead users
Key: FLINK-24159
URL: https://issues.apache.org/jira/browse/FLINK-24159
Project: Flink
Issue Type: Improvement
Components: Documentation, Runtime / Checkpointing
Reporter: Feifan Wang
FLINK-9061 incroduce entropy inject to s3 path for better scalability, but in
document of
[entropy-injection-for-s3-file-systems|https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/filesystems/s3/#entropy-injection-for-s3-file-systems]
use a example with checkpoint directory
"{color:#FF0000}s3://my-bucket/checkpoints/_entropy_/dashboard-job/{color}",
with this configuration every checkpoint key will still start with constant
checkpoints/ prefix which actually reduces scalability.
Thanks to dmtolpeko for describing this issue in his blog (
[flink-and-s3-entropy-injection-for-checkpoints
|http://cloudsqale.com/2021/01/02/flink-and-s3-entropy-injection-for-checkpoints/]).
h3. Proposal
alter the checkpoint directory in document of
[entropy-injection-for-s3-file-systems|https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/filesystems/s3/#entropy-injection-for-s3-file-systems]
to "{color:#FF0000}s3://my-bucket/_entropy_/checkpoints/dashboard-job/{color}"
(make entropy key at start of keys).
If this proposal is appropriate, I am glad to submit a PR to modify the
document here. Any other ideas for this ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
