flink 1.17 ???? s3 ???? ???????? ??????

Thu, 01 Jun 2023 01:40:17 -0700 

Hi All,


?????????? ???? S3 ???? ???????????????? ?? ???????? ??Ha storage ?? filesink?? 
?????? ?????????????????? ???????????? flink-s3-fs-presto 
?? flink-s3-fs-hadoop, ???? flink-s3-fs-presto ???? ?????????????? 
??URI?????? s3p://, flink-s3-fs-hadoop ???? ha storage ?? filesink ??URI 
?????? s3a://



?????? ???????? ?????????? standalone ???????????????? ???????? jm ?? tm?? 
?????????? zk ??


????flink??????????????????????????????????????









flink-conf.yml ???? s3 ????????????


# high-availability.zookeeper.quorum: localhost:2181
state.backend: rocksdb
# ?????? ?? s3://flink_uat ??????????????
state.checkpoints.dir: s3p://flink_uat/flink-checkpoints # 
"s3://<your-bucket&gt;/<endpoint&gt;"
state.savepoints.dir: s3p://flink_uat/flink-savepoints
state.backend.incremental: true
s3.access-key: BC560C10*********
s3.secret-key: Pitk2**********
s3.ssl.enabled: false
s3.path.style.access: true
s3.endpoint: xxxxx:80




high-availability.type: zookeeper
high-availability.zookeeper.quorum: xxxxxxx:2181,xxxxx:2181,xxxxx:2181
high-availability.zookeeper.path.root: /flink_uat
high-availability.cluster-id: /cluster_two # ????: ??????????????
high-availability.storageDir: s3a://flink_uat/recovery


flink-s3-fs-presto ??&nbsp;flink-s3-fs-hadoop ???????? 
????????????????????????????







???????? ????????????????&nbsp;


????????


Luke Yan

回复