Re: flink oss ha

Mon, 30 Aug 2021 20:02:11 -0700 

Hi

这个看上去更像是oss配置的问题，你能使用目前配置的 oss.endpoint，accessKeyId以及accessKeySecret 
结合ossutil等工具访问对应的 oss://bucket-logcenter/flink-state/flink-session-recovery 么？

祝好
唐云
________________________________
From: dker eandei <[email protected]>
Sent: Monday, August 30, 2021 12:36
To: [email protected] <[email protected]>
Subject: 回复: flink oss ha

您好:
     附件是使用oss作高可用时的报错，以下是启动flink时的脚本:

../bin/kubernetes-session.sh \
    -Dkubernetes.cluster-id=flink-session-1 \
    -Dkubernetes.container.image=test/flink:1.13.2-scala_2.12-oss \
    -Dkubernetes.container.image.pull-policy=Always \
    -Dkubernetes.namespace=flink-session \
    -Dkubernetes.service-account=flink-session-sa \
    -Dkubernetes.rest-service.exposed.type=ClusterIP \
    -Dtaskmanager.numberOfTaskSlots=6 \
    -Djobmanager.memory.process.size=1024m \
    -Dtaskmanager.memory.process.size=2048m \
    -Dkubernetes.jobmanager.cpu=1 \
    -Dkubernetes.taskmanager.cpu=2 \
    -Dfs.oss.endpoint="http://oss-xxxx.local"; \
    -Dfs.oss.accessKeyId="j0BAJxxxx" \
    -Dfs.oss.accessKeySecret="7mzTPiC4wxxxx" \
    
-Dhigh-availability=org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory
 \
    
-Dhigh-availability.storageDir=oss://bucket-logcenter/flink-state/flink-session-recovery
 \
    
-Dcontainerized.master.env.ENABLE_BUILT_IN_PLUGINS=flink-oss-fs-hadoop-1.13.2.jar
 \
    
-Dcontainerized.taskmanager.env.ENABLE_BUILT_IN_PLUGINS=flink-oss-fs-hadoop-1.13.2.jar

-----邮件原件-----
发件人: Yun Tang <[email protected]>
发送时间: 2021年8月30日 11:36
收件人: [email protected]
主题: Re: flink oss ha

Hi,
你好，图片无法加载，可以直接粘贴文字出来

祝好
唐云
________________________________
From: dker eandei <[email protected]>
Sent: Friday, August 27, 2021 14:58
To: [email protected] <[email protected]>
Subject: flink oss ha


您好:

        看文档OSS可以用作 FsStatebackend，那么Flink on k8s 
做高可用时，high-availability.storageDir可以配置成oss吗，我试了下，报以下错误:

[cid:[email protected]]



从 Windows 
版邮件<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgo.microsoft.com%2Ffwlink%2F%3FLinkId%3D550986&amp;data=04%7C01%7C%7Cd552b12a5a1f4a92aaee08d96b674cd0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637658913686219405%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=9Z9mxYxfkKqWfTCHYPThD3I97KFAFRMKINYExBuge80%3D&amp;reserved=0>发送

回复