[
https://issues.apache.org/jira/browse/FLINK-28915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17855243#comment-17855243
]
Robert Metzger commented on FLINK-28915:
----------------------------------------
Thanks a lot for the gist. This was the missing piece.
The difference between our approaches is the following: I'm loading S3 as a
plugin.
This is my job defintion:
{code}
apiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
metadata:
name: basic-example
spec:
image: flink:1.19.0-java11
flinkVersion: v1_19
mode: standalone
flinkConfiguration:
taskmanager.numberOfTaskSlots: "2"
s3.endpoint: http://minio:9000
s3.path.style.access: "true"
s3.access.key: root
s3.secret.key: password
podTemplate:
spec:
containers:
- name: flink-main-container
env:
- name: ENABLE_BUILT_IN_PLUGINS
value: "flink-s3-fs-presto-1.19.0.jar"
serviceAccount: flink
jobManager:
resource:
memory: "2048m"
cpu: 1
taskManager:
resource:
memory: "2048m"
cpu: 1
job:
jarURI: s3://dev/kafka-1.19.rocks.jar
parallelism: 1
upgradeMode: stateless
{code}
I'd suspect the plugins are not properly initialized when the code tries to
load the jar.
> Make application mode could support remote DFS schema(e.g. S3, OSS, HDFS,
> etc.)
> -------------------------------------------------------------------------------
>
> Key: FLINK-28915
> URL: https://issues.apache.org/jira/browse/FLINK-28915
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / Kubernetes, flink-contrib
> Reporter: hjw
> Assignee: Ferenc Csaky
> Priority: Major
> Labels: pull-request-available, stale-assigned
> Fix For: 1.19.0
>
>
> As the Flink document show , local is the only supported scheme in Native k8s
> deployment.
> Is there have a plan to support s3 filesystem? thx.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
