[
https://issues.apache.org/jira/browse/FLINK-30851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682520#comment-17682520
]
Vincent Chenal commented on FLINK-30851:
----------------------------------------
I agree that if we add s3 why not adding other ones as well.. I guess it's not
for me to decide
What do you mean by " I think it would be better to use the existing image for
custom docker creation" ?
Right now this is what I do but it does not feel right to have to copy the jar
from another image
{code:java}
FROM apache/flink:1.16.0-scala_2.12-java11 AS builder
FROM apache/flink-kubernetes-operator:1.3.1
RUN mkdir -p /opt/flink/plugins/s3-fs-presto
COPY --from=builder /opt/flink/plugins/s3-fs-presto/*
/opt/flink/plugins/s3-fs-presto/{code}
With flink images, as explained in the doc we must copy plugins from opt/
folder to plugins/ folder but in flink-kubernetes-operator image there's no
opt/ folder
> flink-kubernetes-operator - operator image should provide s3 support for
> jarUri
> -------------------------------------------------------------------------------
>
> Key: FLINK-30851
> URL: https://issues.apache.org/jira/browse/FLINK-30851
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Affects Versions: kubernetes-operator-1.3.1
> Reporter: Vincent Chenal
> Priority: Minor
>
> The docker image does not embed filesystem plugins.
> Neither in plugins/ neither in opt/ folders.
> I'd like to use jarUri this way:
> {code:java}
> apiVersion: flink.apache.org/v1beta1
> kind: FlinkSessionJob
> metadata:
> name: myjob
> spec:
> deploymentName: mydeployment
> job:
> jarURI: s3://somebucket/myjar.jar {code}
> But I'm getting this error:
> {code:java}
> Could not find a file system implementation for scheme 's3'. The scheme is
> directly supported by Flink through the following plugins:
> flink-s3-fs-hadoop, flink-s3-fs-presto. Please ensure that each plugin
> resides within its own subfolder within the plugins directory. See
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for
> more information. If you want to use a Hadoop file system for that scheme,
> please add the scheme to the configuration fs.allowed-fallback-filesystems.
> For a full list of supported file systems, please see
> https://nightlies.apache.org/flink/flink-docs-stable/ops/filesystems/. {code}
> I was able to make it work by putting flink 1.16 s3-presto-fs jar within the
> flink-kubernetes-operator image but it would make sense to have it natively.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
