Fabian Paul created FLINK-36464:
-----------------------------------
Summary: Local artifact fetcher should not require base directory
Key: FLINK-36464
URL: https://issues.apache.org/jira/browse/FLINK-36464
Project: Flink
Issue Type: Improvement
Components: Deployment / Kubernetes
Affects Versions: 1.20.0, 2.0.0
Reporter: Fabian Paul
Currently Flink tries to fetch all artifacts referenced by the job into a
central base directory.
[https://github.com/apache/flink/blob/6e7bf9b6f939cd1d183e416c0187ad24270e0bc1/flink-clients/src/main/java/org/apache/flink/client/program/artifact/ArtifactFetchManager.java#L76]
In case, users only rely on artifacts that are resolvable locally this base
directory is ignored.
[https://github.com/apache/flink/blob/6e7bf9b6f939cd1d183e416c0187ad24270e0bc1/flink-clients/src/main/java/org/apache/flink/client/program/artifact/LocalArtifactFetcher.java#L35]
Unfortunately, Flink always tries to provision the base directory leading to
strange errors in case Flink has no write privileges on the base directory
path.
[https://github.com/apache/flink/blob/6e7bf9b6f939cd1d183e416c0187ad24270e0bc1/flink-clients/src/main/java/org/apache/flink/client/program/artifact/ArtifactFetchManager.java#L123]
Since the base directory is not needed for local execution/local artifact
fetcher, we should allow the creation of the base directory to be disabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
