[
https://issues.apache.org/jira/browse/FLINK-36464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17888189#comment-17888189
]
Mate Czagany commented on FLINK-36464:
--------------------------------------
I have looked into this yesterday, and I think we could move the call to
ArtifactUtils.createMissingParents to HttpArtifactFetcher and FsArtifactFetcher
as advised in the other ticket. Or we could leave it like this, and don't crash
the program upon any IO exceptions during creating that directory, just log it.
WDYT [~fcsaky] ?
> Local artifact fetcher should not require base directory
> --------------------------------------------------------
>
> Key: FLINK-36464
> URL: https://issues.apache.org/jira/browse/FLINK-36464
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / Kubernetes
> Affects Versions: 2.0.0, 1.20.0
> Reporter: Fabian Paul
> Priority: Major
>
> Currently Flink tries to fetch all artifacts referenced by the job into a
> central base directory.
> [https://github.com/apache/flink/blob/6e7bf9b6f939cd1d183e416c0187ad24270e0bc1/flink-clients/src/main/java/org/apache/flink/client/program/artifact/ArtifactFetchManager.java#L76]
>
> In case, users only rely on artifacts that are resolvable locally this base
> directory is ignored.
> [https://github.com/apache/flink/blob/6e7bf9b6f939cd1d183e416c0187ad24270e0bc1/flink-clients/src/main/java/org/apache/flink/client/program/artifact/LocalArtifactFetcher.java#L35]
>
>
> Unfortunately, Flink always tries to provision the base directory leading to
> strange errors in case Flink has no write privileges on the base directory
> path.
> [https://github.com/apache/flink/blob/6e7bf9b6f939cd1d183e416c0187ad24270e0bc1/flink-clients/src/main/java/org/apache/flink/client/program/artifact/ArtifactFetchManager.java#L123]
>
> Since the base directory is not needed for local execution/local artifact
> fetcher, we should allow the creation of the base directory to be disabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
