[
https://issues.apache.org/jira/browse/FLINK-33424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17841454#comment-17841454
]
slankka commented on FLINK-33424:
---------------------------------
[[FLINK-29797][flink-yarn] Fix fs.default-scheme will accidentally cau… by
slankka · Pull Request #24733 · apache/flink
(github.com)|https://github.com/apache/flink/pull/24733] And please assign to
me.
> Resolved an issue in YarnClusterDescriptor where temporary files created
> locally by flink-conf.yaml are treated as remote files
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-33424
> URL: https://issues.apache.org/jira/browse/FLINK-33424
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission
> Affects Versions: 1.17.1
> Reporter: zhengzhili
> Priority: Major
> Fix For: 1.19.0
>
>
> After creating the flink-conf.yaml temporary file in
> YarnClusterDescriptor#startAppMaster, Through YarnApplicationFileUploader#
> registerSingleLocalResource registered local files,
> {quote}fileUploader.registerSingleLocalResource(
> flinkConfigKey,
> new Path(tmpConfigurationFile.getAbsolutePath()),
> "",
> LocalResourceType.FILE,
> true,
> true);
> {quote}
>
> In this case, if (Utils.isRemotePath(resourcePath.toString())) is used to
> determine whether the file is remote based on the path.
> {quote}if (Utils.isRemotePath(resourcePath.toString())) {
> final FileStatus fileStatus = fileSystem.getFileStatus(resourcePath);
> LOG.debug("Using remote file {} to register local resource",
> fileStatus.getPath());
> final YarnLocalResourceDescriptor descriptor =
> YarnLocalResourceDescriptor.fromFileStatus(
> key, fileStatus, LocalResourceVisibility.APPLICATION, resourceType);
> addToEnvShipResourceList(whetherToAddToEnvShipResourceList, descriptor);
> localResources.put(key, descriptor.toLocalResource());
> return descriptor;
> }
> {quote}
>
> The incoming tmpConfigurationFile#getAbsolutePath makes Utils. IsRemotePath
> considered remote file
> When FileSystem is S3AFileSystem, the getFileStatus method displays an error
> indicating that the file does not exist
>
>
> Incoming parameters should be new Path (tmpConfigurationFile.toURI ()) to the
> new Path (tmpConfigurationFile. getAbsolutePath ())
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
