[
https://issues.apache.org/jira/browse/FLINK-38210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18044695#comment-18044695
]
Zackary Naas edited comment on FLINK-38210 at 12/12/25 3:00 PM:
----------------------------------------------------------------
I think I see how to fix this - seems like the `convertToSpells`
[function|https://github.com/apache/flink/blob/7ba97419d200adef672b67c52c2b5a79f277cfda/flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/FlinkS3FileSystem.java#L304]
just needs to account for URIs that start with s3p or s3a and convert them to
s3. I can open a PR for this today.
was (Author: JIRAUSER311614):
I think I see how to fix this - seems like the `convertToSpells`
[function|https://github.com/apache/flink/blob/7ba97419d200adef672b67c52c2b5a79f277cfda/flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/FlinkS3FileSystem.java#L304]
just needs to account for URIs that start with s3p or s3a and convert them to
s3. I can open a PR for this today.
> s5cmd somtimes fails due to path starting with s3p://
> -----------------------------------------------------
>
> Key: FLINK-38210
> URL: https://issues.apache.org/jira/browse/FLINK-38210
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
> Affects Versions: 2.1.0
> Reporter: Grzegorz Liter
> Priority: Major
>
> s5cmd fails to startup job due to path starting with s3p:// which is
> recommended way when both Hadoop and Presto implementations are in use.
>
> {code:java}
> Caused by: java.io.IOException: Failed to cast s5cmd spell
> [/opt/flink/opt/s5cmd/s5cmd -r 0 run] [exit code = 1] [cfg:
> S5CmdConfiguration{path='/opt/flink/opt/s5cmd/s5cmd', args=[-r, 0],
> accessArtifact='null', secretArtifact='null', endpoint='null'}] maybe due
> to:ERROR "cp
> s3p://bucket/checkpoints/9d2739a593435f30a044965427d3d53e/shared/6c726cc2-f51c-42e3-8647-f6de0e32040d
>
> /tmp/tm_dema-flink-stream-processor-taskmanager-1-2/tmp/job_9d2739a593435f30a044965427d3d53e_op_WindowOperator_4ddf880b77ad626da96b1c364685b172__1_6__uuid_96cd1a7e-52ba-46f9-8c3d-4674f4d573f5/942ab49c-d8ae-4d3e-b5df-0d0c700705c9/000019.sst":
> s3 url should start with "s3://" {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
