> Can you check if your proposed implementation and defaults work well in the
> case where the remote artifact fetching for session clusters is stalling
> indefinetly? Will we see the right error message on the client ("artifact
> fetching timed out") instead of ("job submission timed out")?
Yes, clients will get an error that explains the fetch timed out, and
identifying the configured timeout that applied. I've updated the FLIP document
to clarify that the user.artifacts.fetch-timeout value will be applied to the
fetch only, not running the job or generating the JobGraph.
For HTTP/HTTPS artifacts, this timeout will apply to both the request (how long
before the submitter receives the timeout error notification) and the fetch
itself.
For other schemes (e.g. s3:// or abfs://), the timeout only applies to the
request - it controls when the submitter receives an error telling it that the
fetch timeout was exceeded. The actual fetch will continue until the filesystem
plugin's own config tells it to give up.
I've also added a couple of other related controls to tighten up the impact of
handling fetch requests, bringing the proposed controls to:
user.artifacts.job-jar - identifying the artifact to fetch
web.submit.jar-uri.enable - deciding whether the cluster should accept fetch
requests
user.artifacts.fetch-timeout - limiting how long to spend on a fetch
user.artifacts.max-concurrent-fetches - limiting the number of fetches in
flight
user.artifacts.max-size - limiting the size of artifacts to fetch
Kind regards
D
--
dalelane.co.uk
On Tuesday, 8 September 2026 at 10:28, Robert Metzger <[email protected]>
wrote:
> Overall, I'm +1 on the idea.
>
> The only thing that I'm not fully decided on is the fact that the job
> submission would be blocking on the remote download. We could immdiately
> return the submission call and have the job in `INITIALIZING` state, if
> everything is ok, it would transition into CREATED, RUNNING, etc.
> If not, it would go into FAILED, and clients would have to fetch the error
> from the job's exception endpoint.
>
> It will make client implementations more difficult .. but the kubernetes
> operator is anyways polling already, so that should be fine.
>
> A long time ago, I worked on
> https://issues.apache.org/jira/browse/FLINK-16866. Where a
> low `web.timeout` caused issues with slow job submissions. Can you check if
> your proposed implementation and defaults work well in the case where the
> remote artifact fetching for session clusters is stalling indefinetly? Will
> we see the right error message on the client ("artifact fetching timed
> out") instead of ("job submission timed out")?
>
>
> On Mon, Aug 24, 2026 at 12:41 AM Dale Lane <[email protected]>
> wrote:
>
> > I'd like to start a discussion on
> > FLIP-XXX : Remote Artifact Fetch for Session-Mode Job Submission
> >
> > https://docs.google.com/document/d/1drZkXU875-j7lsyJaLp-VkVG_1PRietlfO2ETLeEWJc/edit?usp=sharing
> >
> > This FLIP proposes extending Flink's existing remote-artifact-fetch
> > capability to session-mode job submission, so that a job can be submitted
> > by URI, with the Job Manager performing the fetch itself, the same way an
> > Application Mode cluster already does at start-up.
> >
> > Looking forward to discussion - please let me know what you think!
> >
> > Kind regards
> >
> > Dale
> > --
> > dalelane.co.uk
> >
> >
>
