[
https://issues.apache.org/jira/browse/FLINK-29501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17614120#comment-17614120
]
Chesnay Schepler commented on FLINK-29501:
------------------------------------------
??the Recale API is both disabled and broken at the moment??
In practice the entire mechanism doesn't exist. I'd ignore that there are still
some legacy API fragments around that we haven't removed IIRC exclusively so
that users don't hit a 404 when attempting to do a rescale.
??If you think it is already robust enough to support rescaling requests, we
can re-enable the rescale rest API and also add job vertex overrides to it??
It's definitely robust enough to be used in production I think.
IIRC Till had a prototype for adding a REST endpoint that adjusts the target
parallelism somewhere.
I don't think it worked on a per-vertex basis though; just a global parallelism
increase for all vertices (since in reactive mode everything scales uniformly
anyway). But that shouldn't be difficult to change.
> Allow overriding JobVertex parallelisms during job submission
> -------------------------------------------------------------
>
> Key: FLINK-29501
> URL: https://issues.apache.org/jira/browse/FLINK-29501
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Configuration, Runtime / REST
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Minor
> Labels: pull-request-available
>
> It is a common scenario that users want to make changes to the parallelisms
> in the JobGraph. For example, because they discover that the job needs more
> or less resources. There is the option to do this globally via the job
> parallelism. However, for fine-tuned jobs jobs with potentially many
> branches, tuning on the job vertex level is required.
> This is to propose a way such that users can apply a mapping \{{jobVertexId
> => parallelism}} before the job is submitted without having to modify the
> JobGraph manually.
> One way to achieving this would be to add an optional map field to the Rest
> API jobs endpoint. However, in deployment modes like the application mode,
> this might not make sense because users do not have control the rest endpoint.
> Similarly to how other job parameters are passed in the application mode, we
> propose to add the overrides as a configuration parameter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
