Bobby Wang created SPARK-45250:
----------------------------------
Summary: Support stage level task resource profile for yarn
cluster when dynamic allocation disabled
Key: SPARK-45250
URL: https://issues.apache.org/jira/browse/SPARK-45250
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.4.1
Reporter: Bobby Wang
[https://github.com/apache/spark/pull/37268] has introduced a new feature that
supports stage-level schedule task resource profile for standalone cluster when
dynamic allocation is disabled. It's really cool feature, especially for ML/DL
cases, more details can be found in that PR.
The problem here is that the feature is only available for standalone cluster
for now, but most users would also expect it can be used for other spark
clusters like yarn and k8s.
So I file this issue to track this task.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
