[ 
https://issues.apache.org/jira/browse/SPARK-62036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Zhilko updated SPARK-62036:
--------------------------------
    Description: 
On multi-zone clusters, the driver and executors of one application often land 
in different zones. Traffic between zones (driver-executor RPC, 
executor-executor block transfer, and push-based remote shuffle services such 
as Apache Celeborn) is slower and, on public clouds, billed per GB.

Today users work around this with executor {{podAffinity}} to the driver's 
zone. Shuffle-service settings that depend on the zone, such as Celeborn's 
{{spark.celeborn.tags.tagsExpr}}, can't be set that way: the zone is only known 
after the driver is scheduled, so each application has to look up its own 
node's zone at runtime.

Proposal: the operator picks one zone per application attempt before it creates 
the driver.
* Zone choice: a {{spark.operator/zone}} annotation pins an explicit zone. 
Otherwise the operator picks the configured zone with the fewest active pinned 
applications.
* Placement: it adds a node selector on the configured topology key (default 
{{topology.kubernetes.io/zone}}) to the driver and executors, using 
{{spark.kubernetes.\{driver,executor\}.node.selector.*}}.
* Spark conf: it optionally fills operator-configured conf templates with the 
chosen zone, e.g. {{spark.celeborn.tags.tagsExpr=az-\{zone\}}}. Keys the 
application sets itself are left alone.
* Status: it records the zone in the current attempt's status. Rebuilding the 
spec keeps the same zone, and a restart picks again.
* Scope: it is off by default. Applications queued through Kueue are left to 
Kueue.

> Support pinning a SparkApplication attempt to a single topology zone
> --------------------------------------------------------------------
>
>                 Key: SPARK-62036
>                 URL: https://issues.apache.org/jira/browse/SPARK-62036
>             Project: Spark
>          Issue Type: Improvement
>          Components: Kubernetes
>    Affects Versions: kubernetes-operator-2.0.0
>            Reporter: Alex Zhilko
>            Priority: Major
>
> On multi-zone clusters, the driver and executors of one application often 
> land in different zones. Traffic between zones (driver-executor RPC, 
> executor-executor block transfer, and push-based remote shuffle services such 
> as Apache Celeborn) is slower and, on public clouds, billed per GB.
> Today users work around this with executor {{podAffinity}} to the driver's 
> zone. Shuffle-service settings that depend on the zone, such as Celeborn's 
> {{spark.celeborn.tags.tagsExpr}}, can't be set that way: the zone is only 
> known after the driver is scheduled, so each application has to look up its 
> own node's zone at runtime.
> Proposal: the operator picks one zone per application attempt before it 
> creates the driver.
> * Zone choice: a {{spark.operator/zone}} annotation pins an explicit zone. 
> Otherwise the operator picks the configured zone with the fewest active 
> pinned applications.
> * Placement: it adds a node selector on the configured topology key (default 
> {{topology.kubernetes.io/zone}}) to the driver and executors, using 
> {{spark.kubernetes.\{driver,executor\}.node.selector.*}}.
> * Spark conf: it optionally fills operator-configured conf templates with the 
> chosen zone, e.g. {{spark.celeborn.tags.tagsExpr=az-\{zone\}}}. Keys the 
> application sets itself are left alone.
> * Status: it records the zone in the current attempt's status. Rebuilding the 
> spec keeps the same zone, and a restart picks again.
> * Scope: it is off by default. Applications queued through Kueue are left to 
> Kueue.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to