Holden Karau created SPARK-44529:
------------------------------------
Summary: Add a flag to resolve docker tags to hashes at launch time
Key: SPARK-44529
URL: https://issues.apache.org/jira/browse/SPARK-44529
Project: Spark
Issue Type: Improvement
Components: Kubernetes
Affects Versions: 3.5.0, 4.0.0
Reporter: Holden Karau
If you have an Spark docker tag (like say 3.3) you might want to update the
container but only for newly launched jobs not existing jobs. To allow this we
can resolve the tag to hash at launch time.
In some environments this may also offer a small performance improvement as it
saves K8s from having to re-resolve the tag with additional executor launches.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
