[
https://issues.apache.org/jira/browse/SPARK-35534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michal Luscon updated SPARK-35534:
----------------------------------
Description:
It seems there is currently a hard requirement on uploading files in k8s mode
via {{spark.kubernetes.file.upload.path config option that actually doesn't
make sense if your docker image already contains all of the necessary files.}}
I would propose a local:// extension to spark.kubernetes.file.upload.path, that
will point to the local folder containing files without their actual upload.
was:
It seems there is currently a hard requirement on uploading files in k8s mode
via {{spark.kubernetes.file.upload.path config option that actually doesn't
make sense if your docker image already contains all of the necessary files.}}
{{I would propose a local:// extension to
{color:#000000}spark.kubernetes.file.upload.path, that will point to the local
folder containing files without uploading.{color}
}}
{{}}
> Optional source codes uploading in k8s
> --------------------------------------
>
> Key: SPARK-35534
> URL: https://issues.apache.org/jira/browse/SPARK-35534
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 3.1.1
> Reporter: Michal Luscon
> Priority: Major
>
> It seems there is currently a hard requirement on uploading files in k8s mode
> via {{spark.kubernetes.file.upload.path config option that actually doesn't
> make sense if your docker image already contains all of the necessary files.}}
> I would propose a local:// extension to spark.kubernetes.file.upload.path,
> that will point to the local folder containing files without their actual
> upload.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
