[
https://issues.apache.org/jira/browse/SPARK-28245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alberto Andreotti updated SPARK-28245:
--------------------------------------
Description:
This works in PySpark,
from pyspark.sql import SparkSession
spark = SparkSession.builder \
.master('k8s://https://kubernetes.default') \
But in Scala,
val spark = SparkSession.builder()
.master("k8s://https://kubernetes.default";)
I get the following exception,
Caused by: org.apache.spark.SparkException: Could not parse Master URL:
'k8s://https://kubernetes.default'
> Problem to recognize Kubernetes cluster address from Scala
> ----------------------------------------------------------
>
> Key: SPARK-28245
> URL: https://issues.apache.org/jira/browse/SPARK-28245
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.4.0
> Reporter: Alberto Andreotti
> Priority: Major
>
> This works in PySpark,
>
> from pyspark.sql import SparkSession
> spark = SparkSession.builder \
> .master('k8s://https://kubernetes.default') \
> But in Scala,
> val spark = SparkSession.builder()
> .master("k8s://https://kubernetes.default";)
> I get the following exception,
> Caused by: org.apache.spark.SparkException: Could not parse Master URL:
> 'k8s://https://kubernetes.default'
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
