[
https://issues.apache.org/jira/browse/AIRFLOW-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601539#comment-16601539
]
Apache Spark commented on AIRFLOW-1409:
---------------------------------------
User 'lvicentesanchez' has created a pull request for this issue:
https://github.com/apache/incubator-airflow/pull/2451
> SparkSubmitHook doesn't generate a valid master URL
> ---------------------------------------------------
>
> Key: AIRFLOW-1409
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1409
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Luis Angel Vicente Sanchez
> Priority: Major
>
> If you configure the connection using an ENV variable (and possibly through
> the UI), SparkSubmitHook is ignoring the scheme and doesn't produce valid
> master URL. Example, this ENV variable,
> AIRFLOW_CONN_SPARK_CLUSTER=spark://spark-master:7077
> will produce the following master URL:
> spark-master:7077
> This URL is not valid and the job submission fails. The issue is how the url
> is parsed to create a connection, I have a partial fix working and I will
> submit a PR to github soon.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
