Alexander Kazarin created AIRFLOW-5517:
------------------------------------------
Summary: SparkSubmitHook: spark-binary parameter no longer taken
from extra
Key: AIRFLOW-5517
URL: https://issues.apache.org/jira/browse/AIRFLOW-5517
Project: Apache Airflow
Issue Type: Bug
Components: contrib
Affects Versions: 1.10.5, 1.10.4
Reporter: Alexander Kazarin
We have an extra parameters in spark connection:
{code:java}
{"deploy-mode": "cluster", "spark-binary": "spark2-submit"}
{code}
After upgrade to 1.10.5 from 1.10.3 parameter 'spark-binary' in extra is no
longer take effect.
Broken after
[this|https://github.com/apache/airflow/commit/8be59fb4edf0f2a132b13d0ffd1df0b8908191ab]
commit, I think
Workaround: call SparkSubmitHook with spark_binary=None argument
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
