[ https://issues.apache.org/jira/browse/AIRFLOW-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Kazarin updated AIRFLOW-5517: --------------------------------------- Description: We have an extra parameters in spark connection: {code:java} {"deploy-mode": "cluster", "spark-binary": "spark2-submit"} {code} After upgrade to 1.10.5 from 1.10.3 parameter 'spark-binary' in extra is no longer take effect. Broken after [this|https://github.com/apache/airflow/commit/8be59fb4edf0f2a132b13d0ffd1df0b8908191ab] commit, I think Workaround: call SparkSubmitOperator with spark_binary=None argument was: We have an extra parameters in spark connection: {code:java} {"deploy-mode": "cluster", "spark-binary": "spark2-submit"} {code} After upgrade to 1.10.5 from 1.10.3 parameter 'spark-binary' in extra is no longer take effect. Broken after [this|https://github.com/apache/airflow/commit/8be59fb4edf0f2a132b13d0ffd1df0b8908191ab] commit, I think Workaround: call SparkSubmitHook with spark_binary=None argument > SparkSubmitHook: spark-binary parameter no longer taken from extra > ------------------------------------------------------------------ > > Key: AIRFLOW-5517 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5517 > Project: Apache Airflow > Issue Type: Bug > Components: contrib > Affects Versions: 1.10.4, 1.10.5 > Reporter: Alexander Kazarin > Priority: Major > > We have an extra parameters in spark connection: > {code:java} > {"deploy-mode": "cluster", "spark-binary": "spark2-submit"} > {code} > After upgrade to 1.10.5 from 1.10.3 parameter 'spark-binary' in extra is no > longer take effect. > Broken after > [this|https://github.com/apache/airflow/commit/8be59fb4edf0f2a132b13d0ffd1df0b8908191ab] > commit, I think > Workaround: call SparkSubmitOperator with spark_binary=None argument -- This message was sent by Atlassian Jira (v8.3.4#803005)