albertusk95 commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#issuecomment-571462963 > @tooptoop4 the tests pass only for the case when the connection information (host, port, conn_type, etc.) are stored in **database**. I tried this hook by storing the connection info as an **environment variable**. > > This failed because the URI parser returned irrelevant results for all types of cluster mode deployment. For instance, `URI=spark://host:port` will be parsed into `host:port` without the `spark://`. Obviously it returns this exception: > > ``` > airflow.exceptions.AirflowException: Cannot execute: [path/to/spark-submit, '--master', host:port, job_file.py] > ``` @tooptoop4
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
