uranusjr commented on pull request #15794: URL: https://github.com/apache/airflow/pull/15794#issuecomment-843941647
So the scheme is an issue for both (and only) `SparkSqlHook` and `SparkSubmitHook`. Given that `SparkSubmitHook` already didn't provide a scheme field prior to this PR, and nobody ever complained, I'm inclined to split this into a separate feature request and not deal with it in this PR, because YAGNI. And *if* this is ever requested, we have two options: 1. Explicitly document you can prepend the scheme in the `host` field in the connection. (This works as-is, but probably unintuitive.) 2. Add an extra field `scheme` in connection and apply it to both `SparkSqlHook` and `SparkSubmitHook`. (With documentation, of course.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
