[
https://issues.apache.org/jira/browse/AIRFLOW-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16829063#comment-16829063
]
Ash Berlin-Taylor commented on AIRFLOW-3863:
--------------------------------------------
This would probably be a welcome addition - could you ask on the mailing list
(https://airflow.apache.org/project.html#resources-links) to see what features
others would want.
> Make SparkSubmitHook capable of executing spark-submit through SSH Connection
> -----------------------------------------------------------------------------
>
> Key: AIRFLOW-3863
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3863
> Project: Apache Airflow
> Issue Type: Improvement
> Components: hooks, operators
> Reporter: Felipe Lolas
> Assignee: Felipe Lolas
> Priority: Trivial
>
> Hi!
> I want to add a functionality in SparkSubmitHook; connect to a remote server
> though SSH and execute spark-submit. This would be great when working with
> multiple spark clusters with an edge node for each cluster and installing a
> airflow's worker onto edge node is not possibly.
> Im currently implementing, but I wanna hear some thoughts from the airflow
> community about the solution and if should be commited or not!
> Cheers!
> Felipe
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
