[
https://issues.apache.org/jira/browse/AIRFLOW-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055914#comment-17055914
]
ASF subversion and git services commented on AIRFLOW-6542:
----------------------------------------------------------
Commit 6c39a3bf97414ba2438669894db65c36ccbeb61a in airflow's branch
refs/heads/master from roitvt
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=6c39a3b ]
[AIRFLOW-6542] Add spark-on-k8s operator/hook/sensor (#7163)
> sparkKubernetes operator for
> https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
> -----------------------------------------------------------------------------------------
>
> Key: AIRFLOW-6542
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6542
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Affects Versions: 1.10.7
> Reporter: Roi Teveth
> Priority: Minor
>
> hi,
> we working on spark on Kubernetes POC using the google cloud platform
> spark-k8s-operator
> [https://github.com/GoogleCloudPlatform/spark-on-k8s-operator] and haven't
> found native airflow integration for it so we wrote one:
> kubernetes_hook which create and get kuberenetes crd object
> spark_kubernetes_operator which sends sparkapplication crd to kubernetes
> cluster
> spark_kubernetes_sensor which poke sparkapplication state
> operator example with spark-pi
> application:[https://github.com/roitvt/airflow-spark-on-k8s-operator]
> I'll be glad to contribute our operator to airflow contrib
> Thanks
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
