[
https://issues.apache.org/jira/browse/AIRFLOW-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906015#comment-16906015
]
ASF subversion and git services commented on AIRFLOW-5165:
----------------------------------------------------------
Commit a7a39003f97451347836acbc205d4ee2c35cbfd6 in airflow's branch
refs/heads/master from Maulik Soneji
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=a7a3900 ]
[AIRFLOW-5165] make number of dataproc masters configurable (#5781)
> Make Dataproc highly available
> ------------------------------
>
> Key: AIRFLOW-5165
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5165
> Project: Apache Airflow
> Issue Type: Bug
> Components: plugins
> Affects Versions: 1.10.1, 1.10.2, 1.10.3, 1.10.4
> Reporter: Maulik Soneji
> Assignee: Maulik Soneji
> Priority: Trivial
>
> Currently the number of master nodes in dataproc create operator is set to 1.
> There is no way to increase the number of master nodes in dataproc.
> Inorder to have HA dataproc, Google Cloud suggests to have 3 master nodes
> [https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability|https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability]
> The number of masters should be a config in DataprocCreateOperator so that
> the created dataproc is HA
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
