[
https://issues.apache.org/jira/browse/AIRFLOW-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919118#comment-16919118
]
ASF subversion and git services commented on AIRFLOW-5165:
----------------------------------------------------------
Commit 5061e0418117ea75a6a2596f0471870a55d58012 in airflow's branch
refs/heads/v1-10-stable from Maulik Soneji
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=5061e04 ]
[AIRFLOW-5165] make number of dataproc masters configurable (#5781)
(cherry picked from commit a7a39003f97451347836acbc205d4ee2c35cbfd6)
> Make Dataproc highly available
> ------------------------------
>
> Key: AIRFLOW-5165
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5165
> Project: Apache Airflow
> Issue Type: Improvement
> Components: plugins
> Affects Versions: 1.10.1, 1.10.2, 1.10.3, 1.10.4
> Reporter: Maulik Soneji
> Assignee: Maulik Soneji
> Priority: Trivial
> Fix For: 1.10.5
>
>
> Currently the number of master nodes in dataproc create operator is set to 1.
> There is no way to increase the number of master nodes in dataproc.
> Inorder to have HA dataproc, Google Cloud suggests to have 3 master nodes
> [https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability|https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability]
> The number of masters should be a config in DataprocCreateOperator so that
> the created dataproc is HA
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
