[
https://issues.apache.org/jira/browse/AIRFLOW-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906014#comment-16906014
]
ASF GitHub Bot commented on AIRFLOW-5165:
-----------------------------------------
mik-laj commented on pull request #5781: [AIRFLOW-5165] make number of dataproc
masters configurable
URL: https://github.com/apache/airflow/pull/5781
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Make Dataproc highly available
> ------------------------------
>
> Key: AIRFLOW-5165
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5165
> Project: Apache Airflow
> Issue Type: Bug
> Components: plugins
> Affects Versions: 1.10.1, 1.10.2, 1.10.3, 1.10.4
> Reporter: Maulik Soneji
> Assignee: Maulik Soneji
> Priority: Trivial
>
> Currently the number of master nodes in dataproc create operator is set to 1.
> There is no way to increase the number of master nodes in dataproc.
> Inorder to have HA dataproc, Google Cloud suggests to have 3 master nodes
> [https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability|https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/high-availability]
> The number of masters should be a config in DataprocCreateOperator so that
> the created dataproc is HA
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
