exploy commented on a change in pull request #4015: [AIRFLOW-2789] Create
single node DataProc cluster
URL: https://github.com/apache/incubator-airflow/pull/4015#discussion_r224614716
##########
File path: airflow/contrib/operators/dataproc_operator.py
##########
@@ -170,6 +172,7 @@ def __init__(self,
idle_delete_ttl=None,
auto_delete_time=None,
auto_delete_ttl=None,
+ single_node=False,
Review comment:
I based the behavior on how the `gcloud dataproc clusters create` command
works.
While the cluster is in a single node configuration, user has to keep in
mind that resouces avaiable to the spark are not the same as in case of regular
workers - single node acts both as a worker and as the master, so part of the
resources is consumed by master process.
And last but not least, I prefer to follow the rule ["Explicit is better
than implicit"](https://www.python.org/dev/peps/pep-0020/).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
