jigneshjain25 commented on code in PR #74753:
URL: https://github.com/apache/airflow/pull/74753#discussion_r4288157408
##########
providers/google/src/airflow/providers/google/cloud/operators/dataproc.py:
##########
@@ -230,6 +400,28 @@ class ClusterGenerator:
:param secondary_worker_accelerator_type: Type of the accelerator card
(GPU) to attach to the secondary workers,
see
https://cloud.google.com/dataproc/docs/reference/rest/v1/InstanceGroupConfig#acceleratorconfig
:param secondary_worker_accelerator_count: Number of accelerator cards
(GPUs) to attach to the secondary workers
+ :param master_flexible_config: High Obtainability (HO) mode configuration
of the master node group,
Review Comment:
"Thanks @Colin-Cheng, that's a great catch! You are completely
right—ClusterGenerator is deprecated and scheduled for removal after September
2027, so adding new parameters there would trigger deprecation warnings for
users adopting High Obtainability mode.
We have removed all HO additions from ClusterGenerator and moved validation
and normalization of flexible configs (master_flexible_config,
worker_flexible_config, secondary_worker_flexible_config) directly into
DataprocCreateClusterOperator(cluster_config=...). We have also updated the
documentation, example DAG, and unit tests accordingly."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
