hussein-awala commented on code in PR #31644:
URL: https://github.com/apache/airflow/pull/31644#discussion_r1233359887
##########
airflow/providers/google/cloud/operators/dataproc.py:
##########
@@ -329,6 +361,7 @@ def _build_cluster_data(self):
"boot_disk_size_gb": self.worker_disk_size,
},
"is_preemptible": True,
+ "preemptibility": self.preemptibility,
Review Comment:
```suggestion
"preemptibility": self.preemptibility.value,
```
##########
airflow/providers/google/cloud/operators/dataproc.py:
##########
@@ -63,6 +73,13 @@
from airflow.utils.context import Context
+class PreemptibilityType(Enum):
+ """Contains possible Type values of Preemptibility applicable for every
secondary worker of Cluster."""
+
+ PREEMPTIBLE = "PREEMPTIBLE"
+ SPOT = "SPOT"
Review Comment:
Why is `PREEMPTIBILITY_UNSPECIFIED` not an acceptable value?
##########
airflow/providers/google/cloud/operators/dataproc.py:
##########
@@ -221,6 +246,13 @@ def __init__(
if self.single_node and self.num_preemptible_workers > 0:
raise ValueError("Single node cannot have preemptible workers.")
+ def _set_preemptibility_type(self, preemptibility):
+ if self.num_preemptible_workers > 0:
+ if preemptibility.lower() == "spot":
+ return PreemptibilityType.SPOT.value
+ else:
+ return preemptibility
Review Comment:
```suggestion
return PreemptibilityType(preemptibility.upper())
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
