turbaszek commented on a change in pull request #13256: URL: https://github.com/apache/airflow/pull/13256#discussion_r553806448
########## File path: airflow/providers/google/ADDITIONAL_INFO.md ########## @@ -34,6 +34,7 @@ Details are covered in the UPDATING.md files for each library, but there are som | [``google-cloud-os-login``](https://pypi.org/project/google-cloud-os-login/) | ``>=1.0.0,<2.0.0`` | ``>=2.0.0,<3.0.0`` | [`UPGRADING.md`](https://github.com/googleapis/python-oslogin/blob/master/UPGRADING.md) | | [``google-cloud-pubsub``](https://pypi.org/project/google-cloud-pubsub/) | ``>=1.0.0,<2.0.0`` | ``>=2.0.0,<3.0.0`` | [`UPGRADING.md`](https://github.com/googleapis/python-pubsub/blob/master/UPGRADING.md) | | [``google-cloud-kms``](https://pypi.org/project/google-cloud-os-login/) | ``>=1.2.1,<2.0.0`` | ``>=2.0.0,<3.0.0`` | [`UPGRADING.md`](https://github.com/googleapis/python-kms/blob/master/UPGRADING.md) | +| [``google-cloud-dataproc``](https://pypi.org/project/google-cloud-dataproc/) | ``>=1.0.1,<2.0.0`` | ``>=2.2.0,<3.0.0`` | [`UPGRADING.md`](https://github.com/googleapis/python-dataproc/blob/master/UPGRADING.md) | Review comment: I will introduce the ordering then. There was no a-z order so I added this just as a next item 😄 ########## File path: airflow/providers/google/cloud/operators/dataproc.py ########## @@ -613,18 +609,18 @@ def execute(self, context) -> dict: # Check if cluster is not in ERROR state self._handle_error_state(hook, cluster) - if cluster.status.state == cluster.status.CREATING: + if cluster.status.state == cluster.status.State.CREATING: # Wait for cluster to be be created cluster = self._wait_for_cluster_in_creating_state(hook) self._handle_error_state(hook, cluster) - elif cluster.status.state == cluster.status.DELETING: + elif cluster.status.state == cluster.status.State.DELETING: # Wait for cluster to be deleted self._wait_for_cluster_in_deleting_state(hook) # Create new cluster cluster = self._create_cluster(hook) self._handle_error_state(hook, cluster) - return MessageToDict(cluster) + return Cluster.to_json(cluster) Review comment: Nice catch! ########## File path: airflow/providers/google/cloud/hooks/dataproc.py ########## @@ -26,18 +26,16 @@ from google.api_core.exceptions import ServerError from google.api_core.retry import Retry from google.cloud.dataproc_v1beta2 import ( # pylint: disable=no-name-in-module - ClusterControllerClient, - JobControllerClient, - WorkflowTemplateServiceClient, -) -from google.cloud.dataproc_v1beta2.types import ( # pylint: disable=no-name-in-module Cluster, - Duration, - FieldMask, + ClusterControllerClient, Job, + JobControllerClient, JobStatus, WorkflowTemplate, + WorkflowTemplateServiceClient, ) +from google.protobuf.duration_pb2 import Duration +from google.protobuf.field_mask_pb2 import FieldMask Review comment: Fixed 🆗 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
