jaketf edited a comment on issue #10687: URL: https://github.com/apache/airflow/issues/10687#issuecomment-686848519
@turbaszek I do not think we should unify naming on location as this would not line up with the REST API or the python client lib. the right thing to do here is a bit confusing looking at the [REST API docs](https://cloud.google.com/dataproc/docs/reference/rest) Some dataproc resources (e.g. job / cluster) only have `regions/` endpoints. Other dataproc resources (workflow templates) have BOTH `regions/` and `locations/` endpoints. In these cases the python client lib uses full `parent` parameter ([example](https://googleapis.dev/python/dataproc/latest/dataproc_v1beta2/services.html#google.cloud.dataproc_v1beta2.services.workflow_template_service.WorkflowTemplateServiceClient.instantiate_inline_workflow_template)). This is clearly intentional but I'm not entirely sure the distinction of when one would use `locations/` over the `regions/` endpoint. I imagine a user who understands this difference would want the flexibility to use either endpoint from their airflow DAGs and thus for hook methods / operators that interact with those resources we should provide mutually exclusive `region` AND `location` parameters OR take the same approach as python client lib and use`parent`. It's my hunch that this locations endpoint is a future proofing thing. Any given dataproc job runs only on a single dataproc cluster which will in only live in a single zone but we submit jobs to a regional endpoint and can OPTIONALLY specify a zone in the job / cluster config in the request or use [dataproc auto-zone placement](https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/auto-zone) hence why regional endpoints for ultimately zonal resources. This means "unifying naming on location" will be less explicitly mapping to the API endpoint used for those resources that actually use the region endpoint and this might lead to confusing code like `submit_job(region=self.location,...)` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
