This is an automated email from the ASF dual-hosted git repository.
shahar1 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new ff53593e698 Document Dataproc limitations in Sovereign Cloud from
Google (#66706)
ff53593e698 is described below
commit ff53593e6982621484539b1f2705e746c3bbf5b6
Author: Ulada Zakharava <[email protected]>
AuthorDate: Mon May 11 22:15:50 2026 +0200
Document Dataproc limitations in Sovereign Cloud from Google (#66706)
---
providers/google/docs/operators/cloud/dataproc.rst | 46 ++++++++++++++++++++++
1 file changed, 46 insertions(+)
diff --git a/providers/google/docs/operators/cloud/dataproc.rst
b/providers/google/docs/operators/cloud/dataproc.rst
index 21da2929f30..039a0e8e4af 100644
--- a/providers/google/docs/operators/cloud/dataproc.rst
+++ b/providers/google/docs/operators/cloud/dataproc.rst
@@ -31,6 +31,52 @@ Prerequisite Tasks
.. include:: /operators/_partials/prerequisite_tasks.rst
+Sovereign Cloud from Google guidance
+------------------------------------
+
+Sovereign Cloud from Google is a Google Cloud deployment with a separate
service endpoint surface from
+standard Google Cloud regions. Dataproc support in Sovereign Cloud from Google
can therefore differ from
+Dataproc support in standard Google Cloud. If a Dataproc API method or job
type is not implemented in
+Sovereign Cloud from Google, the corresponding Airflow operator or job
configuration will not run there.
+This is a Dataproc service availability limitation, not a Cloud Composer or
Airflow operator limitation.
+
+The following Dataproc job configurations were available in the tested
Sovereign Cloud from Google environment:
+
+* Hadoop jobs.
+* Hive jobs.
+* PySpark jobs.
+* Spark jobs.
+* Spark SQL jobs.
+* Trino jobs.
+
+The following Dataproc job configurations were not available in the tested
environment:
+
+* ``PigJob``.
+* ``PrestoJob``.
+* ``SparkRJob``.
+* ``FlinkJob``.
+
+The following operators do not work in Sovereign Cloud from Google because the
corresponding Dataproc API
+methods are not implemented there yet:
+
+* Workflow template operators:
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocCreateWorkflowTemplateOperator`,
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocInstantiateWorkflowTemplateOperator`,
and
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocInstantiateInlineWorkflowTemplateOperator`.
+* Batch operators:
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocCreateBatchOperator`,
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocGetBatchOperator`,
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocListBatchesOperator`,
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocDeleteBatchOperator`,
and
+
:class:`~airflow.providers.google.cloud.operators.dataproc.DataprocCancelOperationOperator`.
+
+The same limitation was observed with ``gcloud`` commands, so this is not a
Cloud Composer or Airflow
+operator limitation.
+
+When running in Sovereign Cloud from Google, verify that the Dataproc API
method and job type are available
+in that environment before relying on the corresponding Airflow operator in a
production DAG.
+
+
.. _howto/operator:DataprocCreateClusterOperator:
.. _howto/operator:DataprocInstantiateInlineWorkflowTemplateOperator:
