This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 25b192e043d Document proxy setup for Google provider using PySocks
(#71941)
25b192e043d is described below
commit 25b192e043d5df9fdfe7c767d64d1a53655ebd0d
Author: David Blain <[email protected]>
AuthorDate: Mon Sep 21 00:02:03 2026 +0200
Document proxy setup for Google provider using PySocks (#71941)
Adds a "Using the Google Cloud Connection Behind a Corporate Proxy" section
explaining why httplib2-based discovery hooks need PySocks: without it
ProxyInfo.isgood() is falsy, HTTPSConnectionWithTimeout.connect() has no
ProxiesUnavailableError guard, and the worker silently opens a direct socket
and fails with socket.gaierror.
Generated-by: Claude Opus 5
Co-authored-by: David Blain <[email protected]>
---
docs/spelling_wordlist.txt | 1 +
providers/google/docs/connections/gcp.rst | 86 +++++++++++++++++++++++++++++++
2 files changed, 87 insertions(+)
diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt
index dedf87bc4fe..f719c597bf7 100644
--- a/docs/spelling_wordlist.txt
+++ b/docs/spelling_wordlist.txt
@@ -1348,6 +1348,7 @@ protobuf
provisioner
proxied
proxies
+proxying
psql
psrp
psycopg
diff --git a/providers/google/docs/connections/gcp.rst
b/providers/google/docs/connections/gcp.rst
index b0dd6eae2c4..fe2dbe57e38 100644
--- a/providers/google/docs/connections/gcp.rst
+++ b/providers/google/docs/connections/gcp.rst
@@ -393,6 +393,92 @@ Using a quota project affects where API usage is billed,
which quotas are applie
usage is reported for monitoring and auditing.
+.. _howto/connection:google_cloud_platform:corporate_proxy:
+
+Using the Google Cloud Connection Behind a Corporate Proxy
+----------------------------------------------------------
+
+If Airflow workers are deployed behind a corporate HTTP proxy, two things are
required for
+Google API calls to reach the internet.
+
+**1. Set the standard proxy environment variables on each worker:**
+
+.. code-block:: bash
+
+ export HTTPS_PROXY=http://<proxy-host>:<port>
+ export HTTP_PROXY=http://<proxy-host>:<port>
+ export NO_PROXY=localhost,127.0.0.1,.cluster.local
+
+In a Kubernetes / Helm deployment add them to ``values.yaml``:
+
+.. code-block:: yaml
+
+ env:
+ - name: HTTPS_PROXY
+ value: "http://<proxy-host>:<port>"
+ - name: HTTP_PROXY
+ value: "http://<proxy-host>:<port>"
+ - name: NO_PROXY
+ value: "localhost,127.0.0.1,.cluster.local"
+
+**2. Install the** ``PySocks`` **package in the worker image — this is
mandatory.**
+
+For a bare-metal or custom Docker image, add it at build time:
+
+.. code-block:: dockerfile
+
+ RUN pip install pysocks
+
+In a Kubernetes / Helm deployment, bake ``pysocks`` into your custom worker
image the same way as
+above, then point the chart at that image in ``values.yaml``:
+
+.. code-block:: yaml
+
+ images:
+ airflow:
+ repository: your-registry/airflow-with-pysocks
+ tag: "3.x.x"
+
+Why PySocks is required even for an HTTP proxy
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+All Google API endpoints (``oauth2.googleapis.com``,
``bigquery.googleapis.com``, etc.) are
+HTTPS. To route an HTTPS request through an HTTP proxy, the client must first
open an
+``HTTP CONNECT`` tunnel and then do TLS end-to-end inside it.
+
+The ``httplib2`` library — used by ``google-api-python-client`` and the
+``_authorize()`` path of ``GoogleBaseHook`` for services such as BigQuery Jobs
API,
+Dataflow, Compute, Cloud SQL, Datastore, Cloud Functions, and Marketing
Platform — does
+**not** implement ``CONNECT`` tunneling itself. It delegates proxying to the
external
+`PySocks <https://pypi.org/project/PySocks/>`_ package, which must be
importable at runtime
+to activate its HTTP ``CONNECT`` tunnel path via a ``socks.socksocket``.
+If PySocks is not installed, ``httplib2`` silently falls back to a direct
connection even
+when ``HTTPS_PROXY`` is set, and the worker attempts a direct DNS lookup of
the Google
+endpoint — which fails in a network-restricted environment.
+
+The symptom is a ``socket.gaierror: [Errno -2] Name or service not known`` (or
+``[Errno 11001] getaddrinfo failed`` on Windows) when a task first tries to
authenticate
+or call a Google service.
+
+.. note::
+ Hooks backed by the newer ``google-cloud-*`` client libraries (Cloud
Storage / GCS,
+ Pub/Sub, Spanner, BigQuery Storage API, etc.) use ``google-auth`` with the
``requests``
+ transport, which has native ``CONNECT``-tunnel support and does **not**
need PySocks.
+ Only the ``discovery``-based hooks that go through ``_authorize()`` are
affected.
+
+Quick verification
+~~~~~~~~~~~~~~~~~~
+
+Run the following inside the worker container to confirm the proxy and PySocks
are working:
+
+.. code-block:: bash
+
+ # Proxy can reach Google (a 404 from Google = tunnel succeeded)
+ curl -x http://<proxy-host>:<port> https://oauth2.googleapis.com
+
+ # PySocks is importable
+ python -c "import socks; print('PySocks OK')"
+
Authenticating to Google Sovereign Cloud in Airflow
---------------------------------------------------
