[ 
https://issues.apache.org/jira/browse/SPARK-61773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hyukjin Kwon reassigned SPARK-61773:
------------------------------------

    Assignee: Kaicheng Zhou

> Connect through the HTTP proxy of the environment, as grpc-python does
> ----------------------------------------------------------------------
>
>                 Key: SPARK-61773
>                 URL: https://issues.apache.org/jira/browse/SPARK-61773
>             Project: Spark
>          Issue Type: Improvement
>          Components: Connect
>    Affects Versions: connect-rust-4.3.0
>            Reporter: Kaicheng Zhou
>            Assignee: Kaicheng Zhou
>            Priority: Major
>             Fix For: connect-rust-4.3.0
>
>
> The Rust client (pyspark-client-rust and the native Rust API) always dials 
> the Spark Connect server directly, so it cannot connect from behind an egress 
> HTTP proxy. PySpark's client can: grpcio reads the proxy of the environment 
> for every channel and opens an HTTP/1.1 CONNECT tunnel. A user whose PySpark 
> connects through a proxy today only gets retries from the drop-in.
> Honor the proxy of the environment with grpcio's rules:
> - the proxy is the first of {{grpc_proxy}}, {{https_proxy}} and 
> {{http_proxy}} that is set (an empty value means no proxy); only the 
> lowercase names are read
> - {{no_grpc_proxy}}, or else {{no_proxy}}, lists the hosts that are not 
> proxied: the host or a parent domain, or an IP in a CIDR range
> - only an {{http://}} proxy is supported; credentials in the URI are sent as 
> Proxy-Authorization: Basic
> - TLS is negotiated with the Spark server inside the tunnel, so the proxy 
> sees neither the token nor the data
> Two deliberate differences from grpcio: a loopback host is never proxied, 
> because a token is sent to localhost without TLS (grpcio proxies it, which 
> shows the token to the proxy in clear), and a proxy address with an invalid 
> host or port is an error instead of a channel that never connects.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to