cshuo commented on issue #20605:
URL: https://github.com/apache/hudi/issues/20605#issuecomment-7003257997
### Validation summary
I validated the proposal on a local kind cluster using **Spark 4.1.3**, the
version pinned in the chart’s default Spark Connect image. I recommend keeping
bearer-token validation in Spark and making TLS termination configurable. Both
TLS deployments passed the same checks:
| Deployment | Valid token and trusted CA | Missing or incorrect token |
Untrusted CA |
|---|---|---|---|
| HAProxy sidecar forwarding to Spark on Pod loopback | `SELECT 1` passed |
`UNAUTHENTICATED` | Certificate verification failed |
| Separate gRPC gateway forwarding to a private Spark Service | `SELECT 1`
passed | `UNAUTHENTICATED` | Certificate verification failed |
### Server configuration
I used the following commands with the local chart changes. `$certs`
contains the test CA (`ca.crt`), a server certificate valid for `localhost`
(`tls.crt`), and its private key (`tls.key`).
```bash
# Restrict newly created files to the current user.
umask 077
# Generate a shared token from 32 random bytes. kubectl copies the file
# verbatim, so remove the trailing newline before creating the Secret.
openssl rand -hex 32 | tr -d '\n' > "$certs/token"
# The chart injects this Secret's "token" key into the Spark driver as
# SPARK_CONNECT_AUTHENTICATE_TOKEN. Clients present the same bearer token.
kubectl -n hudi-lakehouse create secret generic connect-auth \
--from-file=token="$certs/token"
# HAProxy uses this certificate and private key for the TLS handshake.
# Clients use ca.crt to verify the server certificate.
kubectl -n hudi-lakehouse create secret tls connect-tls \
--cert="$certs/tls.crt" --key="$certs/tls.key"
# These are independent Helm chart values.
# auth.enabled turns on Spark's built-in token check.
# tls.enabled adds the HAProxy sidecar to the Spark Pod and routes the
# Service to its TLS port. Helm receives only the Secret names.
helm upgrade hudi-spark-connect hudi-lakehouse/charts/hudi-spark-connect \
-n hudi-lakehouse --reuse-values \
--set auth.enabled=true,auth.existingSecret=connect-auth \
--set tls.enabled=true,tls.existingSecret=connect-tls \
--wait
```
For the separate-gateway test, I kept `auth.enabled=true` and set
`tls.enabled=false`. The Spark Pod then ran a single container. The gateway
terminated TLS and forwarded gRPC requests, including the `authorization`
metadata, to the private Spark Service. That backend connection needs a trusted
network or separate transport encryption.
### How clients passed the token
For the sidecar test, I forwarded the Service to the host:
```bash
# Local port 19443 reaches the HAProxy TLS listener through the Service.
kubectl -n hudi-lakehouse port-forward svc/hudi-spark-connect 19443:15002
```
In a second shell, I configured the clients:
```bash
export SPARK_CONNECT_AUTHENTICATE_TOKEN="$(cat "$certs/token")"
export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH="$certs/ca.crt" # PySpark
export SSL_CERT_FILE="$certs/ca.crt" # Foundry ADBC
```
```python
import os
from pyspark.sql import SparkSession
from adbc_driver_manager import dbapi
# PySpark reads the bearer token from SPARK_CONNECT_AUTHENTICATE_TOKEN
# and the trusted CA from GRPC_DEFAULT_SSL_ROOTS_FILE_PATH.
spark =
SparkSession.builder.remote("sc://localhost:19443/;use_ssl=true").getOrCreate()
try:
# Run a query so the test exercises both TLS and Spark's token check.
assert spark.sql("SELECT 1").first()[0] == 1
finally:
spark.stop()
# Foundry ADBC 0.2.1 supports auth_type=none and auth_type=token for Connect.
# In token mode, pass the bearer token in the ADBC password field.
# The URI stays credential-free, and SSL_CERT_FILE supplies the trusted CA.
options = {
"uri": "spark://localhost:19443?api=connect&auth_type=token&tls=true",
"username": "test",
"password": os.environ["SPARK_CONNECT_AUTHENTICATE_TOKEN"],
}
with dbapi.connect(driver="spark", db_kwargs=options, autocommit=True) as
conn:
with conn.cursor() as cur:
cur.execute("SELECT 1")
assert cur.fetchone() == (1,)
```
### Application checks
I tested these integrations against the sidecar deployment with certificate
verification enabled:
| Application | How it received the token | Live result |
|---|---|---|
| Superset 6.1.0 | Superset stored `connect_args.spark_connect_token` in its
encrypted database settings. The dialect passed it to ADBC. | A query through
the SQL Lab API returned `1`. The database API response omitted the plaintext
token. |
| Zeppelin 0.12.0 | The Python interpreter read the token from its
environment and passed it as the ADBC password. | A notebook completed Hudi
CRUD and `MERGE` and returned the expected table. The token stayed out of the
saved notebook. |
| `hudi-agent-gateway` | The Helm deployment read
`sc://…/;use_ssl=true;token=…` from a Kubernetes Secret and mounted the private
CA from a ConfigMap. | The `query_lakehouse` handler successfully queried the
test table. |
### 2.0 status and scope
**I found no client compatibility blocker for 2.0 in these tests.** The
authenticated readiness probe passed its SQL, catalog, and storage checks. With
authentication and TLS at their defaults, the quickstart passed its SQL matrix
and restart-persistence test. I also ran the new CI acceptance step locally. PR
review and remote CI are still pending.
The scope remains endpoint authentication and transport security. All token
holders share the driver’s storage credentials. Per-user identity and
authorization, multi-tenant isolation, and access controls for the Spark UI and
History Server remain follow-up work.
cc @siva
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]