younsl opened a new pull request, #440:
URL: https://github.com/apache/superset-kubernetes-operator/pull/440
## Summary
When spec.valkey.ssl is set without any certificate file (ssl: {}, which the
configuration guide recommends for simple TLS), the operator renders rediss://
URLs but leaves out the shared SSL options. CeleryConfig then has no
broker_use_ssl or redis_backend_use_ssl, and the Celery result backend refuses
to start because ssl_cert_reqs is missing, so the Celery worker crash loops.
The same check also makes ssl.certRequired silently ignored unless a
certificate path is set. This is the common setup for managed Valkey or Redis
with in-transit encryption and a public CA, such as ElastiCache. This PR
renders the SSL options, including ssl_cert_reqs, whenever SSL is enabled,
which matches what the configuration guide already describes.
## Details
- Renderer: the shared SSL options dict is now rendered whenever SSL is
enabled, not only when keyFile, certFile, or caCertFile is set. Certificate
paths are still added only when they are set, and certRequired keeps its
default of required.
- The options flow to the same places as before for mTLS: CACHE_OPTIONS on
every Flask-Caching section, broker_use_ssl and redis_backend_use_ssl on
CeleryConfig, and the RESULTS_BACKEND keyword arguments.
- Flask-Caching and the results backend already used required verification
by default through redis-py, so for them nothing changes unless certRequired is
set to optional or none.
- Existing Deployments that use SSL roll once on upgrade because the
rendered config changes.
- Unit tests: the SSL test without certificate files used to assert that no
SSL options are rendered, which locked in the bug. It now asserts the options
are rendered and wired into every client. A new test covers certRequired none
without certificate files.
Verification: make lint-go (0 issues), make test-unit and make
test-integration (all packages pass), go vet with the integration tag, and the
Kind reproduction below.
<details>
<summary>Kind reproduction and verification</summary>
Setup: Kind cluster, PostgreSQL, and a TLS only Valkey 8 (plaintext port
disabled, self-signed server certificate, client certificates not required).
Superset 7.0.0rc1 with a Celery worker and spec.valkey.ssl set as below. Before
the fix the released 0.3.0 operator reconciled the resource; after the fix a
locally built operator from this branch did.
Before the fix, ssl: {}
The worker crash loops on start:
```
File "/app/.venv/lib/python3.11/site-packages/celery/backends/redis.py",
line 296, in __init__
raise ValueError(E_REDIS_SSL_CERT_REQS_MISSING_INVALID)
ValueError:
A rediss:// URL must have parameter ssl_cert_reqs and this must be set to
CERT_REQUIRED, CERT_OPTIONAL, or CERT_NONE
```
Rendered CeleryConfig:
```python
class CeleryConfig:
broker_url = f"{_vk_base}/0"
result_backend = f"{_vk_base}/0"
CELERY_CONFIG = CeleryConfig
```
Before the fix, ssl: {certRequired: none}
Same ValueError. The rendered superset_config.py is byte for byte identical
to the ssl: {} case, so certRequired is ignored.
After the fix, ssl: {certRequired: none}
Config diff against the previous case:
```diff
+_vk_ssl_opts = {
+ "ssl_cert_reqs": "none",
+}
...
+ "CACHE_OPTIONS": _vk_ssl_opts, (all six cache sections)
...
+ broker_use_ssl = _vk_ssl_opts
+ redis_backend_use_ssl = _vk_ssl_opts
...
+ **_vk_ssl_opts, (RESULTS_BACKEND)
```
The worker is Running and every client works over TLS:
```
$ valkey-cli --tls --cacert /tls/ca.crt -n 0 keys '*'
_kombu.binding.celery
_kombu.binding.celery.pidbox
_kombu.binding.celeryev
ping: {'celery@ssl-repro-celery-worker-bc7875d78-xcm5l': {'ok': 'pong'}}
result backend get: b'ok'
flask cache get: ok
results backend get: ok
```
After the fix, ssl: {} (certRequired defaults to required)
The ValueError is gone, and certificate verification is enforced: the
self-signed test certificate is rejected, as expected without caCertFile. A
server certificate from a public CA verifies against the image's system CA
bundle.
```
consumer: Cannot connect to rediss://:**@valkey:6379/0: Error 1 connecting
to valkey:6379. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
self-signed certificate in certificate chain
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
