karsten-wagner opened a new pull request, #23816: URL: https://github.com/apache/superset/pull/23816
### SUMMARY The `clickhouse-connect` library `0.4.1` did not handle a `no_proxy` environment variable. This is eventually problematic when using the [Docker Compose Installation Guide](https://superset.apache.org/docs/installation/installing-superset-using-docker-compose) in an environment that requires a forward proxy (e.g., in an enterprise environment) and where Clickhouse should be used as a database. In such case the call to the database should remain inside the docker network to allow addressing the database via its container name. Calls to external data sources should be rerouted via the forward proxy. Version `0.4.1` handles all connections via the proxy and thus misses the local network connection inside docker network and thus eventually fails host name resolution, given the forward proxy sits outside the docker network and is not aware of the inside docker network container names. The issue has been fixed in the `clickhouse-connect` library in version `0.5.19` via [#PR163](https://github.com/ClickHouse/clickhouse-connect/issues/163), hence bumping version here in the superset docs accordingly. ### TESTING INSTRUCTIONS * Follow the [Docker Compose Installation Guide](https://superset.apache.org/docs/installation/installing-superset-using-docker-compose) * Follow the existing [Clickhouse Installation Guide](https://superset.apache.org/docs/databases/clickhouse) referencing version `0.4.1` * Add a clickhouse container named `clickhouse` to `docker-compose-non-dev.yml`, e.g.: ```yaml x-clickhouse-volumes: &clickhouse-volumes - clickhouse_home:/var/lib/clickhouse clickhouse: image: clickhouse/clickhouse-server:23 container_name: clickhouse env_file: .env user: "root" restart: unless-stopped ports: - "8123:8123" - "9003:9000" volumes: *clickhouse-volumes volumes: clickhouse_home: external: false ``` * Update `docker/.env` to contain `http_proxy`, `https_proxy` and `no_proxy` environment variables, e.g.: ```yaml # Local Proxy Settings http_proxy="http://host.docker.internal:8888"; https_proxy="http://host.docker.internal:8888"; no_proxy="localhost,127.0.0.1,db,redis,superset,superset-init,superset-worker,superset-beat,clickhouse" ``` * Provide the forward proxy on the host OS (e.g., Fiddler - in above example on port 8888) * Start all containers and log in to superset * Try to add Clickhouse database via connection string `clickhousedb://clickhouse/default` The connection will fail. See eventually logs in the ´superset_app` container. Superset UI shows an error message on the dialog with the connection string, but not on the one where you provide host, port etc. in separate fields. * Change `docker/requirements-local.txt` to contain new version `clickhouse-connect>=0.5.19` * Repeat the test to add the database and all works ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
