mistercrunch opened a new pull request, #34831: URL: https://github.com/apache/superset/pull/34831
This change migrates the Docker development setup to use DuckDB for examples data storage while keeping PostgreSQL for Superset metadata. This significantly improves development environment startup time and resource usage. Key changes: - Add duckdb-engine to development dependencies and upgrade to v0.17.0 - Configure Docker to download pre-built examples.duckdb during image build - Update docker-compose files to use DuckDB URIs for examples data - Add environment variable override support for SQLALCHEMY_EXAMPLES_URI - Fix DuckDB engine spec default_driver (was empty, now "duckdb_engine") - Work around cursor.description bug in duckdb-engine driver The examples.duckdb file (~37MB) is downloaded from apache-superset/examples-data during Docker build using the LOAD_EXAMPLES_DUCKDB build arg. We considered alternative approaches like downloading during container startup, but injecting the file deep in the Docker layer stack provides better Docker layer caching and faster subsequent builds. This hybrid approach uses PostgreSQL for reliable metadata storage and DuckDB for fast analytical examples data, eliminating examples loading time while maintaining full functionality. Technical notes: - Includes workaround for duckdb-engine cursor.description bug: https://github.com/Mause/duckdb_engine/issues/1322 - Examples data baked into container image, no runtime loading needed - Compatible with existing Claudette workflows and testing - Build arg defaults to true for development environments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
