sadpandajoe commented on code in PR #43916: URL: https://github.com/apache/superset/pull/43916#discussion_r3940658832
########## docs/admin_docs/installation/docker-compose.mdx: ########## @@ -196,7 +196,10 @@ One important variable is `SUPERSET_LOAD_EXAMPLES` which determines whether the container will populate example data and visualizations into the metadata database. These examples are helpful for learning and testing out Superset but unnecessary for experienced users and production deployments. The loading process can sometimes take a few minutes and a good amount of -CPU, so you may want to disable it on a resource-constrained device. +CPU, so you may want to disable it on a resource-constrained device. Once examples have been loaded +successfully, later `superset_init` runs against the same `superset_home` volume only refresh +example metadata rather than reloading the data; set `SUPERSET_FORCE_LOAD_EXAMPLES=yes` to force a +full reload. Review Comment: Good catch. `SUPERSET_FORCE_LOAD_EXAMPLES` is now mapped into the `superset-init` / `superset-init-light` environment in both compose files, so setting it from the shell reaches the init script. _🤖 Addressed by [Claude Code](https://claude.com/claude-code)_ ########## docker/docker-init.sh: ########## @@ -70,12 +70,28 @@ if [ "$SUPERSET_LOAD_EXAMPLES" = "yes" ]; then # Load some data to play with echo_step "4" "Starting" "Loading examples" + EXAMPLES_LOADED_MARKER="${SUPERSET_HOME}/.examples-loaded" - # If Cypress run which consumes superset_test_config – load required data for tests - if [ "$CYPRESS_CONFIG" == "true" ]; then - superset load_examples --load-test-data + # Loading examples parses and inserts every example dataset/dashboard/chart + # and is one of the slowest steps of `docker compose up`. Once it has + # succeeded, subsequent `docker-init.sh` runs against the same + # superset_home volume only need to refresh metadata (e.g. after a + # `superset db upgrade`), not reload the data itself. Set + # SUPERSET_FORCE_LOAD_EXAMPLES=yes to force a full reload regardless. + # Cypress runs always do a full reload since they load a distinct set of + # test data (`--load-test-data`) into a separate `superset_cypress` + # database that the metadata-only marker doesn't track. + if [ -f "$EXAMPLES_LOADED_MARKER" ] && [ "$SUPERSET_FORCE_LOAD_EXAMPLES" != "yes" ] && [ "$CYPRESS_CONFIG" != "true" ]; then Review Comment: Agreed. The marker file is gone; the script now asks the databases directly (example table present in the examples DB and the `world_health` dashboard present in the metadata DB) and only skips the load when both exist, so a recreated database volume triggers a full reload. _🤖 Addressed by [Claude Code](https://claude.com/claude-code)_ ########## docker/docker-init.sh: ########## @@ -70,12 +70,28 @@ if [ "$SUPERSET_LOAD_EXAMPLES" = "yes" ]; then # Load some data to play with echo_step "4" "Starting" "Loading examples" + EXAMPLES_LOADED_MARKER="${SUPERSET_HOME}/.examples-loaded" - # If Cypress run which consumes superset_test_config – load required data for tests - if [ "$CYPRESS_CONFIG" == "true" ]; then - superset load_examples --load-test-data + # Loading examples parses and inserts every example dataset/dashboard/chart + # and is one of the slowest steps of `docker compose up`. Once it has + # succeeded, subsequent `docker-init.sh` runs against the same + # superset_home volume only need to refresh metadata (e.g. after a + # `superset db upgrade`), not reload the data itself. Set + # SUPERSET_FORCE_LOAD_EXAMPLES=yes to force a full reload regardless. + # Cypress runs always do a full reload since they load a distinct set of + # test data (`--load-test-data`) into a separate `superset_cypress` + # database that the metadata-only marker doesn't track. + if [ -f "$EXAMPLES_LOADED_MARKER" ] && [ "$SUPERSET_FORCE_LOAD_EXAMPLES" != "yes" ] && [ "$CYPRESS_CONFIG" != "true" ]; then + echo "Examples already loaded, refreshing metadata only (set SUPERSET_FORCE_LOAD_EXAMPLES=yes to force a full reload)" + superset load_examples --only-metadata else - superset load_examples + # If Cypress run which consumes superset_test_config – load required data for tests + if [ "$CYPRESS_CONFIG" == "true" ]; then + superset load_examples --load-test-data + else + superset load_examples + fi + touch "$EXAMPLES_LOADED_MARKER" Review Comment: The marker file is replaced by an in-DB check, so there is no longer a flag written after a partially failed run. The check itself is also treated as best-effort: any error falls through to a full load, and `SUPERSET_FORCE_LOAD_EXAMPLES=yes` remains available to retry a partial load. _🤖 Addressed by [Claude Code](https://claude.com/claude-code)_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
