rusackas commented on code in PR #45588:
URL: https://github.com/apache/superset/pull/45588#discussion_r4344417362
##########
.github/workflows/update-country-maps.yml:
##########
@@ -0,0 +1,166 @@
+name: Update Country Maps
+
+# Regenerates plugins/plugin-chart-country-map's GeoJSON files, countries.ts
+# and docs/data/countries.json from upstream Natural Earth + community geo
+# data, by running the same notebook `npm run update-maps` runs locally.
+#
+# Runs weekly and on workflow_dispatch, but only opens or updates a PR when
+# the regenerated output actually differs from what's on `master` - most
+# weeks this is a no-op. Unlike a translation-string catch-up, nothing here
+# auto-merges: a border/region change is the kind of thing that needs an
+# actual human look at the rendered maps. Community geo sources are pinned
+# to a commit where practical; the live WHO ArcGIS query can't be pinned
+# (it's a live service, not a repo), so a run can still legitimately pick up
+# an unreviewed upstream change there. The job itself fails (separately from
+# whether a PR was opened) if the notebook hits a cell error, so a broken
+# upstream source doesn't just quietly go green.
+on:
+ schedule:
+ - cron: "0 7 * * 1" # Mondays 07:00 UTC, an hour after
sync-translation-catalogs.yml
+ workflow_dispatch: {}
+
+permissions:
+ contents: read
+
+concurrency:
+ group: update-country-maps
+ cancel-in-progress: false
+
+jobs:
+ update-country-maps:
+ runs-on: ubuntu-26.04
Review Comment:
Done, gated scheduled runs on `github.repository == 'apache/superset'`;
`workflow_dispatch` still works anywhere.
##########
.github/workflows/update-country-maps.yml:
##########
@@ -0,0 +1,166 @@
+name: Update Country Maps
+
+# Regenerates plugins/plugin-chart-country-map's GeoJSON files, countries.ts
+# and docs/data/countries.json from upstream Natural Earth + community geo
+# data, by running the same notebook `npm run update-maps` runs locally.
+#
+# Runs weekly and on workflow_dispatch, but only opens or updates a PR when
+# the regenerated output actually differs from what's on `master` - most
+# weeks this is a no-op. Unlike a translation-string catch-up, nothing here
+# auto-merges: a border/region change is the kind of thing that needs an
+# actual human look at the rendered maps. Community geo sources are pinned
+# to a commit where practical; the live WHO ArcGIS query can't be pinned
+# (it's a live service, not a repo), so a run can still legitimately pick up
+# an unreviewed upstream change there. The job itself fails (separately from
+# whether a PR was opened) if the notebook hits a cell error, so a broken
+# upstream source doesn't just quietly go green.
+on:
+ schedule:
+ - cron: "0 7 * * 1" # Mondays 07:00 UTC, an hour after
sync-translation-catalogs.yml
+ workflow_dispatch: {}
+
+permissions:
+ contents: read
+
+concurrency:
+ group: update-country-maps
+ cancel-in-progress: false
+
+jobs:
+ update-country-maps:
+ runs-on: ubuntu-26.04
+ permissions:
+ contents: write
+ pull-requests: write
+ steps:
+ - name: "Checkout master"
+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #
v7.0.1
+ with:
+ ref: master
+ persist-credentials: true
+
+ - name: Setup Python
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 #
v7.0.0
+ with:
+ python-version: "3.11"
+
+ - name: Install notebook dependencies
+ run: |
+ pip install jupyter nbconvert ipykernel geopandas shapely matplotlib
requests
+
+ - name: Force speed_run (skip interactive preview rendering)
+ working-directory:
superset-frontend/plugins/plugin-chart-country-map/scripts
+ run: |
+ python3 - <<'PYEOF'
+ import json
+
+ path = "Country Map GeoJSON Generator.ipynb"
+ with open(path, encoding="utf-8") as f:
+ nb = json.load(f)
+
+ for cell in nb["cells"]:
+ if cell["cell_type"] != "code":
+ continue
+ if "".join(cell["source"]).strip().startswith("speed_run =
False"):
+ cell["source"] = [
+ "speed_run = True\n",
+ "# forced True for unattended CI regeneration; not
committed back\n",
+ ]
+ break
+ else:
+ raise SystemExit("Could not find the speed_run cell to patch")
+
+ with open(".ci-run.ipynb", "w", encoding="utf-8") as f:
+ json.dump(nb, f)
+ PYEOF
+
+ - name: Run notebook
+ working-directory:
superset-frontend/plugins/plugin-chart-country-map/scripts
+ run: |
+ jupyter nbconvert --to notebook --execute --inplace --allow-errors \
+ --ExecutePreprocessor.timeout=1200 .ci-run.ipynb
+
+ - name: Check for cell errors and record a summary
+ id: notebook
+ working-directory:
superset-frontend/plugins/plugin-chart-country-map/scripts
+ run: |
+ python3 - <<'PYEOF' >> "$GITHUB_OUTPUT"
+ import json
+
+ with open(".ci-run.ipynb", encoding="utf-8") as f:
+ nb = json.load(f)
+
+ errors = []
+ for i, cell in enumerate(nb["cells"]):
+ for output in cell.get("outputs", []):
+ if output.get("output_type") == "error":
+ errors.append(f"cell {i}: {output.get('ename')}:
{output.get('evalue')}")
+
+ if errors:
+ print("had_errors=true")
+ summary = "\n".join(f"- {e}" for e in errors[:20])
+ else:
+ print("had_errors=false")
+ summary = ""
+
+ with open("notebook-errors.txt", "w", encoding="utf-8") as f:
+ f.write(summary)
+ PYEOF
+ rm -f .ci-run.ipynb
+ if [ -s notebook-errors.txt ]; then
+ echo "::warning::Notebook hit cell errors; generated data may be
stale."
+ {
+ echo "### Country map notebook cell errors"
+ cat notebook-errors.txt
+ } >> "$GITHUB_STEP_SUMMARY"
+ fi
+
+ - name: Check for drift
+ id: diff
+ run: |
+
PATHS="superset-frontend/plugins/plugin-chart-country-map/src/countries
superset-frontend/plugins/plugin-chart-country-map/src/countries.ts
docs/data/countries.json"
+ if git diff --quiet -- $PATHS; then
Review Comment:
Good catch. Added a step on the no-drift path that closes the open bot PR
and deletes its branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
