sadpandajoe opened a new pull request, #44467:
URL: https://github.com/apache/superset/pull/44467
### SUMMARY
`superset/translations/messages.pot` is the extraction template every
language
catalog is generated from, but nothing in CI ever compared the committed
template against a fresh extraction. The `babel-extract` job regenerates
`messages.pot` from scratch in throwaway worktrees before comparing
translated/fuzzy counts — a check scoped to catching a reworded source string
invalidating an existing translation, not template/source parity. The
template
had drifted 432 strings behind source and carried 109 stale entries, so those
432 strings could not reach any of the 30 `.po` catalogs and were
untranslatable
in every language.
This PR does three things.
**1. Adds a drift guard.** `scripts/translations/check_pot_drift.py`
re-extracts
from source with the same `pybabel extract` invocation as `babel_update.sh`
and
fails when the committed template's msgid set differs from a fresh
extraction.
It is wired into the `babel-extract` job as a hard gate, placed before
`babel_update.sh` overwrites the workspace template. Comparison is on msgid
sets
only, so it is insensitive to `POT-Creation-Date` churn and to the
do-not-translate / `python-format` post-processing steps. Line wrapping is
deliberately *not* normalized away: gettext continuation lines concatenate
back
to the exact original, so any whitespace difference that survives parsing is
a
real source reword that gettext's exact-match lookup would miss.
**2. Unbreaks extraction of the explore additional-actions menu.**
`useExploreAdditionalActionsMenu/index.tsx` was almost entirely invisible to
`pybabel extract`: `return /[",\r\n]/.test(s) ? ... : s;` puts a regex
literal
directly after `return`, and babel's JavaScript lexer reads that `/` as
division, so the `"` inside the character class opens a phantom string
literal
that swallows the next ~290 lines. Only 3 of the module's 34 translatable
strings survived extraction. The stale template still carried the other 31
only
because it predated the file's current shape — so regenerating it would have
deleted 10 still-live, already-translated strings ("Export All Data", "Export
Current View", "New chart", "Superset Chart", …) from all 30 catalogs.
Hoisting
the regex into a module constant fixes extraction with no behavior change.
**3. Regenerates the template and all 30 catalogs** via `babel_update.sh`'s
own
sequence, so the new guard starts green. Adding the guard without
regenerating
would fail CI immediately against the already-stale template.
Regeneration changes **zero existing translation values** (verified by
parsing
both revisions with `babel.messages.pofile.read_po` for all 30 catalogs, and
independently re-verified in review): the diff is additions plus the removal
of
100 msgids that no longer exist in source, none of which still appear inside
a
`t()`/`_()`/`__()`/`tct()` call. The `it/messages.po` catalog's diff looks
large
because `babel_update.sh` also normalizes it from CRLF to LF line endings
(master's copy was the only CRLF-encoded catalog) — that's a whitespace-only
normalization, not a retranslation; the zero-changed-values check above
covers
it.
Note: `babel_update.sh`'s `msgcat --sort-by-msgid --no-wrap` step is a no-op
in
CI today — `--sort-by-msgid` is not an msgcat option, the script has no
`set -e`, so msgcat exits without writing. This PR reproduces current CI
output
rather than introducing a new convention; the flag itself is addressed
separately in #44395. Its other two effects are redundant, since
`--sort-output`
and `--no-location` are already on the `pybabel extract` call.
Out of scope, left untouched (separate, pre-existing issues):
`flask_babel.lazy_gettext` call sites (e.g. `superset/views/filters.py:330`)
are not reachable by `pybabel extract` at all, since neither babel's default
keywords nor this project's `-k` list include `lazy_gettext` — worth its own
follow-up issue.
Fixes #44403
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
### TESTING INSTRUCTIONS
Guard passes on this branch:
python scripts/translations/check_pot_drift.py
# superset/translations/messages.pot matches a fresh extraction.
Guard catches the bug it exists for — restore the pre-fix template and
re-run:
git show master:superset/translations/messages.pot >
superset/translations/messages.pot
python scripts/translations/check_pot_drift.py
# exits 1: strings missing/stale
git checkout -- superset/translations/messages.pot
Unit tests (mocked cases plus a real end-to-end extraction against this
checkout):
pytest tests/unit_tests/scripts/translations/
Frontend behavior of the hoisted regex is unchanged:
cd superset-frontend
npm run test -- src/explore/components/useExploreAdditionalActionsMenu
Regeneration is idempotent — re-running
`./scripts/translations/babel_update.sh`
produces no diff beyond `POT-Creation-Date`.
### ADDITIONAL INFORMATION
- [x] Has associated issue: #44403
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
