This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new a0ebb5427cd ci(i18n): label translation PRs, use ten-page batches, and
validate before build (#4331)
a0ebb5427cd is described below
commit a0ebb5427cd0e4820f06f369c9a80846a40fa3f8
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Fri Oct 16 15:37:20 2026 +0800
ci(i18n): label translation PRs, use ten-page batches, and validate before
build (#4331)
Generated translation PRs currently have no locale label, and normal
generation defaults to two target pages. Add `translate-<locale>`
(`translate-ja` for Japanese) and raise the normal default to at most
ten target pages. Live `validate` remains limited to one exact source
page; the planner still considers up to twenty tasks.
Build Check now installs dependencies and runs unit tests before the
translation gate, then builds the selected website scope and verifies
static pages. The gate reads native source metadata and state, so it can
reject invalid translations before a site build. Keep its full-scope
environment separate from the build's version and locale filters.
Validation:
- Node 20.20.2: all 106 i18n tests pass. The new regression supplies
eleven eligible pages and verifies that only ten targets/state records
are written.
- Both workflow files parse; the normal/validation limits and
dependency-test-gate-build-static ordering are checked.
- A fresh checkout of the real machine candidate PR is running the gate
with no site build or build directory.
- The local publication simulation will verify that the actual PR
command carries `translate-ja` before this PR leaves draft.
The label already exists in this repository. There are no new token
permissions, automatic-generation switches, source edits or
translation-state changes in this PR.
## Versions
- [x] dev
- [x] 4.x
- [ ] 3.x
- [ ] 2.1 or older (not covered by version/language sync gate)
The managed translation scope stays Dev, 4.x and community. No
documentation routes or navigation change.
## Languages
- [ ] Chinese
- [x] English
Only the tooling README changes; there are no localized product-doc
counterparts.
## Docs Checklist
- [x] Checked by AI
- [x] Test Cases Built
- [x] Updated required version and language counterparts, or explained
why not
- [x] If only one language changed, confirmed whether source/translation
counterparts need sync
Co-authored-by: morningman <[email protected]>
---
.github/workflows/build-check.yml | 44 ++++++++++++++++++------------
.github/workflows/japanese-translation.yml | 2 +-
scripts/i18n/__tests__/workflow.test.cjs | 41 ++++++++++++++++++++++++++++
scripts/i18n/simulate-workflow.cjs | 5 ++--
scripts/i18n/workflow-generate.cjs | 4 +--
scripts/i18n/workflow-publish.cjs | 3 +-
translations/README.md | 10 +++++++
7 files changed, 86 insertions(+), 23 deletions(-)
diff --git a/.github/workflows/build-check.yml
b/.github/workflows/build-check.yml
index 0ab7003af0d..b1217ad1d0d 100644
--- a/.github/workflows/build-check.yml
+++ b/.github/workflows/build-check.yml
@@ -84,17 +84,20 @@ jobs:
sudo mkswap /mnt/japanese-build-check-swap
sudo swapon /mnt/japanese-build-check-swap
- - name: Build
+ # Dependencies are required even when the scoped site build is
skipped.
+ - name: Install dependencies
env:
- DOCS_VERSIONS: ${{ steps.detect.outputs.docs_versions }}
- DORIS_ENABLE_LOCALES: ${{
steps.detect.outputs.enabled_locales }}
- PWA_SERVICE_WORKER_URL: https://doris.apache.org/sw.js
+ NODE_OPTIONS: --max-old-space-size=8192
run: |
npm install -g yarn
yarn cache clean
- export NODE_OPTIONS=--max-old-space-size=8192
yarn
yarn key-features:generate
+
+ - name: Japanese translation unit tests
+ env:
+ NODE_OPTIONS: --max-old-space-size=8192
+ run: |
node --test scripts/i18n/__tests__/t04.test.cjs
node --test scripts/i18n/__tests__/heading-ids.test.cjs
node --test scripts/i18n/__tests__/search-input.test.cjs
@@ -105,9 +108,18 @@ jobs:
node --test scripts/i18n/__tests__/workflow.test.cjs
node --test scripts/i18n/__tests__/codex-adapter.test.cjs
- # Dependencies are installed unconditionally above because
the
- # governance steps below need node_modules; only the site
build
- # itself is scoped by the detect step.
+ - name: Japanese translation gate
+ run: |
+ BASE_SHA=$(git merge-base origin/${{ github.base_ref }} HEAD)
+ node scripts/i18n/check-translations.cjs --base "$BASE_SHA"
+
+ - name: Build
+ env:
+ DOCS_VERSIONS: ${{ steps.detect.outputs.docs_versions }}
+ DORIS_ENABLE_LOCALES: ${{
steps.detect.outputs.enabled_locales }}
+ PWA_SERVICE_WORKER_URL: https://doris.apache.org/sw.js
+ NODE_OPTIONS: --max-old-space-size=8192
+ run: |
if [ "${{ steps.detect.outputs.skip_build }}" = "true" ];
then
echo "No site-affecting changes detected — skipping the
docusaurus build."
exit 0
@@ -118,12 +130,15 @@ jobs:
PWA_SERVICE_WORKER_URL=https://doris.apache.org/sw.js
yarn docusaurus build ${{ steps.detect.outputs.locale_args }}
else
# Every locale gets a separate Node process and its real
prefix.
- # Changes that can only affect some locales build just
those.
yarn build:locales --locales "${{
steps.detect.outputs.locale_csv }}"
fi
- # A locale subset cannot render the English or Chinese
- # counterparts, so cross-locale assertions are reported as
- # skipped here. Publishing still verifies the whole site.
+
+ - name: Verify localized static pages
+ if: steps.detect.outputs.skip_build != 'true'
+ env:
+ NODE_OPTIONS: --max-old-space-size=8192
+ run: |
+ # A locale subset cannot prove cross-locale reciprocity.
VERIFY_PARTIAL=""
if [ "${{ steps.detect.outputs.partial_build }}" = "true" ];
then VERIFY_PARTIAL="--partial"; fi
for LOCALE in $(echo "${{
steps.detect.outputs.enabled_locales }}" | tr ',' ' '); do
@@ -131,11 +146,6 @@ jobs:
done
rm -rf build
- - name: Japanese translation gate
- run: |
- BASE_SHA=$(git merge-base origin/${{ github.base_ref }} HEAD)
- node scripts/i18n/check-translations.cjs --base "$BASE_SHA"
-
- name: Docs changed link gate
continue-on-error: true
run: |
diff --git a/.github/workflows/japanese-translation.yml
b/.github/workflows/japanese-translation.yml
index 2ff3c62f108..29b89b2005c 100644
--- a/.github/workflows/japanese-translation.yml
+++ b/.github/workflows/japanese-translation.yml
@@ -203,7 +203,7 @@ jobs:
I18N_LIVE_VERIFIED: ${{ vars.I18N_LIVE_VERIFIED }}
I18N_VALIDATION_RUN: ${{ github.event_name == 'workflow_dispatch' &&
inputs.mode == 'validate' }}
I18N_PATH: ${{ inputs.path }}
- I18N_MAX_PAGES: ${{ inputs.mode == 'validate' && '1' || '2' }}
+ I18N_MAX_PAGES: ${{ inputs.mode == 'validate' && '1' || '10' }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: node scripts/i18n/workflow-generate.cjs
"$RUNNER_TEMP/japanese-translation"
- uses: actions/create-github-app-token@v2
diff --git a/scripts/i18n/__tests__/workflow.test.cjs
b/scripts/i18n/__tests__/workflow.test.cjs
index 3b5bf8fc995..fc2825f1cf7 100644
--- a/scripts/i18n/__tests__/workflow.test.cjs
+++ b/scripts/i18n/__tests__/workflow.test.cjs
@@ -413,3 +413,44 @@ test('publication deduplicates shared state and rejects an
unrelated page shard'
assert.deepEqual(expectedFiles({schemaVersion: 1, provider: 'codex',
candidates: [record, second]}, scope),
[record.targetPath, second.targetPath,
'translations/state/ja/docs.json'].sort());
});
+
+test('normal generation defaults to ten target pages and leaves the next task
untouched', async t => {
+ const root = fs.mkdtempSync(path.join(os.tmpdir(),
'translation-ten-page-batch-'));
+ t.after(() => fs.rmSync(root, {recursive: true, force: true}));
+ const put = (file, text) => {
+ fs.mkdirSync(path.dirname(path.join(root, file)), {recursive: true});
+ fs.writeFileSync(path.join(root, file), text);
+ };
+ put('translations/scope/ja.json', JSON.stringify(scope));
+ put('translations/state/ja/docs.json', JSON.stringify({schemaVersion: 1,
pages: []}));
+ put('translations/state/ja/shards.schema.json',
JSON.stringify({schemaVersion: 1, format: 'page-record-v1'}));
+ const tasks = Array.from({length: 11}, (_, index) => {
+ const source = `# Guide ${index}\n\nA guide.\n`;
+ const sourcePath = `docs/page-${index}.md`;
+ put(sourcePath, source);
+ return {pluginId: 'default', version: 'current', locale: 'ja',
groupId: 'DEV',
+ id: `T06-DEV-${String(index + 1).padStart(15, '0')}`, docId:
`page-${index}`,
+ sourcePath, sourceHash: hash(source), kind: 'translate_new',
+ targetPath:
`i18n/ja/docusaurus-plugin-content-docs/current/page-${index}.md`,
+ requiresHumanReview: false, activePr: null};
+ });
+ const git = (...args) => execFileSync('git', args, {cwd: root, encoding:
'utf8'}).trim();
+ git('init', '-q'); git('add', '.');
+ git('-c', 'user.name=Test', '-c', '[email protected]', 'commit',
'-qm', 'eleven pending pages');
+ const sourceCommit = git('rev-parse', 'HEAD');
+ let calls = 0;
+ const adapter = {model: 'fixture', usedTokens: 0, translate: async
({units}) => {
+ calls++;
+ return {translations: units.map(unit => ({id: unit.id, text: '訳' +
unit.text}))};
+ }};
+ const manifest = await generate({rootDir: root, target, output:
path.join(root, 'output'),
+ plan: {schemaVersion: 1, locale: 'ja', sourceCommit, selected:
tasks.map(task => task.id), tasks, exceptions: []},
+ reconciliation: {botPrs: [], botDetails: [], blockedTargets: []},
ensureFresh: () => {},
+ adapter, glossary: require('../../../translations/glossary/ja.json')});
+ assert.equal(manifest.candidates.length, 10);
+ assert.equal(manifest.failures.length, 0);
+ assert.equal(calls, 10);
+ assert.equal(loadRecords(root, 'ja').size, 10);
+ assert(!fs.existsSync(path.join(root, tasks[10].targetPath)));
+ assert(!fs.existsSync(path.join(root, stateFileFor(tasks[10]))));
+});
diff --git a/scripts/i18n/simulate-workflow.cjs
b/scripts/i18n/simulate-workflow.cjs
index dec7944379a..9422b3a3c70 100644
--- a/scripts/i18n/simulate-workflow.cjs
+++ b/scripts/i18n/simulate-workflow.cjs
@@ -53,9 +53,10 @@ if (args[0] === 'api' && args.some(arg =>
arg.startsWith('repos/apache/doris-web
assert(args.includes('--draft'));
assert.equal(value('--base'), 'master');
assert.equal(value('--repo'), 'apache/doris-website');
+ assert.equal(value('--label'), 'translate-ja');
const body = fs.readFileSync(value('--body-file'), 'utf8');
assert.match(body, /japanese-translation-automation:v1
bot-head=[a-f0-9]{40}/);
- const record = {draft: true, base: value('--base'), head: value('--head'),
body,
+ const record = {draft: true, base: value('--base'), head: value('--head'),
labels: [value('--label')], body,
url: 'https://example.invalid/apache/doris-website/pull/1'};
fs.writeFileSync(process.env.I18N_SIM_PR_RECORD, JSON.stringify(record,
null, 2) + '\\n');
process.stdout.write(record.url + '\\n');
@@ -145,7 +146,7 @@ function main() {
sourceCommit: plan.sourceCommit, selected: plan.selected, sourcePath,
invalidModel: {exitCode: 1, error: failure.failures[0].error,
usedTokens: failure.usedTokens},
generation: {candidates: generation.candidates.length, usedTokens:
generation.usedTokens},
- draftPr: {url: pr.url, branch: pr.head, commit: branch, files:
committedFiles}};
+ draftPr: {url: pr.url, branch: pr.head, commit: branch, files:
committedFiles, labels: pr.labels}};
fs.writeFileSync(path.join(output, 'report.json'), JSON.stringify(report,
null, 2) + '\n');
process.stdout.write(JSON.stringify(report, null, 2) + '\n');
}
diff --git a/scripts/i18n/workflow-generate.cjs
b/scripts/i18n/workflow-generate.cjs
index fbf319e8a3d..35276255640 100644
--- a/scripts/i18n/workflow-generate.cjs
+++ b/scripts/i18n/workflow-generate.cjs
@@ -81,7 +81,7 @@ function bodyFor(manifest) {
'', `${markerPrefixFor(manifest.language)}PENDING -->`, ''].join('\n');
}
-async function generate({plan, reconciliation, maxPages = 2, target,
+async function generate({plan, reconciliation, maxPages = 10, target,
adapter, glossary, rootDir = root, ensureFresh = ensureMasterFresh,
output}) {
assert(Number.isInteger(maxPages) && maxPages > 0 && maxPages <= 20,
'Invalid page limit');
assert(plan.schemaVersion === 1 && plan.sourceCommit ===
execFileSync('git', ['rev-parse', 'HEAD'],
@@ -173,7 +173,7 @@ async function main() {
assert(process.env.CODEX_BIN && fs.existsSync(process.env.CODEX_BIN),
'Pinned Codex runtime is required');
const adapter = codexAdapter({model: process.env.I18N_CODEX_MODEL ||
'gpt-6-sol', language: languageName(root, locale), locale,
codexBin: process.env.CODEX_BIN, codexHome: process.env.CODEX_HOME});
- const manifest = await generate({plan, reconciliation, maxPages:
Number(process.env.I18N_MAX_PAGES || '2'),
+ const manifest = await generate({plan, reconciliation, maxPages:
Number(process.env.I18N_MAX_PAGES || '10'),
target, adapter, glossary, output});
if (process.env.GITHUB_OUTPUT) fs.appendFileSync(process.env.GITHUB_OUTPUT,
`created=${manifest.candidates.length > 0}\n`);
diff --git a/scripts/i18n/workflow-publish.cjs
b/scripts/i18n/workflow-publish.cjs
index 3584ee178d1..5b26f5b8fe2 100644
--- a/scripts/i18n/workflow-publish.cjs
+++ b/scripts/i18n/workflow-publish.cjs
@@ -91,7 +91,8 @@ function main() {
}
git('push', 'origin', `HEAD:refs/heads/${branch}`);
const url = execFileSync('gh', ['pr', 'create', '--repo',
'apache/doris-website', '--base', 'master',
- '--head', branch, '--draft', '--title', `docs(i18n): machine
${language} translation candidates`,
+ '--head', branch, '--draft', '--label', `translate-${locale}`,
+ '--title', `docs(i18n): machine ${language} translation candidates`,
'--body-file', bodyFile], {cwd: root, encoding: 'utf8'}).trim();
if (process.env.GITHUB_STEP_SUMMARY)
fs.appendFileSync(process.env.GITHUB_STEP_SUMMARY,
`\nOpened draft translation PR: ${url}; ${manifest.candidates.length}
candidate pages.\n`);
diff --git a/translations/README.md b/translations/README.md
index a87bbb8f768..668878fa167 100644
--- a/translations/README.md
+++ b/translations/README.md
@@ -18,6 +18,16 @@ registered by the English docs plugin. A Chinese-only file
is reported as
`community`) to the label that locale's search page shows; a locale without it
shows the raw context path.
+The translation workflow's `validate` mode selects one exact source page for a
+live check. Normal `generate` runs produce at most 10 target pages per batch,
+including any reused Dev/4.x partner; the planner can consider up to 20 tasks.
+Automated draft PRs receive the `translate-<locale>` repository label, such as
+`translate-ja`. Create that label before enabling publication for a new locale.
+
+Build Check installs dependencies and runs the i18n unit tests before the
+translation gate. The gate checks source metadata and state without a site
+build; the scoped website build and static-page verification run afterward.
+
## State schema, version 1
`state/ja/docs.json` is the legacy shared ledger. New Japanese records can live
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
