This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 012e6a918be2 [fix](website) Prevent Apache 500 errors from oversized
redirect rules (#4566)
012e6a918be2 is described below
commit 012e6a918be232be241485742438e267fd4a546f
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Sat Oct 31 08:01:02 2026 +0000
[fix](website) Prevent Apache 500 errors from oversized redirect rules
(#4566)
Archiving the historical documentation made thousands of aliases point
to the same archive page. The publication optimizer combined each
destination's aliases into one `RewriteRule`, producing a 263,259-byte
line in the artifact from [run
41149820126](https://github.com/apache/doris-website/actions/runs/41149820126).
Apache rejects `.htaccess` lines above [8,190
characters](https://httpd.apache.org/docs/2.4/configuring.html#syntax),
so uncached website requests returned HTTP 500 while cached pages still
showed the old 2.1 navigation entry.
This change splits grouped redirects into complete directives of at most
8,000 UTF-8 bytes, including escaped paths, targets and flags. It
applies to both root and static-resource controls. Individual redirects
that cannot fit fail the build. The existing publication verifier also
rejects oversized control lines, including continued lines, before the
artifact can be uploaded.
## Validation
- `DORIS_TEST_APACHE=true node --test
scripts/publish/__tests__/*.test.cjs`: 30 passed, zero skips. New
regressions cover 9,000 archive/asset aliases across five locales, UTF-8
byte limits, oversized individual sources/targets, nested controls,
continued lines, query/fragment handling and real Apache responses.
- `node --test scripts/__tests__/build-locales.test.cjs`: 11 passed,
including preservation of the previous artifact after publication
verification fails.
- `NODE_OPTIONS=--max-old-space-size=2048 node
scripts/publish/verify-fixture.cjs --apache`: five-locale Rspack
production fixture and Apache HTTP checks passed.
- Replayed all 92,059 redirect entries from deployment commit `8321964`
locally, using its actual homepage/archive HTML and placeholder redirect
destinations. The original controls reproduced HTTP 500 with `Line too
long`; regenerated controls have a maximum line length of 8,000 bytes.
All ten locale homepage/archive requests returned 200, and 479 sampled
redirect requests matched their expected status and destination.
- `git diff --check`: passed. The full website build is left to the
existing Build Check workflow.
## Self-review
- Goal: the deployed failure is reproduced and resolved with the same
redirect inventory.
- Scope: changes are confined to publication controls, artifact
verification, regression tests and their README.
- Information architecture: active documentation versions, localized
destinations and archive routing retain their existing behavior.
- URL compatibility: exact aliases, trailing-slash/index variants,
query/fragment behavior and temporary asset redirects are covered by
Apache checks.
- Frontend and workflow conventions: existing build/verification entry
points are reused; React, styling and workflow permissions are outside
this change.
- Verification: the tests cover the failure at both fixture and
production-inventory scale; CI will run the complete publishing build.
- Other correctness, accessibility, SEO and maintainability checks: no
additional issues found in this focused review.
## Versions
- [x] dev
- [x] 4.x
- [x] 3.x
- [x] 2.1 or older (not covered by version/language sync gate)
## Languages
- [x] Chinese
- [x] English
The common publication path also covers Japanese, Korean and Russian. No
per-language documentation edits are required.
## Docs Checklist
- [x] Checked by AI
- [x] Test Cases Built
- [x] Updated required version and language counterparts, or explained
why not
- [x] If only one language changed, confirmed whether source/translation
counterparts need sync
Co-authored-by: morningman <[email protected]>
---
scripts/publish/README.md | 11 +-
scripts/publish/__tests__/apache-controls.test.cjs | 114 +++++++++++++++++++++
scripts/publish/apache-controls.cjs | 21 ++++
scripts/publish/optimize.cjs | 39 +++++--
scripts/publish/verify.cjs | 5 +-
5 files changed, 175 insertions(+), 15 deletions(-)
diff --git a/scripts/publish/README.md b/scripts/publish/README.md
index 024a7548afa2..ec777c4e6a34 100644
--- a/scripts/publish/README.md
+++ b/scripts/publish/README.md
@@ -20,8 +20,10 @@ retain their existing asset layout.
present because the asset CDN does not interpret Apache rules.
- Exact Docusaurus client-redirect templates become HTTP 301 rules. Large
`.html`/`.htm`, 3.x and release-note alias families use guarded prefix rules;
- other aliases are grouped by destination. Real files retain priority, missing
- destinations do not redirect through a generic rule, and query/fragment
+ other aliases are grouped by destination and split into directives of at most
+ 8,000 UTF-8 bytes, below Apache's 8,190-byte `.htaccess` line limit. The
bound
+ includes escaped paths, redirect targets and flags. Real files retain
priority,
+ missing destinations do not redirect through a generic rule, and
query/fragment
forwarding follows the original template. Custom HTML such as the tracked
Slack redirect remains a page.
Binary compatibility rules live in their static-resource directories, so
@@ -80,8 +82,9 @@ yarn serve:publish --site .i18n-build/dev-publish
redirect pages, rule counts and verification. `.i18n-build/build-result.json`
also records per-locale timing and build success. Counts describe raw files,
not compressed network transfer or Git pack size. Source changes, invalid
-redirects, resource verification failures and asset collisions preserve the
-previous complete output.
+redirects, oversized Apache control lines (including continued lines and
+static-resource controls), resource verification failures and asset collisions
+preserve the previous complete output.
Changes to publication infrastructure make Build Check build every active
version and enabled locale using the optimized mode on CI, with the existing
diff --git a/scripts/publish/__tests__/apache-controls.test.cjs
b/scripts/publish/__tests__/apache-controls.test.cjs
new file mode 100644
index 000000000000..780a2c8770ad
--- /dev/null
+++ b/scripts/publish/__tests__/apache-controls.test.cjs
@@ -0,0 +1,114 @@
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const {writeApache, inventory} = require('../optimize.cjs');
+const {verify} = require('../verify.cjs');
+
+const locales = ['en','zh-CN','ja','ko','ru'].map(locale =>
({locale,directory:locale === 'en' ? '.' : locale}));
+
+function fixture(t, bulk = false) {
+ const site =
fs.mkdtempSync(path.join(os.tmpdir(),'doris-apache-controls-'));
+ t.after(() => fs.rmSync(site,{recursive:true,force:true}));
+ const write = (file,content) => {
+ fs.mkdirSync(path.dirname(path.join(site,file)),{recursive:true});
+ fs.writeFileSync(path.join(site,file),content);
+ };
+ const redirects = new Map();
+ write('.htaccess','# Original controls\n');
+ write('assets/shared/image.png','image');
+ for (const locale of locales) {
+ const prefix = locale.directory === '.' ? '' : locale.directory+'/';
+ write(prefix+'index.html','<main>Home</main>');
+ write(prefix+'archived-docs/index.html','<main>Archived
Versions</main>');
+ if (!bulk) continue;
+ for (let i = 0; i < 600; i++) {
+ redirects.set('/'+prefix+`docs/2.1/sql-manual/历史函数-${i}/`,
+
{to:'/'+prefix+'archived-docs/',preserveQuery:true,status:301});
+ redirects.set('/'+prefix+`docs/2.0/sql-manual/历史函数-${i}/`,
+
{to:'/'+prefix+'archived-docs/?version=2.0#downloads',preserveQuery:false,status:301});
+ redirects.set('/'+prefix+`images/历史图片-${i}.png`,
+ {to:'/assets/shared/image.png',preserveQuery:true,status:302});
+ }
+ }
+
write('.publish/redirects.json',JSON.stringify({assetPrefixes:[],redirects:[...redirects]
+ .map(([from,item]) =>
[from,item.to,item.preserveQuery,item.status])}));
+ return {site,write,redirects};
+}
+
+test('bounds many-to-one archive and Unicode asset rules in every locale by
UTF-8 bytes', t => {
+ const {site,redirects} = fixture(t,true);
+ writeApache({site,locales,redirects});
+ const controls = inventory(site).filter(item => path.basename(item.file)
=== '.htaccess');
+ assert.equal(controls.length,6);
+ for (const item of controls) {
+ for (const line of fs.readFileSync(item.file,'utf8').split('\n')) {
+ assert(Buffer.byteLength(line) <= 8000,`${item.relative}:
${Buffer.byteLength(line)} bytes`);
+ }
+ }
+ verify(site);
+});
+
+test('rejects an individual redirect that cannot fit into an Apache rule', t
=> {
+ for (const field of ['source','target']) {
+ const {site} = fixture(t);
+ const source = field === 'source' ? '/'+ '长'.repeat(3000) : '/legacy/';
+ const to = field === 'target' ? '/'+ 'x'.repeat(8000) :
'/archived-docs/';
+ assert.throws(() => writeApache({site,locales,redirects:new Map([
+ [source,{to,preserveQuery:true,status:301}],
+ ])}),/Apache.*line/i,field);
+ }
+});
+
+test('publication verification rejects oversized root and nested Apache
controls', t => {
+ const {site,write,redirects} = fixture(t);
+ writeApache({site,locales,redirects});
+ for (const file of ['.htaccess','zh-CN/images/.htaccess']) {
+ // Fewer than 8,000 characters can still exceed the limit in UTF-8.
+ write(file,'#'+'长'.repeat(3000)+'\n');
+ assert.throws(() => verify(site),error => error.message.includes(file)
&& /Apache.*line/i.test(error.message));
+ write(file,'# Valid controls\n');
+ }
+ // Apache joins continuation lines before checking the line limit.
+ write('.htaccess','#'+'x'.repeat(5000)+'\\\n'+'x'.repeat(5000)+'\n');
+ assert.throws(() => verify(site),/Apache.*line/i);
+});
+
+test('Apache serves pages and large archive/asset redirect groups after
splitting',
+ {skip:process.env.DORIS_TEST_APACHE !== 'true'}, async t => {
+ const {site,redirects} = fixture(t,true);
+ writeApache({site,locales,redirects});
+ const server = await require('../apache.cjs').apache(site);
+ t.after(() => server.close());
+ const probe = async (source,options) => {
+ const response = await fetch(server.origin+source,options);
+ await response.arrayBuffer();
+ return response;
+ };
+ for (const locale of locales) {
+ const prefix = locale.directory === '.' ? '' : locale.directory+'/';
+ assert.equal((await probe('/'+prefix)).status,200);
+ assert.equal((await probe('/'+prefix+'archived-docs/')).status,200);
+ for (const i of [0,99,100,299,599]) {
+ const from = '/'+prefix+`docs/2.1/sql-manual/历史函数-${i}/`;
+ for (const source of [from,from.slice(0,-1),from+'index.html']) {
+ const response = await
probe(source+'?probe=1',{redirect:'manual'});
+ assert.equal(response.status,301,source);
+ assert.equal(new
URL(response.headers.get('location')).pathname,'/'+prefix+'archived-docs/');
+ assert.equal(new
URL(response.headers.get('location')).search,'?probe=1');
+ }
+ const fixed = await
probe('/'+prefix+`docs/2.0/sql-manual/历史函数-${i}/?probe=1`,{redirect:'manual'});
+ assert.equal(fixed.status,301);
+ assert.equal(new
URL(fixed.headers.get('location')).pathname,'/'+prefix+'archived-docs/');
+ assert.equal(new
URL(fixed.headers.get('location')).search,'?version=2.0');
+ assert.equal(new
URL(fixed.headers.get('location')).hash,'#downloads');
+ const image = await
probe('/'+prefix+`images/历史图片-${i}.png?probe=1`,{redirect:'manual'});
+ assert.equal(image.status,302);
+ assert.equal(new
URL(image.headers.get('location')).pathname,'/assets/shared/image.png');
+ assert.equal(new
URL(image.headers.get('location')).search,'?probe=1');
+ assert.equal(image.headers.get('cache-control'),'no-store');
+ }
+ assert.equal((await
probe('/'+prefix+'docs/2.1/unknown/',{redirect:'manual'})).status,404);
+ }
+});
diff --git a/scripts/publish/apache-controls.cjs
b/scripts/publish/apache-controls.cjs
new file mode 100644
index 000000000000..7b7170f3f388
--- /dev/null
+++ b/scripts/publish/apache-controls.cjs
@@ -0,0 +1,21 @@
+const assert = require('node:assert/strict');
+
+// Apache limits .htaccess lines to 8,190 bytes. Leave room for line endings
+// and count the complete escaped directive, including its target and flags.
+const maxLineBytes = 8000;
+
+function validateApacheControls(content, source) {
+ let bytes = 0, start = 1;
+ for (const [index,line] of content.split(/\r?\n/).entries()) {
+ bytes += Buffer.byteLength(line);
+ assert(bytes <= maxLineBytes,
+ `Apache control line exceeds ${maxLineBytes} bytes:
${source}:${start} (${bytes} bytes)`);
+ // Continued physical lines form one logical configuration line.
+ if (!line.endsWith('\\')) {
+ bytes = 0;
+ start = index+2;
+ }
+ }
+}
+
+module.exports = {maxLineBytes,validateApacheControls};
diff --git a/scripts/publish/optimize.cjs b/scripts/publish/optimize.cjs
index f07e3ccb5c94..aaab1c8d1c2b 100644
--- a/scripts/publish/optimize.cjs
+++ b/scripts/publish/optimize.cjs
@@ -2,6 +2,7 @@ const fs = require('node:fs');
const path = require('node:path');
const {createHash} = require('node:crypto');
const assert = require('node:assert/strict');
+const {maxLineBytes,validateApacheControls} = require('./apache-controls.cjs');
const binary =
/\.(?:png|jpe?g|gif|webp|avif|svg|ico|ttf|otf|woff2?|eot|pdf|zip|docx?|xlsx?|mp[34]|webm|wav|ogg)$/i;
const begin = '# BEGIN DORIS PUBLISH OPTIMIZATION';
@@ -238,13 +239,27 @@ function
writeApache({site,locales,redirects,replaceExisting = false}) {
// Quote and escape config syntax, while leaving the already encoded
// redirect target intact (NE avoids double encoding it).
const target =
item.to.replace(/\\/g,'\\\\').replace(/"/g,'\\"').replace(/\$/g,'\\$').replace(/%/g,'\\%');
- const pattern = from.map(value => escapeRegex(value.path)
- + (value.directory ? '(?:/index\\.html|/)?' :
'')).join('|').replace(/"/g,'\\"');
- const rule = `RewriteRule "^(?:${pattern})$" "${target}"
[R=${item.status},L,NE${item.preserveQuery ? '' : ',QSD'}${item.status === 302
? ',E=DORIS_ASSET_REDIRECT:1' : ''}]`;
- if (directory) {
- if (!assetRules.has(directory)) assetRules.set(directory,[]);
- assetRules.get(directory).push(rule);
- } else rules.push(rule);
+ const prefix = 'RewriteRule "^(?:';
+ const suffix = `)$" "${target}"
[R=${item.status},L,NE${item.preserveQuery ? '' : ',QSD'}${item.status === 302
? ',E=DORIS_ASSET_REDIRECT:1' : ''}]`;
+ const overhead = Buffer.byteLength(prefix+suffix);
+ if (directory && !assetRules.has(directory))
assetRules.set(directory,[]);
+ const output = directory ? assetRules.get(directory) : rules;
+ let patterns = [], bytes = overhead;
+ for (const value of from) {
+ const pattern = (escapeRegex(value.path)
+ + (value.directory ? '(?:/index\\.html|/)?' :
'')).replace(/"/g,'\\"');
+ const size = Buffer.byteLength(pattern);
+ assert(overhead+size <= maxLineBytes,
+ `Publication redirect cannot fit in an Apache control line of
${maxLineBytes} bytes`);
+ if (bytes+size+(patterns.length ? 1 : 0) > maxLineBytes) {
+ output.push(prefix+patterns.join('|')+suffix);
+ patterns = [];
+ bytes = overhead;
+ }
+ bytes += size+(patterns.length ? 1 : 0);
+ patterns.push(pattern);
+ }
+ if (patterns.length) output.push(prefix+patterns.join('|')+suffix);
}
rules.push('</IfModule>','<IfModule mod_headers.c>',
'Header always set Cache-Control "no-store"
env=DORIS_ASSET_REDIRECT','</IfModule>',end);
@@ -255,14 +270,18 @@ function
writeApache({site,locales,redirects,replaceExisting = false}) {
assert(finish >= end.length, 'Incomplete publication controls');
original =
original.slice(0,original.indexOf(begin))+original.slice(finish).replace(/^\n+/,'');
}
- fs.writeFileSync(controls,rules.join('\n')+'\n\n'+original);
+ const content = rules.join('\n')+'\n\n'+original;
+ validateApacheControls(content,controls);
+ fs.writeFileSync(controls,content);
for (const [directory,lines] of assetRules) {
const file = path.join(site,directory,'.htaccess');
assert(!fs.existsSync(file) || (replaceExisting &&
fs.readFileSync(file,'utf8').startsWith(begin)),
`Cannot replace existing asset controls: ${directory}`);
fs.mkdirSync(path.dirname(file),{recursive:true});
- fs.writeFileSync(file,[begin,'<IfModule mod_rewrite.c>','RewriteEngine
On',
- 'RewriteOptions
IgnoreInherit',...lines,'</IfModule>',end,''].join('\n'));
+ const content = [begin,'<IfModule mod_rewrite.c>','RewriteEngine On',
+ 'RewriteOptions
IgnoreInherit',...lines,'</IfModule>',end,''].join('\n');
+ validateApacheControls(content,file);
+ fs.writeFileSync(file,content);
}
return {root:rules.filter(line => line.startsWith('RewriteRule')).length,
total:rules.filter(line => line.startsWith('RewriteRule')).length
diff --git a/scripts/publish/verify.cjs b/scripts/publish/verify.cjs
index 38bf8e7f9d27..d81acd15507d 100644
--- a/scripts/publish/verify.cjs
+++ b/scripts/publish/verify.cjs
@@ -3,6 +3,7 @@ const path = require('node:path');
const assert = require('node:assert/strict');
const {Script} = require('node:vm');
const {inventory} = require('./optimize.cjs');
+const {validateApacheControls} = require('./apache-controls.cjs');
function verify(site) {
site = path.resolve(site);
@@ -36,7 +37,9 @@ function verify(site) {
references++;
};
for (const item of files) {
- if (item.relative.endsWith('.html')) {
+ if (path.basename(item.file) === '.htaccess') {
+
validateApacheControls(fs.readFileSync(item.file,'utf8'),item.relative);
+ } else if (item.relative.endsWith('.html')) {
const content = fs.readFileSync(item.file,'utf8');
for (const tag of
content.matchAll(/<(?:script|link|img|source|video|audio)\b[^>]*>/g)) {
for (const attribute of
tag[0].matchAll(/\b(?:src|href)=(?:"([^"]*)"|'([^']*)'|([^\s>]+))/g)) {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
