DanielLeens opened a new pull request, #479: URL: https://github.com/apache/seatunnel-website/pull/479
## Summary - add a postbuild generator that derives `sitemap.xml`, `zh-CN/sitemap.xml`, `llms.txt`, `llms-full.txt`, and `robots.txt` from the final built website output - keep the generated sitemap scope aligned with the existing Docusaurus `ignorePatterns`, including `/blog/**` and non-latest versioned docs, while also excluding `noindex` and search pages - verify the generated discovery artifacts in the scheduled deploy workflow with both positive route checks and negative regression checks ## Root Cause The scheduled website deploy syncs fresh docs from `apache/seatunnel`, but the build pipeline did not publish dedicated LLM discovery artifacts or enforce the OpenAI crawler policy checks needed for the new discovery requirements. ## Validation - `npm run sync` - `GH_TOKEN=$(gh auth token) GITHUB_TOKEN=$(gh auth token) npm run build` - reused the successful `build/` output from that full build after the final postbuild-only filtering change, then reran `node tools/generate-llm-artifacts.js` - reused the same successful `build/` output to rerun the workflow-equivalent artifact verification block, including en/zh key docs routes, `OAI-SearchBot` / `GPTBot` / `ChatGPT-User`, `search` exclusion, `/blog/**` exclusion, and non-latest versioned docs exclusion ## Notes - This repo does not use Maven, so `mvn spotless:apply` is not applicable here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
