goutamadwant opened a new pull request, #12469: URL: https://github.com/apache/seatunnel/pull/12469
### Purpose of this pull request The `Dead links` job runs `markdown-link-check` on every `*.md` file in the repository (about 1,090 files) on every push. It is `continue-on-error`, so it does not block merges, but it still shows as a failed job on unrelated PRs whenever any external link in the docs is briefly unavailable. **Before** - Fork job [106620300149](https://github.com/goutamadwant/seatunnel/actions/runs/35688494412/job/106620300149) (PR #12429, a Cassandra connector change) failed on a link in `docs/zh/connectors/source/FakeSource.md`, a file the PR did not touch: ``` [✖] https://github.com/lightbend/config/blob/main/HOCON.md → Status: 503 ``` An external 503 also failed this job on #12367. - The loop also stops at the first file with a dead link. GitHub runs `run:` steps with `bash -e`, so the failing `markdown-link-check` ends the step. In that job the scan stopped at file 1,030 of 1,087, and the remaining files were never checked. The file list comes from `$(find ...)`, so a path with a space would also be split. - A full scan takes about 6.5–7.5 minutes per run (nightly runs 2026-09-13..24). **After** - On branch pushes, which is how PR builds run in the contributor's fork, the job checks only markdown files added or modified relative to `apache/seatunnel` `dev`. It fetches `dev` the same way the `changes` job does, then diffs the branch against its merge base with `dev`. Deleted files are skipped. If there is no changed markdown file, the job ends without installing the tool. If the merge base cannot be found, it prints a warning and falls back to the full scan. - The nightly run (`schedule`) and pushes to `apache/seatunnel` `dev` still check every markdown file. Link rot in unchanged files is still reported there. - Every listed file is checked. A file that fails is checked once more after 30 seconds, because `markdown-link-check` 3.8.7 only retries HTTP 429. `.dlc.json` already enables that retry (`retryOn429`, `retryCount`, `fallbackRetryDelay`); the tool has no retry for 5xx or timeouts. At the end the job lists every file that still has dead links and fails. - `.dlc.json` and `aliveStatusCodes` are unchanged, so no new status code is treated as alive. This removes most of the dead-link noise from PR builds and keeps the check meaningful for PRs that edit docs. It also saves about 6 runner-minutes on pushes that don't touch markdown. ### Does this PR introduce _any_ user-facing change? No. CI only. ### How was this patch tested? - `actionlint` 1.7.7 on `backend.yml`: no findings. YAML parsed with PyYAML, and `bash -n` on both extracted step scripts. - I ran the two step scripts from the workflow locally in a scratch clone, with `[email protected]` and a local mirror standing in for `apache/seatunnel`. The test branch changed two docs, deleted a third and changed `pom.xml`: - Branch mode: 2 files listed; the deleted file and `pom.xml` were excluded. The file with a broken link (404 and a missing relative file) was retried once and reported, the other file was still checked, and the step exited 1. - Branch with only a `pom.xml` change: `Markdown files to check: 0`, then `No markdown files to check.`, exit 0. - `FULL_SCAN=true`: 1,087 files listed. - Fetch failure: warning printed, full list of 1,087 files. - In a second scratch repo, a branch that added `-cfoo.md` (a name that looks like an option) and `文档.md`: both paths were listed unquoted and checked, and the dead link in `-cfoo.md` was reported. - For comparison, the old loop under `bash -e` with the broken file first checked only that file and stopped. - Fork run of this branch: the change is under `.github/workflows/`, which is lightweight in `ci_scope.py`, so it does not start the connector or engine matrix. The `Dead links` job there should show `Markdown files to check: 0` and finish in under a minute. Not verified: the `schedule` and `apache/seatunnel` `dev` full-scan paths on GitHub. They only run upstream. Also, a PR that renames or deletes a doc can break links in files it did not change. The nightly full scan still catches that, but the PR run does not. ### Check list * [ ] If any new Jar binary package adding in your PR, please add License Notice according [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md) * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs * [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR. * [ ] If you are contributing the connector code, please check that the following files are updated: 1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it 2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml) 3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml) 4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/) 5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
