This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow-steward.git
The following commit(s) were added to refs/heads/main by this push:
new 926480ae docs(spec): correct stale eval-coverage Known Gap in
meta-and-quality-tooling (#561)
926480ae is described below
commit 926480aec6ee66199565f826dc80addf304ac686
Author: Justin Mclean <[email protected]>
AuthorDate: Sat Jun 27 06:15:55 2026 +1000
docs(spec): correct stale eval-coverage Known Gap in
meta-and-quality-tooling (#561)
The Known Gaps section claimed "~15 suites, coverage incomplete". All 44
shipped skills now have a matching suite in tools/skill-evals/evals/; the
soft eval-coverage check (check #8) in skill-and-tool-validator enforces
this going forward. Remove the stale back-fill claim and state the actual
coverage.
Generated-by: Claude (Opus 4.7)
---
tools/spec-loop/specs/meta-and-quality-tooling.md | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/tools/spec-loop/specs/meta-and-quality-tooling.md
b/tools/spec-loop/specs/meta-and-quality-tooling.md
index 9f4b0823..fd15bd79 100644
--- a/tools/spec-loop/specs/meta-and-quality-tooling.md
+++ b/tools/spec-loop/specs/meta-and-quality-tooling.md
@@ -82,9 +82,10 @@ uv run --project tools/skill-and-tool-validator --group dev
skill-and-tool-valid
## Known gaps
-- **Eval coverage is incomplete** — the harness has ~15 suites but the
- repo has more skills than that; skills added before the per-skill-eval
- convention have no suite. Back-filling one suite per uncovered skill is
- a tracked work item.
-- Other gaps appear as new quality checks worth adding (e.g. a spec
- validator analogous to the skill validator) — recorded by the plan pass.
+- **Eval coverage is complete.** All 44 shipped skills have a matching
+ suite in `tools/skill-evals/evals/`; the soft eval-coverage check in
+ `skill-and-tool-validator` (check #8) warns when a newly added skill
+ has no suite, keeping coverage complete going forward.
+- Other gaps appear as new quality checks worth adding — recorded by the
+ plan pass. The spec validator (analogous to the skill validator) and
+ the ASF-coupling advisory lint are two recent additions to this surface.
