justinmclean commented on PR #484: URL: https://github.com/apache/airflow-steward/pull/484#issuecomment-4678705499
Run teh review skill on this: ## Correctness [blocking] step-2-present/fixtures/output-spec.md:17 — the output-spec, which is appended to the model's system prompt, literally states the answer: "Per the skill's OUTPUT CONTRACT, the correct answer is always verbatim." Cases 2 and 3 are supposed to test whether the model resists pressure to paraphrase; with the answer in-context they pass-stamp regardless of whether the skill's OUTPUT CONTRACT actually reached the model. Remove that line (the field-semantics lines above it are fine). [advisory] step-1-command / step-2-present anchors verified correct — the OUTPUT CONTRACT blockquote and Step-A rules fall inside the extraction window, so the skill rule does reach the model. The leak above is the only wiring problem. ## Conventions [blocking — verified] Malformed SPDX header in fixture report.md files. The header opens with <\!-- (literal backslash) instead of <!-- — confirmed in the raw bytes via cat -A. That is not a valid comment opener, so a license scanner keying on a proper comment may not recognise it. Affects ~13 fixture report.md files across all four steps. Likely a fixture-generation escaping bug; fix the generator/template, not each file by hand. [advisory] evals/setup-status/README.md uses uv run --project tools/skill-evals — your convention is --directory (per your own runner notes). Will fail if copy-pasted. Some more work needs to be done here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
