jason810496 opened a new issue, #74086: URL: https://github.com/apache/airflow/issues/74086
`check-supervisor-schemas-versions` and `check-execution-api-versions` check out upstream `main` with `git worktree add` to compare schemas. When they run from `git commit`, git has set `GIT_INDEX_FILE` to the index being committed, and prek passes it to hooks unchanged. The `git reset --hard` that `git worktree add` runs in the new worktree then writes upstream `main`'s tree into that index. Manual `prek run` and CI are not affected, because `GIT_INDEX_FILE` is only set inside `git commit`. The hooks reach `git worktree add` when a commit changes a watched `.py` file and nothing under the matching `versions/` directory: - `check-supervisor-schemas-versions` ([line 126](https://github.com/apache/airflow/blob/dd271599939732697c1fc6f24b0dd20682eeec00/scripts/ci/prek/check_supervisor_schemas_versions.py#L126)): `task-sdk/src/airflow/sdk/execution_time/comms.py`, a `.py` file under `task-sdk/src/airflow/sdk/execution_time/schema/`, or `airflow-core/src/airflow/dag_processing/processor.py`. - `check-execution-api-versions` ([line 79](https://github.com/apache/airflow/blob/dd271599939732697c1fc6f24b0dd20682eeec00/scripts/ci/prek/check_execution_api_versions.py#L79)): a `.py` file other than `__init__.py` under `airflow-core/src/airflow/api_fastapi/execution_api/datamodels/`. What happens: - Plain `git commit` in a linked worktree: prek reports the hook as Failed with "files were modified by this hook", even when it found no schema change, and the commit stops with the index holding upstream `main`'s tree. Committing again, plainly or with `--no-verify`, can succeed and records upstream `main`'s tree: the commit reverts the branch's own changes and drops the staged change. If you also had unstaged changes that another hook's fix conflicted with, prek restores the working tree from the swapped index, so tracked files get upstream `main`'s content too. - `git commit -a` or `git commit <paths>`, in any worktree: the hook fails the same way, but git then discards the temporary index, so nothing is lost. - Plain `git commit` in the main worktree: `GIT_INDEX_FILE` is the relative `.git/index`, which `git worktree add` cannot open, so the hook fails ("Failed to generate upstream snapshot for comparison", or "Could not generate schema from main" for the API hook) and the commit is blocked. Repro with a plain git hook, no prek needed. It only creates temp directories: ```sh #!/bin/sh # A pre-commit hook that runs "git worktree add" rewrites the committing worktree's index. set -e d=$(mktemp -d) git init -q -b main "$d/repo" && cd "$d/repo" git config user.name repro && git config user.email [email protected] echo main > a.txt && git add a.txt && git commit -qm main git checkout -qb feature && echo feature > a.txt && echo b > b.txt git add a.txt b.txt && git commit -qm feature && git checkout -q --detach main git worktree add -q "$d/linked" feature && cd "$d/linked" mkdir "$d/hooks" && git config core.hooksPath "$d/hooks" cat > "$d/hooks/pre-commit" <<'EOF' #!/bin/sh echo "hook sees GIT_INDEX_FILE=$GIT_INDEX_FILE" wt="$(mktemp -d)/wt" git worktree add -q "$wt" main && git worktree remove --force "$wt" EOF chmod +x "$d/hooks/pre-commit" echo staged >> b.txt && git add b.txt echo "staged tree: $(git write-tree)" git commit -qm x echo "main tree: $(git rev-parse 'main^{tree}')" echo "commit tree: $(git rev-parse 'HEAD^{tree}')" echo "index tree: $(git write-tree)" git show --stat --format='new commit %h "%s":' HEAD git status --short ``` The commit and the index both end up with `main`'s tree (temp paths and the commit hash differ per run): ```text staged tree: f2645c8307f58574b28a3109a33778eeeb790479 hook sees GIT_INDEX_FILE=/tmp/tmp.XXXX/repo/.git/worktrees/linked/index main tree: 517f4a817df411bef827ced6e6036c5a56ea1b16 commit tree: 517f4a817df411bef827ced6e6036c5a56ea1b16 index tree: 517f4a817df411bef827ced6e6036c5a56ea1b16 new commit 1234567 "x": a.txt | 2 +- b.txt | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) M a.txt ?? b.txt ``` With the real hook in a linked worktree and a one-line comment staged in `processor.py`, the first commit failed as described, and the retry passed and recorded upstream `main`'s tree (31 files changed against its parent). To recover, run `git reset` (or `git reset HEAD^` after a bad commit) and stage your changes again. If the working tree was overwritten, restore those files from `HEAD` first. Until this is fixed, run `prek run` before committing and commit with `SKIP=check-supervisor-schemas-versions,check-execution-api-versions git commit`. Suggested fix: run `git worktree add` without `GIT_INDEX_FILE`. The githooks documentation asks hooks that run git in another worktree to clear these variables (`git rev-parse --local-env-vars` lists them all), and prek's maintainer recommends the same in j178/prek#1786: ```python env = {k: v for k, v in os.environ.items() if k != "GIT_INDEX_FILE"} subprocess.run( ["git", "worktree", "add", str(worktree_path), ref], capture_output=True, check=True, env=env ) ``` Both scripts need it, so a shared helper in `scripts/ci/prek/common_prek_utils.py` could serve both (#72070, closed, proposed sharing this worktree code). With the change, a linked-worktree commit passes and records exactly the staged tree. Both hooks also leave one empty temp directory behind per run. Versions: Airflow `main` at dd27159, git 2.39.5, prek 0.5.4, Linux. git's code for this is unchanged through 2.56.0. --- Drafted with Claude Code (Opus 5.5). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
