MichalJaroslawKrzywanski-TomTom opened a new pull request, #73878:
URL: https://github.com/apache/airflow/pull/73878

   GitHub rejects a just-issued App installation token with `remote: Repository 
not found.` for a few seconds; the same clone succeeds unchanged about 10 s 
later. Every KubernetesExecutor task pod mints its own token in `GitHook` and 
immediately calls `_clone_bare_repo_if_required`, whose retry allowed two 
attempts with no wait, so both landed inside that window and the task died at 
startup with `RuntimeError: Error cloning repository`. #73877 has the log 
evidence: over 7 days every occurrence in two deployments was within 1 s of 
`Successfully obtained GitHub App installation access token`, and 3 of 51 
task-pod clones failed this way.
   
   The fix gives the bare clone five attempts with exponential backoff (2, 4, 
8, 15 s), so at least one attempt lands after the token is usable. Worst case a 
task that can genuinely never clone now fails after ~29 s instead of 
immediately. `refresh()` in the dag-processor hits the same window on 
`_fetch_bare_repo`, but it already recovers on the next processor cycle, so 
this PR leaves it alone.
   
   ##### Tests
   
   * An autouse fixture stubs the retry's `sleep`, so the existing 
persistent-failure tests do not spend the backoff time; 
`test_clone_bare_repo_invalid_repository_error_retry_fails` now expects five 
attempts.
   * `test_clone_bare_repo_waits_out_transient_repository_not_found` drives the 
real bundle code against a `clone_from` that raises `Repository not found` 
twice, and asserts three attempts, a real bare repo afterwards, and two 
positive, non-decreasing waits.
   
   ##### Verification
   
   Run in `apache/airflow:3.2.2-python3.10` (Debian 12, git 2.39.5) with the 
provider installed from this branch:
   
   ```
   providers/git/tests/unit/git/bundles/test_git.py   117 passed
   providers/git/tests/unit/git/hooks/test_git.py     55 passed
   ```
   
   Before the source change, the new test fails on the second `clone_from` 
call, as expected. `ruff check` and `ruff format --check` are clean on both 
changed files.
   
   closes: #73877
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Code following [the 
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions).
 The diagnosis from production logs, the fix, the tests and the test runs above 
were reviewed by me.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to