Aditya-XR commented on PR #76085:
URL: https://github.com/apache/airflow/pull/76085#issuecomment-6914040897

   Thanks for testing this on macOS and for putting together the comparison 
table, @bingqin2!
   
   On the numbering difference in rows 2 and 3 (`2 / 2` and `3 / 2`):
   
   1. **Why `main` gave `eu.ingest__2` / `eu.ingest__3` there:**
      On `main`, `_check_for_group_id_collisions` searched `used_group_ids` 
with the unprefixed pattern `^ingest__\d+$`. That pattern accidentally matched 
top-level `ingest__1` (while ignoring sibling groups like `us.ingest__1`), 
which caused the second call inside `eu` to skip `eu.ingest__1` and jump 
straight to `eu.ingest__2`.
   
   2. **Consistency with `@task` (`get_unique_task_id`) and isolation between 
scopes:**
      For decorated tasks, `get_unique_task_id` in 
`task-sdk/src/airflow/sdk/bases/decorator.py` already builds the suffix regex 
from the prefixed `tg_task_id` (`prefix = re.split(r"__\d+$", tg_task_id)[0]`), 
so repeated `@task` calls inside `eu` are always numbered `eu.t`, `eu.t__1`, 
`eu.t__2` regardless of how many times `t()` is called at the top level.
      With the bare-id lookup + `while self.group_id in self.used_group_ids:` 
fallback:
      - Calling `ingest()` inside `eu` when top-level `ingest()` was called 
twice produces `eu.ingest`, `eu.ingest__2`, `eu.ingest__3` (skipping 
`eu.ingest__1`).
      - Adding or removing a top-level `ingest()` call outside `eu` shifts all 
the suffixed group IDs *inside* `eu` (e.g., `eu.ingest__2` becomes 
`eu.ingest__3`), which would rename `eu`'s tasks whenever top-level calls 
change.
   
   Scoping the suffix lookup to `prefixed_base` keeps each parent `TaskGroup`'s 
numbering self-contained (`eu.ingest`, `eu.ingest__1`, `eu.ingest__2`) and 
consistent with `get_unique_task_id`. That said, if maintainers prefer 
preserving the top-level offset for existing 2-call Dags via the `while` loop 
fallback, I am happy to switch to that approach!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to