potiuk commented on PR #56187:
URL: https://github.com/apache/airflow/pull/56187#issuecomment-3517336909
I think I fixed both issues (two fixups) (let's see if CI agrees) - also you
will need to rebase:
Issue 1) -> the shared project's stastd extra was not present in `dev`
dependency group. The `dev` dependency group is the one that is automatically
added by `uv` when you run `uv sync` (i.e. in editable mode from sources when
pyproject.toml is present) -> adding the `self` project (it's perfectly ok to
refer to your own project by it's name) with `statsd` extra `dev` dependency
group will make statsd extra deps to be addeed before tests are run -> this was
the reason why tests failed for the shared distrbution. I ended up in adding
`apache-airflow-shared-observability[all]` to dev dependency group for it -
because also datadog tests started to fail when I only added
`apache-airflow-shared-observability[statsd].
Issue 2) -> the problem was that you added extras that had the same names in
task-sdk like the ones we already added automatically from providers. This
cause duplicate entries in optional-dependencies and ulitmately a failure when
parsing pyproject.toml (unfortunately the only error we get from tomli when
parsing such pyproject.toml is nothing-telling
````
Cannot overwrite a value
````
This error is raised by toml parser when it sees the same value specified
twiice in toml without declaring it as array of values :( .
But the root cause of the problem was that we actually copied those optional
extras from task-sdk to airflow-core - which was only done to copy the "sentry"
extra. This was to avoid duplication, but eventually we do not need that
synchronisation. In Airlfow 3.2 we hope task-sdk will be fully decoupled from
`airflow` - so we will also be able to remove all the coupling between the two
so it's perfectly fine to have duplication and WET (Write Everything Twice)
code rather than DRY.
I removed copying the optional extras from task-sdk completely (only left
this to be automatically generated) so that we can automatically install all
the dependencies from all the packages in the workspace - but this one is
really a temporary measure until we decouple task-sdk - then we wil remove even
that one:
```
"all-task-sdk" = [
"apache-airflow-task-sdk[all]"
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
