IKholopov opened a new pull request, #46844:
URL: https://github.com/apache/airflow/pull/46844
Related: #45623
Making DagCode model use relative fileloc of a Dag instead of the absolute
one*.
*unit test mode fallbacks to absolute paths still
The main motivation for allowing fallbacks is that there are still a lot of
options to define a Dag outside of a bundle and sync it to DB (DagBag without
bundle_path, Dag defined directly in a test code or imported module). If bundle
path would've been required to be passed for those definition directly - that
would be a hard to automate breaking change for (end users) tests code.
Compared to `bundle_name` or `code_reader` (introduced here) there is no
universal semi-valid path that could've been substituted to cover almost all
cases with some fixture.
One alternative that I've considered - instantiate fallback `relative_path
-> absolute_path` even earlier and consider `bundle_path == Path("")`, but
found it even more error-obscuring compared to explicit allow-listing fallbacks
in a fixed number of places.
If my assessment of the issue with using `DagBag/SDM.write_dag` in Dag test
code is incorrect, and, for ex. we would be better if we forced everyone to set
bundle_path in DagBag & pass dags with fully initialized relative paths, please
let me know.
Other changes:
- Inverting control of resolving the full path and reading a code from file
to `dag_processing` and DagBag (to avoid tying DagCode model with bundle paths
and simplify supplying the src code in tests)
<!-- Please keep an empty line above the dashes. -->
---
^ Add meaningful description above
Read the [Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named {pr_number}.significant.rst or
{issue_number}.significant.rst, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
