Oh one thing to note: In order to get both mypy and IntelliJ working wit this, we needed to commit the typestubs, and in order to make Python not get confused by the src/airflow/_vendor/airflow_shared directory existing and treating it as a namespace package, so the loader installer code now looks like this:
https://github.com/apache/airflow/pull/53149/files#diff-734233b1301f78bec9035c8be3f55be15b9f6f1662da8e56897cb2e72b281a76R56-R71 ``` def _install_loader(): # Add a new module Finder to find the imports when the vendored sources aren't found. We only add it when # something that looks like the airflow repo layout is already in the sys path i.e. when we are running in # a repo checkout. # # We need to add this to the dir first as we now keep the `.pyi` files for the vendored libs, and for dir in sys.path: if dir.endswith("shared/core-and-tasksdk/src"): sys.meta_path.insert(0, _NonVendoredAirflowSharedFinder(dir)) break if __debug__: # i.e. we are not running with PYTHONOPTMIZE. This is almost never set, but if someone _does_ run it, we # def don't want to load code from the wrong place _install_loader() ``` > On 10 Jul 2025, at 17:39, Ash Berlin-Taylor <a...@apache.org> wrote: > > Not quite final PR, but good enough that I want to see how it behaves on CI, > and other IDEs etc https://github.com/apache/airflow/pull/53149 (We updated > the `setup_idea.py`, so either re-run that or add the new src root manually) > > Let the naming discussions start! > > -ash > >> On 10 Jul 2025, at 12:51, Jarek Potiuk <ja...@potiuk.com> wrote: >> >> Yeah. when we get the final PR I will also want to to test more scenarios - >> with IDE/mypy integration switching branches, uv syncing etc. and will be >> happy to help and document the contributor's doc to explain what and how to >> work with it. This would be a super cool thing if we get it to >> work seamlessly for everyone :) >> >> On Thu, Jul 10, 2025 at 7:44 AM Amogh Desai <amoghdesai....@gmail.com> >> wrote: >> >>> Agreed! >>> >>> Once the PR is up, we can have these implementation level discussions >>> over there. Good chat however! >>> >>> Thanks & Regards, >>> Amogh Desai >>> >>> >>> On Wed, Jul 9, 2025 at 3:56 PM Jarek Potiuk <ja...@potiuk.com> wrote: >>> >>>> Yeah. I think extracting one-by-one, feature-by-feature that we want to >>>> share to a separate distribution is the best approach - it will actually >>>> also help with the "__init__.py" cleanup - because almost by definition - >>>> those distributions will not be able to "reach" outside - i.e. they only >>>> can be "used" not "use" something else. Which means that - for example as >>>> it is now - configuration using logging which is using configuration >>>> (leading to circular dependencies and partially initialized modules) will >>>> not happen and we will have to figure out rather how to inject >>>> configuration into logging (from task-sdk, airflow-ctl, airflow-core) and >>>> to get the right sequence of initialization - rather than have the >>>> inter-feature-dependencies. >>>> >>>> And that is precisely what low-cyclomatic complexity is about as well - >>>> it generally leads to easier-to-maintain software that has well defined >>>> functionality and does not have those weird circular dependencies we have >>>> now. That's a kind-of side-effect of such a "per feature" split, but a very >>>> desirable one. >>>> >>>> J. >>>> >>>> On Wed, Jul 9, 2025 at 10:17 AM Amogh Desai <amoghdesai....@gmail.com> >>>> wrote: >>>> >>>>> Probably, you make a valid point. >>>>> >>>>> Maybe this is an implementation detail, so we could figure it out as we >>>>> start on a POC and factor in these things >>>>> as we move along? >>>>> >>>>> But from an initial guess, I would think that execution time related >>>>> items (if we manage to enumerate them) would be something >>>>> that would be better off in that "core_and_task_sdk" bundle. >>>>> >>>>> Thanks & Regards, >>>>> Amogh Desai >>>>> >>>>> >>>>> On Tue, Jul 8, 2025 at 3:46 PM Jarek Potiuk <ja...@potiuk.com> wrote: >>>>> >>>>>>> Not that I am against your idea and we can surely expand as we need >>>>>> but we would not need to expand the >>>>>> "core_and_task_sdk" if we put only the relevant items into it. >>>>>> >>>>>> So if we move logging and config out, my question is what is really >>>>>> relevant to "stay" in "core_and_task_sdk" ? And what we know will not be >>>>>> needed by other distributions in the future ? >>>>>> >>>>>> What would be the content of "core_and_task_sdk" ? Can we enumerate >>>>>> what should go there now ? >>>>>> >>>>>> My bet is that if we enumerate them, it will turn out that they are >>>>>> good candidates to make separate "shared features" that can be logically >>>>>> named and modularised, and that there is no real need to have shared >>>>>> "core_and_task_sdk" which sounds like "bag of everything else". >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Jul 8, 2025 at 11:48 AM Amogh Desai <amoghdesai....@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Yeah, I think what you are showcasing here is a step ahead of the >>>>>>> initial proposal from Ash. >>>>>>> >>>>>>> From the original proposal, the `core_and_task_sdk` *can* have the >>>>>>> things relevant to just those two >>>>>>> distros. Logging, Config are modules that might be needed by >>>>>>> airflow-ctl for example, so ideally, those >>>>>>> would not be good candidates to be put in there, ideally speaking. >>>>>>> >>>>>>> The example of Kubernetes Utils sounds to be a good example, it will >>>>>>> be used by KubeExecutor (lets say this is a >>>>>>> module called "executors") and by KPO (providers), and the >>>>>>> "shared/kubernetes" would probably be a good >>>>>>> candidate for that. >>>>>>> >>>>>>> Not that I am against your idea and we can surely expand as we need >>>>>>> but we would not need to expand the >>>>>>> "core_and_task_sdk" if we put only the relevant items into it. >>>>>>> >>>>>>> Thanks & Regards, >>>>>>> Amogh Desai >>>>>>> >>>>>>> >>>>>>> On Tue, Jul 8, 2025 at 12:28 PM Jarek Potiuk <ja...@potiuk.com> wrote: >>>>>>> >>>>>>>>> @Jarek Potiuk <ja...@potiuk.com> a little confused on what you >>>>>>>> mean there, I am understanding the direction >>>>>>>> but could you elaborate a bit more please? >>>>>>>> >>>>>>>> Let me elaborate: >>>>>>>> >>>>>>>> As I understand (maybe I am wrong?), the proposal is that we have a >>>>>>>> "core-and-task-sdk" folder which is a shared distribution that is >>>>>>>> vendored-in into both "airflow-core" and "airflow-task-sdk". This >>>>>>>> contains some shared code that we want to include in both distributions >>>>>>>> (note that we never ever release "core-and-task-sdk" >>>>>>>> distribution because it only contains code that is shared between the >>>>>>>> two >>>>>>>> distributions we release. >>>>>>>> >>>>>>>> That's fine and cool. >>>>>>>> >>>>>>>> Imagine that this distribution contains "logging" (shared code >>>>>>>> for logging) and "config" (shared code for configuration). - both >>>>>>>> needed in >>>>>>>> "airflow-core" and "airflow-task-sdk". So far so good. But what happen >>>>>>>> if >>>>>>>> we want to use logging in the same fashion in say "airflow-ctl" (that >>>>>>>> also >>>>>>>> applies for other distributions that we might come up with) ? Are we >>>>>>>> going >>>>>>>> to vendor in the whole "core-and-task-sdk" distribution in >>>>>>>> "airflow-ctl" ? >>>>>>>> It would be far better if we just vendor in "logging" and do not >>>>>>>> vendor-in >>>>>>>> "config". >>>>>>>> >>>>>>>> And if we are going to have a mechanism to vendor-in "a distribution" >>>>>>>> - there is nothing wrong with having the same way to vendor-in multiple >>>>>>>> distributions - so we can easily do it this way (i added "orm" , >>>>>>>> "serialization". and "fast_api" as an example thing that we might want >>>>>>>> to >>>>>>>> share - not sure if that is really something we want to do but it will >>>>>>>> allow to illustrate my idea better) >>>>>>>> >>>>>>>> / >>>>>>>> airflow-ctl/ >>>>>>>> task-sdk/... >>>>>>>> airflow-core/... >>>>>>>> .... >>>>>>>> shared/ >>>>>>>> kubernetes/ >>>>>>>> pyproject.toml >>>>>>>> src/ >>>>>>>> airflow_shared_kubernetes/__init__.py >>>>>>>> logging/ >>>>>>>> pyproject.toml >>>>>>>> src/ >>>>>>>> airflow_shared_logging/__init__.py >>>>>>>> config/ >>>>>>>> pyproject.toml >>>>>>>> src/ >>>>>>>> airflow_shared_config/__init__.py >>>>>>>> orm/ >>>>>>>> pyproject.toml >>>>>>>> src/ >>>>>>>> airflow_shared_orm/__init__.py >>>>>>>> serialization/ >>>>>>>> pyproject.toml >>>>>>>> src/ >>>>>>>> airflow_shared_serialization/__init__.py >>>>>>>> fast_api/ >>>>>>>> pyproject.toml >>>>>>>> src/ >>>>>>>> airflow_shared_fast_api/__init__.py >>>>>>>> ... >>>>>>>> >>>>>>>> This has multiple benefits (and I see no real drawbacks): >>>>>>>> >>>>>>>> * the code can be really well modularised. Those "things" we share >>>>>>>> (and this also connects to the __init__.py discussion) - can be >>>>>>>> independent >>>>>>>> - and (it follow Jens comment) it allows to keep low cyclomatic >>>>>>>> complexity >>>>>>>> https://en.wikipedia.org/wiki/Cyclomatic_complexity . It will be way >>>>>>>> easier to implement logging in the way that it does not import or use >>>>>>>> config. This means for example that configuration for logging will >>>>>>>> need to >>>>>>>> be injected when logging is initialized - and that's exactly what we >>>>>>>> want, >>>>>>>> we do not want logging code to use configuration code directly - they >>>>>>>> should be independent from each other and you should be free to >>>>>>>> vendor-in >>>>>>>> either logging or config independently if you also vendored-in the >>>>>>>> other. >>>>>>>> >>>>>>>> * it's much more logical. We split based on functionality we want to >>>>>>>> share - not about the "distributions" we want to produce. That allows >>>>>>>> us - >>>>>>>> in the future - to make different decisions on how we split our >>>>>>>> distributions. For example (i do not tell we have to do it, or that we >>>>>>>> will >>>>>>>> do it but we will have such a possibility) - we can add more shared >>>>>>>> utilities we find useful in the same way and decide that "scheduler", >>>>>>>> "api_server" or "scheduler" or "triggerer" or "dag processor" are >>>>>>>> split to >>>>>>>> separate distributions - because for example we want to keep a number >>>>>>>> of >>>>>>>> dependencies down. And for example "api_server" might use "fast_api", >>>>>>>> "config", "logging", "orm" and "fast_api" and "serialization" , where >>>>>>>> the >>>>>>>> scheduler should not need "fast_api". The "dag_processor" eventually >>>>>>>> might >>>>>>>> not need "orm" nor "fast_api" and only use the other >>>>>>>> >>>>>>>> This seems like a natural approach. If we have a mechanism to "share" >>>>>>>> the code, it does not add complexity, but allows us to isolate >>>>>>>> independent >>>>>>>> functionality into "isolated" boxes and use them >>>>>>>> >>>>>>>> Also for cyclomatic complexity that is a complex word (badly chosen >>>>>>>> as it scares people away) and has some math behind, but it really boils >>>>>>>> down to very simple "rules of thumb" (and yes I am a big proponent of >>>>>>>> having low cyclomatic complexity). >>>>>>>> >>>>>>>> a) when you are building the "final" product (i.e. distribution you >>>>>>>> want to release) - make sure that you only "use" things - that nothing >>>>>>>> else >>>>>>>> is "using" you as a library. >>>>>>>> b) when you are building a "shared" thing (a library) - make sure >>>>>>>> that library is only "used" by others but it does not really "use" >>>>>>>> anything >>>>>>>> else. For example in the case I explained above - we can achieve >>>>>>>> low-cyclomatic complexity when: >>>>>>>> >>>>>>>> * airflow-core uses: logging, config, orm, serialization, fast_api >>>>>>>> * none of the "logging, config, orm, serialization, fast_api" use >>>>>>>> each other - they are at the bottom of the "user -> used" tree >>>>>>>> >>>>>>>> J. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jul 8, 2025 at 8:16 AM Amogh Desai <amoghdesai....@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I like the folder structure proposed by Ash and have no objections >>>>>>>>> with it. >>>>>>>>> >>>>>>>>> "core_and_task_sdk" sounds good to me and justifies what it should >>>>>>>>> do pretty well. >>>>>>>>> >>>>>>>>> @Jarek Potiuk <ja...@potiuk.com> a little confused on what you mean >>>>>>>>> there, I am understanding the direction >>>>>>>>> but could you elaborate a bit more please? >>>>>>>>> >>>>>>>>> Naming is REALLY hard! >>>>>>>>> >>>>>>>>> Thanks & Regards, >>>>>>>>> Amogh Desai >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jul 8, 2025 at 2:52 AM Jarek Potiuk <ja...@potiuk.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> How about splitting it even more and having each shared "thing" >>>>>>>>>> named? >>>>>>>>>> "logging", "config" and sharing them explicitly and separately with >>>>>>>>>> the >>>>>>>>>> right "user" ? >>>>>>>>>> That sounds way more modular and we will be able to choose which >>>>>>>>>> of the >>>>>>>>>> shared "utils" we use where. >>>>>>>>>> >>>>>>>>>> J. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Jul 7, 2025 at 11:13 PM Jens Scheffler >>>>>>>>>> <j_scheff...@gmx.de.invalid> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I like "core_and_task_sdk the same like core-and-task-sdk - I >>>>>>>>>> have no >>>>>>>>>>> problem and it is a path only. >>>>>>>>>>> >>>>>>>>>>> if we get to "dag-parser-scheduler-task-sdk-and-triggerer" which >>>>>>>>>> is a >>>>>>>>>>> bit bulky we then should name it "all-not-api-server" :-D >>>>>>>>>>> >>>>>>>>>>> On 07.07.25 22:57, Ash Berlin-Taylor wrote: >>>>>>>>>>>> In case I did a bad job explaining it, the “core and task sdk” >>>>>>>>>> is not in >>>>>>>>>>> the module name/import name, just in the file path. >>>>>>>>>>>> >>>>>>>>>>>> Anyone have other ideas? >>>>>>>>>>>> >>>>>>>>>>>>> On 7 Jul 2025, at 21:37, Buğra Öztürk <ozturkbugr...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks Ash! Looks cool! I like the structure. This will >>>>>>>>>> enable all the >>>>>>>>>>>>> combinations and structure looks easy to grasp. No strong >>>>>>>>>> stance on the >>>>>>>>>>>>> naming other than maybe it is a bit long with `and`, >>>>>>>>>> `core_ctl` could be >>>>>>>>>>>>> shorter, since no import path is defined like that, we can >>>>>>>>>> give any name >>>>>>>>>>>>> for sure. >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, 7 Jul 2025, 21:51 Jarek Potiuk, <ja...@potiuk.com> >>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Looks good but I think we should find some better logical >>>>>>>>>> name for >>>>>>>>>>>>>> core_and_sdk :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> pon., 7 lip 2025, 21:44 użytkownik Jens Scheffler >>>>>>>>>>>>>> <j_scheff...@gmx.de.invalid> napisał: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cool! Especially the "shared" folder with the ability to have >>>>>>>>>>>>>>> N-combinations w/o exploding project repo root! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 07.07.25 14:43, Ash Berlin-Taylor wrote: >>>>>>>>>>>>>>>> Oh, and all of this will be explain in shared/README.md >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On 7 Jul 2025, at 13:41, Ash Berlin-Taylor <a...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Okay, so it seems we have agreement on the approach here, >>>>>>>>>> so I’ll >>>>>>>>>>>>>>> continue with this, and on the dev call it was mentioned that >>>>>>>>>>>>>>> “airflow-common” wasn’t a great name, so here is my proposal >>>>>>>>>> for the >>>>>>>>>>> file >>>>>>>>>>>>>>> structure; >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> / >>>>>>>>>>>>>>>>> task-sdk/... >>>>>>>>>>>>>>>>> airflow-core/... >>>>>>>>>>>>>>>>> shared/ >>>>>>>>>>>>>>>>> kuberenetes/ >>>>>>>>>>>>>>>>> pyproject.toml >>>>>>>>>>>>>>>>> src/ >>>>>>>>>>>>>>>>> airflow_kube/__init__.py >>>>>>>>>>>>>>>>> core-and-tasksdk/ >>>>>>>>>>>>>>>>> pyproject.toml >>>>>>>>>>>>>>>>> src/ >>>>>>>>>>>>>>>>> airflow_shared/__init__.py >>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Things to note here: the “shared” folder has (the >>>>>>>>>> possibility) of >>>>>>>>>>>>>>> having multiple different shared “libraries” in it, in this >>>>>>>>>> example I >>>>>>>>>>> am >>>>>>>>>>>>>>> supposing a hypothetical shared kuberenetes folder a world >>>>>>>>>> in which we >>>>>>>>>>>>>>> split the KubePodOperator and the KubeExecutor in to two >>>>>>>>>> separate >>>>>>>>>>>>>>> distributions (example only, not proposing we do that right >>>>>>>>>> now, and >>>>>>>>>>> that >>>>>>>>>>>>>>> will be a separate discussion) >>>>>>>>>>>>>>>>> The other things to note here: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> - the folder name in shared aims to be “self-documenting”, >>>>>>>>>> hence the >>>>>>>>>>>>>>> verbose “core-and-tasksdk” to say where the shared library is >>>>>>>>>>> intended to >>>>>>>>>>>>>>> be used. >>>>>>>>>>>>>>>>> - the python module itself should almost always have an >>>>>>>>>> `airflow_` >>>>>>>>>>> (or >>>>>>>>>>>>>>> maybe `_airflow_`?) prefix so that it does not conflict with >>>>>>>>>> anything >>>>>>>>>>>>>> else >>>>>>>>>>>>>>> we might use. It won’t matter “in production” as those will >>>>>>>>>> be >>>>>>>>>>> vendored >>>>>>>>>>>>>> in >>>>>>>>>>>>>>> to be imported as `airflow/_vendor/airflow_shared` etc, but >>>>>>>>>> avoiding >>>>>>>>>>>>>>> conflicts at dev time with the Finder approach is a good >>>>>>>>>> safety >>>>>>>>>>> measure. >>>>>>>>>>>>>>>>> I will start making a real PR for this proposal now, but >>>>>>>>>> I’m open to >>>>>>>>>>>>>>> feedback (either here, or in the PR when I open it) >>>>>>>>>>>>>>>>> -ash >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 4 Jul 2025, at 16:55, Jarek Potiuk <ja...@potiuk.com> >>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yeah we have to try it and test - also building packages >>>>>>>>>> happens >>>>>>>>>>> semi >>>>>>>>>>>>>>>>>> frequently when you run `uv sync` (they use some kind of >>>>>>>>>> heuristics >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>> decide when) and you can force it with `--reinstall` or >>>>>>>>>>> `--refresh`. >>>>>>>>>>>>>>>>>> Package build also happens every time when you run >>>>>>>>>> "ci-image build` >>>>>>>>>>>>>>> now in >>>>>>>>>>>>>>>>>> breeze so it seems like it will nicely integrate in our >>>>>>>>>> workflows. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Looks really cool Ash. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Fri, Jul 4, 2025 at 5:14 PM Ash Berlin-Taylor < >>>>>>>>>> a...@apache.org> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>> It’s not just release time, but any time we build a >>>>>>>>>> package which >>>>>>>>>>>>>>> happens >>>>>>>>>>>>>>>>>>> on “every” CI run. The normal unit tests will use code >>>>>>>>>> from >>>>>>>>>>>>>>>>>>> airflow-common/src/airflow_common; the kube tests which >>>>>>>>>> build an >>>>>>>>>>>>>>> image will >>>>>>>>>>>>>>>>>>> build the dists and vendor in the code from that commit. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> There is only a single copy of the shared code committed >>>>>>>>>> to the >>>>>>>>>>>>>> repo, >>>>>>>>>>>>>>> so >>>>>>>>>>>>>>>>>>> there is never anything to synchronise. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On 4 Jul 2025, at 15:53, Amogh Desai < >>>>>>>>>> amoghdesai....@gmail.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> Thanks Ash. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This is really cool and helpful that you were able to >>>>>>>>>> test both >>>>>>>>>>>>>>> scenarios >>>>>>>>>>>>>>>>>>>> -- repo checkout >>>>>>>>>>>>>>>>>>>> and also installing from the vendored package and the >>>>>>>>>> resolution >>>>>>>>>>>>>>> worked >>>>>>>>>>>>>>>>>>>> fine too. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I like this idea compared the to relative import one >>>>>>>>>> for few >>>>>>>>>>>>>> reasons: >>>>>>>>>>>>>>>>>>>> - It feels like it will take some time to adjust to the >>>>>>>>>> new >>>>>>>>>>> coding >>>>>>>>>>>>>>>>>>> standard >>>>>>>>>>>>>>>>>>>> that we will lay >>>>>>>>>>>>>>>>>>>> if we impose relative imports in the shared dist >>>>>>>>>>>>>>>>>>>> - We can continue using repo wise absolute import >>>>>>>>>> standards, it >>>>>>>>>>> is >>>>>>>>>>>>>>> also >>>>>>>>>>>>>>>>>>>> much easier for situations >>>>>>>>>>>>>>>>>>>> when we do global search in IDE to find + replace, this >>>>>>>>>> could >>>>>>>>>>> mean >>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>>>>> there >>>>>>>>>>>>>>>>>>>> - The vendoring work is a proven and established >>>>>>>>>> paradigm across >>>>>>>>>>>>>>> projects >>>>>>>>>>>>>>>>>>>> and would >>>>>>>>>>>>>>>>>>>> out of box give us the build tooling we need also >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Nothing too against the relative import but with the >>>>>>>>>> evidence >>>>>>>>>>>>>>> provided >>>>>>>>>>>>>>>>>>>> above, vendored approach >>>>>>>>>>>>>>>>>>>> seems to only do us good. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Regarding synchronizing it, release time should be fine >>>>>>>>>> as long >>>>>>>>>>> as >>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>> a good CI workflow to probably >>>>>>>>>>>>>>>>>>>> catch such issues per PR if changes are made in shared >>>>>>>>>> dist? >>>>>>>>>>>>>>> (precommit >>>>>>>>>>>>>>>>>>>> would make it really slow i guess) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> If we can run our tests with vendored code we should be >>>>>>>>>> mostly >>>>>>>>>>>>>>> covered. >>>>>>>>>>>>>>>>>>>> Good effort all! >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>>>>>>>>>> Amogh Desai >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Fri, Jul 4, 2025 at 7:23 PM Ash Berlin-Taylor < >>>>>>>>>>> a...@apache.org> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>> Okay, I think I’ve got something that works and I’m >>>>>>>>>> happy with. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> https://github.com/astronomer/airflow/tree/shared-vendored-lib-tasksdk-and-core >>>>>>>>>>>>>>>>>>>>> This produces the following from `uv build task-sdk` >>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> https://github.com/user-attachments/files/21058976/apache_airflow_task_sdk-1.1.0.tar.gz >>>>>>>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> https://github.com/user-attachments/files/21058996/apache_airflow_task_sdk-1.1.0-py3-none-any.whl.zip >>>>>>>>>>>>>>>>>>>>> (`.whl.zip` as GH won't allow .whl upload, but will >>>>>>>>>> .zip) >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> ❯ unzip -l >>>>>>>>>>>>>> dist/apache_airflow_task_sdk-1.1.0-py3-none-any.whl.zip | >>>>>>>>>>>>>>>>>>> grep >>>>>>>>>>>>>>>>>>>>> _vendor >>>>>>>>>>>>>>>>>>>>> 50 02-02-2020 00:00 >>>>>>>>>> airflow/sdk/_vendor/.gitignore >>>>>>>>>>>>>>>>>>>>> 2082 02-02-2020 00:00 >>>>>>>>>> airflow/sdk/_vendor/__init__.py >>>>>>>>>>>>>>>>>>>>> 28 02-02-2020 00:00 >>>>>>>>>>> airflow/sdk/_vendor/airflow_common.pyi >>>>>>>>>>>>>>>>>>>>> 18 02-02-2020 00:00 >>>>>>>>>> airflow/sdk/_vendor/vendor.txt >>>>>>>>>>>>>>>>>>>>> 785 02-02-2020 00:00 >>>>>>>>>>>>>>>>>>>>> airflow/sdk/_vendor/airflow_common/__init__.py >>>>>>>>>>>>>>>>>>>>> 10628 02-02-2020 00:00 >>>>>>>>>>>>>>>>>>>>> airflow/sdk/_vendor/airflow_common/timezone.py >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> And similarly in the .tar.gz, so our “sdist” is >>>>>>>>>> complete too: >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> ❯ tar -tzf dist/apache_airflow_task_sdk-1.1.0.tar.gz >>>>>>>>>> |grep >>>>>>>>>>> _vendor >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>> apache_airflow_task_sdk-1.1.0/src/airflow/sdk/_vendor/.gitignore >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>> apache_airflow_task_sdk-1.1.0/src/airflow/sdk/_vendor/__init__.py >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> apache_airflow_task_sdk-1.1.0/src/airflow/sdk/_vendor/airflow_common.pyi >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>> apache_airflow_task_sdk-1.1.0/src/airflow/sdk/_vendor/vendor.txt >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> apache_airflow_task_sdk-1.1.0/src/airflow/sdk/_vendor/airflow_common/__init__.py >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> apache_airflow_task_sdk-1.1.0/src/airflow/sdk/_vendor/airflow_common/timezone.py >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The plugin works at build time by including/copying >>>>>>>>>> the libs >>>>>>>>>>>>>>> specified >>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>> vendor.txt into place (and let `vendoring` take care >>>>>>>>>> of import >>>>>>>>>>>>>>>>>>> rewrites.) >>>>>>>>>>>>>>>>>>>>> For the imports to continue to work at “dev” time/from >>>>>>>>>> a repo >>>>>>>>>>>>>>> checkout, >>>>>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>>>> have added a import finder to `sys.meta_path`, and >>>>>>>>>> since it’s at >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> end of >>>>>>>>>>>>>>>>>>>>> the list it will only be used if the normal import >>>>>>>>>> can’t find >>>>>>>>>>>>>>> things. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> https://github.com/astronomer/airflow/blob/996817782be6071b306a87af9f36fe1cf2d3aaa3/task-sdk/src/airflow/sdk/_vendor/__init__.py >>>>>>>>>>>>>>>>>>>>> This doesn’t quite give us the same runtime effect >>>>>>>>>> “import >>>>>>>>>>>>>>> rewriting” >>>>>>>>>>>>>>>>>>>>> affect, as in this approach `airflow_common` is >>>>>>>>>> directly loaded >>>>>>>>>>>>>>> (i.e. >>>>>>>>>>>>>>>>>>>>> airflow.sdk._vendor.airflow_common and airflow_common >>>>>>>>>> exist in >>>>>>>>>>>>>>>>>>>>> sys.modules), but it does work for everything that I >>>>>>>>>> was able to >>>>>>>>>>>>>>> test.. >>>>>>>>>>>>>>>>>>>>> I tested it with the diff at the end of this message. >>>>>>>>>> My test >>>>>>>>>>>>>>> ipython >>>>>>>>>>>>>>>>>>>>> shell: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> In [1]: from >>>>>>>>>> airflow.sdk._vendor.airflow_common.timezone import >>>>>>>>>>>>>> foo >>>>>>>>>>>>>>>>>>>>> In [2]: foo >>>>>>>>>>>>>>>>>>>>> Out[2]: 1 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In [3]: import airflow.sdk._vendor.airflow_common >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In [4]: import >>>>>>>>>> airflow.sdk._vendor.airflow_common.timezone >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In [5]: airflow.sdk._vendor.airflow_common.__file__ >>>>>>>>>>>>>>>>>>>>> Out[5]: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> '/Users/ash/code/airflow/airflow/airflow-common/src/airflow_common/__init__.py' >>>>>>>>>>>>>>>>>>>>> In [6]: >>>>>>>>>> airflow.sdk._vendor.airflow_common.timezone.__file__ >>>>>>>>>>>>>>>>>>>>> Out[6]: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> '/Users/ash/code/airflow/airflow/airflow-common/src/airflow_common/timezone.py' >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> And in an standalone environment with the SDK dist I >>>>>>>>>> built (it >>>>>>>>>>>>>>> needed >>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>> matching airflow-core right now, but that is nothing >>>>>>>>>> to do with >>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>> discussion): >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> ❯ _AIRFLOW__AS_LIBRARY=1 uvx --python 3.12 --with >>>>>>>>>>>>>>>>>>>>> dist/apache_airflow_core-3.1.0-py3-none-any.whl --with >>>>>>>>>>>>>>>>>>>>> dist/apache_airflow_task_sdk-1.1.0-py3-none-any.whl >>>>>>>>>> ipython >>>>>>>>>>>>>>>>>>>>> Python 3.12.7 (main, Oct 16 2024, 07:12:08) [Clang >>>>>>>>>> 18.1.8 ] >>>>>>>>>>>>>>>>>>>>> Type 'copyright', 'credits' or 'license' for more >>>>>>>>>> information >>>>>>>>>>>>>>>>>>>>> IPython 9.4.0 -- An enhanced Interactive Python. Type >>>>>>>>>> '?' for >>>>>>>>>>>>>> help. >>>>>>>>>>>>>>>>>>>>> Tip: You can use `%hist` to view history, see the >>>>>>>>>> options with >>>>>>>>>>>>>>>>>>> `%history?` >>>>>>>>>>>>>>>>>>>>> In [1]: import >>>>>>>>>> airflow.sdk._vendor.airflow_common.timezone >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> In [2]: >>>>>>>>>> airflow.sdk._vendor.airflow_common.timezone.__file__ >>>>>>>>>>>>>>>>>>>>> Out[2]: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> '/Users/ash/.cache/uv/archive-v0/WWq6r65aPto2eJOyPObEH/lib/python3.12/site-packages/airflow/sdk/_vendor/airflow_common/timezone.py’ >>>>>>>>>>>>>>>>>>>>> `` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ```diff >>>>>>>>>>>>>>>>>>>>> diff --git >>>>>>>>>> a/airflow-common/src/airflow_common/__init__.py >>>>>>>>>>>>>>>>>>>>> b/airflow-common/src/airflow_common/__init__.py >>>>>>>>>>>>>>>>>>>>> index 13a83393a9..927b7c6b61 100644 >>>>>>>>>>>>>>>>>>>>> --- a/airflow-common/src/airflow_common/__init__.py >>>>>>>>>>>>>>>>>>>>> +++ b/airflow-common/src/airflow_common/__init__.py >>>>>>>>>>>>>>>>>>>>> @@ -14,3 +14,5 @@ >>>>>>>>>>>>>>>>>>>>> # KIND, either express or implied. See the License >>>>>>>>>> for the >>>>>>>>>>>>>>>>>>>>> # specific language governing permissions and >>>>>>>>>> limitations >>>>>>>>>>>>>>>>>>>>> # under the License. >>>>>>>>>>>>>>>>>>>>> + >>>>>>>>>>>>>>>>>>>>> +foo = 1 >>>>>>>>>>>>>>>>>>>>> diff --git >>>>>>>>>> a/airflow-common/src/airflow_common/timezone.py >>>>>>>>>>>>>>>>>>>>> b/airflow-common/src/airflow_common/timezone.py >>>>>>>>>>>>>>>>>>>>> index 340b924c66..58384ef20f 100644 >>>>>>>>>>>>>>>>>>>>> --- a/airflow-common/src/airflow_common/timezone.py >>>>>>>>>>>>>>>>>>>>> +++ b/airflow-common/src/airflow_common/timezone.py >>>>>>>>>>>>>>>>>>>>> @@ -36,6 +36,9 @@ _PENDULUM3 = >>>>>>>>>>>>>>>>>>>>> version.parse(metadata.version("pendulum")).major == 3 >>>>>>>>>>>>>>>>>>>>> # - FixedTimezone(0, "UTC") in pendulum 2 >>>>>>>>>>>>>>>>>>>>> utc = pendulum.UTC >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> + >>>>>>>>>>>>>>>>>>>>> +from airflow_common import foo >>>>>>>>>>>>>>>>>>>>> + >>>>>>>>>>>>>>>>>>>>> TIMEZONE: Timezone >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 3 Jul 2025, at 12:43, Jarek Potiuk < >>>>>>>>>> ja...@potiuk.com> >>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>> I think both approaches are doable: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 1) -> We can very easily prevent bad imports by >>>>>>>>>> pre-commit when >>>>>>>>>>>>>>>>>>> importing >>>>>>>>>>>>>>>>>>>>>> from different distributions and make sure we are >>>>>>>>>> only doing >>>>>>>>>>>>>>> relative >>>>>>>>>>>>>>>>>>>>>> imports in the shared modules. We are doing plenty of >>>>>>>>>> this >>>>>>>>>>>>>>> already. And >>>>>>>>>>>>>>>>>>>>> yes >>>>>>>>>>>>>>>>>>>>>> it would require relative links we currently do not >>>>>>>>>> allow. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 2) -> has one disadvantage that someone at some point >>>>>>>>>> in time >>>>>>>>>>>>>> will >>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>> decide to synchronize this and if it happens just >>>>>>>>>> before >>>>>>>>>>> release >>>>>>>>>>>>>>> (I bet >>>>>>>>>>>>>>>>>>>>>> this is going to happen) this will lead to solving >>>>>>>>>> problems >>>>>>>>>>> that >>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>> normally be solved during PR when you make a change >>>>>>>>>> (i.e. >>>>>>>>>>>>>> symbolic >>>>>>>>>>>>>>> link >>>>>>>>>>>>>>>>>>>>> has >>>>>>>>>>>>>>>>>>>>>> the advantage that whoever modifies shared code will >>>>>>>>>> be >>>>>>>>>>>>>> immediately >>>>>>>>>>>>>>>>>>>>>> notified in their PR - that they broke something >>>>>>>>>> because either >>>>>>>>>>>>>>> static >>>>>>>>>>>>>>>>>>>>>> checks or mypy or tests fail. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Ash, do you have an idea of a process (who and when) >>>>>>>>>> does the >>>>>>>>>>>>>>>>>>>>>> synchronisation in case of vendoring? Maybe we could >>>>>>>>>> solve it >>>>>>>>>>> if >>>>>>>>>>>>>>> it is >>>>>>>>>>>>>>>>>>>>> done >>>>>>>>>>>>>>>>>>>>>> more frequently and with some regularity? We could >>>>>>>>>> potentially >>>>>>>>>>>>>>> force >>>>>>>>>>>>>>>>>>>>>> re-vendoring at PR time as well any time shared code >>>>>>>>>> changes >>>>>>>>>>> (and >>>>>>>>>>>>>>>>>>> prevent >>>>>>>>>>>>>>>>>>>>>> it by pre-commit. And I can't think of some place >>>>>>>>>> (other than >>>>>>>>>>>>>>> releases) >>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>> our development workflow and that seems to be a bit >>>>>>>>>> too late as >>>>>>>>>>>>>>> puts an >>>>>>>>>>>>>>>>>>>>>> extra effort on fixing potential incompatibilities >>>>>>>>>> introduced >>>>>>>>>>> on >>>>>>>>>>>>>>>>>>> release >>>>>>>>>>>>>>>>>>>>>> manager and delays the release. WDYT? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Re: relative links. I think for a shared library we >>>>>>>>>> could >>>>>>>>>>>>>>> potentially >>>>>>>>>>>>>>>>>>>>> relax >>>>>>>>>>>>>>>>>>>>>> this and allow them (and actually disallow absolute >>>>>>>>>> links in >>>>>>>>>>> the >>>>>>>>>>>>>>> pieces >>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>> code that are shared - again, by pre-commit). As I >>>>>>>>>> recall, the >>>>>>>>>>>>>> only >>>>>>>>>>>>>>>>>>>>> reason >>>>>>>>>>>>>>>>>>>>>> we forbade the relative links is because of how we >>>>>>>>>> are (or >>>>>>>>>>> maybe >>>>>>>>>>>>>>> were) >>>>>>>>>>>>>>>>>>>>>> doing DAG parsing and failures resulting from it. So >>>>>>>>>> we decided >>>>>>>>>>>>>> to >>>>>>>>>>>>>>> just >>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>> allow it to keep consistency. The way how Dag parsing >>>>>>>>>> works is >>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>> when >>>>>>>>>>>>>>>>>>>>>> you are using importlib to read the Dag from a file, >>>>>>>>>> the >>>>>>>>>>> relative >>>>>>>>>>>>>>>>>>> imports >>>>>>>>>>>>>>>>>>>>>> do not work as it does not know what they should be >>>>>>>>>> relative >>>>>>>>>>> to. >>>>>>>>>>>>>>> But if >>>>>>>>>>>>>>>>>>>>>> relative import is done from an imported package, it >>>>>>>>>> should be >>>>>>>>>>> no >>>>>>>>>>>>>>>>>>>>> problem, >>>>>>>>>>>>>>>>>>>>>> I think - otherwise our Dags would not be able to >>>>>>>>>> import any >>>>>>>>>>>>>>> library >>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>> uses relative imports. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Of course consistency might be the reason why we do >>>>>>>>>> not want to >>>>>>>>>>>>>>>>>>> introduce >>>>>>>>>>>>>>>>>>>>>> relative imports. I don't see it as an issue if it is >>>>>>>>>> guarded >>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>> pre-commit >>>>>>>>>>>>>>>>>>>>>> though. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> J. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> J. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> czw., 3 lip 2025, 12:11 użytkownik Ash Berlin-Taylor < >>>>>>>>>>>>>>> a...@apache.org> >>>>>>>>>>>>>>>>>>>>>> napisał: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Oh yes, symlinks will work, with one big caveat: It >>>>>>>>>> does mean >>>>>>>>>>>>>> you >>>>>>>>>>>>>>>>>>> can’t >>>>>>>>>>>>>>>>>>>>>>> use absolute imports in one common module to another. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> For example >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> https://github.com/apache/airflow/blob/4c66ebd06/airflow-core/src/airflow/utils/serve_logs.py#L41 >>>>>>>>>>>>>>>>>>>>>>> where we have >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> from airflow.utils.module_loading import >>>>>>>>>> import_string >>>>>>>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> if we want to move serve_logs into this common lib >>>>>>>>>> that is >>>>>>>>>>> then >>>>>>>>>>>>>>>>>>>>> symlinked >>>>>>>>>>>>>>>>>>>>>>> then we wouldn’t be able to have `from >>>>>>>>>>>>>>> airflow_common.module_loading >>>>>>>>>>>>>>>>>>>>> import >>>>>>>>>>>>>>>>>>>>>>> import_string`. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I can think of two possible solutions here. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> 1) is to allow/require relative imports in this >>>>>>>>>> shared lib, so >>>>>>>>>>>>>>> `from >>>>>>>>>>>>>>>>>>>>>>> .module_loading import import_string` >>>>>>>>>>>>>>>>>>>>>>> 2) is to use `vendoring`[1] (from the pip >>>>>>>>>> maintainers) which >>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>>>> handle >>>>>>>>>>>>>>>>>>>>>>> import-rewriting for us. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I’d entirely forgot that symlinks in repos was a >>>>>>>>>> thing, so I >>>>>>>>>>>>>>> prepared >>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>> minimal POC/demo of what vendoring approach could >>>>>>>>>> look like >>>>>>>>>>> here >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> https://github.com/apache/airflow/commit/996817782be6071b306a87af9f36fe1cf2d3aaa3 >>>>>>>>>>>>>>>>>>>>>>> Now personally I am more than happy with relative >>>>>>>>>> imports, but >>>>>>>>>>>>>>>>>>> generally >>>>>>>>>>>>>>>>>>>>>>> as a project we have avoided them, so I think that >>>>>>>>>> limits what >>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>> could >>>>>>>>>>>>>>>>>>>>> do >>>>>>>>>>>>>>>>>>>>>>> with a symlink based approach. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -ash >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> [1] https://github.com/pradyunsg/vendoring >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On 3 Jul 2025, at 10:30, Pavankumar Gopidesu < >>>>>>>>>>>>>>>>>>> gopidesupa...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>> Thanks Ash >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yes agree option 2 would be preferred for me. >>>>>>>>>> Making sure we >>>>>>>>>>>>>>> have all >>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>> gaurdriles to protect any unwanted behaviour in >>>>>>>>>> code sharing >>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>> executing >>>>>>>>>>>>>>>>>>>>>>>> right of tests between the packages. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Agree with others, option 2 would be >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jul 3, 2025 at 10:02 AM Amogh Desai < >>>>>>>>>>>>>>>>>>> amoghdesai....@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks for starting this discussion, Ash. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> I would prefer option 2 here with proper tooling >>>>>>>>>> to handle >>>>>>>>>>> the >>>>>>>>>>>>>>> code >>>>>>>>>>>>>>>>>>>>>>>>> duplication at *release* time. >>>>>>>>>>>>>>>>>>>>>>>>> It is best to have a dist that has all it needs in >>>>>>>>>> itself. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Option 1 could very quickly get out of hand and if >>>>>>>>>> we decide >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>> separate >>>>>>>>>>>>>>>>>>>>>>>>> triggerer / >>>>>>>>>>>>>>>>>>>>>>>>> dag processor / config etc etc as separate >>>>>>>>>> packages, back >>>>>>>>>>>>>>> compat is >>>>>>>>>>>>>>>>>>>>>>> going >>>>>>>>>>>>>>>>>>>>>>>>> to be a nightmare >>>>>>>>>>>>>>>>>>>>>>>>> and will bite us harder than we anticipate. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks & Regards, >>>>>>>>>>>>>>>>>>>>>>>>> Amogh Desai >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Jul 3, 2025 at 1:12 AM Kaxil Naik < >>>>>>>>>>>>>> kaxiln...@gmail.com> >>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> I prefer Option 2 as well to avoid matrix of >>>>>>>>>> dependencies >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, 3 Jul 2025 at 01:03, Jens Scheffler >>>>>>>>>>>>>>>>>>>>> <j_scheff...@gmx.de.invalid >>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I'd also rather prefer option 2 - reason here is >>>>>>>>>> it is >>>>>>>>>>>>>> rather >>>>>>>>>>>>>>>>>>>>>>> pragmatic >>>>>>>>>>>>>>>>>>>>>>>>>>> and we no not need to cut another package and >>>>>>>>>> have less >>>>>>>>>>>>>>> package >>>>>>>>>>>>>>>>>>>>> counts >>>>>>>>>>>>>>>>>>>>>>>>>>> and dependencies. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I remember some time ago I was checking >>>>>>>>>> (together with >>>>>>>>>>>>>> Jarek, >>>>>>>>>>>>>>> I am >>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>> sure anymore...) if the usage of symlinks would >>>>>>>>>> be >>>>>>>>>>> possible. >>>>>>>>>>>>>>> To >>>>>>>>>>>>>>>>>>> keep >>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>> source in one package but "symlink" it into >>>>>>>>>> another. If >>>>>>>>>>> then >>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>>> point >>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>> packaging/release the files are materialized we >>>>>>>>>> have 1 set >>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>> code. >>>>>>>>>>>>>>>>>>>>>>>>>>> Otherwise if not possible still the redundancy >>>>>>>>>> could be >>>>>>>>>>>>>>> solved by >>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>>>> pre-commit hook - and in Git the files are >>>>>>>>>> de-duplicated >>>>>>>>>>>>>>> anyway >>>>>>>>>>>>>>>>>>>>> based >>>>>>>>>>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>>>>>>>> content hash, so this does not hurt. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 02.07.25 18:49, Shahar Epstein wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> I support option 2 with proper automation & CI >>>>>>>>>> - the >>>>>>>>>>>>>>> reasonings >>>>>>>>>>>>>>>>>>>>>>>>> you've >>>>>>>>>>>>>>>>>>>>>>>>>>>> shown for that make sense to me. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Shahar >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 2, 2025 at 3:36 PM Ash >>>>>>>>>> Berlin-Taylor < >>>>>>>>>>>>>>> a...@apache.org >>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello everyone, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> As we work on finishing off the code-level >>>>>>>>>> separation of >>>>>>>>>>>>>>> Task >>>>>>>>>>>>>>>>>>> SDK >>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>> Core >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (scheduler etc) we have come across some >>>>>>>>>> situations >>>>>>>>>>> where >>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>>>>>> share code between these. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However it’s not as straight forward of “just >>>>>>>>>> put it in >>>>>>>>>>> a >>>>>>>>>>>>>>> common >>>>>>>>>>>>>>>>>>>>>>>>> dist >>>>>>>>>>>>>>>>>>>>>>>>>>> they >>>>>>>>>>>>>>>>>>>>>>>>>>>>> both depend upon” because one of the goals of >>>>>>>>>> the Task >>>>>>>>>>> SDK >>>>>>>>>>>>>>>>>>>>>>>>> separation >>>>>>>>>>>>>>>>>>>>>>>>>>> was >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to have 100% complete version independence >>>>>>>>>> between the >>>>>>>>>>>>>> two, >>>>>>>>>>>>>>>>>>>>> ideally >>>>>>>>>>>>>>>>>>>>>>>>>>> even if >>>>>>>>>>>>>>>>>>>>>>>>>>>>> they are built into the same image and venv. >>>>>>>>>> Most of the >>>>>>>>>>>>>>> reason >>>>>>>>>>>>>>>>>>>>> why >>>>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>>>> isn’t straight forward comes down to backwards >>>>>>>>>>>>>>> compatibility - >>>>>>>>>>>>>>>>>>> if >>>>>>>>>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>>>>>>>>>>>> make >>>>>>>>>>>>>>>>>>>>>>>>>>>>> an change to the common/shared distribution >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> We’ve listed the options we have thought about >>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/apache/airflow/issues/51545 >>>>>>>>>> (but >>>>>>>>>>> that >>>>>>>>>>>>>>> covers >>>>>>>>>>>>>>>>>>>>>>>>> some >>>>>>>>>>>>>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>>>>>>>>>>> things that I don’t want to get in to in this >>>>>>>>>> discussion >>>>>>>>>>>>>>> such as >>>>>>>>>>>>>>>>>>>>>>>>>>> possibly >>>>>>>>>>>>>>>>>>>>>>>>>>>>> separating operators and executors out of a >>>>>>>>>> single >>>>>>>>>>>>>> provider >>>>>>>>>>>>>>>>>>> dist.) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> To give a concrete example of some code I >>>>>>>>>> would like to >>>>>>>>>>>>>>> share >>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> https://github.com/apache/airflow/blob/84897570bf7e438afb157ba4700768ea74824295/airflow-core/src/airflow/_logging/structlog.py >>>>>>>>>>>>>>>>>>>>>>>>>>>>> — logging config. Another thing we will want >>>>>>>>>> to share >>>>>>>>>>> will >>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AirflowConfigParser class from >>>>>>>>>> airflow.configuration >>>>>>>>>>> (but >>>>>>>>>>>>>>>>>>> notably: >>>>>>>>>>>>>>>>>>>>>>>>>> only >>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> parser class, _not_ the default config values, >>>>>>>>>> again, >>>>>>>>>>> lets >>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>> dwell >>>>>>>>>>>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> specifics of that) >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> So to bring the options listed in the issue >>>>>>>>>> here for >>>>>>>>>>>>>>> discussion, >>>>>>>>>>>>>>>>>>>>>>>>>> broadly >>>>>>>>>>>>>>>>>>>>>>>>>>>>> speaking there are two high-level approaches: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. A single shared distribution >>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. No shared package and copy/duplicate code >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The advantage of Approach 1 is that we only >>>>>>>>>> have the >>>>>>>>>>> code >>>>>>>>>>>>>>> in one >>>>>>>>>>>>>>>>>>>>>>>>>> place. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> However for me, at least in this specific case >>>>>>>>>> of >>>>>>>>>>> Logging >>>>>>>>>>>>>>> config >>>>>>>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>>>>>>>>>>> AirflowConfigParser class is that backwards >>>>>>>>>>> compatibility >>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>> much >>>>>>>>>>>>>>>>>>>>>>>>> much >>>>>>>>>>>>>>>>>>>>>>>>>>>>> harder. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> The main advantage of Approach 2 is the the >>>>>>>>>> code is >>>>>>>>>>>>>> released >>>>>>>>>>>>>>>>>>>>>>>>>>> with/embedded >>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the dist (i.e. apache-airflow-task-sdk >>>>>>>>>> would contain >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> right >>>>>>>>>>>>>>>>>>>>>>>>>>> version >>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the logging config and ConfigParser etc). >>>>>>>>>> The >>>>>>>>>>> downside >>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>>> either >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the code will need to be duplicated in the >>>>>>>>>> repo, or >>>>>>>>>>> better >>>>>>>>>>>>>>> yet >>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>>>>>>> would >>>>>>>>>>>>>>>>>>>>>>>>>>>>> live in a single place in the repo, but some >>>>>>>>>> tooling >>>>>>>>>>> (TBD) >>>>>>>>>>>>>>> will >>>>>>>>>>>>>>>>>>>>>>>>>>>>> automatically handle the duplication, either >>>>>>>>>> at commit >>>>>>>>>>>>>>> time, or >>>>>>>>>>>>>>>>>>> my >>>>>>>>>>>>>>>>>>>>>>>>>>>>> preference, at release time. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> For this kind of shared “utility” code I am >>>>>>>>>> very >>>>>>>>>>> strongly >>>>>>>>>>>>>>>>>>> leaning >>>>>>>>>>>>>>>>>>>>>>>>>>> towards >>>>>>>>>>>>>>>>>>>>>>>>>>>>> option 2 with automation, as otherwise I think >>>>>>>>>> the >>>>>>>>>>>>>> backwards >>>>>>>>>>>>>>>>>>>>>>>>>>> compatibility >>>>>>>>>>>>>>>>>>>>>>>>>>>>> requirements would make it unworkable (very >>>>>>>>>> quickly over >>>>>>>>>>>>>>> time >>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> combinations we would have to test would just >>>>>>>>>> be >>>>>>>>>>>>>>> unreasonable) >>>>>>>>>>>>>>>>>>>>> and I >>>>>>>>>>>>>>>>>>>>>>>>>>> don’t >>>>>>>>>>>>>>>>>>>>>>>>>>>>> feel confident we can have things as stable as >>>>>>>>>> we need >>>>>>>>>>> to >>>>>>>>>>>>>>> really >>>>>>>>>>>>>>>>>>>>>>>>>> deliver >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the version separation/independency I want to >>>>>>>>>> delivery >>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>> AIP-72. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> So unless someone feels very strongly about >>>>>>>>>> this, I will >>>>>>>>>>>>>>> come up >>>>>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> draft PR for further discussion that will >>>>>>>>>> implement code >>>>>>>>>>>>>>> sharing >>>>>>>>>>>>>>>>>>>>> via >>>>>>>>>>>>>>>>>>>>>>>>>>>>> “vendoring” it at build time. I have an idea >>>>>>>>>> of how I >>>>>>>>>>> can >>>>>>>>>>>>>>>>>>> achieve >>>>>>>>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>>>>>> so >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we have a single version in the repo and it’ll >>>>>>>>>> work >>>>>>>>>>> there, >>>>>>>>>>>>>>> but >>>>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>>>>>>>>>>>> runtime >>>>>>>>>>>>>>>>>>>>>>>>>>>>> we vendor it in to the shipped dist so it >>>>>>>>>> lives at >>>>>>>>>>>>>> something >>>>>>>>>>>>>>>>>>> like >>>>>>>>>>>>>>>>>>>>>>>>>>>>> `airflow.sdk._vendor` etc. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> In terms of repo layout, this likely means we >>>>>>>>>> would end >>>>>>>>>>> up >>>>>>>>>>>>>>> with: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> airflow-core/pyproject.toml >>>>>>>>>>>>>>>>>>>>>>>>>>>>> airflow-core/src/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> airflow-core/tests/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> task-sdk/pyproject.toml >>>>>>>>>>>>>>>>>>>>>>>>>>>>> task-sdk/src/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> task-sdk/tests/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> airflow-common/src >>>>>>>>>>>>>>>>>>>>>>>>>>>>> airflow-common/tests/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> # Possibly no airflow-common/pyproject.toml, >>>>>>>>>> as deps >>>>>>>>>>> would >>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>> included >>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the downstream projects. TBD. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thoughts and feedback welcomed. >>>>>>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>>>>>>>>>>>>>> To unsubscribe, e-mail: >>>>>>>>>>> dev-unsubscr...@airflow.apache.org >>>>>>>>>>>>>>>>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>>>>>> dev-h...@airflow.apache.org >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>>>>>> To unsubscribe, e-mail: >>>>>>>>>> dev-unsubscr...@airflow.apache.org >>>>>>>>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>> dev-h...@airflow.apache.org >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>> dev-h...@airflow.apache.org >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>> dev-h...@airflow.apache.org >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>>>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
