potiuk commented on PR #56187: URL: https://github.com/apache/airflow/pull/56187#issuecomment-3538530821
i am sorry to delay this one, but It really makes little sense to make it this way with those dependencies becoming more complex and intertwined than less. The goal for Airflow 3.2 is that `task-sdk` and `airflow-core` are completely isolated from each other. And this is the **only** purpose of having things in `shared` library to allow that. It's ok to remove some of the other things (3. 4.) and keep that one rebased for a while. Yes. I understand that it's more difficult and painful, but we have a history of keeping such PRs (and even PRS depending on those) open and continuously rebasing them on top of each other for a while. That's kind of painful, I agree, but the pain is localised to your area of expertise and knowledge and what you really trade-off here is that **someone** (possibly even you) will anyhow have to solve the same issues in the future, but it will be more messy and difficult to understand what to do. Having a "clear isolation" goal makes it relatively **easy** for you to understand what's the goal. When you merge it to main, the actual goal and problems to solve will be way more blurry. Cross dependencies will creep in shared libraries and some of the assertions we have in the shared libraries (only depend on external deps, other distributions in airflow should only "use" shared dependencies, and shared dependencies can depend on other "shared" dependnecies - but without circular dependencies) - are no longer held and it makes it difficult for others to understand what is right and what is wrong. Deep down it's all about cyclomatic complexity (at distribution level) and keeping dependencies as DAG (yeah really - we absolutely want to avoid cycles of dependencies when it comes to shared dependencies). From that point of view, it's way healthier (for a project) to only merge it when those problems are solved - even if it is more painful for you, personally to keep the PRs rebased. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
