potiuk commented on issue #58006: URL: https://github.com/apache/airflow/issues/58006#issuecomment-3500155341
Actually - I think this is one of the cases where I think we should come back to my original idea - where I proposed that - until we truly separate task-sdk from airflow and have a set of compatibiliy tests that test various versions of airlfow and task-sdk together (reasonable subset) - we should absolutely bind airflow and task-sdk with ==. This is what I proposed originally - to have airflow-core and airflow 3.1.N to have `task-sdk==1.1.N` as dependency, We have absolutely no idea if task-sdk 1.1.2 (or 1.1.3) will work with 3.1.1 for example, and that combination is perfectly valid when we used in airflow 3.1.1 -> `task-sdk<1,2.0,>1.1.1` We do not test it now, and by my standards it means that it `does not work` (and as we see now it also caused problems with structlog - so cadwyn is not really providing1 100% certainty that things will work across the versions, there are other things at play such as common dependencies for example. This will be **way** more viable after we truly separate task-sdk and complete the `shared-libraries` separation - still some reasonable tests will have to run with newer task-sdk versions and older Airflow versions (again because of potentially future upgrades to shared dependencies - like structlog that will not be limited for older airflow might break cooperation between the two - if only because some internals of data passed by the API will change. My proposal for Airflow 3.1.3 will be to set task-sdk==1.1.3. If we had it in 3.1.2 we would imediately find out that task-sdk 1.1.2 has not been bumped in pyproject.toml - if only for that reason I think it's way better approach. cc: @ashb @amoghrajesh -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
