potiuk commented on PR #41463: URL: https://github.com/apache/airflow/pull/41463#issuecomment-2308905483
> hi, I did a compatible 2.8 2.9 2.10 implementation https://github.com/apache/airflow/pull/41047 but you said that it was not a feature welcome on 2.10 branch. I think there was no (easy) way to make it really 2.8-2.10 compatible. It was still changing behaviour for 2.10 - as I understood it and it was main reason why I hesitated with merging it - because we did not want to suddenly change behaviour of those sensors whe users will move between 2.8 and 2.10. The problem with the change is it is implemented now is that you can't import "AirflowPokeFailException" And "FailPolicy" from "airflow 2" in providers - because it is just missing in Airflow 2.8 - 2.10 and providers simply will not work for 2.8. 2.9. 2.10. And since we do not have Airlfow 3 for quite some time (6 months) - it means they won't work at all. And having it in 2.10 would not help to much (because the providers will still have to work for 2.8-2.9 where it is missing). But the future implementation of it might be (or at least that's how I see it). Example SFTP provider: * airflow.providers.sftp depends on airflow.sdlk (which is independent from Airflow Version - and can be installed also in Airflow 2.8 - 2.10 (or whatever versions will be supported then) * the PokeFailException , Fail Policy and - most importantly - BaseSensorOperator will also come from the "task-sdk", not from "airflow". * the "BaseSensorOperator" from "task-sdk" - at least as long as Airflow 2 is supported - should support both - soft fail (with deprecation) and Fail Policy - and deprecation on soft_fail will suggest to switch to Airflow 3's Fail Policy and we might be able to drop it some day - and the thing there is that for those future providers, the user will be able to see the deprecation to Fail Policy even when running on Airflow 2. I am not sure if it makes it clearer - but with "task.sdk" and code for airflow BaseOperators / Sensor operators etc. coming from there, rather than from "airflow" we will be able to handle it quite a bit better. Or so I hope. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
