vincbeck opened a new pull request, #22737: URL: https://github.com/apache/airflow/pull/22737
Deprecate `S3PrefixSensor` and use `S3KeySensor` instead. ## Why deprecating `S3PrefixSensor`? `S3PrefixSensor` and `S3KeySensor` are doing pretty much the same thing so to avoid duplicates, we should deprecate one. Also `S3PrefixSensor` does not behave the way it is described by the documentation in comments and the way you expect by the name. `S3PrefixSensor` does not wait for a given prefix in S3 to exist but it waits for a given folder in S3 to exist (given the delimiter is `/`). Here are some examples I ran for testing. - `prefix="test"`. `true` when directory `test` exists - `prefix="test"`. `false` when directory `test` does not exist and file `test` exists - `prefix="test"`. `false` when directory `test` does not exist and file `test2` exists - `prefix="tes"`. `false` when directory `test` exists This misalignment between expected behavior and actual one is confusing for users. [Example of thread](https://stackoverflow.com/questions/71511535/s3prefixsensor-in-mwaa?noredirect=1#comment126395890_71511535) where a user does not understand why `S3PrefixSensor` behave this way. ## Why updating `S3KeySensor`? In order to be backward compatible we want to use `S3KeySensor` instead. `S3PrefixSensor` accept a list of files as input, so `S3KeySensor` should be too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
