sunank200 opened a new pull request #16821:
URL: https://github.com/apache/airflow/pull/16821
Add more filter options to list_keys of S3Hook
This commit adds following filters to list the keys in list_keys of S3Hook:
- `start_after_key` filters the any keys after the specified key.
start_after_key can be any key in the bucket.
- `start_after_datetime` filters all the keys with last modified
date-time greater than or equal to the start_after_datetime.
- `to_datetime` filters all the keys with last modified date-time less
than or equal to the to_datetime
closes: #16627
This change wouldn't affect dependencies for other operators like
S3DeleteObjectsOperator, S3ListOperator, S3Hook methods:get_wildcard_key,
delete_bucket and S3KeysUnchangedSensor.
Corresponding unittest has been added to test_s3.py.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #16627
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
