potiuk commented on a change in pull request #7791: Add ability to specify a
maximum modified time for objects in GCSToGCSOperator
URL: https://github.com/apache/airflow/pull/7791#discussion_r395973548
##########
File path: airflow/providers/google/cloud/operators/gcs_to_gcs.py
##########
@@ -87,6 +87,10 @@ class GCSToGCSOperator(BaseOperator):
only if they were modified after last_modified_time.
If tzinfo has not been set, UTC will be assumed.
:type last_modified_time: datetime.datetime
+ :param maximum_modified_time: When specified, the objects will be copied
or moved,
Review comment:
What I meant was to add an option to specify how old the object should be -
i.e. 3600 = older than 1 hour. So the thing is to specify the "age" (relative
to current time) rather than modified time (absolute UTC). -> it boils down to
"max_modified < current_time - age
(so it is a simple modification) but I think it is much more useful this
way - otherwise you wil have to do this calculation in DAGs (you will rarely
pass absolute time as parameter hard-coded in DAG.).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
