idantepper commented on code in PR #39151:
URL: https://github.com/apache/airflow/pull/39151#discussion_r1575309487
##########
airflow/providers/amazon/aws/operators/s3.py:
##########
@@ -503,23 +509,37 @@ def __init__(
self.bucket = bucket
self.keys = keys
self.prefix = prefix
+ self.from_datetime = from_datetime
+ self.to_datetime = to_datetime
self.aws_conn_id = aws_conn_id
self.verify = verify
self._keys: str | list[str] = ""
- if not exactly_one(prefix is None, keys is None):
- raise AirflowException("Either keys or prefix should be set.")
+ if not exactly_one(keys is None, all([prefix is None, from_datetime is
None, to_datetime is None])):
+ raise AirflowException(
+ "Either keys or at least one of prefix, from_datetime,
to_datetime should be set."
+ )
def execute(self, context: Context):
- if not exactly_one(self.keys is None, self.prefix is None):
- raise AirflowException("Either keys or prefix should be set.")
+ if not exactly_one(
+ self.keys is None,
+ all([self.prefix is None, self.from_datetime is None,
self.to_datetime is None]),
+ ):
+ raise AirflowException(
+ "Either keys or at least one of prefix, from_datetime,
to_datetime should be set."
+ )
Review Comment:
actually I was also curious about that at the beginning so I checked why
this was done. in this pr
[https://github.com/apache/airflow/pull/21458#pullrequestreview-878651357
](url) @potiuk mentioned that passing also prefix and keys in init should
fail (as early as possible) but we should also have this in the execute because
we would like this validation also to happen for cases that the values are
XComArgs or templated/Jinja expressions that are evaluated only at the execute.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
