uranusjr commented on a change in pull request #18807:
URL: https://github.com/apache/airflow/pull/18807#discussion_r726882552
##########
File path: airflow/providers/amazon/aws/sensors/s3_prefix.py
##########
@@ -67,18 +67,15 @@ def __init__(
super().__init__(**kwargs)
# Parse
self.bucket_name = bucket_name
- self.prefix = prefix
+ self.prefix = [prefix] if isinstance(prefix, str) else prefix
self.delimiter = delimiter
- self.full_url = "s3://" + bucket_name + '/' + prefix
self.aws_conn_id = aws_conn_id
self.verify = verify
self.hook: Optional[S3Hook] = None
- def poke(self, context):
+ def poke(self, context: Dict[str, Any]):
self.log.info('Poking for prefix : %s in bucket s3://%s', self.prefix,
self.bucket_name)
- return self.get_hook().check_for_prefix(
- prefix=self.prefix, delimiter=self.delimiter,
bucket_name=self.bucket_name
- )
+ return all(self._check_for_prefix(prefix) for prefix in self.prefix)
Review comment:
> I'm curious how we handle such backward incompatible changes.
Do you mean changing `prefix` to `prefixes`? As the change stands right now,
it's already backwards incoimpatible, so changing the name is not really a
concern. It's extremely unlikely (and not idiomatic) to access attributes in a
hook, so I shouldn't really matter much. We could even change the attribute to
`_prefixes` to discourage access even further.
As for the `all` issue, you can actually fix it quite easily with
`all([...])` (explicitly cast the result of all calls to a list).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
