shahar1 commented on issue #29115: URL: https://github.com/apache/airflow/issues/29115#issuecomment-1528808991
> Due to name of argument as `delimiter`, description and code of Operator and [google-cloud-storage](https://cloud.google.com/python/docs/reference/storage/latest/buckets#listblobsmaxresultsnone-pagetokennone-prefixnone-delimiternone-startoffsetnone-endoffsetnone-includetrailingdelimiternone-versionsnone-projectionnoacl-fieldsnone-clientnone-timeout60-retrygoogleapicoreretryretry-object) I thought it work correctly. However have no idea who actually need this functionality. > > I thought better implement new argument with proper implementation and with name `suffix` Based on the following resources: [1] https://cloud.google.com/storage/docs/json_api/v1/objects/list?apix_params=%7B%22bucket%22%3A%22reproducing_issue_11874%22%2C%22delimiter%22%3A%22.PNG_2%22%7D#try-it [2] https://stackoverflow.com/questions/44702835/google-cloud-storage-ignoring-delimiter It seems that treating `delimiter` as `suffix` is a misuse, and filtering according to suffixes shouldn't be done as part of the API call to GCS but as a separate logic within the hook by simple list filtering. Therefore, I suggest deprecating the `delimiter` parameter (as it should almost always be `"/") and adding a `suffix` parameter instead for use as a filter, as follows: If `delimiter` is set and `suffix` is not already provided, warn that `delimiter` is deprecated and assign `self.suffix` its value, while `self.delimiter` will be set to a constant `"/"`. If `suffix` is provided, it will behave as expected regardless of the value of `delimiter`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
