szczeles commented on issue #5428: AIRFLOW-4809 | s3_delete_objects_operator should not fail on empty list of keys URL: https://github.com/apache/airflow/pull/5428#issuecomment-503445558 @OmerJog I like your point of view! However, I checked boto3 documentation and it looks the list of objects is marked as REQUIRED (see: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.delete_objects) so I suspect the correct library usage is to "not call the `delete_objects` if there is nothing to delete". An interesting fact is that In boto2 function `delete_keys()` works well on empty array provided. According to your second question - in unix shell the result depends on the "force" flag, see: ``` ~ $ mkdir /tmp/e ~ $ rm /tmp/e/file # deleting non-existing keys rm: /tmp/e/file: No such file or directory ~ $ echo $? 1 ~ $ rm -f /tmp/e/file ~ $ echo $? 0 ~ $ rm # deleting empty keys usage: rm [-f | -i] [-dPRrvW] file ... unlink file ~ $ echo $? 64 ~ $ rm -f # deleting empty keys ~ $ echo $? 0 ``` I just checked that the operator already works as in the "force" mode because deleting non-existing key results in task success. So currently it works in mixed-force mode: for deleting nonexisting key it works exactly as `rm -f`, for deleting no keys it works as `rm`. This PR applies "force" mode for both cases. @ashb thank you for your suggestion, it makes 100% sense to add a flag here. Should it be named `not_found_mode` as in IMAP or maybe `force` to match syntax for unix tools that in general "remove things"?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
