[
https://issues.apache.org/jira/browse/AIRFLOW-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor closed AIRFLOW-4809.
--------------------------------------
Resolution: Abandoned
> s3_delete_objects_operator fails on empty list of keys
> ------------------------------------------------------
>
> Key: AIRFLOW-4809
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4809
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib, operators
> Affects Versions: 1.10.3
> Reporter: Mariusz Strzelecki
> Assignee: Mariusz Strzelecki
> Priority: Major
>
> When s3_delete_objects_operator is used in a dynamic way (for example list of
> keys comes from s3_list_operator via XCom) there might be a case when the
> list of keys is empty. In my case it happens when chained operators are
> removing old files from S3 and there are no old files yet (because this is
> very first run of DAG).
> In case of empty `keys` hook raises an exception (via boto3):
> {noformat}
> [2019-06-18 13:23:53,790] {{base_task_runner.py:101}} INFO - Job 115: Subtask
> delete_old_files [2019-06-18 13:23:53,790] {{cli.py:517}} INFO - Running
> <TaskInstance: xxxxxx.delete_old_files 2019-06-17T00:00:00+00:00 [running]>
> on host 82f571f444f5
> [2019-06-18 13:23:56,199] {{__init__.py:1580}} ERROR - An error occurred
> (MalformedXML) when calling the DeleteObjects operation: The XML you provided
> was not well-formed or did not validate against our published schema
> Traceback (most recent call last):
> File "/usr/local/lib/python3.6/site-packages/airflow/models/__init__.py",
> line 1441, in _run_raw_task
> result = task_copy.execute(context=context)
> File
> "/usr/local/lib/python3.6/site-packages/airflow/contrib/operators/s3_delete_objects_operator.py",
> line 80, in execute
> response = s3_hook.delete_objects(bucket=self.bucket, keys=self.keys)
> File "/usr/local/lib/python3.6/site-packages/airflow/hooks/S3_hook.py",
> line 542, in delete_objects
> Delete=delete_dict)
> File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 357,
> in _api_call
> return self._make_api_call(operation_name, kwargs)
> File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 661,
> in _make_api_call
> raise error_class(parsed_response, operation_name)
> botocore.exceptions.ClientError: An error occurred (MalformedXML) when
> calling the DeleteObjects operation: The XML you provided was not well-formed
> or did not validate against our published schema
> {noformat}
> I already have a patch that checks if there is anything to delete and if not,
> just returns from the operator.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
