[
https://issues.apache.org/jira/browse/AIRFLOW-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963217#comment-16963217
]
Dom edited comment on AIRFLOW-5819 at 10/30/19 4:55 PM:
--------------------------------------------------------
I am not entirely sure why this issue is not capture by the existing test case
(which does have None in the arguments):
[https://github.com/apache/airflow/blob/73bf718358c01cc41e5f23b914a8824a2665a28c/tests/contrib/operators/test_awsbatch_operator.py#L39]
was (Author: domantasjurkus):
I am not entirely sure why this issue is not capture by the existing test case:
https://github.com/apache/airflow/blob/73bf718358c01cc41e5f23b914a8824a2665a28c/tests/contrib/operators/test_awsbatch_operator.py#L39
> AWSBatchOperator has invalid default value for array_properties
> ---------------------------------------------------------------
>
> Key: AIRFLOW-5819
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5819
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
> Affects Versions: 1.10.6
> Reporter: Dom
> Priority: Major
>
> We upgraded our Airflow version from 1.10.3 to 1.10.6 and saw our
> AWSBatchOperator throwing the following error (easily fixed by changing the
> default param from None to {}):
> {code:java}
> [2019-10-30 13:10:15.392] INFO:airflow.task.operators.execute:93 Running AWS
> Batch Job - Job definition: batch-jobdef-1 - on queue AwsBatch-batch-queue
> [2019-10-30 13:10:15.393] INFO:airflow.task.operators.execute:95
> AWSBatchOperator overrides: {'command': ['--start_datetime', '2019-10-29']}
> [2019-10-30 13:10:15.433] INFO:airflow.task.operators.execute:121 AWS Batch
> Job has failed executed
> [2019-10-30 13:10:15.445] ERROR:airflow.task.handle_failure:1058 Parameter
> validation failed:
> Invalid type for parameter arrayProperties, value: None, type: <class
> 'NoneType'>, valid types: <class 'dict'>
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/awsbatch_operator.py",
> line 108, in execute
> containerOverrides=self.overrides)
> File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 357,
> in _api_call
> return self._make_api_call(operation_name, kwargs)
> File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 634,
> in _make_api_call
> api_params, operation_model, context=request_context)
> File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 682,
> in _convert_to_request_dict
> api_params, operation_model)
> File "/usr/local/lib/python3.7/site-packages/botocore/validate.py", line
> 297, in serialize_to_request
> raise ParamValidationError(report=report.generate_report())
> botocore.exceptions.ParamValidationError: Parameter validation failed:
> Invalid type for parameter arrayProperties, value: None, type: <class
> 'NoneType'>, valid types: <class 'dict'>During handling of the above
> exception, another exception occurred:Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line
> 930, in _run_raw_task
> result = task_copy.execute(context=context)
> File
> "/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/awsbatch_operator.py",
> line 122, in execute
> raise AirflowException(e)
> airflow.exceptions.AirflowException: Parameter validation failed:
> Invalid type for parameter arrayProperties, value: None, type: <class
> 'NoneType'>, valid types: <class 'dict'>
> [2019-10-30 13:10:15.447] INFO:airflow.task.handle_failure:1087 All retries
> failed; marking task as FAILED{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
