Yuvaneshnm opened a new issue #15271:
URL: https://github.com/apache/airflow/issues/15271
**Description**
When we try to use AWS secret manager as backend to store airflow
connections(in URI format), Our DAG fails with error
**Error:**
{{emr_create_job_flow_operator.py:66}} INFO - Creating JobFlow using
aws-conn-id: AWS-S3-TCDS, emr-conn-id: AWS-EMR-TCDS-AGG-HOURLY-STG
[2021-04-02 18:41:19,041] {{taskinstance.py:1150}} ERROR - Parameter
validation failed:
Invalid type for parameter Applications, value: [{'Name': 'Hadoop'},
{'Name': 'Spark'}, {'Name': 'Zeppelin'}, {'Name': 'Hive'}, {'Name':
'Ganglia'}], type: <class 'str'>, valid types: <class 'list'>, <class 'tuple'>
Note : All the values in the connection are retrieved as String and we need
emr_hook.py to convert strings to their expected format.
**Use case / motivation**
We need emr_hook.py to convert strings to their expected format so that we
will be able use our connections stored in AWS secret manager backend which is
recommended by AWS Managed airflow(MWAA)
**Are you willing to submit a PR?**
No
**Related Issues**
Could not find any related issue at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
