Tuan Vu created AIRFLOW-3889:
--------------------------------
Summary: Handle bytes type in render_template_from_field for
PubSubPublishOperator
Key: AIRFLOW-3889
URL: https://issues.apache.org/jira/browse/AIRFLOW-3889
Project: Apache Airflow
Issue Type: Bug
Components: models
Affects Versions: 1.10.0
Environment: composer-1.3.0-airflow-1.10.0
Google Cloud Composer v1.3.0
Airflow v.10.0
Reporter: Tuan Vu
The render_template_from_field method of the BaseOperator class raises an
exception when it encounters content that is bytes type. Specifically, the
PubSubPublishOperator requires the message data to be encoded as base64.
Error:
{code:java}
airflow.exceptions.AirflowException: Type '<class 'bytes'>' used for parameter
'messages[data]' is not supported for templating
{code}
Steps to reproduce:
* Airflow version: 1.10.0 - I use Google Cloud Composer with Airflow v1.10.0
* Run through the example_pubsub_flow:
[https://github.com/apache/airflow/blob/53b89b98371c7bb993b242c341d3941e9ce09f9a/airflow/contrib/example_dags/example_pubsub_flow.py]
* Encounter errors on task t3 in the DAG.
Input:
{code:java}
messages = [
{'data': b64encode(b'Hello World')},
{'data': b64encode(b'Another message')},
{'data': b64encode(b'A final message')}
]
{code}
Task:
{code:java}
t3 = PubSubPublishOperator(
task_id='publish-messages', messages=messages)
{code}
Output error:
{code:java}
airflow.exceptions.AirflowException: Type '<class 'bytes'>' used for parameter
'messages[data]' is not supported for templating{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
