[ https://issues.apache.org/jira/browse/AIRFLOW-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ash Berlin-Taylor updated AIRFLOW-3419: --------------------------------------- Summary: S3_hook.select_key is broken on Python3 (was: S3_hook.select_key is broken) > S3_hook.select_key is broken on Python3 > --------------------------------------- > > Key: AIRFLOW-3419 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3419 > Project: Apache Airflow > Issue Type: Bug > Components: boto3, hooks > Affects Versions: 1.10.1 > Reporter: Maria Rebelka > Priority: Major > > Hello, > Using select_key throws an error: > {quote}text = S3Hook('aws_conn').select_key(key='my_key', > bucket_name='my_bucket', > expression='SELECT * FROM S3Object s', > expression_type='SQL', > input_serialization={'JSON': \{'Type': > 'DOCUMENT'}}, > output_serialization={'JSON': {}}){quote} > Traceback (most recent call last): > {quote} File "db.py", line 31, in <module> > output_serialization={'JSON': {}}) > File "/usr/local/lib/python3.4/site-packages/airflow/hooks/S3_hook.py", > line 262, in select_key > for event in response['Payload'] > TypeError: sequence item 0: expected str instance, bytes found{quote} > Seems that the problem is in this line: > S3_hook.py, line 262: return ''.join(event['Records']['Payload'] > which probably should be return > ''.join(event['Records']['Payload'].decode('utf-8') > From example in Amazon blog: > https://aws.amazon.com/blogs/aws/s3-glacier-select/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)