[ https://issues.apache.org/jira/browse/AIRFLOW-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802908#comment-16802908 ]
ASF subversion and git services commented on AIRFLOW-3419: ---------------------------------------------------------- Commit 8eaaec638d711229a7923feb1968e9c5f638bfcc in airflow's branch refs/heads/master from Ash Berlin-Taylor [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=8eaaec6 ] [AIRFLOW-3419] Fix S3Hook.select_key on Python3 (#4970) > S3_hook.select_key is broken on Python3 > --------------------------------------- > > Key: AIRFLOW-3419 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3419 > Project: Apache Airflow > Issue Type: Bug > Components: boto3, hooks > Affects Versions: 1.10.1 > Reporter: Maria Rebelka > Priority: Major > Fix For: 1.10.3 > > > Hello, > Using select_key throws an error: > {quote}text = S3Hook('aws_conn').select_key(key='my_key', > bucket_name='my_bucket', > expression='SELECT * FROM S3Object s', > expression_type='SQL', > input_serialization={'JSON': \{'Type': > 'DOCUMENT'}}, > output_serialization={'JSON': {}}){quote} > Traceback (most recent call last): > {quote} File "db.py", line 31, in <module> > output_serialization={'JSON': {}}) > File "/usr/local/lib/python3.4/site-packages/airflow/hooks/S3_hook.py", > line 262, in select_key > for event in response['Payload'] > TypeError: sequence item 0: expected str instance, bytes found{quote} > Seems that the problem is in this line: > S3_hook.py, line 262: return ''.join(event['Records']['Payload'] > which probably should be return > ''.join(event['Records']['Payload'].decode('utf-8') > From example in Amazon blog: > https://aws.amazon.com/blogs/aws/s3-glacier-select/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)