[ https://issues.apache.org/jira/browse/AIRFLOW-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joy Gao updated AIRFLOW-1613: ----------------------------- Description: In Python 3, map(...) returns an iterator, which can only be iterated over once. Therefore the current implementation will return an empty list after the first iteration of schema: {code:python} schema = map(lambda schema_tuple: schema_tuple[0], cursor.description) file_no = 0 tmp_file_handle = NamedTemporaryFile(delete=True) tmp_file_handles = {self.filename.format(file_no): tmp_file_handle} for row in cursor: # Convert datetime objects to utc seconds, and decimals to floats row = map(self.convert_types, row) row_dict = dict(zip(schema, row)) {code} Moving it inside the loop for re-use. was: In Python 3, map(...) returns an iterator, which can only be iterated over once. Therefore the current implementation will return an empty list after the first iteration of schema: {code:python} schema = map(lambda schema_tuple: schema_tuple[0], cursor.description) file_no = 0 tmp_file_handle = NamedTemporaryFile(delete=True) tmp_file_handles = {self.filename.format(file_no): tmp_file_handle} for row in cursor: # Convert datetime objects to utc seconds, and decimals to floats row = map(self.convert_types, row) row_dict = dict(zip(schema, row)) {code} Moving it inside the loop for re-use. > Make MySqlToGoogleCloudStorageOperator compaitible with python3 > --------------------------------------------------------------- > > Key: AIRFLOW-1613 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1613 > Project: Apache Airflow > Issue Type: Bug > Components: contrib > Reporter: Joy Gao > > In Python 3, map(...) returns an iterator, which can only be iterated over > once. > Therefore the current implementation will return an empty list after the > first iteration of schema: > {code:python} > schema = map(lambda schema_tuple: schema_tuple[0], cursor.description) > file_no = 0 > tmp_file_handle = NamedTemporaryFile(delete=True) > tmp_file_handles = {self.filename.format(file_no): tmp_file_handle} > for row in cursor: > # Convert datetime objects to utc seconds, and decimals to floats > row = map(self.convert_types, row) > row_dict = dict(zip(schema, row)) > {code} > Moving it inside the loop for re-use. -- This message was sent by Atlassian JIRA (v6.4.14#64029)