rsg17 commented on pull request #21704:
URL: https://github.com/apache/airflow/pull/21704#issuecomment-1047046996
> Tests are failing:
> ```
>
> self = <Task(GCSToTrinoOperator): test_gcs_to_trino>, context = None
>
> def execute(self, context: 'Context') -> None:
> gcs_hook = GCSHook(
> gcp_conn_id=self.gcp_conn_id,
> delegate_to=self.delegate_to,
> impersonation_chain=self.impersonation_chain,
> )
>
> trino_hook = TrinoHook(trino_conn_id=self.trino_conn_id)
>
> with NamedTemporaryFile("w+") as temp_file:
> self.log.info("Downloading data from %s", self.source_object)
> gcs_hook.download(
> bucket_name=self.source_bucket,
> object_name=self.source_object,
> filename=temp_file.name,
> )
>
> data = csv.reader(temp_file)
> > fields = tuple(next(data))
> E TypeError: 'tuple' object is not an iterator
>
> airflow/providers/trino/transfers/gcs_to_trino.py:97: TypeError
> ```
Yes. I made the change based on @uranusjr suggestion. This logic is more
efficient than reading in the whole csv at once; but it causes the unit tests
to fail.
I pushed the change upstream so that I can get suggestions on why this
happens.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
