jaegwonseo commented on code in PR #23695:
URL: https://github.com/apache/airflow/pull/23695#discussion_r874526859
##########
airflow/providers/google/cloud/transfers/sql_to_gcs.py:
##########
@@ -165,7 +168,9 @@ def _write_local_data_files(self, cursor):
names in GCS, and values are file handles to local files that
contain the data for the GCS objects.
"""
- schema = list(map(lambda schema_tuple: schema_tuple[0],
cursor.description))
+ org_schema = list(map(lambda schema_tuple: schema_tuple[0],
cursor.description))
+ schema = [column for column in org_schema if column not in
self.exclude_columns]
+
Review Comment:
If schema is changed to set type, the order of the columns is changed.
It seems better to use a list to keep the order of the columns.
however, it seems good to receive the input value as a set to prevent
duplication of the input exclude column.
```python
delegate_to: Optional[str] = None,
impersonation_chain: Optional[Union[str, Sequence[str]]] = None,
exclude_columns=None,
**kwargs,
) -> None:
super().__init__(**kwargs)
if exclude_columns is None:
exclude_columns = set()
```
using exclude_columns as a set to prevent duplication
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
