jaegwonseo opened a new issue, #23694:
URL: https://github.com/apache/airflow/issues/23694
### Description
when transfer data from rdb to gcs, some column should be droped for
security issue
### Use case/motivation
When retrieving data from each table, mainly query to get all columns as
shown below.
```sql
select * from table
```
we can use specify the columns to import for each table, but in this case,
we have to know the schema of all tables.
By adding exclude_column args to the operator as shown below, data from
various tables can be sent to gcs with the same query.
```python
to_tcs_task = MySQLToGCSOperator(
task_id='',
sql=f'select * from target_table,
bucket='',
filename='',
schema_filename='',
export_format='json',
exclude_column=target_table_info.exclude_column,
mysql_conn_id=source_connect_id,
gcp_conn_id=target_connect_id
)
```
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
