This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new faf3c4fe47 Fix PostgresToGCSOperat bool dtype (#25475)
faf3c4fe47 is described below
commit faf3c4fe474733965ab301465f695e3cc311169c
Author: dclandau <[email protected]>
AuthorDate: Tue Aug 2 21:28:56 2022 +0100
Fix PostgresToGCSOperat bool dtype (#25475)
When converting postgres boolean data type it was defaulting to the
parquet parquet.string() data type. Pyarrow would then raise an error
when attempting to convert the boolean data type into its string.
Changing the PostgresToGCSOperator class map `data_type` to convert
postgres boolean type to bigquery `BOOL` data type which then maps to
the parquet `pa.bool_()` data type when `_convert_parquet_schema` is
called.
---
airflow/providers/google/cloud/transfers/postgres_to_gcs.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/airflow/providers/google/cloud/transfers/postgres_to_gcs.py
b/airflow/providers/google/cloud/transfers/postgres_to_gcs.py
index 3f3012514d..97748003db 100644
--- a/airflow/providers/google/cloud/transfers/postgres_to_gcs.py
+++ b/airflow/providers/google/cloud/transfers/postgres_to_gcs.py
@@ -88,7 +88,7 @@ class PostgresToGCSOperator(BaseSQLToGCSOperator):
20: 'INTEGER',
21: 'INTEGER',
23: 'INTEGER',
- 16: 'BOOLEAN',
+ 16: 'BOOL',
700: 'FLOAT',
701: 'FLOAT',
1700: 'FLOAT',
