[
https://issues.apache.org/jira/browse/AIRFLOW-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamil Bregula updated AIRFLOW-3487:
-----------------------------------
Component/s: gcp
> PostgresToGoogleCloudStorageOperator can't handle big sql result
> ----------------------------------------------------------------
>
> Key: AIRFLOW-3487
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3487
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib, gcp, operators
> Affects Versions: 1.10.1
> Reporter: Duan Shiqiang
> Priority: Major
>
> PostgresToGoogleCloudStorageOperator is using psycopg2's default cursor to
> query postgres which will execute and fetch all results in memory. For bigger
> sql result it won't work if the airflow worker doesn't have that much memory.
> The operator should consider using server side cursor to handle big volume of
> data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
