[
https://issues.apache.org/jira/browse/AIRFLOW-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491033#comment-16491033
]
ASF subversion and git services commented on AIRFLOW-2525:
----------------------------------------------------------
Commit dabf1b962dcd4323a6ea723076afcd2a20fcb354 in incubator-airflow's branch
refs/heads/master from [~sekikn]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=dabf1b9 ]
[AIRFLOW-2525] Fix PostgresHook.copy_expert to work with "COPY FROM"
For now PostgresHook.copy_expert supports
"COPY TO" but not "COPY FROM", because it
opens a file with write mode and doesn't
commit operations. This PR fixes it by
opening a file with read and write mode
and committing operations at last.
> Fix PostgresHook.copy_expert to work with "COPY FROM"
> -----------------------------------------------------
>
> Key: AIRFLOW-2525
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2525
> Project: Apache Airflow
> Issue Type: Bug
> Components: hooks
> Reporter: Kengo Seki
> Assignee: Kengo Seki
> Priority: Major
> Fix For: 2.0.0
>
>
> psycopg2's {{copy_expert}} [works with both "COPY FROM" and "COPY
> TO"|http://initd.org/psycopg/docs/cursor.html#cursor.copy_expert].
> But {{PostgresHook.copy_expert}} fails with "COPY FROM" as follows:
> {code}
> In [1]: from airflow.hooks.postgres_hook import PostgresHook
> In [2]: hook = PostgresHook()
> In [3]: hook.copy_expert("COPY t FROM STDIN", "/tmp/t")
> [2018-05-24 23:37:54,767] {base_hook.py:83} INFO - Using connection to:
> localhost
> ---------------------------------------------------------------------------
> QueryCanceledError Traceback (most recent call last)
> <ipython-input-3-7d9c6c8c57fc> in <module>()
> ----> 1 hook.copy_expert("COPY t FROM STDIN", "/tmp/t")
> ~/dev/incubator-airflow/airflow/hooks/postgres_hook.py in copy_expert(self,
> sql, filename, open)
> 68 with closing(self.get_conn()) as conn:
> 69 with closing(conn.cursor()) as cur:
> ---> 70 cur.copy_expert(sql, f)
> 71
> 72 @staticmethod
> QueryCanceledError: COPY from stdin failed: error in .read() call:
> UnsupportedOperation not readable
> CONTEXT: COPY t, line 1
> {code}
> This is because the file used in this method is opened with 'w' mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
