Kengo Seki created AIRFLOW-2525:
-----------------------------------
Summary: Fix PostgresHook.copy_expert to work with "COPY FROM"
Key: AIRFLOW-2525
URL: https://issues.apache.org/jira/browse/AIRFLOW-2525
Project: Apache Airflow
Issue Type: Bug
Components: hooks
Reporter: Kengo Seki
Assignee: Kengo Seki
psycopg2's {{copy_expert}} [works with both "COPY FROM" and "COPY
TO"|http://initd.org/psycopg/docs/cursor.html#cursor.copy_expert].
But {{PostgresHook.copy_expert}} fails with "COPY FROM" as follows:
{code}
In [1]: from airflow.hooks.postgres_hook import PostgresHook
In [2]: hook = PostgresHook()
In [3]: hook.copy_expert("COPY t FROM STDIN", "/tmp/t")
[2018-05-24 23:37:54,767] {base_hook.py:83} INFO - Using connection to:
localhost
---------------------------------------------------------------------------
QueryCanceledError Traceback (most recent call last)
<ipython-input-3-7d9c6c8c57fc> in <module>()
----> 1 hook.copy_expert("COPY t FROM STDIN", "/tmp/t")
~/dev/incubator-airflow/airflow/hooks/postgres_hook.py in copy_expert(self,
sql, filename, open)
68 with closing(self.get_conn()) as conn:
69 with closing(conn.cursor()) as cur:
---> 70 cur.copy_expert(sql, f)
71
72 @staticmethod
QueryCanceledError: COPY from stdin failed: error in .read() call:
UnsupportedOperation not readable
CONTEXT: COPY t, line 1
{code}
This is because the file used in this method is opened with 'w' mode.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
