Kaxil Naik created AIRFLOW-6192:
-----------------------------------
Summary: Stop creating Hook from SFTPSensor.__init__
Key: AIRFLOW-6192
URL: https://issues.apache.org/jira/browse/AIRFLOW-6192
Project: Apache Airflow
Issue Type: Improvement
Components: contrib
Affects Versions: 1.10.6
Reporter: Kaxil Naik
Assignee: Kaxil Naik
Fix For: 1.10.7
>From Slack
>(https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1575647500146200):
When I create a task based upon the SFTPSensor and then parse the Python file,
do I see two log entries coming from base_hook.py referencing the default sftp
connection id? I also see the same log messages whenever the DAGs are
refreshed.
airflow@0030a4ae49df:~$ python dags/sftp_sensor.py
[2019-12-06 15:45:16,440] {{settings.py:252}} INFO - settings.configure_orm():
Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=9685
/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning:
The psycopg2 wheel package will be renamed from release 2.8; in order to keep
installing from binary please use "pip install psycopg2-binary" instead. For
details see:
<http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
""")
[2019-12-06 15:45:17,535] {{base_hook.py:84}} INFO - Using connection to: id:
sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow,
Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True}
[2019-12-06 15:45:17,539] {{base_hook.py:84}} INFO - Using connection to: id:
sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow,
Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True}
airflow@0030a4ae49df:~$ airflow list_dags
[2019-12-06 15:47:43,529] {{settings.py:252}} INFO - settings.configure_orm():
Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=11223
/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning:
The psycopg2 wheel package will be renamed from release 2.8; in order to keep
installing from binary please use "pip install psycopg2-binary" instead. For
details see:
<http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
""")
[2019-12-06 15:47:44,631] {{__init__.py:51}} INFO - Using executor LocalExecutor
[2019-12-06 15:47:44,631] {{dagbag.py:92}} INFO - Filling up the DagBag from
/usr/local/airflow/dags
[2019-12-06 15:47:48,998] {{base_hook.py:84}} INFO - Using connection to: id:
sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow,
Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True}
[2019-12-06 15:47:49,005] {{base_hook.py:84}} INFO - Using connection to: id:
sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow,
Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True}
This is on v1.10.6
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
