[ https://issues.apache.org/jira/browse/AIRFLOW-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaxil Naik updated AIRFLOW-6192: -------------------------------- Description: >From Slack >(https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1575647500146200): When I create a task based upon the SFTPSensor and then parse the Python file, do I see two log entries coming from base_hook.py referencing the default sftp connection id? I also see the same log messages whenever the DAGs are refreshed. {noformat} airflow@0030a4ae49df:~$ python dags/sftp_sensor.py [2019-12-06 15:45:16,440] {{settings.py:252}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=9685 /usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. """) [2019-12-06 15:45:17,535] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} [2019-12-06 15:45:17,539] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} airflow@0030a4ae49df:~$ airflow list_dags [2019-12-06 15:47:43,529] {{settings.py:252}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=11223 /usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. """) [2019-12-06 15:47:44,631] {{__init__.py:51}} INFO - Using executor LocalExecutor [2019-12-06 15:47:44,631] {{dagbag.py:92}} INFO - Filling up the DagBag from /usr/local/airflow/dags [2019-12-06 15:47:48,998] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} [2019-12-06 15:47:49,005] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} {noformat} This is on v1.10.6 was: >From Slack >(https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1575647500146200): When I create a task based upon the SFTPSensor and then parse the Python file, do I see two log entries coming from base_hook.py referencing the default sftp connection id? I also see the same log messages whenever the DAGs are refreshed. airflow@0030a4ae49df:~$ python dags/sftp_sensor.py [2019-12-06 15:45:16,440] {{settings.py:252}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=9685 /usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. """) [2019-12-06 15:45:17,535] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} [2019-12-06 15:45:17,539] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} airflow@0030a4ae49df:~$ airflow list_dags [2019-12-06 15:47:43,529] {{settings.py:252}} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=11223 /usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. """) [2019-12-06 15:47:44,631] {{__init__.py:51}} INFO - Using executor LocalExecutor [2019-12-06 15:47:44,631] {{dagbag.py:92}} INFO - Filling up the DagBag from /usr/local/airflow/dags [2019-12-06 15:47:48,998] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} [2019-12-06 15:47:49,005] {{base_hook.py:84}} INFO - Using connection to: id: sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': True} This is on v1.10.6 > Stop creating Hook from SFTPSensor.__init__ > ------------------------------------------- > > Key: AIRFLOW-6192 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6192 > Project: Apache Airflow > Issue Type: Improvement > Components: contrib > Affects Versions: 1.10.6 > Reporter: Kaxil Naik > Assignee: Kaxil Naik > Priority: Minor > Fix For: 1.10.7 > > > From Slack > (https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1575647500146200): > When I create a task based upon the SFTPSensor and then parse the Python > file, do I see two log entries coming from base_hook.py referencing the > default sftp connection id? I also see the same log messages whenever the > DAGs are refreshed. > {noformat} > airflow@0030a4ae49df:~$ python dags/sftp_sensor.py > [2019-12-06 15:45:16,440] {{settings.py:252}} INFO - > settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, > pool_recycle=1800, pid=9685 > /usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: > The psycopg2 wheel package will be renamed from release 2.8; in order to keep > installing from binary please use "pip install psycopg2-binary" instead. For > details see: > <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. > """) > [2019-12-06 15:45:17,535] {{base_hook.py:84}} INFO - Using connection to: id: > sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, > Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': > True} > [2019-12-06 15:45:17,539] {{base_hook.py:84}} INFO - Using connection to: id: > sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, > Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': > True} > airflow@0030a4ae49df:~$ airflow list_dags > [2019-12-06 15:47:43,529] {{settings.py:252}} INFO - > settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, > pool_recycle=1800, pid=11223 > /usr/local/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: > The psycopg2 wheel package will be renamed from release 2.8; in order to keep > installing from binary please use "pip install psycopg2-binary" instead. For > details see: > <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>. > """) > [2019-12-06 15:47:44,631] {{__init__.py:51}} INFO - Using executor > LocalExecutor > [2019-12-06 15:47:44,631] {{dagbag.py:92}} INFO - Filling up the DagBag from > /usr/local/airflow/dags > [2019-12-06 15:47:48,998] {{base_hook.py:84}} INFO - Using connection to: id: > sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, > Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': > True} > [2019-12-06 15:47:49,005] {{base_hook.py:84}} INFO - Using connection to: id: > sftp_default. Host: localhost, Port: 22, Schema: None, Login: airflow, > Password: None, extra: {'key_file': '~/.ssh/id_rsa', 'no_host_key_check': > True} > {noformat} > This is on v1.10.6 -- This message was sent by Atlassian Jira (v8.3.4#803005)