Kengo Seki created AIRFLOW-2705:
-----------------------------------
Summary: Move class-level moto decorator to method-level
Key: AIRFLOW-2705
URL: https://issues.apache.org/jira/browse/AIRFLOW-2705
Project: Apache Airflow
Issue Type: Bug
Components: redshift, tests
Reporter: Kengo Seki
Assignee: Kengo Seki
In tests/contrib/hooks/test_redshift_hook.py and
tests/contrib/sensors/test_aws_redshift_cluster_sensor.py, moto decorators are
at class-level, whereas they are at method-level in other tests using moto.
{code:title=tests/contrib/hooks/test_redshift_hook.py}
@mock_redshift
class TestRedshiftHook(unittest.TestCase):
{code}
I found it can affect other tests using HTTP request. Let's say the following
test is added to tests/hooks/test_http_hook.py:
{code:title=tests/hooks/test_http_hook.py}
def test_sample(self):
HttpHook("GET").run("")
{code}
This test sends a GET request to https://www.google.com/ and succeeds of course:
{code}
$ ./run_unit_tests.sh tests.hooks.test_http_hook:TestHttpHook.test_sample
(snip)
/home/sekikn/a3/lib/python3.5/site-packages/urllib3/connectionpool.py:858:
InsecureRequestWarning: Unverified HTTPS request is being made. Adding
certificate verification is strongly advised. See:
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
.
----------------------------------------------------------------------
Ran 1 test in 0.189s
OK
{code}
But strangely, it fails with ConnectionError when it's executed after
tests/contrib/hooks/test_redshift_hook.py or
tests/contrib/sensors/test_aws_redshift_cluster_sensor.py:
{code}
$ ./run_unit_tests.sh tests.contrib.hooks.test_redshift_hook
tests.hooks.test_http_hook:TestHttpHook.test_sample
(snip)
E
======================================================================
ERROR: test_sample (tests.hooks.test_http_hook.TestHttpHook)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/sekikn/dev/incubator-airflow/tests/hooks/test_http_hook.py", line
250, in test_sample
HttpHook("GET").run("")
File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line
119, in run
return self.run_and_check(session, prepped_request, extra_options)
File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line
167, in run_and_check
raise ex
File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line
159, in run_and_check
allow_redirects=extra_options.get("allow_redirects", True))
File "/home/sekikn/a3/lib/python3.5/site-packages/requests/sessions.py", line
618, in send
r = adapter.send(request, **kwargs)
File
"/home/sekikn/a3/lib/python3.5/site-packages/moto/packages/responses/responses.py",
line 308, in unbound_on_send
return self._on_request(adapter, request, *a, **kwargs)
File
"/home/sekikn/a3/lib/python3.5/site-packages/moto/packages/responses/responses.py",
line 250, in _on_request
raise response
requests.exceptions.ConnectionError: Connection refused: GET
https://www.google.com/
-------------------- >> begin captured stdout << ---------------------
[2018-06-30 09:41:21,029] {base_hook.py:83} INFO - Using connection to:
https://www.google.com/
[2018-06-30 09:41:21,030] {http_hook.py:118} INFO - Sending 'GET' to url:
https://www.google.com/
[2018-06-30 09:41:21,030] {http_hook.py:166} WARNING - Connection refused: GET
https://www.google.com/ Tenacity will retry to execute the operation
--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to:
https://www.google.com/
airflow.hooks.http_hook.HttpHook: INFO: Sending 'GET' to url:
https://www.google.com/
airflow.hooks.http_hook.HttpHook: WARNING: Connection refused: GET
https://www.google.com/ Tenacity will retry to execute the operation
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 7 tests in 1.838s
FAILED (errors=1)
{code}
This problem is solved by using method-level moto decorator instead of
class-level one. I suspect class-level decorator patches method that sends HTTP
request and doesn't restore the original state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
