[
https://issues.apache.org/jira/browse/AIRFLOW-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16089412#comment-16089412
]
ASF subversion and git services commented on AIRFLOW-1393:
----------------------------------------------------------
Commit 751e936ac2d18aec0315d2ed4f307c6b04ea431e in incubator-airflow's branch
refs/heads/master from [~ashb]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=751e936 ]
[AIRFLOW-1393][[AIRFLOW-1393] Enable Py3 tests in contrib/spark_submit_hook[
The unit tests in
`tests/contrib/hooks/test_spark_submit_hook.py`
were skiped if run in Python3 because some test
cases loop forever
due to a mismatch/misunderstanding about bytes vs
string that didn't
matter under Py2 (i.e. the mocked data for
subprocess.Popen was
returning a String, but the actual Popen call
would return bytes.)
The fix is to use bytes and `six.ByteIO` so that
the tests work on Py2
and Py3. Alsowe had to patch `subprocess.Popen` in
the right place so
the mocks are picked up.
Closes #2427 from ashb/enable-
spark_submit_hook_tests-py3
> Enable Python3 tests on contrib Spark Submit Hook
> -------------------------------------------------
>
> Key: AIRFLOW-1393
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1393
> Project: Apache Airflow
> Issue Type: Bug
> Components: tests
> Reporter: Ash Berlin-Taylor
> Priority: Minor
>
> The unit tests in {{tests/contrib/hooks/test_spark_submit_hook.py}} currently
> skip if run in Python3 because some test cases loop forever due to a
> mismatch/misunderstanding about bytes vs string (i.e. the mocked data for
> {{subprocess.Popen}} doesn't behave the same as actually running Popen)
> The fix is to use bytes and {{six.ByteIO}} so that the tests work on Py2 and
> Py3.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
