[ 
https://issues.apache.org/jira/browse/AIRFLOW-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16089411#comment-16089411
 ] 

ASF subversion and git services commented on AIRFLOW-1393:
----------------------------------------------------------

Commit 751e936ac2d18aec0315d2ed4f307c6b04ea431e in incubator-airflow's branch 
refs/heads/master from [~ashb]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=751e936 ]

[AIRFLOW-1393][[AIRFLOW-1393] Enable Py3 tests in contrib/spark_submit_hook[

The unit tests in
`tests/contrib/hooks/test_spark_submit_hook.py`
were skiped if run in Python3 because some test
cases loop forever
due to a mismatch/misunderstanding about bytes vs
string that didn't
matter under Py2 (i.e. the mocked data for
subprocess.Popen was
returning a String, but the actual Popen call
would return bytes.)

The fix is to use bytes and `six.ByteIO` so that
the tests work on Py2
and Py3. Alsowe had to patch `subprocess.Popen` in
the right place so
the mocks are picked up.

Closes #2427 from ashb/enable-
spark_submit_hook_tests-py3


> Enable Python3 tests on contrib Spark Submit Hook
> -------------------------------------------------
>
>                 Key: AIRFLOW-1393
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1393
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: tests
>            Reporter: Ash Berlin-Taylor
>            Priority: Minor
>
> The unit tests in {{tests/contrib/hooks/test_spark_submit_hook.py}} currently 
> skip if run in Python3 because some test cases  loop forever due to a 
> mismatch/misunderstanding about bytes vs string (i.e. the mocked data for 
> {{subprocess.Popen}} doesn't behave the same as actually running Popen)
> The fix is to use bytes and {{six.ByteIO}} so that the tests work on Py2 and 
> Py3.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to