[
https://issues.apache.org/jira/browse/LIVY-795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shiquan Wang updated LIVY-795:
------------------------------
Description:
When working with PySpark with Python >= 3.8, below error is raised:
{code:java}
TypeError: required field "type_ignores" missing from Module{code}
The error points to code in
[https://github.com/apache/incubator-livy/blob/4d8a912699683b973eee76d4e91447d769a0cb0d/repl/src/main/resources/fake_shell.py#L223]
The error is caused by an API change in Python 3.8:
[https://github.com/python/cpython/pull/11645/files] and was reported here
[https://bugs.python.org/issue35894]
IPython has faced similar issue:
[https://github.com/ipython/ipython/issues/11590]
IPython solved the issue in this PR:
[https://github.com/ipython/ipython/pull/11593]
was:
When working with PySpark with Python >= 3.8, below error is raised:
{code:java}
TypeError: required field "type_ignores" missing from Module{code}
The error points to code in
[https://github.com/apache/incubator-livy/blob/4d8a912699683b973eee76d4e91447d769a0cb0d/repl/src/main/resources/fake_shell.py#L223]
The error is caused by an API change in Python 3.8:
[https://github.com/python/cpython/pull/11645/files] and was reported here
[https://bugs.python.org/issue35894]
IPython has faced similar issue:
[https://github.com/ipython/ipython/issues/11590]
IPython solved the issue in this PR:
[https://github.com/ipython/ipython/pull/11593]
> Can't work with PySpark with Python >= 3.8 due to
> https://bugs.python.org/issue35894
> ------------------------------------------------------------------------------------
>
> Key: LIVY-795
> URL: https://issues.apache.org/jira/browse/LIVY-795
> Project: Livy
> Issue Type: Bug
> Components: REPL
> Affects Versions: 0.7.0
> Environment: EMR 6.1.0
> Reporter: Shiquan Wang
> Priority: Blocker
> Labels: easyfix
>
>
> When working with PySpark with Python >= 3.8, below error is raised:
> {code:java}
> TypeError: required field "type_ignores" missing from Module{code}
> The error points to code in
> [https://github.com/apache/incubator-livy/blob/4d8a912699683b973eee76d4e91447d769a0cb0d/repl/src/main/resources/fake_shell.py#L223]
>
> The error is caused by an API change in Python 3.8:
> [https://github.com/python/cpython/pull/11645/files] and was reported here
> [https://bugs.python.org/issue35894]
> IPython has faced similar issue:
> [https://github.com/ipython/ipython/issues/11590]
> IPython solved the issue in this PR:
> [https://github.com/ipython/ipython/pull/11593]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
