[incubator-livy] branch master updated: [LIVY-795] Add support for PySpark with Python >= 3.8 (#314)

[leesf]

This is an automated email from the ASF dual-hosted git repository.

leesf pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-livy.git


The following commit(s) were added to refs/heads/master by this push:
     new 2b25c710 [LIVY-795] Add support for PySpark with Python >= 3.8 (#314)
2b25c710 is described below

commit 2b25c71008e98dc645f6117d379e1ca790688730
Author: Gabriel Magno <gabrielmag...@gmail.com>
AuthorDate: Tue Nov 8 22:03:34 2022 -0300

    [LIVY-795] Add support for PySpark with Python >= 3.8 (#314)
---
 repl/src/main/resources/fake_shell.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/repl/src/main/resources/fake_shell.py 
b/repl/src/main/resources/fake_shell.py
index 2a99185b..b5b284d8 100644
--- a/repl/src/main/resources/fake_shell.py
+++ b/repl/src/main/resources/fake_shell.py
@@ -40,6 +40,14 @@ else:
     import cStringIO
     import StringIO
 
+if sys.version_info > (3,8):
+    from ast import Module
+else :
+    # mock the new API, ignore second argument
+    # see https://github.com/ipython/ipython/issues/11590
+    from ast import Module as OriginalModule
+    Module = lambda nodelist, type_ignores: OriginalModule(nodelist)
+
 logging.basicConfig()
 LOG = logging.getLogger('fake_shell')
 
@@ -219,7 +227,7 @@ class NormalNode(object):
 
         try:
             for node in to_run_exec:
-                mod = ast.Module([node])
+                mod = Module([node], [])
                 code = compile(mod, '<stdin>', 'exec')
                 exec(code, global_dict)