Repository: zeppelin Updated Branches: refs/heads/branch-0.8 b682ce16b -> e133507da
[zeppelin-3625] [branch-0.8] Pandasql interpreter fails to query over python dataframe ### What is this PR for? fix pandasql query over dataframe for `branch-0.8`. the reason in this branch seems to be different from `master`. first of all there's NPE with python context being used from pandas without being initialised. after fixing it there's another problem with `_displayhook`. ### What type of PR is it? [Bug Fix] ### Todos * [x] - fix npe * [x] - fix `_displayhook` * [x] - test ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3625 ### How should this be tested? explained in the issue ### Screenshots (if appropriate) original NPE different from `master`: <img width="1232" alt="screen shot 2018-07-15 at 7 26 39 pm" src="https://user-images.githubusercontent.com/1642088/42735603-f01172a0-8891-11e8-87a5-8b359209b889.png"> after fixing npe, issue with `_displayhook`: <img width="1271" alt="screen shot 2018-07-15 at 7 32 54 pm" src="https://user-images.githubusercontent.com/1642088/42735612-1bc13a34-8892-11e8-9379-af56c46a2ecb.png"> and interpreter logs show <img width="884" alt="screen shot 2018-07-15 at 7 33 47 pm" src="https://user-images.githubusercontent.com/1642088/42735615-3563e108-8892-11e8-979a-73d9fcf28c0a.png"> at this point not sure why can't execute `__zeppelin__._displayhook()` in ipython kernel and not sure what's the purpose of this command on pandasql->interpret. removing this command resolves issue as below. opinions on this are appreciated in case there're better options <img width="1270" alt="screen shot 2018-07-15 at 7 37 01 pm" src="https://user-images.githubusercontent.com/1642088/42735636-d83db566-8892-11e8-92fc-383957cbcee1.png"> ### Questions: * Does the licenses files need update? * Is there breaking changes for older versions? * Does this needs documentation? Author: Khalid Huseynov <[email protected]> Closes #3073 from khalidhuseynov/fix-0.8/ZEPPELIN-3625 and squashes the following commits: d48418790 [Khalid Huseynov] enable all python tests bce568971 [Khalid Huseynov] remove _displayhook from pandasql interpret af11abd1b [Khalid Huseynov] fix npe Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/e133507d Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/e133507d Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/e133507d Branch: refs/heads/branch-0.8 Commit: e133507dac4e16bdb749e822e0d00ab7fba17782 Parents: b682ce1 Author: Khalid Huseynov <[email protected]> Authored: Thu Jul 19 12:12:24 2018 +0900 Committer: Jongyoul Lee <[email protected]> Committed: Fri Jul 20 08:01:16 2018 +0900 ---------------------------------------------------------------------- python/pom.xml | 5 ----- .../main/java/org/apache/zeppelin/python/PythonInterpreter.java | 2 +- .../org/apache/zeppelin/python/PythonInterpreterPandasSql.java | 2 +- 3 files changed, 2 insertions(+), 7 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/pom.xml ---------------------------------------------------------------------- diff --git a/python/pom.xml b/python/pom.xml index 468a7bc..2e6912c 100644 --- a/python/pom.xml +++ b/python/pom.xml @@ -35,11 +35,6 @@ <properties> <interpreter.name>python</interpreter.name> <python.py4j.version>0.9.2</python.py4j.version> - <python.test.exclude> - **/PythonInterpreterWithPythonInstalledTest.java, - **/PythonInterpreterPandasSqlTest.java, - **/PythonInterpreterMatplotlibTest.java - </python.test.exclude> <pypi.repo.url>https://pypi.python.org/packages</pypi.repo.url> <python.py4j.repo.folder>/64/5c/01e13b68e8caafece40d549f232c9b5677ad1016071a48d04cc3895acaa3</python.py4j.repo.folder> <grpc.version>1.4.0</grpc.version> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java ---------------------------------------------------------------------- diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java index fab2ed9..cac743d 100644 --- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java +++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java @@ -554,7 +554,7 @@ public class PythonInterpreter extends Interpreter implements ExecuteResultHandl } try { - interpret(bootstrapCode, context); + interpret(bootstrapCode, InterpreterContext.get()); } catch (InterpreterException e) { throw new IOException(e); } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java ---------------------------------------------------------------------- diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java index 54984c3..ea05ce7 100644 --- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java +++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java @@ -90,7 +90,7 @@ public class PythonInterpreterPandasSql extends Interpreter { Interpreter python = getPythonInterpreter(); return python.interpret( - "__zeppelin__.show(pysqldf('" + st + "'))\n__zeppelin__._displayhook()", context); + "__zeppelin__.show(pysqldf('" + st + "'))", context); } @Override
