Thank you for detailed explanation, Alex! I see what you mean now and that would be splendid to have such level of integration.
I think a very good start will be - to create an umbrella JIRA issue, put the detailed description there, and then post the link to the dev@z.i.a.o mailing list, moving further discussion there. I have some experience with Python, but not familiar with internals of matplotlib yet. Would be happy to help on the JVM\Javascript side of things. -- Alex On Sun, Aug 14, 2016 at 1:07 PM, Goodman, Alexander (398K) < alexander.good...@jpl.nasa.gov> wrote: > Hi Alex, > > On Sat, Aug 13, 2016 at 7:53 PM, Alexander Bezzubov <b...@apache.org> > wrote: > > > > > > Hi Alex, > > answer inline > > > > On Sat, Aug 13, 2016 at 1:15 PM, Goodman, Alexander (398K) < > alexander.good...@jpl.nasa.gov> wrote: > > >> > > >> Thank you guys for the responses. > >> > >> I wonder how much the editor has to do with it since I was able to get > my changes to work (ie the ones in my PR) after manually adding them in a > vanilla editor (I was previously using Atom). However I did later have > issues similar to those mentioned in ZEPPELIN-1133 when importing some > notebooks that I had previously used with the pyspark interpreter in > zeppelin 0.5 and replacing each cell with the python interpreter. If you > are interested, I would be happy to share them with you. > > > > > > It would be very appreciated indeed if you could followup on the issues > that you had \w some reproducible examples i.e in comments in ZEPPELIN-1133 > > Will do. I'll post the notebook in question in the JIRA thread a bit later > and how to reproduce the errors I was getting. > > > > > > > > >> In general I also think we should continue to try taking matplotlib > integration much further, hopefully to the point where its inline plotting > features in zeppelin are comparable to those found in Jupyter. > > > > > > > > I totally agree. Could you elaborate a bit on what "inline plotting > features found in Jupyter" do you mean here? > > Essentially at we should continue to make matplotlib plotting feel more > seamless. That means we should at the minimum try to accomplish what we > have now using plt.show() instead of z.show(plt) (plt being > matplotlib.pyplot of course). This will require implementing a custom > matplotlib backend which defines the show() function specific for a > zeppelin notebook. Jupyter / ipython accomplishes this through a custom > "inline" matplotlib backend. It could also help pave the way for using > matplotlib's interactive plotting features, which will require some > javascript integration. matplotlib ships a backend that does this called > "nbagg" which accomplishes this for Jupyter. > > > > > > > > > >> > >> The latter obviously has the advantage of having more direct python and > javascript integration, but nonetheless this is a subject I would > personally be very interested in exploring in more detail. > > > > > > I'll be happy to help here, let's join forces. What kind of > python\javascript integration do you think we should have in Apache > Zeppelin? May be we should start a new thread discussing it in more details. > > Sure thing. Should I make a thread about this on the dev mailing list or > make a new JIRA issue? I think I have already described the gist of it > above, so for now I am thinking we should prioritize the potential > development of a new matplotlib backend for zeppelin that shows static > images. I am mainly a python specialist so I would be happy to collaborate. > > > > Thanks, > Alex > >> On Fri, Aug 12, 2016 at 8:53 PM, Alexander Bezzubov <b...@apache.org> > wrote: > > >>> > > >>> Hi Alexander, > >>> > >>> thank you for contributing Matplotlib integration improvements! > >>> > >>> You are right, it's odd and this is not your text editor's fault - our > current Python interpreter implementation is quite early stages and these > are artefacts of the way it currently works. Sometimes it does not accept > valid Python files [1]. > >>> > >>> I'm working on improving it right now [2], by changing the underneath > mechanism of communication with Python process, which should address this > issues, as well as few others [3], like windows support, better error > detection, etc. > >>> > >>> I hope to submit a PR next week, meanwhile the best way to debug your > changes is to enable logging [4] by adding a line to log4j.properties " > log4j.logger.org.apache.zeppelin.python.PythonInterpreter=DEBUG" (i.e by > creating it in ./python/src/test/resources/ if you want it to work only in > unit-tests, or just changing code to `log.warn` to always log it) in order > to be able to see what exactly actual Python repl process is not happy > about. > >>> > >>> Hope this helps! > >>> > >>> 1. https://issues.apache.org/jira/browse/ZEPPELIN-1133 > >>> 2. https://issues.apache.org/jira/browse/ZEPPELIN-1325 > >>> 3. https://issues.apache.org/jira/browse/ZEPPELIN-1133?jql= > project%20%3D%20ZEPPELIN%20AND%20component%20%3D% > 20python-interpreter%20AND%20resolution%20%3D%20Unresolved > >>> 4. https://github.com/apache/zeppelin/blob/master/python/ > src/main/java/org/apache/zeppelin/python/PythonInterpreter.java#L221 > >>> > >>> -- > >>> Alex > >>> > >>> On Sat, Aug 13, 2016, 01:04 Goodman, Alexander (398K) < > alexander.good...@jpl.nasa.gov> wrote: > > >>>> > > >>>> An update: It seems like the problem might lie in my editor. After a > bit of hacking into the log fiIes, I found that there were several > IndentationErrors being reported, starting at the PyZeppelinContext > constructor, resulting in a seemingly empty class once bootstrap.py is > processed. It's still odd because again I can import bootstrap.py in my > local python installation and use z.show() without IndentationErrors. > >>>> > >>>> On Fri, Aug 12, 2016 at 7:19 AM, Goodman, Alexander (398K) < > alexander.good...@jpl.nasa.gov> wrote: > > >>>>> > > >>>>> Hi, > >>>>> > >>>>> I am attempting to improve the matplotlib plotting capabilities in > the python interpreter (see the ZEPPELIN-1318 JIRA issue) and have ran into > some issues. Specifically, after saving my changes to bootstrap.py and > rebuilding, I get the following error message after running the unit tests: > >>>>> > >>>>> AttributeError: 'PyZeppelinContext' object has no attribute 'show' > expected:<SUCCESS> but was:<ERROR> > >>>>> > >>>>> PythonInterpreterPandasSqlTest.sqlOverTestDataPrintsTable:129 > Traceback (most recent call last): > >>>>> > >>>>> File "<stdin>", line 1, in <module> > >>>>> > >>>>> The same error message also appeared when I attempted to run the > simple matplotlib example notebook. This seems to happen if I make any > changes to the show_matplotlib() method in bootstrap.py, including one line > changes that don't actually do anything. I also tested the module for > syntax errors in my local python installation and everything seems to check > out, so I can't understand where the problem lies. I am fairly certain my > changes are sound (see [1]). I tested this for a zeppelin build without any > additional options (mvn clean package -DskipTests) both with the modified > and unmodified source code, the latter having no issues. > >>>>> > >>>>> [1] - https://github.com/agoodm/zeppelin/commit/ > 10c06a9ceb7be94085ea39ebaad8beab7e0ff024 > >>>>> > >>>>> Any help would be appreciated. > >>>>> > >>>>> Thanks, > >>>>> Alex > >>>>> > >>>>> -- > >>>>> Alex Goodman > >>>>> Data Scientist I > >>>>> Science Data Modeling and Computing (398K) > >>>>> Jet Propulsion Laboratory > >>>>> California Institute of Technology > >>>>> Tel: +1-818-354-6012 > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Alex Goodman > >>>> Data Scientist I > >>>> Science Data Modeling and Computing (398K) > >>>> Jet Propulsion Laboratory > >>>> California Institute of Technology > >>>> Tel: +1-818-354-6012 > >> > >> > >> > >> > >> -- > >> Alex Goodman > >> Data Scientist I > >> Science Data Modeling and Computing (398K) > >> Jet Propulsion Laboratory > >> California Institute of Technology > >> Tel: +1-818-354-6012 > > > > > > -- > Alex Goodman > Data Scientist I > Science Data Modeling and Computing (398K) > Jet Propulsion Laboratory > California Institute of Technology > Tel: +1-818-354-6012 >
