Hi Bruno, This is a known issue with matplotlib that's probably out of our control. Essentially, figure drawing callbacks are handled differently depending on whether or not your shell environment is ipython. I have described the issue in more detail here:
https://issues.apache.org/jira/browse/ZEPPELIN-1904 In my post I also gave a workaround, which is setting your zeppelin.pyspark.python property value to ipython instead of python in your interpreter settings. All of this of course assumes you have ipython. Let me know if this helps. Thanks, Alex On Fri, Feb 24, 2017 at 2:14 AM, Bruno Aranda <[email protected]> wrote: > Hi, > > I am trying to use matplotlib to render some plots using the plot_date > function in PySpark, where the X axis are datetimes. With a sample code > like this one: > > ////// > > %pyspark > > import matplotlib.pyplot as plt > import matplotlib.dates as mdates > import pandas as pd > import numpy as np > > sx = pd.Series(np.random.randn(5)) > sy = pd.Series(['1487930047000', '1487930048000', '1487930049000', > '1487930050000', '1487930051000']) > > df = DataFrame({"value": sx, "time": sy}) > > # df = sqlContext.table("clipinParams").toPandas() > df['time'] = pd.to_datetime(df['time'],unit='ms') > > plt.plot_date(df['time'], df['value']) > plt.show() > plt.close() > > ////// > > However, it seems to fail every other run and we cannot get why. The > exception is the following: > > ////// > > Traceback (most recent call last): > File "/tmp/zeppelin_pyspark-6333777839230958463.py", line 346, in <module> > raise Exception(traceback.format_exc()) > Exception: Traceback (most recent call last): > File "/tmp/zeppelin_pyspark-6333777839230958463.py", line 334, in <module> > exec(code) > File "<stdin>", line 9, in <module> > File "/usr/local/lib64/python2.7/site-packages/matplotlib/pyplot.py", > line 3341, in plot_date > data=data, **kwargs) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/__init__.py", > line 1892, in inner > return func(ax, *args, **kwargs) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axes/_axes.py", > line 1480, in plot_date > self.xaxis_date(tz) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axes/_base.py", > line 3315, in xaxis_date > self.xaxis.axis_date(tz) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axis.py", line > 1674, in axis_date > self.update_units(datetime.datetime(2009, 1, 1, 0, 0, 0, 0, tz)) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axis.py", line > 1424, in update_units > self._update_axisinfo() > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axis.py", line > 1442, in _update_axisinfo > self.set_major_locator(info.majloc) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axis.py", line > 1543, in set_major_locator > self.stale = True > File "/usr/local/lib64/python2.7/site-packages/matplotlib/artist.py", > line 279, in stale > self.stale_callback(self, val) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/artist.py", > line 76, in _stale_axes_callback > self.axes.stale = val > File "/usr/local/lib64/python2.7/site-packages/matplotlib/artist.py", > line 279, in stale > self.stale_callback(self, val) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/figure.py", > line 56, in _stale_figure_callback > self.figure.stale = val > File "/usr/local/lib64/python2.7/site-packages/matplotlib/artist.py", > line 279, in stale > self.stale_callback(self, val) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/pyplot.py", > line 576, in _auto_draw_if_interactive > fig.canvas.draw_idle() > File "/usr/lib/zeppelin/interpreter/lib/python/backend_zinline.py", line > 132, in draw_idle > self.draw(*args, **kwargs) > File > "/usr/local/lib64/python2.7/site-packages/matplotlib/backends/backend_agg.py", > line 464, in draw > self.figure.draw(self.renderer) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/artist.py", > line 63, in draw_wrapper > draw(artist, renderer, *args, **kwargs) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/figure.py", > line 1143, in draw > renderer, self, dsu, self.suppressComposite) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/image.py", line > 139, in _draw_list_compositing_images > a.draw(renderer) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/artist.py", > line 63, in draw_wrapper > draw(artist, renderer, *args, **kwargs) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axes/_base.py", > line 2409, in draw > mimage._draw_list_compositing_images(renderer, self, dsu) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/image.py", line > 139, in _draw_list_compositing_images > a.draw(renderer) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/artist.py", > line 63, in draw_wrapper > draw(artist, renderer, *args, **kwargs) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axis.py", line > 1136, in draw > ticks_to_draw = self._update_ticks(renderer) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axis.py", line > 969, in _update_ticks > tick_tups = [t for t in self.iter_ticks()] > File "/usr/local/lib64/python2.7/site-packages/matplotlib/axis.py", line > 912, in iter_ticks > majorLocs = self.major.locator() > File "/usr/local/lib64/python2.7/site-packages/matplotlib/dates.py", line > 983, in __call__ > self.refresh() > File "/usr/local/lib64/python2.7/site-packages/matplotlib/dates.py", line > 1003, in refresh > dmin, dmax = self.viewlim_to_dt() > File "/usr/local/lib64/python2.7/site-packages/matplotlib/dates.py", line > 760, in viewlim_to_dt > return num2date(vmin, self.tz), num2date(vmax, self.tz) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/dates.py", line > 401, in num2date > return _from_ordinalf(x, tz) > File "/usr/local/lib64/python2.7/site-packages/matplotlib/dates.py", line > 254, in _from_ordinalf > dt = datetime.datetime.fromordinal(ix).replace(tzinfo=UTC) > ValueError: ordinal must be >= 1 > > ////// > > Any ideas of things we can look at? > > Cheers, > > Bruno > -- Alex Goodman Data Scientist I Science Data Modeling and Computing (398K) Jet Propulsion Laboratory California Institute of Technology Tel: +1-818-354-6012
