I¹ve been having some issues with this logic. Once I get to sizable
datasets (~ 40,000), it no longer works, while with smaller ones ~1000 it
works. I¹m thinking the HTML becomes too much, since this works fine using
seaborn/matplotlib alone:
%%pyspark
# seaborn stuff
sns.set(style='darkgrid')
sm = sns.PairGrid(df_sub.tail(150),
hue='label', palette={0: '#931D3D', 1: '#54B847'},
size=4, hue_kws={'marker': ['o', 'D']})
sm = sm.map_offdiag(plt.scatter, alpha=0.25, s=25)
sm = sm.map_diag(plt.hist, bins=10)
sm = sm.add_legend()
# trying to get it to show up
img = StringIO.StringIO()
sm.savefig(img, format='svg')
img.seek(0)
html_txt = "<div style='width:600px'>" + img.buf + "</div>"
kernel.display().html(html_txt)
Any ideas how to get that second block leveraging Toree in a better way?
Thanks,
Ian
On 11/2/16, 7:46 PM, "Marius van Niekerk" <[email protected]>
wrote:
>So matplotlib isn't very well supported at the moment. Toree's pyspark
>kernel is not ipykernel -- so the magics that work with that (and
>matplotlib integration) sadly won't work.
>
>You can do it by calling the kernel.display methods afaik.
>On Wed, Nov 2, 2016 at 15:22 <[email protected]> wrote:
>
>> Hi,
>>
>> I just noticed that simple plots with matplotlib do not work in Toree. I
>> get this error in the UI:
>>
>>
>> Magic pyspark failed to execute with error:
>> null was reset!
>>
>>
>> In the logs I see:
>>
>> 16/11/02 14:12:15 ERROR PySparkProcessHandler: null process failed:
>> org.apache.commons.exec.ExecuteException: Process exited with an error:
>>134
>> (Exit value: 134)
>> 16/11/02 14:12:15 WARN MagicManager: Magic pyspark failed to execute
>>with
>> error:
>> null was reset!
>> res12:
>>
>>Either[org.apache.toree.magic.CellMagicOutput,org.apache.toree.magic.Line
>>MagicOutput]
>> =
>> Left(Map(text/plain -> Magic pyspark failed to execute with error:
>> null was reset!))
>>
>> Is there a fix for this or some kind of work around? Otherwise, if you
>>let
>> me know what issue type, I can create an issue in Jira.
>>
>> Thanks,
>>
>> Ian
>>
>--
>regards
>Marius van Niekerk
