Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

2010-09-14 Thread Eric Firing 
On 09/13/2010 08:54 AM, Brian Granger wrote:
> Eric,
>
> Sorry about the delay, I was on vacation last week...comments inline below...
>
> On Tue, Sep 7, 2010 at 2:26 PM, Eric Firing  wrote:
>> On 09/07/2010 11:07 AM, Fernando Perez wrote:
>>> Hi Eric,
>>>
>>> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firingwrote:

 I have been doing a little testing with ipython 0.10 versus
 ipython-newkernel, both modes, and with mpl svn versus your guisupport.
 There are so many possible modes of operation and combinations of
 versions and backends that all this will take some time to sort out.

 Can you give me simple examples of what does *not* work correctly when
 you use mpl *svn* with ipython-newkernel, in either or both of the
 console or gui modes, but *does* work with your guisupport version?
>>>
>>> Thanks for your testing, Eric.
>>>
>>> With matplotlib *alone*, I can't find a way to crash/lock/whatever the
>>> combo of matplotlib(svn)+ipython-newkernel.
>>>
>>> The reason, i believe, is that guisupport.py is available in ipython
>>> itself, and it goes out of its way to avoid creating a second main qt
>>> app, letting matplotlib create it. Since that main app is alive all
>>> the time, there's only one app and one event loop and life is good.
>>> But if I were to open another library that uses Qt and makes a new
>>> main qApp unconditionally, we'd have problems.
>>>
>>> Brian and Evan have recently just added the guisupport.py patch to
>>> Enthought's ETS, so that now it probably will be pretty hard to
>>> actually see the problem: if both ipython and ets go out of their way
>>> to avoid the nested main app issue, mpl can get away with making one
>>> unconditionally and things will probably work OK.
>>>
>>> But the idea is for all of us (ipython, ets, mpl, etc) to agree on a
>>> collaborative protocol with a simple api: check for one special
>>> '_in_event_loop' flag in the main toolkit before making one.  That
>>> will make it easier to have interactive code that uses Wx or Qt from
>>> more than one library coexisting in the same process.
>>
>> Fernando,
>
>
>> There are two parts to guisupport: ensure a single main app, and ensure
>> no more than one call to the mainloop.
>
> Yes, that is a good summary.
>
>>   The first makes perfect sense,
>> and cannot cause any problems that I can see.  The second one is the one
>> that I think may be both unnecessary and undesirable.  The reason is
>> that the gui toolkit mainloop functions or methods are designed for
>> nested calls.  This permits blocking within a running mainloop, and
>> allows show() to block when pyplot is not in interactive mode.  This is
>> what is lost with the guisupport mods.  Some changes to mainloop logic
>> may well be needed, but I don't think that prohibiting nested calls to
>> the underlying toolkit mainloop function is necessary or desirable.
>
> This is a very good point and is something that we have thought
> carefully about.  You are very correct, that the functions in
> guisupport cannot be used to do a nested mainloop.  Nested calls to
> the mainloop should be done in the usual manner by simply calling the
> appropriate gui toolkit method for doing so.  We probably need to
> clarify this point, but the focus of the functions in guisupport are
> *only* the first and main invocation of the event loop.  Basically, we
> want to ensure that:
>
> * Projects don't accidentally do nested mainloops because there were
> not aware that the outermost eventloop was already running.
> * Projects start the outermost eventloop in a manner that other
> projects will be able to reliably detect.
>
> I should mention the other approach that we have tried, but that failed:
>
> * Have IPython startup, create an app and start the main loop.
> * Then monkeypatch the gui toolkit so that the mainloop calls are no-ops.
> * Further monkeypatch the gui toolkit so that it appears that the
> mainloop is running (even when it may not be because of PyOS_InputHook
> magic).
>
> This allowed us to do everything, BUT obviously, nested mainloops
> failed.  Thus, making sure that nested mainloops still work was the
> main reason we have created guisupport.  We should better document
> these details though.

Brian,

Thanks for the clarification.  Your basic approach sounds fine. I need 
to think a little more about how to handle this in mpl.  The proposed 
modification is not quite right yet.  I probably can't work on it until 
the weekend.

Eric

>
> Cheers,
>
> Brian
>
>> Eric
>>
>>>
>>> I'll let Brian fill in with more details when he has some
>>> availability, but I think that's the gist of it.
>>>
>>> Regards,
>>>
>>> f
>>
>>
>> --
>> This SF.net Dev2Dev email is sponsored by:
>>
>> Show off your parallel programming skills.
>> Enter the Intel(R) Threading Challenge 2010.
>> http://p.sf.net/sfu/intel-thread-sfd
>> __

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

2010-09-14 Thread Eric Firing 
On 09/13/2010 05:46 PM, Brian Granger wrote:
> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing  wrote:
>> On 09/03/2010 12:37 PM, Brian Granger wrote:
>>> Hello all,
>>>
>>> I would like to submit the following branch on github for review and
>>> merging into matplotlib trunk:
>>>
>>> http://github.com/ellisonbg/matplotlib/commits/guisupport
>>>
>>> This branch implements the logic needed for the qt4 and wx backends to
>>> fully work with the upcoming IPython 0.11 release.  In our testing we
>>> have run many of the mpl examples (including the new animation
>>> examples) in both qt4/wx in both the terminal based IPython and the
>>> new IPython Qt GUI.  For background on these changes please see this
>>> thread:
>>>
>>> http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel
>>>
>>> It is important to note that we have not updated the other matplotlib
>>> backends (gtk, tk, etc.) to have this logic.  This is mainly because
>>> we know almost nothing about these toolkits and could really use some
>>> help from folks who are experts at the respective toolkits.  We have
>>> done some minimal testing and these other backends do work for simple
>>> examples in the terminal IPython, but they won't work in all cases and
>>> will definitely not work in the new IPython Qt based GUI.
>>>
>>> We would love feedback and help testing as these changes are
>>> significant (even though only a few lines of code).  To test this
>>> stuff you will need to grab the following IPython development branch:
>>>
>>> http://github.com/ipython/ipython/tree/newkernel
>>>
>>> You should be able to run the examples in regular ipython:
>>>
>>> ipython --pylab qt4|wx
>>>
>>> Or the new GUI
>>>
>>> ipythonqt --pylab qt4|wx
>>
>> Brian, Fernando,
>>
>> I have been doing a little testing with ipython 0.10 versus
>> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
>> There are so many possible modes of operation and combinations of
>> versions and backends that all this will take some time to sort out.
>>
>> Can you give me simple examples of what does *not* work correctly when
>> you use mpl *svn* with ipython-newkernel, in either or both of the
>> console or gui modes, but *does* work with your guisupport version?
>
> The problems are when matplotlib and enthought.traits/pyface/mayavi
> stuff are used together.  There are two types of problems:
>
> * Multiple apps are created.  The enthought stuff used to not check
> for existing apps, but that has been fixed.

That one is easy.

> * Event loop is started multiple times.  This one is more subtle and
> on some toolkits the error is not fatal.  This problem shows up when
> IPython is run in terminal mode and event loop integration is handled
> through PyOS_InputHook.  In these situations, if matplotlib or ets
> start the event loop themselves, IPython will hang.

So, this can be checked with nothing but IPython and mpl. I think I may 
have seen this with some combination of configurations, though not with 
what I typically use.

>
> Unfortunately, I am having trouble getting an install of both
> matplotlib svn and ets svn on the same machine, so I can't reproduce
> any of the failures ATM.  I am trying to get things installed so I can
> reproduce the problems.

At least twice in the last couple of years I have tried to get mayavi 
compiled and running, without fouling up my development versions of 
numpy and mpl.  I never succeeded.  Granted, I didn't allocate much time 
and mental energy to it.

In any case, with the help of your recent explanations, I expect we can 
make mpl play by your suggested rules without sacrificing anything. 
Part of the change for mpl 1.0 was to factor all of the show logic out 
into backend_bases.py, leaving only the core mainloop call in the 
specific backends.  I hope we can keep that aspect, retaining the 
ability to work with earlier ipython (0.10) and the ability for show to 
block or not depending on the interactivity state.

Eric



--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Gökhan Sever 
On Mon, Sep 13, 2010 at 6:44 PM, Fernando Perez wrote:

> Thanks, that's good to know.  But I'm mostly thinking of teaching
> situations, so it would be nice to have this in the source: it's not
> for my use but for the benefit of students who may be in a lab where
> they can't install extensions.  But I don't know if that can even be
> done in html in the first place.
>

I think there might be a couple different approaches that might be useful
for educational purposes of IPython + matplotlib usage. For instance:

1-) When one downloads a script from the matplotlib gallery via an external
script (name it load_into_ipython or open_with_ipython) the contents of that
gallery script (or any python script) can be executed locally inside an
ipython session.

2-) Matplotlib gallery might turn to an interactive environment where you
can execute the script from right within your browser and change parameters
in the same browser window. As far as I know mpl figures can now be drawn on
html canvas. This might for sure boost the number of matplotlib audience.


-- 
Gökhan
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread John Hunter 
On Mon, Sep 13, 2010 at 4:10 PM, Brian Granger  wrote:

>> One small request: is it possible/easy to add to the MPL examples a
>> little 'copy to clipboard' button or link?  Now that one can
>> copy/paste wholesale examples into an interactive session to explore
>> them, it feels annoying to have to highlight the whole text box and
>> then do Ctrl-C or menu->copy.  It would be really nice to have a
>> one-click 'copy to clipboard'...  But I have no idea if that's easy or
>> hard in HTML...
>
> +1 to this!

On a quick googling, there are some IE only Javascript examples to do
this.  Apparently you can enable them in firefox but it requires a
significant amount of about:config hackery
(http://www.febooti.com/support/website-help/website-javascript-copy-clipboard.html).

How about this as an alternative: on my box, I can drag the "source
code" link from the browser into my terminal, which by default pastes
the URL of the referenced *.py into the terminal.  If "run" supported
a -w (web) option, or automatically detected that the URL starts with
http, it could do a web run of the file.  Of course, you may want the
source code pasted in for illustrative purposes... To support this,
you could add a -u (url) option to "paste" which assumes the input is
a url, fetches it, and pastes the contents into ipython.  So you could
type "paste -u" and then drag the link into the terminal, and it would
fetch it and paste the code into an input block.

JDH

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] For review and merging: new GUi support for Qt4 and Wx

2010-09-14 Thread Brian Granger 
On Tue, Sep 14, 2010 at 12:59 AM, Eric Firing  wrote:
> On 09/13/2010 05:46 PM, Brian Granger wrote:
>>
>> On Tue, Sep 7, 2010 at 1:31 PM, Eric Firing  wrote:
>>>
>>> On 09/03/2010 12:37 PM, Brian Granger wrote:

 Hello all,

 I would like to submit the following branch on github for review and
 merging into matplotlib trunk:

 http://github.com/ellisonbg/matplotlib/commits/guisupport

 This branch implements the logic needed for the qt4 and wx backends to
 fully work with the upcoming IPython 0.11 release.  In our testing we
 have run many of the mpl examples (including the new animation
 examples) in both qt4/wx in both the terminal based IPython and the
 new IPython Qt GUI.  For background on these changes please see this
 thread:


 http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTik2SNtXMaezCc0UiMnCYg6LxwEL1eN9YASnmOua%40mail.gmail.com&forum_name=matplotlib-devel

 It is important to note that we have not updated the other matplotlib
 backends (gtk, tk, etc.) to have this logic.  This is mainly because
 we know almost nothing about these toolkits and could really use some
 help from folks who are experts at the respective toolkits.  We have
 done some minimal testing and these other backends do work for simple
 examples in the terminal IPython, but they won't work in all cases and
 will definitely not work in the new IPython Qt based GUI.

 We would love feedback and help testing as these changes are
 significant (even though only a few lines of code).  To test this
 stuff you will need to grab the following IPython development branch:

 http://github.com/ipython/ipython/tree/newkernel

 You should be able to run the examples in regular ipython:

 ipython --pylab qt4|wx

 Or the new GUI

 ipythonqt --pylab qt4|wx
>>>
>>> Brian, Fernando,
>>>
>>> I have been doing a little testing with ipython 0.10 versus
>>> ipython-newkernel, both modes, and with mpl svn versus your guisupport.
>>> There are so many possible modes of operation and combinations of
>>> versions and backends that all this will take some time to sort out.
>>>
>>> Can you give me simple examples of what does *not* work correctly when
>>> you use mpl *svn* with ipython-newkernel, in either or both of the
>>> console or gui modes, but *does* work with your guisupport version?
>>
>> The problems are when matplotlib and enthought.traits/pyface/mayavi
>> stuff are used together.  There are two types of problems:
>>
>> * Multiple apps are created.  The enthought stuff used to not check
>> for existing apps, but that has been fixed.
>
> That one is easy.
>
>> * Event loop is started multiple times.  This one is more subtle and
>> on some toolkits the error is not fatal.  This problem shows up when
>> IPython is run in terminal mode and event loop integration is handled
>> through PyOS_InputHook.  In these situations, if matplotlib or ets
>> start the event loop themselves, IPython will hang.
>
> So, this can be checked with nothing but IPython and mpl. I think I may have
> seen this with some combination of configurations, though not with what I
> typically use.
>
>>
>> Unfortunately, I am having trouble getting an install of both
>> matplotlib svn and ets svn on the same machine, so I can't reproduce
>> any of the failures ATM.  I am trying to get things installed so I can
>> reproduce the problems.
>
> At least twice in the last couple of years I have tried to get mayavi
> compiled and running, without fouling up my development versions of numpy
> and mpl.  I never succeeded.  Granted, I didn't allocate much time and
> mental energy to it.

Yes, it can be a challenge, we will see how far I get...

> In any case, with the help of your recent explanations, I expect we can make
> mpl play by your suggested rules without sacrificing anything. Part of the
> change for mpl 1.0 was to factor all of the show logic out into
> backend_bases.py, leaving only the core mainloop call in the specific
> backends.  I hope we can keep that aspect, retaining the ability to work
> with earlier ipython (0.10) and the ability for show to block or not
> depending on the interactivity state.

The refactored show logic does help a lot.  I am pretty sure that my
branch already does all the things you want though.  If there are
things that it is missing we can definitely add them.  But when you
get a chance to look at this, let us know and we can continue the
discussion.

Cheers,

Brian



> Eric
>
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
[email protected]
[email protected]

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/n

Re: [matplotlib-devel] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Michael Droettboom 
On 09/13/2010 04:58 PM, Fernando Perez wrote:
> Hi folks,
>
> [ sorry for the cross-post, but devs on both lists will care about this]
>
> I just went through the exercise of pasting 100 randomly chosen
> examples from the gallery into the new ipython console with inline
> graphics.  Report:
>
> - 98 worked perfectly: the figures I got were identical to those on the 
> website.
>
> - 1 had minor visual differences:
> http://matplotlib.sourceforge.net/examples/pylab_examples/quadmesh_demo.html:
> in the SVG render, the masked region
> appears black instead of transparent.
>
This is now fixed in r8699.
> - One produced an error:
> http://matplotlib.sourceforge.net/examples/axes_grid/simple_axisline4.html
>
> ...
> ...: plt.draw()
> ...: plt.show()
> ...:
> Received invalid plot data.
>
This is now fixed in r8700.
> But when I save the file and try to load it  into firefox, it seems to
> indeed be bad SVG:
>
> XML Parsing Error: mismatched tag. Expected:.
> Location: file:///home/fperez/ipython/ipython/bad.svg
> Line Number 287, Column 3:
> --^
>
> In summary: we can run pretty much any MPL example by straight
> copy/paste, and the only two glitches I see are in the SVG data
> itself.  Once the other two buglets I reported earlier get fixed up,
> this will be a very nice way to interact with MPL.
>
> One small request: is it possible/easy to add to the MPL examples a
> little 'copy to clipboard' button or link?  Now that one can
> copy/paste wholesale examples into an interactive session to explore
> them, it feels annoying to have to highlight the whole text box and
> then do Ctrl-C or menu->copy.  It would be really nice to have a
> one-click 'copy to clipboard'...  But I have no idea if that's easy or
> hard in HTML...
>
Good idea.  I'll have a look at how hard this would be to add as a 
Sphinx extension.

Cheers,
Mike
> Anyway, I think we're starting to be in pretty good shape!
>
> Cheers,
>
> f
>
> --
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing
> http://p.sf.net/sfu/novell-sfdev2dev
> ___
> Matplotlib-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>


-- 
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA


--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread David Warde-Farley 

On 2010-09-13, at 7:44 PM, Fernando Perez wrote:

> Thanks, that's good to know.  But I'm mostly thinking of teaching
> situations, so it would be nice to have this in the source: it's not
> for my use but for the benefit of students who may be in a lab where
> they can't install extensions.  But I don't know if that can even be
> done in html in the first place.

You can definitely hijack copies with JavaScript:

http://stackoverflow.com/questions/400212/how-to-copy-to-clipboard-in-javascript

Nice work IPython people! I haven't been following too closely but this looks 
exciting.

David
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
On Tue, Sep 14, 2010 at 8:59 AM, Michael Droettboom  wrote:
>
> This is now fixed in r8699.
>> - One produced an error:
>> http://matplotlib.sourceforge.net/examples/axes_grid/simple_axisline4.html
>>
>> ...
>>     ...: plt.draw()
>>     ...: plt.show()
>>     ...:
>> Received invalid plot data.
>>
> This is now fixed in r8700.

Great, many thanks for these fixes!  It means that we're probably
capable now of running just about every pylab example out of the
box... We'll keep testing and will report if we see anything weird.

>> One small request: is it possible/easy to add to the MPL examples a
>> little 'copy to clipboard' button or link?  Now that one can
>> copy/paste wholesale examples into an interactive session to explore
>> them, it feels annoying to have to highlight the whole text box and
>> then do Ctrl-C or menu->copy.  It would be really nice to have a
>> one-click 'copy to clipboard'...  But I have no idea if that's easy or
>> hard in HTML...
>>
> Good idea.  I'll have a look at how hard this would be to add as a
> Sphinx extension.

Great, if it can be done it would be wonderful (Robert indicated it
may require flash, but others provided JS pointers; I'll leave it to
you to navigate those lovely waters :)

Cheers,

f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
On Mon, Sep 13, 2010 at 7:08 PM, David Warde-Farley
 wrote:
>
> Nice work IPython people! I haven't been following too closely but this looks 
> exciting.

Thanks, David.  Hopefully by next week we'll complete our
stabilization so that we're willing to foist this on the ipython-dev
and mpl-dev denizens.  Anyone is welcome to try it out now, but we're
moving things around enough that at any given point it may or may not
work.  We'll move it into trunk once the churn stops.

Regards,

f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
Hey,

On Tue, Sep 14, 2010 at 8:21 AM, John Hunter  wrote:
>
> How about this as an alternative: on my box, I can drag the "source
> code" link from the browser into my terminal, which by default pastes
> the URL of the referenced *.py into the terminal.  If "run" supported
> a -w (web) option, or automatically detected that the URL starts with
> http, it could do a web run of the file.  Of course, you may want the
> source code pasted in for illustrative purposes... To support this,
> you could add a -u (url) option to "paste" which assumes the input is
> a url, fetches it, and pastes the contents into ipython.  So you could
> type "paste -u" and then drag the link into the terminal, and it would
> fetch it and paste the code into an input block.

I'll play with those ideas, good thoughts.  %paste may not be the best
location because paste is now a terminal-only magic, since gui clients
have 'real' paste.  What's a little trickier now is that we really
need to think all the time in terms of completely separate kernel and
client codes, that only communicate via messages.  So a magic can't
muck with code in the client, because the magic executes inside the
kernel and the client is in a separate process (and possibly in a
separate computer).  While this is a bit more constraining, it forces
us to have a clean separation between different kinds of
functionality.

We've toyed with the idea of enabling a special syntax for *purely
client side* commands, something like

:cmd

that would never be sent to the kernel, and would be something for the
client to process internally.  But we'll have to mull this a little
longer...

One very important point in all of this is that the client *may not be
written in Python*.  All we have is a messaging protocol, the client
could be a web browser or anything else.

But in any case, I'll play with ways to expose this that are as easy
as possible for the users thanks for the feedback.

Cheers,

f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Anne Archibald 
On 14 September 2010 11:08, Gökhan Sever  wrote:

> 1-) When one downloads a script from the matplotlib gallery via an external
> script (name it load_into_ipython or open_with_ipython) the contents of that
> gallery script (or any python script) can be executed locally inside an
> ipython session.

Not to be difficult, but I should point out that allowing users to run
code with one click, particularly if that code is from a wiki or other
user-submitted gallery, is just asking for trouble. How long before
someone submits "import os, shutil;
shutil.deltree(os.environ['HOME'])"? Or sneaks it into some otherwise
inoffensive script?

> 2-) Matplotlib gallery might turn to an interactive environment where you
> can execute the script from right within your browser and change parameters
> in the same browser window. As far as I know mpl figures can now be drawn on
> html canvas. This might for sure boost the number of matplotlib audience.

Is there a sandboxed browser plugin? Or server plugin, depending on
where you run the script?

Anne

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
On Tue, Sep 14, 2010 at 11:48 AM, Anne Archibald
 wrote:
> On 14 September 2010 11:08, Gökhan Sever  wrote:
>
>> 1-) When one downloads a script from the matplotlib gallery via an external
>> script (name it load_into_ipython or open_with_ipython) the contents of that
>> gallery script (or any python script) can be executed locally inside an
>> ipython session.
>
> Not to be difficult, but I should point out that allowing users to run
> code with one click, particularly if that code is from a wiki or other
> user-submitted gallery, is just asking for trouble. How long before
> someone submits "import os, shutil;
> shutil.deltree(os.environ['HOME'])"? Or sneaks it into some otherwise
> inoffensive script?

Very valid points.  I'm leaning more towards something like a
combination of (hopefully) a 'copy code' button on the MPL webpages
themselves, so users don't have to scroll/highlight a lot but would
still do paste, execute manually, and a special %mplexample magic.

This would only run examples from the mpl gallery (hardcoding the
path), would display the code to the  user first, and would ask for
confirmation before execution.  Since those html pages are built by
executing those same scripts, there's a layer of sanity already built
into it (the rmtree call would have already nuked the builder's home
directory in the build process if it had been there).  Showing the
code to the user and confirming execution before proceeding adds a
final chance for the person to check her parachute before  jumping off
the cliff.

Does that sound reasonable?

>> 2-) Matplotlib gallery might turn to an interactive environment where you
>> can execute the script from right within your browser and change parameters
>> in the same browser window. As far as I know mpl figures can now be drawn on
>> html canvas. This might for sure boost the number of matplotlib audience.
>
> Is there a sandboxed browser plugin? Or server plugin, depending on
> where you run the script?

This would have to be server-side, and code needs to be written.  Part
of our interest with this explicit separation of ipython kernel and
clients with a well-defined protocol is to make the above possible.
But we haven't written any of the code necessary to have a browser
client, and to serve code read from a sphinx-generated HTML page.
Gokhan, your patches will be welcome, the infrastructure is now ready
and waiting for you :)

Cheers,

f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Gökhan Sever 
On Tue, Sep 14, 2010 at 1:48 PM, Anne Archibald
wrote:

> On 14 September 2010 11:08, Gökhan Sever  wrote:
>
> > 1-) When one downloads a script from the matplotlib gallery via an
> external
> > script (name it load_into_ipython or open_with_ipython) the contents of
> that
> > gallery script (or any python script) can be executed locally inside an
> > ipython session.
>
> Not to be difficult, but I should point out that allowing users to run
> code with one click, particularly if that code is from a wiki or other
> user-submitted gallery, is just asking for trouble. How long before
> someone submits "import os, shutil;
> shutil.deltree(os.environ['HOME'])"? Or sneaks it into some otherwise
> inoffensive script?
>

I was thinking naively --with Python for Science in mind not Python for
Hacking. I accept this is a not good idea considering that harmful effect
but in this brave new world of us there is always danger involved when one
puts their hands on virtual lands --whether the code is executed
intentionally or sneaked by a conic head.


>
> > 2-) Matplotlib gallery might turn to an interactive environment where you
> > can execute the script from right within your browser and change
> parameters
> > in the same browser window. As far as I know mpl figures can now be drawn
> on
> > html canvas. This might for sure boost the number of matplotlib audience.
>
> Is there a sandboxed browser plugin? Or server plugin, depending on
> where you run the script?
>

This one is one of my aloud speculations. Only at ideas level. I will write
more, answering Fernando's reply.


>
> Anne
>



-- 
Gökhan
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Gökhan Sever 
On Tue, Sep 14, 2010 at 1:58 PM, Fernando Perez wrote:

> >> 2-) Matplotlib gallery might turn to an interactive environment where
> you
> >> can execute the script from right within your browser and change
> parameters
> >> in the same browser window. As far as I know mpl figures can now be
> drawn on
> >> html canvas. This might for sure boost the number of matplotlib
> audience.
> >
> > Is there a sandboxed browser plugin? Or server plugin, depending on
> > where you run the script?
>
> This would have to be server-side, and code needs to be written.  Part
> of our interest with this explicit separation of ipython kernel and
> clients with a well-defined protocol is to make the above possible.
> But we haven't written any of the code necessary to have a browser
> client, and to serve code read from a sphinx-generated HTML page.
> Gokhan, your patches will be welcome, the infrastructure is now ready
> and waiting for you :)


Sage provides some level of interaction actually without any deployment made
on local side. Try for instance the following example on sagenb.org

from scipy import stats
import numpy as np
import matplotlib.pyplot as plt

@interact
def plot_gamma(a=(1,(1,10)), loc=(0,(0,10)), scale=(1,(1,10))):
rv = stats.gamma(a, loc, scale)
x = np.linspace(-1,20,1000)
plt.plot(x,rv.pdf(x))
plt.grid(True)
plt.savefig('plt.png')
plt.clf()

This one is very useful for educational and demonstrative purposes. Still
requires a bit Sage syntax manipulations to make things fully interacting on
browser.

Nice that you have matured IPython infra for implementing
such interactive functionality. I was thinking perhaps running something on
top GApss Engine but not sure they allow compiling/running C/C++ extensions
on their servers. Alternatively, like in Sage servers virtual OS'es might be
the way to go with it then possibly there will be user registration and
management issues (not sure about all specifics).

Probably, Ondrej might like experiencing with this idea :) Since he has
similar initiatives and asking help on similar topics. I am trying to
graduate myself working to solve some of my research problems and struggling
with writing especially to move on PhD. This might be a very fun Summer job
if I am wandering around jobless then.

-- 
Gökhan
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
On Tue, Sep 14, 2010 at 12:38 PM, Gökhan Sever  wrote:
>
> Sage provides some level of interaction actually without any deployment made
> on local side. Try for instance the following example on sagenb.org
> from scipy import stats
> import numpy as np
> import matplotlib.pyplot as plt
> @interact
> def plot_gamma(a=(1,(1,10)), loc=(0,(0,10)), scale=(1,(1,10))):
>     rv = stats.gamma(a, loc, scale)
>     x = np.linspace(-1,20,1000)
>     plt.plot(x,rv.pdf(x))
>     plt.grid(True)
>     plt.savefig('plt.png')
>     plt.clf()
> This one is very useful for educational and demonstrative purposes. Still
> requires a bit Sage syntax manipulations to make things fully interacting on
> browser.
> Nice that you have matured IPython infra for implementing
> such interactive functionality. I was thinking perhaps running something on
> top GApss Engine but not sure they allow compiling/running C/C++ extensions
> on their servers. Alternatively, like in Sage servers virtual OS'es might be
> the way to go with it then possibly there will be user registration and
> management issues (not sure about all specifics).

Actually sage does have one *implicit* but very particular 'local
deployment': its notebook execution model is *strictly* tied to the
idea that the client is a web browser.  Try this code in the sage
terminal (i.e. their customized ipython, not the notebook):

sage: @interact
: def x(a=(1, (1, 10))):
: print a
:

and you'll see:

 a 


 ... lots more...






 
 
sage:


So you can see, @interact in sage does basically:

- analyze the inputs of the function
- do some basic 'type inference' and emit javascript/html controls for
each parameter.
- emit an html section that wires the above controls to repeated calls
of the decorated function as the controls are operated.

This is very cool, and it enables great functionality, but it's
hard-coded to an html/javascript client.

What we're doing is a little different, as we've built a *protocol*
that clients can use to talk to the kernel, regardless of how they are
implemented.

As the functionality matures, we'll see who contributes a
browser-based client (that will require wrapping the kernel in an http
server, obviously).  And then the question of things like @interact
will be an interesting one to think about.  @interact by nature is
creating a user interface (Mathematica's Manipulate creates Notebook
controls, sage's @interact creates HTML ones).  I'm not sure yet how
we'll approach that: having per-client implementations? A traits-style
approach where each client renders it differently?  No idea yet.

Cheers,

f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Benjamin Root 
On Tue, Sep 14, 2010 at 1:58 PM, Fernando Perez wrote:

> On Tue, Sep 14, 2010 at 11:48 AM, Anne Archibald
>  wrote:
> > On 14 September 2010 11:08, Gökhan Sever  wrote:
> >
> >> 1-) When one downloads a script from the matplotlib gallery via an
> external
> >> script (name it load_into_ipython or open_with_ipython) the contents of
> that
> >> gallery script (or any python script) can be executed locally inside an
> >> ipython session.
> >
> > Not to be difficult, but I should point out that allowing users to run
> > code with one click, particularly if that code is from a wiki or other
> > user-submitted gallery, is just asking for trouble. How long before
> > someone submits "import os, shutil;
> > shutil.deltree(os.environ['HOME'])"? Or sneaks it into some otherwise
> > inoffensive script?
>
> Very valid points.  I'm leaning more towards something like a
> combination of (hopefully) a 'copy code' button on the MPL webpages
> themselves, so users don't have to scroll/highlight a lot but would
> still do paste, execute manually, and a special %mplexample magic.
>
> This would only run examples from the mpl gallery (hardcoding the
> path), would display the code to the  user first, and would ask for
> confirmation before execution.  Since those html pages are built by
> executing those same scripts, there's a layer of sanity already built
> into it (the rmtree call would have already nuked the builder's home
> directory in the build process if it had been there).  Showing the
> code to the user and confirming execution before proceeding adds a
> final chance for the person to check her parachute before  jumping off
> the cliff.
>
> Does that sound reasonable?
>
> >> 2-) Matplotlib gallery might turn to an interactive environment where
> you
> >> can execute the script from right within your browser and change
> parameters
> >> in the same browser window. As far as I know mpl figures can now be
> drawn on
> >> html canvas. This might for sure boost the number of matplotlib
> audience.
> >
> > Is there a sandboxed browser plugin? Or server plugin, depending on
> > where you run the script?
>
> This would have to be server-side, and code needs to be written.  Part
> of our interest with this explicit separation of ipython kernel and
> clients with a well-defined protocol is to make the above possible.
> But we haven't written any of the code necessary to have a browser
> client, and to serve code read from a sphinx-generated HTML page.
> Gokhan, your patches will be welcome, the infrastructure is now ready
> and waiting for you :)
>
> Cheers,
>
> f
>

Just a crazy idea to consider that would completely bypass this whole
vulnerability issue...

Why not have an examples module that contains function calls to each
example?  On the website, we can show the source code, but also say that one
could just do:

>>> import matplotlib.examples as ex
>>> ex.bars3d_demo()

My 2 cents...

Ben Root
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
On Tue, Sep 14, 2010 at 12:57 PM, Benjamin Root  wrote:
>
> Why not have an examples module that contains function calls to each
> example?  On the website, we can show the source code, but also say that one
> could just do:
>
 import matplotlib.examples as ex
 ex.bars3d_demo()

The idea is to have the *actual code* pasted in your terminal, because
now we can easily edit complex multi-line examples directly in
ipython.  So it's not just a matter of seeing the figure results, but
mostly of having the actual source in your input buffer to play with.

Cheers,

f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Benjamin Root 
On Wed, Sep 15, 2010 at 12:39 AM, Fernando Perez wrote:

> On Tue, Sep 14, 2010 at 12:57 PM, Benjamin Root  wrote:
> >
> > Why not have an examples module that contains function calls to each
> > example?  On the website, we can show the source code, but also say that
> one
> > could just do:
> >
>  import matplotlib.examples as ex
>  ex.bars3d_demo()
>
> The idea is to have the *actual code* pasted in your terminal, because
> now we can easily edit complex multi-line examples directly in
> ipython.  So it's not just a matter of seeing the figure results, but
> mostly of having the actual source in your input buffer to play with.
>
> Cheers,
>
> f
>

True... but, consider this.  ipython can already display the code for a
particular module/function using the '??' idiom.  Why not have some way to
take that text and bring it into the input buffer?

I can imagine this being useful beyond matplotlib where anybody could have
their example codes easily accessed and edited.

Ben Root
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
On Tue, Sep 14, 2010 at 6:01 PM, Benjamin Root  wrote:
> True... but, consider this.  ipython can already display the code for a
> particular module/function using the '??' idiom.  Why not have some way to
> take that text and bring it into the input buffer?

Yes, but that's a separate issue.  The approach you propose would
likely have in ex.demo_somehting() a stub to retrieve the actual
example code as a string from a file elsewhere, because (at least
right now) the mpl examples are written as 100% standalone files, not
as functions inside of some other control module.  What you are saying
does apply to the mayavi.mlab.test_*() functions, that do serve as
examples precisely in that manner, since those *do* contain their code
inside the functions.

So for the matplotlib examples, that live in standalone files, we'd
still need something different.

> I can imagine this being useful beyond matplotlib where anybody could have
> their example codes easily accessed and edited.

Certainly!  Right now the pager is a very simple tool, but I hope that
once we put this code out we'll get contributions from enterprising Qt
coders who may improve it and add things like a button that would copy
the code from the source part of an info pane and paste it in the
interactive area, all with a single click.

We want to settle the core protocol/messaging behavior first, and once
this is ready and people test it a little, I really hope we'll get
contributions that enhance the user experience very much in this
manner.

Cheers,

f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Benjamin Root 
On Wed, Sep 15, 2010 at 1:08 AM, Fernando Perez wrote:

> On Tue, Sep 14, 2010 at 6:01 PM, Benjamin Root  wrote:
> > True... but, consider this.  ipython can already display the code for a
> > particular module/function using the '??' idiom.  Why not have some way
> to
> > take that text and bring it into the input buffer?
>
> Yes, but that's a separate issue.  The approach you propose would
> likely have in ex.demo_somehting() a stub to retrieve the actual
> example code as a string from a file elsewhere, because (at least
> right now) the mpl examples are written as 100% standalone files, not
> as functions inside of some other control module.  What you are saying
> does apply to the mayavi.mlab.test_*() functions, that do serve as
> examples precisely in that manner, since those *do* contain their code
> inside the functions.
>
> So for the matplotlib examples, that live in standalone files, we'd
> still need something different.
>
>
Well, my idea was predicated upon what I said previously that matplotlib
should package the examples into a useful module with function call.  This
way, one could have been able to run the demos and view the code just like
any other piece of code in matplotlib.


> > I can imagine this being useful beyond matplotlib where anybody could
> have
> > their example codes easily accessed and edited.
>
> Certainly!  Right now the pager is a very simple tool, but I hope that
> once we put this code out we'll get contributions from enterprising Qt
> coders who may improve it and add things like a button that would copy
> the code from the source part of an info pane and paste it in the
> interactive area, all with a single click.
>
> We want to settle the core protocol/messaging behavior first, and once
> this is ready and people test it a little, I really hope we'll get
> contributions that enhance the user experience very much in this
> manner.
>
>
That is a good gameplan.  What I have seen of ipython is very good and
really resolves a lot of gripes I have had with my typical python
development workflow.  Keep up the good work!

Ben Root
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Benjamin Root 
On Wed, Sep 15, 2010 at 1:10 AM, Robert Kern  wrote:

> On 9/14/10 8:01 PM, Benjamin Root wrote:
> > On Wed, Sep 15, 2010 at 12:39 AM, Fernando Perez  > @gmail.com > wrote:
> >
> > On Tue, Sep 14, 2010 at 12:57 PM, Benjamin Root  > > wrote:
> >  >
> >  > Why not have an examples module that contains function calls to
> each
> >  > example?  On the website, we can show the source code, but also
> say that one
> >  > could just do:
> >  >
> >   import matplotlib.examples as ex
> >   ex.bars3d_demo()
> >
> > The idea is to have the *actual code* pasted in your terminal,
> because
> > now we can easily edit complex multi-line examples directly in
> > ipython.  So it's not just a matter of seeing the figure results, but
> > mostly of having the actual source in your input buffer to play with.
> >
> > Cheers,
> >
> > f
> >
> >
> > True... but, consider this.  ipython can already display the code for a
> > particular module/function using the '??' idiom.  Why not have some way
> to take
> > that text and bring it into the input buffer?
>
> Regardless of whether or not this is a good idea, it doesn't replace the
> functionality Fernando is requesting. People *will* be looking at the
> matplotlib
> docs on the web and copy-pasting the examples. Placing a link that they can
> click to easily, robustly, and *obviously* copy the code for pasting into
> the
> shell (or anywhere else!) is better than telling them to go type some magic
> commands they've probably never seen before. Magic commands that they will
> probably want to copy-paste. And so the cycle is complete.
>
>
Good point.  I guess I am just a little *too* terminal-oriented.  I tend to
access the examples directly and have to copy-and-paste snippets of code,
which has many issues with indentation and empty lines.

Just the new input buffer of ipython is a significant enough feature to
stand on its own and be beneficial.  How we get text to it can be done in
many different ways.

Ben Root
--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel

Re: [matplotlib-devel] [IPython-dev] IPython (new) + matplotlib report: happy news

2010-09-14 Thread Fernando Perez 
On Tue, Sep 14, 2010 at 6:29 PM, Benjamin Root  wrote:
> Good point.  I guess I am just a little *too* terminal-oriented.

It's probably worth mentioning that we've gone to great lengths to try
to produce in the new console an experience that's as seamless and
fluid as possible to anyone who 'lives in a terminal' (like myself).
We want this to be 'a terminal, but better': multiline editing, inline
graphics, html documentation, popups with call tips, but all the
keyboard friendliness and raw efficiency of a terminal.  Put another
way: this should be 100% usable *without* a mouse, and you should be
more efficient with this in python than with any terminal.  If you're
not, it's a bug :)

Cheers,


f

--
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
___
Matplotlib-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel