Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
Since there don't seem to be any forthcoming answers, I have a somewhat different question. In the matplotlib FAQ, it states that using 'show()' puts you in the GUI mainloop (http://matplotlib.sourceforge.net/faq/howto_faq.html#use-show). However, using plot commands on the ipython command line does not shut down the command line generally. I gathered from some googling that this is because ipython starts up the matplotlib graphics in a different 'thread', but I don't understand how this is done and most of what I've seen says it is bad. So, my question now: How can I exit the GUI mainloop without closing the graphics windows? Thanks, Adam -- View this message in context: http://www.nabble.com/Interactive-backends-very-%28suprisingly-%29-slow-for-multiple-subplots-tp23261074p23295883.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
On Wed, Apr 29, 2009 at 9:07 AM, keflavich keflav...@gmail.com wrote: Since there don't seem to be any forthcoming answers, I have a somewhat different question. In the matplotlib FAQ, it states that using 'show()' puts you in the GUI mainloop (http://matplotlib.sourceforge.net/faq/howto_faq.html#use-show). However, using plot commands on the ipython command line does not shut down the command line generally. I gathered from some googling that this is because ipython starts up the matplotlib graphics in a different 'thread', but I don't understand how this is done and most of what I've seen says it is bad. most GUI mainloops are blocking, so after you start them you cannot issue more commands from an interactive shell. Either you need to run a GUI shell, or in the case of ipython run the GUI in a separate thread. One exception to this is tkinter (tkagg), which plays nicely with a standard python shell. I understand that recent versions of pygtk work also w/o running the mainloop in a separate thread, but I haven't dug into the details. So, my question now: How can I exit the GUI mainloop without closing the graphics windows? This question doesn't really make sense to me. Perhaps you can clearly describe your use case (what you need to do) rather than the proposed solutions. JDH -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
On Wed, Apr 29, 2009 at 8:29 AM, John Hunter jdh2...@gmail.com wrote: On Wed, Apr 29, 2009 at 9:07 AM, keflavich keflav...@gmail.com wrote: Since there don't seem to be any forthcoming answers, I have a somewhat different question. In the matplotlib FAQ, it states that using 'show()' puts you in the GUI mainloop (http://matplotlib.sourceforge.net/faq/howto_faq.html#use-show). However, using plot commands on the ipython command line does not shut down the command line generally. I gathered from some googling that this is because ipython starts up the matplotlib graphics in a different 'thread', but I don't understand how this is done and most of what I've seen says it is bad. most GUI mainloops are blocking, so after you start them you cannot issue more commands from an interactive shell. Either you need to run a GUI shell, or in the case of ipython run the GUI in a separate thread. One exception to this is tkinter (tkagg), which plays nicely with a standard python shell. I understand that recent versions of pygtk work also w/o running the mainloop in a separate thread, but I haven't dug into the details. OK, that's very helpful. So, my question now: How can I exit the GUI mainloop without closing the graphics windows? This question doesn't really make sense to me. Perhaps you can clearly describe your use case (what you need to do) rather than the proposed solutions. I would like to have access to the command line while simultaneously being able to interact with and/or display plots. I think this is what ipython does by default when you pass it a thread keyword (-pylab, -qt4thread, etc.). I had some trouble getting ipython to work correctly, but I think that had to do with passing the thread keyword before/after some other keywords. Part of my question that I hope makes sense: Is there a way to unblock the command line without closing the plot window when using an interactive backend? Thanks, and sorry about the misunderstandings / lack of clarity, Adam -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
On Wed, Apr 29, 2009 at 9:22 AM, John Hunter jdh2...@gmail.com wrote: On Wed, Apr 29, 2009 at 9:50 AM, Adam keflav...@gmail.com wrote: I would like to have access to the command line while simultaneously being able to interact with and/or display plots. I think this is what ipython does by default when you pass it a thread keyword (-pylab, -qt4thread, etc.). I had some trouble getting ipython to work correctly, but I think that had to do with passing the thread keyword before/after some other keywords. From your previous posts, I think you may have been be using ipython incorrectly, ie mixing ipython -pylab with the matplotlib use directive. Start with a canconical simple script, eg:: import matplotlib.pyplot as plt plt.figure(1) plt.plot([1,2,3]) plt.figure(2) plt.plot([4,5,6]) plt.show() and set your matplotlibrc to backend to TkAgg. Start ipython with:: ipython -pylab and run your test script with:: In [61]: run test.py If that works, close everything down and set your backend to QtAgg and try running it again in the same way and let us know what happens. It should just work. I'm suspecting in that as you were testing and trying a lot of things, you got yourself into a situation where multiple GUIs were competing for attention. Part of my question that I hope makes sense: Is there a way to unblock the command line without closing the plot window when using an interactive backend? Yes, that makes sense, and basically you need to either use TkAgg from a regular python shell, use ipython in pylab mode with any supported backend, or use a GUI shell. ipython also has support for embedding in GUI shells. See also http://matplotlib.sourceforge.net/users/shell.html JDH Thanks John. I think you answered my questions completely now. FWIW, I was not using matplotlib.use() with ipython, I was using it when calling 'python test.py' on the command line. My mistake with ipython was using an import command before -pylab, i.e.: ipython -i -c import pyfits,matplotlib -pylab which does not work, whereas ipython -pylab -i -c import pyfits,matplotlib does. Thanks for the help! -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
keflavich keflav...@gmail.com writes: I tried the same series of plot commands using the SVG, PS, and PDF backends and the whole series of 50 plots takes ~1s. Did you produce any output with savefig? 50 plots per second sounds pretty fast - at least on my computer, the matplotlib examples render much slower than that, so I suspect your test script might not have really caused the backends to render anything. Anyway, if you want to compare the execution time of the interactive backends, I suppose the best reference would be the Agg backend, which does the same rendering without any of the interactiveness. -- Jouni K. Seppänen http://www.iki.fi/jks -- Crystal Reports #45; New Free Runtime and 30 Day Trial Check out the new simplified licensign option that enables unlimited royalty#45;free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
Jouni K. Seppänen wrote: keflavich keflav...@gmail.com writes: I tried the same series of plot commands using the SVG, PS, and PDF backends and the whole series of 50 plots takes ~1s. Did you produce any output with savefig? 50 plots per second sounds pretty fast - at least on my computer, the matplotlib examples render much slower than that, so I suspect your test script might not have really caused the backends to render anything. Anyway, if you want to compare the execution time of the interactive backends, I suppose the best reference would be the Agg backend, which does the same rendering without any of the interactiveness. Yes, I'm using savefig. I switched to the 'Paint' backend so I can save as .png and use Mac's preview to view the files as they're updated; that's my cheap workaround for the moment. I tried 'agg' and it performs essentially the same as 'Paint' - pretty fast. Maybe closer to 2-3s than 1s. Adam -- View this message in context: http://www.nabble.com/Interactive-backends-very-%28suprisingly-%29-slow-for-multiple-subplots-tp23261074p23261638.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Crystal Reports #45; New Free Runtime and 30 Day Trial Check out the new simplified licensign option that enables unlimited royalty#45;free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
On Mon, Apr 27, 2009 at 12:24 PM, keflavich keflav...@gmail.com wrote: Hi, I'm trying to plot a series of ~30-50 small plots, each of which contains 3 plots of ~10-20 points (one plot is data, one plot is errorbars, one plot is a model fit). I've tried using GtkAgg and Qt4Agg as backends, and both are extremely slow - they take ~5-10 seconds for the first plot window and then hang. Sometimes they'll plot subsequent windows, but sometimes they will not. That does sound exceedingly slow -- it looks like you are having some problems with the GUI or environment and not just the mpl component. How are you running and profiling your script? Can you post some free-standing example code which exposes the problem? Can you provide some of the environment data detailed at http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#reporting-problems -- Crystal Reports #45; New Free Runtime and 30 Day Trial Check out the new simplified licensign option that enables unlimited royalty#45;free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
John Hunter-4 wrote: That does sound exceedingly slow -- it looks like you are having some problems with the GUI or environment and not just the mpl component. How are you running and profiling your script? Can you post some free-standing example code which exposes the problem? Can you provide some of the environment data detailed at http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#reporting-problems I haven't been doing any profiling; I've never been particularly comfortable with timeit. As for normal problem reporting details... Darwin eta.colorado.edu 9.6.3 Darwin Kernel Version 9.6.3: Tue Jan 20 18:26:40 PST 2009; root:xnu-1228.10.33~1/RELEASE_I386 i386 i386 matplotlib version 0.98.3 My example standalone script (note that it takes the backend as a command line argument and assumes you have not selected a backend / imported pylab): import matplotlib import sys matplotlib.use(sys.argv[1]) from pylab import * i=1; n=1; print Working on figure 0 figure(0); clf() for j in xrange(36): if (i % 10)==0: print Working on figure %i % n figure(n); clf() i=1 n+=1 subplot(3,3,i) plot([1,2,3],[3,2,1]) plot([1,2,1],[1,2,3]) plot([1,2,3],[1,2,3]) i+=1 Results: In [1]: %run -t code/test.py 'Qt4Agg' backend Qt4Agg version 0.9.1 IPython CPU timings (estimated): User : 21.553853 s. System:0.0 s. In [1]: %run -t code/test.py 'pdf' backend pdf version unknown IPython CPU timings (estimated): User : 1.056959 s. System:0.0 s. In [1]: %run -t code/test.py 'svg' backend svg version 0.98.5.2 IPython CPU timings (estimated): User : 1.056702 s. System:0.0 s. Curiously, if I add ioff() at the beginning of the script using the Qt4Agg backend, I get the much nicer result: User : 1.328496 s. and show works quite rapidly. This suggests to me that the placement of ioff() is important, so I'm going to try playing with that a little... Adam -- View this message in context: http://www.nabble.com/Interactive-backends-very-%28suprisingly-%29-slow-for-multiple-subplots-tp23261074p23263295.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Crystal Reports #45; New Free Runtime and 30 Day Trial Check out the new simplified licensign option that enables unlimited royalty#45;free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
So, as the Matplotlib help page suggests, working through a test problem helped me narrow down my problem... but it still hasn't solved it. If I set ioff() at the main level, rather than in a function I call, it works. However, when I show() the plot, the code halts until I close it, which is not helpful. I feel like this is a problem I've resolved before, but googling hasn't helped me so far - I think the closest I have come are show() hangs threads that have no replies. I guess this changes my question to: How can I draw all of my figures non-interactively, then show them and return to the python command prompt? ion() show() should work, I think, but does not. Thanks, Adam -- View this message in context: http://www.nabble.com/Interactive-backends-very-%28suprisingly-%29-slow-for-multiple-subplots-tp23261074p23263639.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Crystal Reports #45; New Free Runtime and 30 Day Trial Check out the new simplified licensign option that enables unlimited royalty#45;free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
On Mon, Apr 27, 2009 at 2:28 PM, keflavich keflav...@gmail.com wrote: John Hunter-4 wrote: That does sound exceedingly slow -- it looks like you are having some problems with the GUI or environment and not just the mpl component. How are you running and profiling your script? Can you post some free-standing example code which exposes the problem? Can you provide some of the environment data detailed at http://matplotlib.sourceforge.net/faq/troubleshooting_faq.html#reporting-problems I haven't been doing any profiling; I've never been particularly comfortable with timeit. As for normal problem reporting details... Darwin eta.colorado.edu 9.6.3 Darwin Kernel Version 9.6.3: Tue Jan 20 18:26:40 PST 2009; root:xnu-1228.10.33~1/RELEASE_I386 i386 i386 matplotlib version 0.98.3 My example standalone script (note that it takes the backend as a command line argument and assumes you have not selected a backend / imported pylab): import matplotlib import sys matplotlib.use(sys.argv[1]) from pylab import * i=1; n=1; print Working on figure 0 figure(0); clf() for j in xrange(36): if (i % 10)==0: print Working on figure %i % n figure(n); clf() i=1 n+=1 subplot(3,3,i) plot([1,2,3],[3,2,1]) plot([1,2,1],[1,2,3]) plot([1,2,3],[1,2,3]) i+=1 Results: In [1]: %run -t code/test.py 'Qt4Agg' Ahh, mixing matplotlib.use from an interactive ipython session -- that is an important detail :-) What is your backend (import matplotlib; print matplotlib.rcParams['backend']) It is quite likely that you are getting cross GUI / cross threading problems from trying to do this inside ipython, which may be using one GUI backend based on rc at pylab startup, and mpl is trying to use another one with the use directive. Basically, use is not supported for switching GUIs, it was designed to be the first line executed in a session but when you do this interactively in ipython pylab mode, you are asking for a world of pain. Try running your example from a standard unix shell rather than from ipython JDH -- Crystal Reports #45; New Free Runtime and 30 Day Trial Check out the new simplified licensign option that enables unlimited royalty#45;free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Interactive backends very (suprisingly?) slow for multiple subplots
John Hunter-4 wrote: Ahh, mixing matplotlib.use from an interactive ipython session -- that is an important detail :-) What is your backend (import matplotlib; print matplotlib.rcParams['backend']) It is quite likely that you are getting cross GUI / cross threading problems from trying to do this inside ipython, which may be using one GUI backend based on rc at pylab startup, and mpl is trying to use another one with the use directive. Basically, use is not supported for switching GUIs, it was designed to be the first line executed in a session but when you do this interactively in ipython pylab mode, you are asking for a world of pain. Try running your example from a standard unix shell rather than from ipython JDH Sorry, I should have specified: I wrote this script to use separate from my interactive sessions as a test. I use Qt4Agg now, set in my matplotlibrc, and I do not use matplotlib.use() at all (since it doesn't work anyway). The faster performance I reported was achieved by setting the backend in my matplotlibrc, not on the command line. So I don't think I'm running into the issues you're talking about. I don't really know how to run a script in which I expect plots to show up from the command line. If I do something simple like 'python test.py' it pops up windows but never shows the plots I expect to see. Adam -- View this message in context: http://www.nabble.com/Interactive-backends-very-%28suprisingly-%29-slow-for-multiple-subplots-tp23261074p23264479.html Sent from the matplotlib - users mailing list archive at Nabble.com. -- Crystal Reports #45; New Free Runtime and 30 Day Trial Check out the new simplified licensign option that enables unlimited royalty#45;free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
