Nicolas,
I'm not sure how you've reached your conclusion.
The 21,000 calls to Line2D.draw (i.e. 21 per frame) are easily explained
since each grid line (or tick) is in fact a line.
The 10,000 calls to Tick.draw (i.e. 10 per frame) are because there are
10 tick labels.
likewise for Text.draw etc. and on down the list.
I don't see how any of these objects could be drawn without calling
draw. ;) There actually already is caching that occurs rendering text,
for example. None of these calls are strictly "useless".
If animation speed is important, there are tricks in the
examples/animation directory that show how to reuse the ticks from a
previous draw and only update the data (i.e. eliminate many of these
"useless" draw calls). But they are tricks -- they don't work for the
general case of "anything in this plot may change at any time".
But I think you are comparing apples to oranges in your speed
comparison. You say a "do-nothing" loop in PyOpenGL runs at around 2000
fps, and a "do-nothing" loop in matplotlib runs about 100fps. But, of
course, the matplotlib test case is actually doing a great deal even
with the backend doing nothing -- it is doing all of the work of laying
out the plot, which is the majority of time spent getting a plot to the
screen. And that all work happens in Python -- its speed is what it is
and is acceptable in many contexts -- but no backend work is going to
improve what's going on in the core.
I should warn you that previous attempts to speed up matplotlib using
hardware acceleration have failed to produce much fruit because the
backends actually do fairly little work by design in matplotlib. The
actual act of rendering paths into pixels on screen (i.e. what happens
in the backend) is a small fraction of the run time, even when using a
software renderer (eg. Agg). Here's a useful benchmark that renders a
plot a bunch of times to memory:
import sys
import matplotlib
matplotlib.use(sys.argv[-1])
from pylab import *
import numpy
plot(numpy.arange(int(sys.argv[-2])))
for i in xrange(1000):
draw()
and the results (on my 2.33GHz Intel E6550):
> time python test_backend_speed.py 3 agg
real 0m15.211s
user 0m15.009s
sys 0m0.136s
# Here "backend_pyglet" is the do-nothing backend you sent to the list
in a previous e-mail
> time python test_backend_speed.py 3 module://backend_pyglet
real 0m14.038s
user 0m13.713s
sys 0m0.256s
> time python test_backend_speed.py 100 agg
real 0m23.038s
user 0m22.845s
sys 0m0.132s
> time python test_backend_speed.py 100 module://backend_pyglet
real 0m15.251s
user 0m14.837s
sys 0m0.304s
So you see the actual work in the backend can be a fairly small fraction
of the total runtime -- that gives one an idea of the upper bound on the
speed improvement that any backend could make without digging into the
matplotlib core and making improvements there.
In fairness, my test is not measuring the time to (once rendering the
plot) blit it to the screen. I suspect OpenGL will have an advantage
there. It may even be possible as a mid-way solution to create an
Agg/OpenGL backend that used Agg for rendering and OpenGL -- that's
something that would be really useful just to have another nice
cross-platform GUI backend.
The other important thing to note about this benchmark is that as the
size of the data increases, the proportion of time spent in the backend
increases.
I'm also worried (and I have no numbers to back this up) that a pyglet
or PyOpenGL backend may actually be slower if the work to convert paths
from matplotlib's path.Path format to the format understood by pyglet
and/or PyOpenGL happens in Python, as your preliminary code
backend_pyglet.py does in draw_path (i.e. looping over each vertex in a
Python loop.) In the Agg backend, that happens in C++ on-the-fly
without copying the data -- see src/path_converters.h. This code is
exposed to Python through matplotlib._path.cleanup_path, but that does
require copying memory, which for large data sets may be a limiting
factor. So you may end up needing to write the backend in C++ to really
beat the current Agg backend, but I'd love to be proven wrong.
I hope this helps to better illustrate what you're seeing, and I don't
mean to discourage you in implementing an OpenGL-based backend (which
would be very nice to have for portability reasons among others). But I
hope this also illustrates that if the end goal is simply to "go
faster", this may be somewhat like putting racing tires on a car without
replacing the engine.
Cheers,
Mike
On 07/29/2011 03:09 AM, Nicolas Rougier wrote:
I just did it using the regular python profiler (and a pyglet backend
because glut cannot be easily profiled).
Here are some results for exactly 1000 frames displayed:
> python -m cProfile -s cumulative test_backend_pyglet.py
7723453 function calls (7596399 primitive calls) in 16.583
seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.004 0.004 16.587 16.587
test_backend_pyglet.py:1(<module>)
1 0.000 0.000 14.000 14.000 pyplot.py:123(show)
1 0.000 0.000 14.000 14.000 backend_pyglet.py:214(show)
1 0.000 0.000 13.951 13.951 __init__.py:115(run)
1 0.000 0.000 13.951 13.951 base.py:117(run)
1 0.009 0.009 13.949 13.949 base.py:149(_run_estimated)
1000 0.018 0.000 13.639 0.014 base.py:244(idle)
1119/1116 0.009 0.000 13.378 0.012 event.py:318(dispatch_event)
1004 0.006 0.000 13.340 0.013
__init__.py:1148(dispatch_event)
1000 0.002 0.000 13.326 0.013
backend_pyglet.py:319(on_draw)
1000 0.013 0.000 13.324 0.013 backend_pyglet.py:324(draw)
1000 0.004 0.000 13.001 0.013
backend_pyglet.py:342(_render_figure)
60000/1000 0.204 0.000 12.977 0.013 artist.py:53(draw_wrapper)
1000 0.023 0.000 12.970 0.013 figure.py:805(draw)
1000 0.046 0.000 12.775 0.013 axes.py:1866(draw)
2000 0.046 0.000 11.791 0.006 axis.py:1029(draw)
10000 0.098 0.000 8.521 0.001 axis.py:219(draw)
21000 0.710 0.000 6.467 0.000 lines.py:463(draw)
20000 0.030 0.000 2.301 0.000
transforms.py:2234(get_transformed_points_and_affine)
21000 0.093 0.000 2.245 0.000
transforms.py:2224(_revalidate)
1 0.001 0.001 2.081 2.081 pylab.py:1(<module>)
1 0.001 0.001 2.080 2.080 pylab.py:215(<module>)
12/6 0.007 0.001 2.036 0.339 {__import__}
1 0.001 0.001 1.937 1.937 pyplot.py:15(<module>)
1 0.000 0.000 1.935 1.935 __init__.py:14(pylab_setup)
1 0.001 0.001 1.931 1.931
backend_pyglet.py:55(<module>)
1 0.011 0.011 1.929 1.929 __init__.py:94(<module>)
68000/64000 0.450 0.000 1.824 0.000
transforms.py:1732(transform)
15000 0.381 0.000 1.726 0.000 text.py:514(draw)
2000 0.035 0.000 1.711 0.001
axis.py:977(_get_tick_bboxes)
10000 0.091 0.000 1.668 0.000
text.py:713(get_window_extent)
64605 0.694 0.000 1.520 0.000 path.py:83(__init__)
24000 0.067 0.000 1.491 0.000
transforms.py:1155(transform_path_non_affine)
20041 0.435 0.000 1.430 0.000 lines.py:386(recache)
44000 0.118 0.000 1.385 0.000
transforms.py:1761(transform_non_affine)
164452 1.340 0.000 1.340 0.000
{numpy.core.multiarray.array}
20000 0.076 0.000 1.137 0.000
transforms.py:1119(transform_point)
It does not seem to have superfluous call to the various methods (even
if the plot is a simple 3 points line, there is a lot to draw) and
maybe this means an efficient OpenGL backend would require some cache
system to avoid repeating "useless" operations.
Nicolas
On Jul 28, 2011, at 3:15 PM, Michael Droettboom wrote:
Have you tried running it in the Python profiler? I find this script
[1] in combination with kcachegrind to be very useful in answering
these kinds of questions.
[1] http://codespeak.net/pypy/dist/pypy/tool/lsprofcalltree.py
Mike
On 07/28/2011 07:16 AM, Nicolas Rougier wrote:
I've created a fork at:
https://github.com/rougier/matplotlib/tree/gl-backend
The name of the backend is glut (it requires OpenGL) and does not
display anything, it only measures fps.
It seems to be stuck at 100fps with the following test script:
import matplotlib
matplotlib.use('glut')
from pylab import *
plot([1,2,3])
show()
while the same do-nothing window directly in pyOpenGL is around
2000fps on the same machine.
I would like to understand why this is so slow and if it can be fixed.
Nicolas
On Jul 27, 2011, at 3:28 PM, Benjamin Root wrote:
On Wednesday, July 27, 2011, Nicolas Rougier
<nicolas.roug...@inria.fr <mailto:nicolas.roug...@inria.fr>> wrote:
>
>
> Hi all,
>
> I've been testing various idea around the idea of a GL backend,
and I would have a few questions.
> First, I tried to use the backend template to quickly test an
empty pyglet backend and I've been quite surprised by the bad
performances. Without drawing anything, I can hardly reach 100FPS
and I wonder if I did something wrong ? (The backend is
available backend_pyglet.py
<http://www.loria.fr/~rougier/tmp/backend_pyglet.py
<http://www.loria.fr/%7Erougier/tmp/backend_pyglet.py>> and the
test file is at test_backend_pyglet.py
<http://www.loria.fr/~rougier/tmp/test_backend_pyglet.py
<http://www.loria.fr/%7Erougier/tmp/test_backend_pyglet.py>>)
>
> Second, I've been experimenting with proper anti-alias technics
(using shaders) and the results are not so bad so far (IMHO) :
> Antialiased line with thickness varying by 0.1 pixels:
> http://www.loria.fr/~rougier/tmp/aa-line.png
<http://www.loria.fr/%7Erougier/tmp/aa-line.png>
> (don't paid attention to the cap, it's not done yet)
>
> Antialiased circles (small circles position is increased by 0.1
pixels)
> http://www.loria.fr/~rougier/tmp/aa-circle.png
<http://www.loria.fr/%7Erougier/tmp/aa-circle.png>
> (I can post source code if anyone is interested)
> I don't know yet if all matplotlib artists can be drawing using
these technics.
>
> My question relates to the cairo backend that now seems to
support gl and shaders. Does anyone know the status of the
gl-backend and how it would improve performances of matplotlib ? (I
had a hard time finding any information).
>
> Nicolas
Nicolas,
I want to immediately encourage you to continue on your efforts.
PLEASE make a fork on github so that we may be able to experiment
better.
Cheers!
Ben Root
------------------------------------------------------------------------------
Got Input? Slashdot Needs You.
Take our quick survey online. Come on, we don't ask for help often.
Plus, you'll get a chance to win $100 to spend on ThinkGeek.
http://p.sf.net/sfu/slashdot-survey
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
--
Michael Droettboom
Science Software Branch
Space Telescope Science Institute
Baltimore, Maryland, USA
------------------------------------------------------------------------------
Got Input? Slashdot Needs You.
Take our quick survey online. Come on, we don't ask for help often.
Plus, you'll get a chance to win $100 to spend on ThinkGeek.
http://p.sf.net/sfu/slashdot-survey_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
------------------------------------------------------------------------------
Got Input? Slashdot Needs You.
Take our quick survey online. Come on, we don't ask for help often.
Plus, you'll get a chance to win $100 to spend on ThinkGeek.
http://p.sf.net/sfu/slashdot-survey
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
------------------------------------------------------------------------------
Got Input? Slashdot Needs You.
Take our quick survey online. Come on, we don't ask for help often.
Plus, you'll get a chance to win $100 to spend on ThinkGeek.
http://p.sf.net/sfu/slashdot-survey
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
