[matplotlib-devel] tick marks and render speed
About a week ago I sent a message to the users mailing list with tick mark performance problems. I now have a proof of concept patch which (mis-)uses the projection keyword in add_subplot to use a custom Axes class. Import one python file, use "projection='fastticks'" -> boring scatter plots render about 2x as fast and plots with lots of minor ticks render 6x faster. The code is at https://github.com/jbmohler/mplfastaxes ; the core idea is in fastaxes.py which uses a single Line2D for all tick marks of a given flavor rather than a Line2D for every single tick mark. There are two reasons this isn't a total win: 1) it's not done yet in all tick/grid configurations. 2) it loses flexibility in tick labeling The lost flexibility may matter a great deal to other people. In my use-case, the labeling flexibility simply seems misguided and the performance is much much preferred. Comments welcome about how this could move forward toward being included in MPL. Joel -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft ___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] tick marks and render speed
This is very interesting. I have also noticed that ticking has been a bottleneck in rendering, but always suspected the computation of the ticks rather than the actual rendering of the marks. Perhaps this might spur some renewed interest in identifying the specific reason for the bottleneck and solving that issue now that we know that there are significant performance gain potential here? Looking forward to reviewing this at some point in the next few weeks. Cheers! Ben Root On Thu, Jun 26, 2014 at 2:10 PM, Joel B. Mohler wrote: > About a week ago I sent a message to the users mailing list with tick mark > performance problems. I now have a proof of concept patch which > (mis-)uses the > projection keyword in add_subplot to use a custom Axes class. Import one > python file, use "projection='fastticks'" -> boring scatter plots render > about > 2x as fast and plots with lots of minor ticks render 6x faster. > > The code is at https://github.com/jbmohler/mplfastaxes ; the core idea is > in > fastaxes.py which uses a single Line2D for all tick marks of a given flavor > rather than a Line2D for every single tick mark. > > There are two reasons this isn't a total win: > 1) it's not done yet in all tick/grid configurations. > 2) it loses flexibility in tick labeling > > The lost flexibility may matter a great deal to other people. In my > use-case, > the labeling flexibility simply seems misguided and the performance is much > much preferred. > > Comments welcome about how this could move forward toward being included in > MPL. > > Joel > > > -- > Open source business process management suite built on Java and Eclipse > Turn processes into business applications with Bonita BPM Community Edition > Quickly connect people, data, and systems into organized workflows > Winner of BOSSIE, CODIE, OW2 and Gartner awards > http://p.sf.net/sfu/Bonitasoft > ___ > Matplotlib-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] nosetests: too slow, too much memory
Just noticed an oddity with the tests on Travis versus the tests on my machine. The test log on Travis for a single run has over 10,000 lines. But, for me, it is over ~4800 lines. At a glance, I can see that test_mlab is not executed for me, but they are for Travis. I am very suspicious of the test_mlab run on Travis because it seems to be running multiple times, but I can't be sure. Michael, can I get the test log for one of the recent Travis runs? Thanks, Ben Root On Wed, Jun 4, 2014 at 1:49 PM, Benjamin Root wrote: > So, I just tried comparing memory usage for a plot displayed via show() > versus savefig() as a PNG. It would seem that saving to pngs uses more > memory. Not sure why, though. > > Ben > On Jun 4, 2014 12:57 PM, "Eric Firing" wrote: > >> On 2014/06/04 6:26 AM, Benjamin Root wrote: >> >>> A theory... >>> >>> If I remember correctly, the nosttests was set up to execute in parallel >>> using the default Multiprocessing settings, which is to have a process >>> worker for each available CPU core. Perhaps this might be the crux of >>> the issue with so many simultaneous tests running that the amount of >>> memory used at the same time becomes too large. Or, am I thinking of the >>> doc build system? >>> >>> Ben Root >>> >> >> Ben, >> >> Top shows a single process. The VM is configured with 2 cores. >> >> Eric >> > -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
Re: [matplotlib-devel] nosetests: too slow, too much memory
False alarm. The Travis logs includes (but hides) the install output, and test_mlab was running for me, but I was looking at the wrong line numbers. Still though, I would be curious as to any differences, but I can't seem to download the log file. Ben On Thu, Jun 26, 2014 at 6:57 PM, Benjamin Root wrote: > Just noticed an oddity with the tests on Travis versus the tests on my > machine. The test log on Travis for a single run has over 10,000 lines. > But, for me, it is over ~4800 lines. At a glance, I can see that test_mlab > is not executed for me, but they are for Travis. I am very suspicious of > the test_mlab run on Travis because it seems to be running multiple times, > but I can't be sure. > > Michael, can I get the test log for one of the recent Travis runs? > > Thanks, > Ben Root > > > > On Wed, Jun 4, 2014 at 1:49 PM, Benjamin Root wrote: > >> So, I just tried comparing memory usage for a plot displayed via show() >> versus savefig() as a PNG. It would seem that saving to pngs uses more >> memory. Not sure why, though. >> >> Ben >> On Jun 4, 2014 12:57 PM, "Eric Firing" wrote: >> >>> On 2014/06/04 6:26 AM, Benjamin Root wrote: >>> A theory... If I remember correctly, the nosttests was set up to execute in parallel using the default Multiprocessing settings, which is to have a process worker for each available CPU core. Perhaps this might be the crux of the issue with so many simultaneous tests running that the amount of memory used at the same time becomes too large. Or, am I thinking of the doc build system? Ben Root >>> >>> Ben, >>> >>> Top shows a single process. The VM is configured with 2 cores. >>> >>> Eric >>> >> > -- Open source business process management suite built on Java and Eclipse Turn processes into business applications with Bonita BPM Community Edition Quickly connect people, data, and systems into organized workflows Winner of BOSSIE, CODIE, OW2 and Gartner awards http://p.sf.net/sfu/Bonitasoft___ Matplotlib-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
