Ok, so it seems from reading comments further down that bug report that FF's garbage collection does not immediately free memory for old jqplot canvases.
As I said, as soon as I find the time I will experiment with more aggressive DOM element destruction/creation and see if it improves how quickly memory is freed. I do think it will at least partly circumvent jqPlot's bug. 2010/12/22 Dima Tisnek <[email protected]>: > Firefox people point the blame squarely at jqplot: > > --- Comment #4 from Boris Zbarsky (:bz) <[email protected]> 2010-12-22 > 13:43:23 PST --- > So what I see this page do, in horizontal mode, is create 17 canvases each of > which is width="816" height="3587". That means that each of them has a > backing > store of 816*3587*4 = 11,707,968 bytes. So that's about 200MB of memory usage > right there. > > I have no idea why they feel a need for 17 huge canvases, but if they want > them, that's how much memory they'll take... > > ( https://bugzilla.mozilla.org/show_bug.cgi?id=620883 ) > > On 22 December 2010 01:24, Miquel Torres <[email protected]> wrote: >> Mmm, yes, it appears that the memory is not properly freed. I can try >> experimenting by destroying the dom element and recreating it each >> time that the plot changes... >> >> Btw., instead of continuing here "polluting" the pypy-dev mailing >> list, we can move to http://groups.google.com/group/codespeed if >> needed. >> >> Thanks for reporting this. >> >> Cheers, >> Miquel >> >> >> 2010/12/22 Dima Tisnek <[email protected]>: >>> I sort of figured it out, although I don't have a ready solution. >>> >>> This affects opera 11, stable ff, ff 4.0b7; amazingly it does not affect >>> ie8. >>> Basically memory consumption of the plot appears to be proportional to >>> plot area. >>> Normal plot that you get by default at >>> http://speed.pypy.org/comparison/ costs about 100M of browser memory >>> consumption: >>> opera 130M, stable ff 70M, beta ff 90M at window size 1680x1050; >>> opera 80M, stable ff 55M, beta ff 70M at window size 1024x600; >>> Switching to "horizontal" produces a tall plot of same width and costs >>> about 300~700M of browser memory: >>> opera 720M, stable ff 370M, beta ff 370M at window wize 1680x1050; >>> opera 350M, stable ff 370M, beta ff 370M at window size 1024x600; >>> >>> Suprisingly window size only matters while javascript produces the >>> plot, and not when window is resized, even though plot it resized with >>> the window correctly. >>> >>> This alone is pretty heavy, but doesn't grind the browser. >>> What really grinds is that every time you change a tickbox on the >>> left, a plot is redrawn and another 200M of browser memory is wasted. >>> This is not double buffering, as next change adds yet another 200M or >>> so and so on, it appears that either js doesn't free something, or >>> browser caches or saves the previous page state. >>> >>> As memory consumption grows, at some point browser hits the wall, >>> causes heavy swapping for some time, and I think garbage collection, >>> because practical (but not virtual) memory usage first drops to >>> something like 20~50M and then returns to "normal" 300M. >>> opera ~30 seconds, stable ff about a minute, beta ff several minutes >>> (total system mem 1G, cpu Atom @1.6GHz) >>> >>> Perhaps OS also plays a role in the grind, as it is clearly swapping >>> and swaps out too much? or triggers gc too late and gc has to pull the >>> pages back from disk to perform collection? >>> >>> ie8 doesn't use that much memory, as a matter of fact memory >>> consumption starts little (40M) and changes very little (only +10M) if >>> you go to horizonatal view; the price is very slow rendering, more >>> than 10 seconds per column change. >>> >>> I'll post this on firefox bugzilla too, let's see if someone has a solution. >>> >>> Meanwhile perhaps pypy speed center could start with a smaller plot >>> area (or fewer columns as that makes horizontal plot smaller) to >>> accomodate varying hardware and system mem usage that users might >>> have? >>> The simplest would be a warning next to "horizontal" checkbox. >>> >>> On 21 December 2010 01:06, Miquel Torres <[email protected]> wrote: >>>> Hi Dima, >>>> >>>>> another temp problem with speed pypy is that it's terrubly slow in ff >>>>> beta. it also occasionally grinds in stable ff and opera, but I guess >>>>> this can be forgiven for the sake of simplicity / developer effort. >>>> >>>> Well, speed.pypy is actually fast in all modern browsers. The problem >>>> you are referring to is probably caused by a bug in the javascript >>>> plotting library (jqPplot) that is triggered in the comparison view >>>> when there are some results with 0 values. It only appears for some >>>> plot types, but it is very annoying because it grinds the browser to a >>>> halt like you say. Is that what you meant? >>>> >>>> We are looking into it, and will fix that library if necessary. >>>> >>>> Cheers, >>>> Miquel >>>> >>>> >>>> 2010/12/21 Dima Tisnek <[email protected]>: >>>>> On 20 December 2010 19:21, William ML Leslie >>>>> <[email protected]> wrote: >>>>>> On 21 December 2010 11:59, Dima Tisnek <[email protected]> wrote: >>>>>>> More visibility for performance achievements would do >>>>>>> good too. >>>>>> >>>>>> Where are pypy's performance achievements *not* visible, but should be? >>>>> >>>>> for example http://shootout.alioth.debian.org/ >>>>> doesn't say which pypy version is used, what options, doesn't have >>>>> performance figures for multithreaded/multicore >>>>> >>>>> also benchmarks are kinda small, most of them are not docuemented, and >>>>> I couldn't find any info if the same python code was used for cpython >>>>> and pypy (both shootout and speed pypy) or interpreter-specific >>>>> verions were used, that is each tweaked for best performance given the >>>>> known tradeoffs for each implementation.further the most standard >>>>> benchmarks, pystone, etc. completely ignore the fact that real world >>>>> programs are large and only a few small paths are execured often. >>>>> >>>>> another temp problem with speed pypy is that it's terrubly slow in ff >>>>> beta. it also occasionally grinds in stable ff and opera, but I guess >>>>> this can be forgiven for the sake of simplicity / developer effort. >>>>> >>>>> if you google for 'python performance' you don't get a single link to >>>>> pypy on the first page, as a matter of fact, codespeak is poorly >>>>> indexed, it took me quite some time to get some of my questions >>>>> answered with a search. also if you look up 'pypy gc' you get a page >>>>> on codespeak, but to interpret what the data actually means is so far >>>>> beyond me. >>>>> >>>>> a good overview is found in the mainling list >>>>> http://codespeak.net/pipermail/pypy-dev/2010q1/005757.html then again >>>>> slowspitfire and spambayes bits are outdated by now. >>>>> >>>>> the definitive good thing about pypy is that it's much easier to find >>>>> out about its inner workings than that of cpython! >>>>> >>>>> hopefully a bit more of end-user stuff get known. >>>>> let's call it pypy public outreach (feature request) >>>>> >>>>>> >>>>>>> Sidetracking... one day when pypy jit/gc/etc are all worked out, how >>>>>>> hard would it be to use same techniques and most of backends for some >>>>>>> unrelated language that doesn't have jit yet, e.g. php? >>>>>> >>>>>> You know that pypy already has implementations of other languages, >>>>>> right - Scheme, Javascript, Prolog, IO and smallTalk? They aren't >>>>>> integrated with the translated pypy-c, but they show that it is not >>>>>> too difficult to write a runtime for any dynamic language you choose. >>>>> >>>>> Oh I didn't know there were so many, and I mistakenly though that js >>>>> was a target, not implmented langauge. In any case I read somewhere >>>>> that js support was retired... >>>>> >>>>>> >>>>>>> And how hard >>>>>>> would it be to marry two dynamic languages, so that modules from one >>>>>>> could be used in the other? Or that modules written in rpython could >>>>>>> be used in several langs? >>>>>> >>>>>> It's in the "interesting problems" bucket, and the effort required >>>>>> depends on the level of integration between languages you want. There >>>>>> are several projects already attempting to do decent integration >>>>>> between several languages, besides the approach used on the JVM, there >>>>>> are also GNU Guile, Racket, and Parrot, among others. It might be >>>>>> worth waiting to see how these different projects pan out, before >>>>>> spending a bunch of effort just to be an also-ran in the >>>>>> multi-language runtime market. >>>>>> >>>>>> However, implementing more languages in rpython has the side-effect of >>>>>> propagating the l * o * p problem: it introduces more and more >>>>>> implementations that then have to be maintained, so good >>>>>> cross-language integration probably belongs /outside/ pypy itself, so >>>>>> existing runtimes can hook into it. >>>>> >>>>> Makes perfect sense, after all any given other language hardly has the >>>>> same data model as python. >>>>> >>>>>> >>>>>> But it would be an interesting experiment (to marry the various >>>>>> interpreters pypy ships with), if you wanted to try it. >>>>>> >>>>>> My two cents. >>>>>> >>>>>> -- >>>>>> William Leslie >>>>>> >>>>> _______________________________________________ >>>>> [email protected] >>>>> http://codespeak.net/mailman/listinfo/pypy-dev >>>> >>> >> > _______________________________________________ [email protected] http://codespeak.net/mailman/listinfo/pypy-dev
