Love this. "a popular micro-blogging site" who shall remain anonmyous .->
D. On Thu, Nov 26, 2009 at 5:00 AM, David Pollak <[email protected]> wrote: > On Wed, Nov 25, 2009 at 7:16 PM, Bechauf, Michael > <[email protected]>wrote: > >> Wasn't this exactly the kind of stuff that the Eclipse Memory Analyzer - >> donated by SAP - was supposed to fix ? A heap of that size for a still >> moderate number of 300 users is crazy, so either there is stuff like >> circular references that hog memory, or the design model is fundamentally >> flawed. I don't understand why ESME needs "sessions" ? How can a scaleable >> server be created if each user will allocate memory until some timeout. In a >> world of stateless browser-based UIs that's not going to work. >> > > You're actually dead wrong about this. "Stateless" is not... it's just > pushing state and cache someplace else (the RDBMS, memcached, etc.). > "Stateless" will lead to radical performance problems. "Stateless" merely > moves the caching decisions into code you don't control. I dealt with this > issue first-hand while helping a popular micro-blogging site migrate from a > "stateless" to a Scala-based backend. I'm dealing with this issue > first-hand helping another popular site that's experiencing exponential > growth migrate away from "push everything back to the RDBMS and hope for the > best." > > My original design for ESME is stateful. My original design for ESME is > based on lessoned learned in this very space and was oriented to have things > intelligently cached so that the caching is not based on RDBMS indexes. I'm > not sure what happened to cause the particular issues, but it seems like > folks are loading messages from the RDBMS rather than asking the message > cache for them. > > >> >> Time for me to look at that code ... >> >> -Michael >> >> >> ----- Original Message ----- >> From: Markus Kohler <[email protected]> >> To: [email protected] <[email protected]> >> Sent: Wed Nov 25 12:14:58 2009 >> Subject: Further analysis of the GC issue >> >> Hi all, >> the Garbage Collector issue I was talking about is reproducible. >> I've uploaded an annotated GC graph to >> >> http://picasaweb.google.com/lh/photo/wB-RRtb0wIVfpxJkTJPNuw?authkey=Gv1sRgCOve7LThpfvXsQE&feat=directlink >> >> I think the "LOGON" phase where I logon all the 300 users looks ok (given >> that probably textile formatting is involved) but the phase where just one >> user sends one message is certainly not looking good. >> >> I took the profiler and the result is a bit shocking. For that one message, >> 881.000.000 objects weighting 23,2 Gbyte where allocated (and reclaimed >> afterwards). My former record was 2Gbyte ;-) >> >> Fortunately I have a theory what happens, without looking into the >> code,yet, >> so take it with a grain of salt. It seems that the public time line for all >> users is re-rendered, because 99% of the allocations come >> from org.apache.esme.comet.PublicTimeline.render(). I guess all the actors >> for all the users are sitting there, not knowing that the user has closed >> the browser, because the user session has not yet expired. >> >> I wonder how we get around this with a real "push" model. If the browser >> would ask for updates this rendering could be done lazily. Or can we "ping" >> the browser and check whether it responds? >> On the other side. It should also not be necessary the re-render the >> message >> again and again because the result will be the same. >> >> I will send Richard some attachments. Not sure whether you will need them, >> they look very similar to the ones we already have. >> >> BTW, we should definitely check the use >> of scala.xml.XML$.loadString(java.lang.String) >> It's creating a new Parser each time, which is a bit costly because it >> allocates a new Buffer each time and also hits the disk, when searching for >> the name of the Java class. >> >> Greetings, >> Markus >> >> >> >> "The best way to predict the future is to invent it" -- Alan Kay >> > > > > -- > Lift, the simply functional web framework http://liftweb.net > Beginning Scala http://www.apress.com/book/view/1430219890 > Follow me: http://twitter.com/dpp > Surf the harmonics >
