I added some stuff to visualize the dump-memory-stats output. I'm finding that...
channel-syncer evt evt-set hash hash-table-bucket installed-finalizers semaphore thread thread-cell unknown,154 unknown,225 unknown,226 unknown,231 unknown,234 unknown,253 weak-box all appear to grow without bound. I made a change to the dispatch server to not construct new evts on every connection, but it didn't make any difference. The code that I'm running doesn't use hashes at all, and it uses the same evt, so this is suspicious to me. My guess, based on nothing really, is that inside of the implementation of sync, there's a call to 'select' or 'poll' that uses a hash table to map the OS fd back to the Racket evt. This thing is leaking and causing leaks in the hash table, the evts inside of it, and maybe the temporary thread that holds the call to select. I push my dump analysis code to my fork of Danny's repo. Just name one of the "dump-*" files to "dump" and run "dump.rkt". You'll get a ton of PNGs in the "dump-pngs" directory. Jay On Mon, Apr 22, 2013 at 10:38 AM, Jay McCarthy <jay.mccar...@gmail.com> wrote: > I'm looking into this... > > I think the connection thing is a bit of red herring. If you allow > HTTP/1.1 then your test uses 6 connections total until the crash, the > first 5 to just load the static content. However, if you remove it, > then every repl test is its own connection. This has the effect of > each repl test getting its own thread and custodian. So, it's not that > connections are leaking, but that by re-using the same connection, > some other thing that is leaking lasts a long time. [BTW, you don't > need to hack the code to do this, just pass #:connection-close? #t to > serve/servlet.] > > I gathered per-connection memory stats and found that it takes about > 160 reqs to GC. It dies after 24 such cycles. Most allocate 30mbs. > Most leak 3 mbs. > > I cut out the entire continuation apparatus of the server and had two > interesting results. First, the memory still grows unboundedly and > second the sandbox doesn't catch it. This suggests to me that the > leaked memory is not accounted to the sandbox custodian. > > The part of the server that is left uses no global data structures, is > only using six threads at most, and does not use continuations or > non-tail recursive loops. It should run in constant memory. > > So, now it's my turn to cry wolf and say that something strange is > happening in the VM. > > My results, including logs and modification to the test server are here: > > https://github.com/jeapostrophe/web-server-under-sandbox > > to get the graphs and analysis, just link one of the logs to "log" and > run "log.rkt" > > Jay > > On Sat, Apr 20, 2013 at 5:57 PM, Danny Yoo <d...@hashcollision.org> wrote: >>> Let me post code and see if anyone agrees: >>> >>> https://github.com/dyoo/web-server-under-sandbox >>> >>> I create a trivial web server with a servlet that immediately errors >>> out. I have it also serve static JavaScript that's essentially a >>> torture test: it keeps sending requests to the web server, over and >>> over. (Ripped out of whalesong's REPL testing code). >> >> >> Followup: I've gotten independent confirmation from Eric Hanchrow that >> he can replicate my results, so at least I'm not hallucinating. >> >> I've been able to trace down the source of the garbage somewhere due >> to the reuse of the connection to support HTTP/1.1. If I hack this >> line in the codebase: >> >> >> https://github.com/plt/racket/blob/master/collects/web-server/private/dispatch-server-unit.rkt#L106 >> >> >> so that it _always_ closes the connection, then the resulting web >> server survives my torture test. >> >> >> So there's some kind of state that's accumulating due to the >> connection reuse to support HTTP/1.1. Still trying to trace the exact >> nature of the problem down. > > > > -- > Jay McCarthy <j...@cs.byu.edu> > Assistant Professor / Brigham Young University > http://faculty.cs.byu.edu/~jay > > "The glory of God is Intelligence" - D&C 93 -- Jay McCarthy <j...@cs.byu.edu> Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 ____________________ Racket Users list: http://lists.racket-lang.org/users
