On Thu, Jun 25, 2009 at 3:26 PM, Mike Beltzner <[email protected]> wrote:
> On 25-Jun-09, at 12:52 PM, Mike Belshe wrote: > > Yeah, the APIs all have constraints. We end up walking the pages and >> adding them up. See process_util_win.cc in the chromium tree. Be sure to >> check about:memory and hover over the little "?" icons to see what we >> measure. >> > > This screen actually confuses me a little, as the Summary statistics don't > match the summation of the process based statistics. Do you mean to say your > summary statistics take into account the memory that's being shared across > the various processes? Correct. The "shared" across all processes is a bit of a hack, because you can't know exactly which pages are shared across every single process. We do a heuristic. > > > If so, is there any command-line switch that will output those summary > stats to console on a regular basis? You can use --enable-logging --log-level=0 - each time you refresh the about:memory page, it will drop a line to the log file. This can be useful for plots over time. > > > >> If someone measures the "Private Bytes" counter, which uses the >> PROCESS_MEMORY_COUNTERS_EX.PrivateUsage structure, that seems to map to the >> Task Manager "Commit Size" which isn't the thing I believe we want to >> measure. >> >> Right. >> > > Interestingly, as I watched this value change while webpages were loading, > it tracked the same pattern of growth/decline as "Memory (Private Working > Set)" in the Task Manager, though the values were usually about 2x or so > more. I suppose this is due to the heap sharing you were speaking of > earlier? I'm not quite sure what you mean. But of course, as you load more tabs, you'll see the memory use go up. > > > The "Working Set - Private" counter doesn't seem to have a structure >> according to the MSDN document; that's what maps to the "Memory (Private >> Working Set)" column in the TaskManager. >> >> Right, I think you have to use QueryWorkingSet, walk the pages and >> categorize them yourself. >> > > OK, I can look into trying that. Though I'm wondering if it's worth the > bother, as the meta-pattern, to me, is more interesting than the precise > megabyte count. For a single process browser, it's not worth the effort; I think it's the only way to know how to account for shared memory. > > > >> The closest thing I can find is the "Working Set" counter, which uses the >> PROCESS_MEMORY_COUNTERS_EX.WorkingSetSize structure and shows up in the >> Vista Task Manager as "Working Set (Memory)" >> >> For multi-proc browsers like chrome, this will way overstate RAM; there is >> a good 5-6MB of shared working set in each process. So for 10 tabs, you'd >> could an extra 50MB for Chrome if you do it this way. >> > > Looking both in Task Manager and about:memory, when I have 30 tabs open I'm > not seeing 30 processes. Are you sure you're right about this point? > You don't always get a new process for every tab. If two tabs are connected via javascript, then they'll be in the same process (the about:memory shows which tabs are in the same process). So, clicking a link, for example, will open in the same tab, but typing the URL in the omnibox will create a new process. Others could tell you more about the exact policy for when you get a new process and when you don't. > > > If you come up with a better way, please let me know! >> > > Well, I can tell you what we've done and what we're doing, just trying to > get confidence on various metrics. We've replicated the "membuster" test we > used around the release of Firefox 3 (see > http://blog.pavlov.net/2008/03/11/firefox-3-memory-usage/ ) as a way of > measuring Firefox's ability to hold a "steady" state of memory across a > browsing session (ie: not leak) and then release that memory when the > session ends (ie: not bogart system resources). The test: > > - runs on Windows 7, using Python and our Standalone Talos code to make > measurements > - uses a local web proxy to ensure we're always viewing the same web > content > - uses the JS at view-source: > http://random.pavlov.net/membuster/index.html to: > --- open 30 pages using window.open() calls > --- load ~300 pages, opening and closing windows with each load (always 30 > open) > --- close all 30 windows when done cycling OK - I think this might basically use one renderer process in chrome? Because of the new-process creation policy, it may not be representative of real world usage. Darin? > The whole while, we measure the amount of memory taken using the > PROCESS_MEMORY_COUNTERS structure, summating over processes when multiple > exist (as they do in the case of Internet Explorer 8 and Chrome 2) Ok - that will double count shared memory. I'd estimate 3-5MB per process. > The results can be seen here, using both the PrivateUsage (shows as "Commit > Size" in Windows 7 Task Manager) and WorkingSet (shows as "Working Set > (Memory)" in Windows 7 Task Manager) counters: > > http://people.mozilla.com/~beltzner/images/private-usage.png > http://people.mozilla.com/~beltzner/images/working-set.png > Interesting. Based on how you described the test, it almost looks like a memory leak in chrome. > > The datapoint we wanted to get out of this was primarily about Firefox 3.5 > vs Firefox 3.0.11, and as you can see we actually take slightly (7-10MB) > more memory during the page cycling, but manage to release more when done. > Safari had previously been unable to complete this test, but now does. > > I'd love to get your feedback on how we can improve our recording here. I > think we're going to try and bundle together the test and files into an > easier-to-use-tool than our delicately cobbled together solution, but if you > wanted to try it yourself, here's a ZIP with the required code (needs Python > 2.5 or later to be installed on your system): BTW - using the about:memory, chrome will log memory stats for all browsers (IE, safari, firefox, chrome), and it uses the same math on all of them, so its apples-to-apples. It will log them all to disk using the mechanism described above. I'll try to take a closer look at your test, but I'm not sure when I'll have time :-( Mike > > Once unzipped, enter the directory and run the following commands: > > \path\to\python\python.exe proxyserver.py -v -l -u proxy-cache.db > \path\to\python\python.exe measure_any_windows_app.py > > \path\to\resultsfile\resultsfile.txt > > You'll want to edit measure_any_windows_app.py to have the correct paths to > the various applications, and you'll also want to make sure your browsers > are set to use 127.0.0.1 port 8000 as a proxy (under Advanced > Network > > Settings in Firefox, Internet Properties > Connections > LAN Settings for > Chrome/IE/Safari) > > (Thanks to David Dahl for modifying the required files to get at those > measurements) > > cheers, > mike > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
