Ben, FYI, bug 1002346 has added swap size to b2g-info.
Ting ----- Original Message ----- > From: "Ben Kelly" <[email protected]> > To: "Dave Huseby" <[email protected]>, "Ting-Yu Chou" <[email protected]> > Cc: [email protected], "Kyle Huey" <[email protected]>, "Harald > Kirschner" <[email protected]>, "Etienne > Segonzac" <[email protected]>, "Vivien Nicolas" <[email protected]>, > "Axel Kratel" <[email protected]>, > [email protected], [email protected] > Sent: Wednesday, May 21, 2014 10:10:57 PM > Subject: Re: [b2g] Firewatch: Realtime B2G info > > Sorry if this is a bit off topic, but it reminded me of few things I > discovered last week looking at facebook on tarako. > > So I realized b2g-info is not the best for measuring leaks on devices > with zram, like tarako. When an app is leaking memory it may just be > inflating zram swap pages which do not show up in RSS. So an app can be > leaking, RSS remains stable, but eventually you run out of zram swap and > OOM. > > A clue that this is happening will be if the LMK kill log message > indicates the process size is much larger than what b2g-info reports. > (The kill log message provides size in number of 4k pages.) > > In addition to b2g-info you probably want to look at the total amount of > swap being used: > > adb shell cat /sys/block/zram0/mem_used_total > > You can see what the current zram limit is with: > > adb shell cat /sys/block/zram0/disksize > > It can also be useful to hack /system/xbin/zram.sh to configure a larger > amount of zram. This might let you run your test case past the point of > a normal OOM and then capture a memory report. > > So to tie this back into the mail thread, we probably need to include > zram swap into any graphing tools to get the full picture. > > Ben > > On 5/21/2014 9:52 AM, Dave Huseby wrote: > > Hi Ting, > > > > I share your frustration with falling back to b2g-info. I myself have > > been debugging OOM kills on Tarako and have just been watching adb > > logcat with grep filters to figure out what is going on. It is less > > than ideal. > > > > Having a realtime line graph of member is a really good idea. Last > > week, Harald Kirschner (cc'd) demonstrated his latest version of > > firewatch with its realtime graphing UI. It's exactly what you're > > looking for. > > > > We had a cross-team meeting between the Gaia, Dev-tools, and Fxos Perf > > teams as well as anybody involved with writing tools (e.g. Harald). > > Axel Kratel is the primary person to talk to about tools features now. > > He's the product manager for the dev tools team. > > > > I am putting together a list of requirements for a memory profiling tool > > *right now*. I was planning to deliver it to Axel this week. His team > > is writing the new memory profiler dev tool right now and if we want to > > have any influence on its design and features we need to communicate > > them to his team ASAP. > > > > I'd ping Harald to get the latest copy of firewatch for you immediate > > needs. Then email me any specific requirements you have for a memory > > profiler so I can include them in the list I'm delivering to Axel. > > > > -dave > > > > On 05/21/2014 03:31 AM, Ting-Yu Chou wrote: > >> Hi Dave, > >> > >> When I was working on memory issues on Tarako, such as Gallery gets killed > >> by > >> LMK while scrolling, I usually asked by Gaia developers that "How should I > >> measure how much memory is the application using while I am ...?" I > >> answered > >> them that I eyeball the output of b2g-info, which is not an easy way. > >> > >> As the loading from Tarako is eased recently, I am thinking maybe a > >> real-time > >> memory line chart will be helpful. Thinker told me there're some > >> discussions on > >> dev-platform, so I found and read your email. > >> > >>> mode" for the profiler. We can then add in Harald's (and :bkelly's > >>> prototype, and :hub's implemented) real-time memory reporting into the > >>> profiler timeline. I'm working on profiler markers from JS > >> > >> Seems you're concentrate on unifying the communication between target and > >> host. But how do you think if I try to add the real-time memory report > >> simultaneously? > >> > >> I haven't thought it completely yet, but I'd like to know are there any > >> chances > >> that I can help on that, or maybe someone is already working on it. > >> > >> Thanks, > >> Ting > >> > >>> On 03/25/2014 11:44 AM, Dietrich Ayala wrote: > >>>> Harald's tool is a bit different - it's about real-time tracking for > >>>> developers. I have an oddly similar tool I wrote for my own use, that > >>>> shows real-time graphs of memory and CPU using the same method, to be > >>>> able to see the impact of specific sequences of user actions across > >>>> multiple processes. > >>>> > >>>> However, on Tarako I've noticed that it greatly affects the performance > >>>> of the device over short periods of time. I haven't looked further into > >>>> it - Harald have you seen that issue? > >>>> > >>>> > >>>> ------------------------------------------------------------------------ > >>>> > >>>> Thanks Harald! > >>>> > >>>> We (+FxOS Perf Team [1]) have a similar memory tracking effort that > >>>> feeds into Datazilla [2]. *Hub*ert Figuiere can speak to its > >>>> details. *Ben Kelly* leads our memory performance efforts and *Dave > >>>> Huseby* leads our tool development. Please chat with all three > >>>> about > >>>> what you've built and how we can work together on this. > >>>> > >>>> Mike > >>>> > >>>> [1]: https://wiki.mozilla.org/B2G/Performance > >>>> [2]: http://mzl.la/1giCtEL > >>>> > >>>> > >>>> ------------------------------------------------------------------------ > >>>> *From: *"Harald Kirschner" <[email protected]> > >>>> *To: *"Apps Partner Engineering @ Mozilla" > >>>> <[email protected]> > >>>> *Cc: *"Dietrich Ayala" <[email protected]>, "Fabrice Desre" > >>>> <[email protected]>, "Thomas Elin" <[email protected]>, "Mike > >>>> Lee" > >>>> <[email protected]> > >>>> *Sent: *Tuesday, March 25, 2014 11:26:04 AM > >>>> *Subject: *Firewatch: Realtime B2G info > >>>> > >>>> To debug memory use a bit better after hearing that the new dev > >>>> tool > >>>> overlay doesn’t report a good memory indicator here a new approach: > >>>> > >>>> https://github.com/digitarald/firewatch > >>>> > >>>> Firewatch is basically a real-time b2g-info, showing only the > >>>> useful > >>>> values (PSS and USS [1]) and free memory. On the roadmap is a > >>>> history feature to see max/min/avg/mean values (spikes) and keeping > >>>> killed apps. > >>>> > >>>> I still have to give it a test ride on Tarako and should have more > >>>> details later. Feel free to cc more people as feedback is much > >>>> appreciated! > >>>> > >>>> /Harald > >>>> > >>>> Partner Engineer & Web Craftsman | [email protected] > >>>> <mailto:[email protected]> > >>>> > >>>> [1]: http://elinux.org/Android_Memory_Usage#procrank > >>>> > >>>> > >>>> > >>> > >>> _______________________________________________ > >>> dev-b2g mailing list > >>> [email protected] > >>> https://lists.mozilla.org/listinfo/dev-b2g > >>> > > > > > > > > _______________________________________________ > > dev-b2g mailing list > > [email protected] > > https://lists.mozilla.org/listinfo/dev-b2g > > > _______________________________________________ dev-b2g mailing list [email protected] https://lists.mozilla.org/listinfo/dev-b2g
