Ah ok. 1) Who's going to define these benchmarks?
2) Do we need a new baseline? 3) Is anyone investigating the memory regressions that we already have? a) We still have not resolved the series of initial bugs going into 2.5/3.0: https://bugzilla.mozilla.org/show_bug.cgi?id=1155854 https://bugzilla.mozilla.org/show_bug.cgi?id=1162535 b) I just did a quick search and found some open bugs in memory leaks; I don't know if they all still occur: 1135278 - shared/js/media/video_player.js leaks memory 1135256 - navigator.mozL10n.ready can cause memory leaks 1032830 - Clock leaks memory while screen is off 1203999 Pin the Web is leaking setting observers--- 1202592 Activity chooser leaks settings observer 1137995 research on power consumption shows privacy info leaks 1118106 SecureElement : Resource management / resources leak to be handled in erroneous conditions such as gecko crash / restart etc. 1111526 [B2G][AudioChannel] Paused media element sound leaks when another media element plays with loop = on . 1046895 Wakelock leaks after several failed registration attempts 1039068 notifications.js leaks the most recently used notification <div> 1020685 Fix leak via contextmenu handler On Tue, Sep 15, 2015 at 2:10 PM, Eli Perelman <[email protected]> wrote: > Naoki, yes, I believe that should probably be updated. It's OK to have > multiple benchmarks, but for something that we are going to base blockers > on, they should be well-established per release. For example: > > v2.2 > flame-kk 319MB > > v2.5 > flame-kk 319MB > flame-kk 1024MB > aries 2048MB > > v3.0 > aries 2048MB > aries-l 2048 > > Not that these are the actual baseline benchmarks, just that they should > be well-defined, and there should be a common denominator between 2 > adjacent releases. > > Thanks, > > Eli Perelman > > On Tue, Sep 15, 2015 at 3:28 PM, Naoki Hirata <[email protected]> wrote: > >> It sounds like : >> https://wiki.mozilla.org/Firefox_OS/Performance/Benchmark needs to be >> updated to include memory allocations? or are we talking about the same >> thing? >> >> On Tue, Sep 15, 2015 at 12:01 PM, Eli Perelman <[email protected]> >> wrote: >> >>> Raptor's performance tests also use 319MB as it is the only real >>> baseline we tracked from v2.2. In order to compare the performance between >>> the two branches of v2.2 and v2.5, we need a comparable baseline. We have >>> been aggregating data for the Flame 1GB for future comparisons, but I would >>> suggest proposing a new baseline sooner rather than later so Raptor can >>> begin to aggregate data against that device and make it trackable between >>> branches. >>> >>> Thanks, >>> >>> Eli Perelman >>> >>> On Tue, Sep 15, 2015 at 12:48 PM, Martijn <[email protected]> >>> wrote: >>> >>>> On Tue, Sep 15, 2015 at 6:32 PM, Naoki Hirata <[email protected]> >>>> wrote: >>>> > >>>> > I believe we still do care. >>>> > >>>> > Running Engineering build will make some difference as the 319 MB >>>> takes into account the camera but not Marionette (adb + devtools) which >>>> the engineering build runs. >>>> > >>>> > Having said that we run automation tests through jenkins, I'll check >>>> with mwargers in regards to the issue to see if he already filed a bug or >>>> not. >>>> >>>> >>>> Yes, we're running the Gaia UI tests with the 319MB configuration and >>>> we have various tests that are failing (intermittently) and disabled >>>> because they just don't work with 319MB: >>>> 1172167 [Flame][Cards View] Cards view loses the list of recently >>>> opened apps >>>> 1176502 Failure in test_homescreen_column_layout.py on the Flame >>>> device, because Settings app gets killed >>>> 1178859 test_browser_bookmark.py: "IOError: Connection to Marionette >>>> server is lost." >>>> 1181344 [Window Management] Settings will be a blank white screen when >>>> returning from the "Built-in Keyboard" settings >>>> 1188080 test_rocketbar_offline_behavior.py: "NoSuchWindowException: >>>> None" >>>> 1189262 Failure in test_browser_share_link.py using Flame 319MB >>>> 1204280 [Flame] Intermittent failure in test_sms_contact.py in >>>> tap_send_sms using Flame 319MB >>>> 1188603 [Settings][Ringtones] Share activity closes when attempting to >>>> add custom ringtone >>>> >>>> I think these bugs need to be fixed, because they stand for lost >>>> functionality in those low memory conditions. >>>> >>>> All the bugs currently out there, regarding supporting 319MB memory >>>> configuration: >>>> >>>> https://bugzilla.mozilla.org/buglist.cgi?list_id=12549261&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=[319MB-Flame-Support] >>>> >>>> Regards, >>>> Martijn >>>> >>>> >>>> > >>>> > >>>> > We still have not resolved the series of initial bugs going into >>>> 2.5/3.0: >>>> > https://bugzilla.mozilla.org/show_bug.cgi?id=1155854 >>>> > https://bugzilla.mozilla.org/show_bug.cgi?id=1162535 >>>> > >>>> > On Tue, Sep 15, 2015 at 2:13 AM, Ting-Yu Chou <[email protected]> >>>> wrote: >>>> >> >>>> >> A bit off-topic. >>>> >> >>>> >> Raptor [1] just records a serious memory regression though, not sure >>>> is it related to the issue you see. >>>> >> >>>> >> Ting >>>> >> >>>> >> [1] >>>> http://raptor.mozilla.org/#/dashboard/script/apps-memory.js?device=flame-kk&branch=master&memory=319 >>>> >> >>>> >> On Tue, Sep 15, 2015 at 4:56 PM, Gabriele Svelto < >>>> [email protected]> wrote: >>>> >>> >>>> >>> I was doing some testing in the past few days on a Flame using the >>>> >>> 319MiB memory switch and found that our core apps are now almost >>>> >>> unusable on it. Opening a single app seems to end up killing the >>>> >>> homescreen and keeping two open at the same time is almost >>>> impossible. >>>> >>> >>>> >>> So I've got two questions: the first one is, did we significantly >>>> >>> regress memory consumption again? Or is it just gecko having grown >>>> >>> significantly in the past few weeks making our minimum process size >>>> >>> larger? The second question is obviously: do we still care about >>>> 256MiB >>>> >>> devices? Because if we still do then it looks like we're not in a >>>> good >>>> >>> spot for a 2.5 release running on those. If we don't care I suppose >>>> it's >>>> >>> fine though we should still keep our memory consumption under >>>> control. >>>> >>> If we move our minimum to 512MiB we'll have plenty of room and this >>>> >>> might cause us to regress even further almost without noticing (*). >>>> >>> >>>> >>> I'm running an engineering build BTW, but I don't think this should >>>> make >>>> >>> much of a difference. >>>> >>> >>>> >>> Gabriele >>>> >>> >>>> >>> *) Unless the screen has a rather high resolution, in which case >>>> 512MiB >>>> >>> might still yield only a small amount of usable memory for apps as >>>> most >>>> >>> of it will go to the framebuffer and accompanying structures >>>> (layers & co). >>>> >>> >>>> >>> >>>> >>> _______________________________________________ >>>> >>> dev-fxos mailing list >>>> >>> [email protected] >>>> >>> https://lists.mozilla.org/listinfo/dev-fxos >>>> >>> >>>> >> >>>> >> >>>> >> _______________________________________________ >>>> >> dev-fxos mailing list >>>> >> [email protected] >>>> >> https://lists.mozilla.org/listinfo/dev-fxos >>>> >> >>>> > >>>> > >>>> > _______________________________________________ >>>> > dev-fxos mailing list >>>> > [email protected] >>>> > https://lists.mozilla.org/listinfo/dev-fxos >>>> > >>>> _______________________________________________ >>>> dev-fxos mailing list >>>> [email protected] >>>> https://lists.mozilla.org/listinfo/dev-fxos >>>> >>> >>> >> >
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
