Re: [Sugar-devel] Sugar Performance testing
2014-03-09 14:46 GMT-03:00 Daniel Narvaez dwnarv...@gmail.com: On 9 March 2014 17:53, Sebastian Silva sebast...@fuentelibre.org wrote: Hi dear Sugar developers. We have participated in the deployment in Peru of Sugar 0.94 (classic) for XO1 and XO1.5. It will be ongoing in 2014 and hopefully we will tighten the feedback circle and work closer with upstream (master). Now we as a team are working in Colombia with XO1.75 and again, the issue of performance creeps in, and they are interested in downgrading to Sugar 0.94 (classic). that way madness lies, if we stay without updating, we break cohesion. It would be best if we could all just work on the same basis. For us to base our work on 0.101+ (new) Sugar, we have to make sure we have solved the performance issues plaguing (new) Sugar and/or OLPC/OS 13.x. Which OLPC version are you comparing with? In the rest of the answer I'll call that X.x :) For this I need your advice. How do I setup an identical environment for (classic) 0.94 Sugar and (new) 0.101+ Sugar? Can I use sugar-build for this, or something else...? On which hardware and on which distribution? Anyway it's probably not going to be trivial. So let me suggest an easier first step. You could test 0.94 activities on the top of OLPC 13.x. If they perform the same as OLPC X.x then we know the issue is the gtk3 toolkit (no change was made to the gtk2 toolkit). If they are bad as stock 13.x activities, then we will know it's something in the system. If it's something in the middle we will have to come up with a more complicated strategy. But I think the data we get from this initial testing will be useful to figure out that strategy. How do I profile the session (CPU usage, memory consumption, timing)? For memory I would try this https://github.com/pixelb/ps_mem For CPU top should be fine, but it depends what exactly you want to test. For timing I usually just print out time.time intervals from the code :) For Python, you can get profiling information with cProfile http://docs.python.org/2/library/profile.html See the treeviews optimization Gonzalo and I been working on http://wiki.sugarlabs.org/go/Features/Optimize_TreeViews Do we have some automated GUI testing? Can I make some? See sugar-build/build/tests/shell.py, you could use something like that to measure startup time I suppose. Anyway you can use the same kind of code to click around in activities UI etc. ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel -- .. manuq .. ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
On Sun, Mar 9, 2014 at 1:53 PM, Sebastian Silva sebast...@fuentelibre.orgwrote: Hi dear Sugar developers. We have participated in the deployment in Peru of Sugar 0.94 (classic) for XO1 and XO1.5. It will be ongoing in 2014 and hopefully we will tighten the feedback circle and work closer with upstream (master). Now we as a team are working in Colombia with XO1.75 and again, the issue of performance creeps in, and they are interested in downgrading to Sugar 0.94 (classic). that way madness lies, if we stay without updating, we break cohesion. It would be best if we could all just work on the same basis. For us to base our work on 0.101+ (new) Sugar, we have to make sure we have solved the performance issues plaguing (new) Sugar and/or OLPC/OS 13.x. +1 in all the rationale. For this I need your advice. *How do I setup an identical environment for (classic) 0.94 Sugar and (new) 0.101+ Sugar? Can I use sugar-build for this, or something else...?* I am not sure if you need a _identical_ environment to compare 0.94 and 0.101. In the end, the user will be affected by the addition of all the changes (system, sugar, activities) When I tried to detect performance problems in activities, the worst regressions were by modifications in the activities itself, because of adding a functionality or similar. We already know there are issues with the port to gtk3 too. Then, IMHO, the first rough test would be identify what are the points with performance regressions. Test activity startup of all the activities you use, comparing the version available for sugar 0.94, and the last version available. Test sugar, and see what is slower, the journal? any control panel section? Anything in particular? Something is getting slower after a time of use? Use the xos to test, your development environment can be faster and is more difficult see any difference. *How do I profile the session (CPU usage, memory consumption, timing)? * Activity startup time is in the shell.log if sugar have debug enabled. You can use [1] to see memory usage. We used cProfile [2] to get more precise numbers, but that required usually program a little test case with the code we wanted to profile. There are a old page about profiling in wiki laptop too [3] Right now, I am trying to use systemtap to profile/detect memory leaks, but I need solve some problems. Will report when have something usable. * Do we have some automated GUI testing? Can I make some? *These questions should help bring me to a point where I can measure and try changes specific to Sugar. I appreciate any advice you can give me on this topic. Next I'd like to try to determine if it's related to the base distribution. What distribution are you using with 0.94 and 0.101? Regards, Sebastian ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel -- Gonzalo Odiard SugarLabs - Learning Software for children [1] https://raw.github.com/pixelb/ps_mem/master/ps_mem.py [2] http://wiki.sugarlabs.org/go/Features/Optimize_TreeViews#How_To_Test [3] http://wiki.laptop.org/go/Performance_tuning ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
On Mon, Mar 10, 2014 at 8:46 AM, Gonzalo Odiard godi...@sugarlabs.org wrote: Activity startup time is in the shell.log if sugar have debug enabled. +100 on Gonzalo's recommendations. Also a couple of notes, from memory, - Gobject Introspection turned out to be slow to instantiate (IIRC, dsd posted analysis this topic). - The network view leaks AP icons. You need a good number of APs visible for this to be noticeable, if ps_mem.py tells you that the memory usage is growing over time, check whether disabling wifi stops the memory leaks. This should be an easy fix -- I researched this couple years ago. - any tests run on SD cards depend on the SD card performance, not two are alike... hth! m -- martin.langh...@gmail.com - ask interesting questions - don't get distracted with shiny stuff - working code first ~ http://docs.moodle.org/en/User:Martin_Langhoff ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
El 09/03/14 12:46, Daniel Narvaez escribió: On 9 March 2014 17:53, Sebastian Silva sebast...@fuentelibre.org mailto:sebast...@fuentelibre.org wrote: For us to base our work on 0.101+ (new) Sugar, we have to make sure we have solved the performance issues plaguing (new) Sugar and/or OLPC/OS 13.x. Which OLPC version are you comparing with? In the rest of the answer I'll call that X.x :) Well we deployed an image based on 11.3 i.e. Fedora 14. However the focus of my research is Sugar itself. I figure base OS issues are a different beast and worth isolating for this first approach, that's why I figured I wanted the same setup with different Sugar versions. For this I need your advice. *How do I setup an identical environment for (classic) 0.94 Sugar and (new) 0.101+ Sugar? Can I use sugar-build for this, or something else...?** * On which hardware and on which distribution? For Christmas I got an Acer Chromebook c720, cost $US200, got 2GB ram and 16GBSSD. Pretty fast Haswell based processor, coming from an Atom netbook. Quite happy with the upgrade, btw. Anyway it's probably not going to be trivial. So let me suggest an easier first step. You could test 0.94 activities on the top of OLPC 13.x. If they perform the same as OLPC X.x then we know the issue is the gtk3 toolkit (no change was made to the gtk2 toolkit). If they are bad as stock 13.x activities, then we will know it's something in the system. If it's something in the middle we will have to come up with a more complicated strategy. But I think the data we get from this initial testing will be useful to figure out that strategy. I will make the time to flash 13.2 on an XO1 and do this informal test. However I has hoping to use my dev environment because I really want a test that can be automated/done quickly, and gives numbers, not just impressions. ***How do I profile the session (CPU usage, memory consumption, timing)? * For memory I would try this https://github.com/pixelb/ps_mem For CPU top should be fine, but it depends what exactly you want to test. For timing I usually just print out time.time intervals from the code :) *Do we have some automated GUI testing? Can I make some?* See sugar-build/build/tests/shell.py, you could use something like that to measure startup time I suppose. Anyway you can use the same kind of code to click around in activities UI etc. Is this what runs when you do ./osbuild check ? I tried but it's failing for me and I'm not finding the error. Below is the output. shell.log and datastore.log in home/dotsugar/logs/ don't show any traceback either. [icarito@aiki sugar-build]$ ./osbuild check = Check = * Running smoke test Command failed: sugar-runner Initializing built-in extension Generic Event Extension Initializing built-in extension SHAPE Initializing built-in extension MIT-SHM Initializing built-in extension XInputExtension Initializing built-in extension XTEST Initializing built-in extension BIG-REQUESTS Initializing built-in extension SYNC Initializing built-in extension XKEYBOARD Initializing built-in extension XC-MISC Initializing built-in extension SECURITY Initializing built-in extension XINERAMA Initializing built-in extension XFIXES Initializing built-in extension RENDER Initializing built-in extension RANDR Initializing built-in extension COMPOSITE Initializing built-in extension DAMAGE Initializing built-in extension MIT-SCREEN-SAVER Initializing built-in extension DOUBLE-BUFFER Initializing built-in extension RECORD Initializing built-in extension DPMS Initializing built-in extension X-Resource Initializing built-in extension XVideo Initializing built-in extension XVideo-MotionCompensation Initializing built-in extension GLX screen 0 shmid 110952503 [dix] Could not init font path element /usr/share/fonts/Type1/, removing from list! xinit: XFree86_VT property unexpectedly has 0 items instead of 1 Xlib: extension RANDR missing on display :99. The XKEYBOARD keymap compiler (xkbcomp) reports: Warning: Compat map for group 2 redefined Using new definition Warning: Compat map for group 3 redefined Using new definition Warning: Compat map for group 4 redefined Using new definition Errors from xkbcomp are not fatal to the X server xinit: connection to X server lost ! Failed to run command check [icarito@aiki sugar-build]$ ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
A quick note, be sure you have the last packages gobject-introspection and pygobject3 from dsd repository [1] or the patches applied to your own packages. Gonzalo [1] http://dev.laptop.org/~dsd/public_rpms/f18/ On Mon, Mar 10, 2014 at 12:22 PM, Sebastian Silva sebast...@fuentelibre.org wrote: El 09/03/14 12:46, Daniel Narvaez escribió: On 9 March 2014 17:53, Sebastian Silva sebast...@fuentelibre.org wrote: For us to base our work on 0.101+ (new) Sugar, we have to make sure we have solved the performance issues plaguing (new) Sugar and/or OLPC/OS 13.x. Which OLPC version are you comparing with? In the rest of the answer I'll call that X.x :) Well we deployed an image based on 11.3 i.e. Fedora 14. However the focus of my research is Sugar itself. I figure base OS issues are a different beast and worth isolating for this first approach, that's why I figured I wanted the same setup with different Sugar versions. For this I need your advice. *How do I setup an identical environment for (classic) 0.94 Sugar and (new) 0.101+ Sugar? Can I use sugar-build for this, or something else...?* On which hardware and on which distribution? For Christmas I got an Acer Chromebook c720, cost $US200, got 2GB ram and 16GBSSD. Pretty fast Haswell based processor, coming from an Atom netbook. Quite happy with the upgrade, btw. Anyway it's probably not going to be trivial. So let me suggest an easier first step. You could test 0.94 activities on the top of OLPC 13.x. If they perform the same as OLPC X.x then we know the issue is the gtk3 toolkit (no change was made to the gtk2 toolkit). If they are bad as stock 13.x activities, then we will know it's something in the system. If it's something in the middle we will have to come up with a more complicated strategy. But I think the data we get from this initial testing will be useful to figure out that strategy. I will make the time to flash 13.2 on an XO1 and do this informal test. However I has hoping to use my dev environment because I really want a test that can be automated/done quickly, and gives numbers, not just impressions. *How do I profile the session (CPU usage, memory consumption, timing)? * For memory I would try this https://github.com/pixelb/ps_mem For CPU top should be fine, but it depends what exactly you want to test. For timing I usually just print out time.time intervals from the code :) * Do we have some automated GUI testing? Can I make some?* See sugar-build/build/tests/shell.py, you could use something like that to measure startup time I suppose. Anyway you can use the same kind of code to click around in activities UI etc. Is this what runs when you do ./osbuild check ? I tried but it's failing for me and I'm not finding the error. Below is the output. shell.log and datastore.log in home/dotsugar/logs/ don't show any traceback either. [icarito@aiki sugar-build]$ ./osbuild check = Check = * Running smoke test Command failed: sugar-runner Initializing built-in extension Generic Event Extension Initializing built-in extension SHAPE Initializing built-in extension MIT-SHM Initializing built-in extension XInputExtension Initializing built-in extension XTEST Initializing built-in extension BIG-REQUESTS Initializing built-in extension SYNC Initializing built-in extension XKEYBOARD Initializing built-in extension XC-MISC Initializing built-in extension SECURITY Initializing built-in extension XINERAMA Initializing built-in extension XFIXES Initializing built-in extension RENDER Initializing built-in extension RANDR Initializing built-in extension COMPOSITE Initializing built-in extension DAMAGE Initializing built-in extension MIT-SCREEN-SAVER Initializing built-in extension DOUBLE-BUFFER Initializing built-in extension RECORD Initializing built-in extension DPMS Initializing built-in extension X-Resource Initializing built-in extension XVideo Initializing built-in extension XVideo-MotionCompensation Initializing built-in extension GLX screen 0 shmid 110952503 [dix] Could not init font path element /usr/share/fonts/Type1/, removing from list! xinit: XFree86_VT property unexpectedly has 0 items instead of 1 Xlib: extension RANDR missing on display :99. The XKEYBOARD keymap compiler (xkbcomp) reports: Warning: Compat map for group 2 redefined Using new definition Warning: Compat map for group 3 redefined Using new definition Warning: Compat map for group 4 redefined Using new definition Errors from xkbcomp are not fatal to the X server xinit: connection to X server lost ! Failed to run command check [icarito@aiki sugar-build]$ ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel -- Gonzalo Odiard SugarLabs - Learning Software for
Re: [Sugar-devel] Sugar Performance testing
Thanks for the notes, do you maybe have a link to the patches that need applying? Sebastian El 10/03/14 10:30, Gonzalo Odiard escribió: A quick note, be sure you have the last packages gobject-introspection and pygobject3 from dsd repository [1] or the patches applied to your own packages. Gonzalo [1] http://dev.laptop.org/~dsd/public_rpms/f18/ http://dev.laptop.org/%7Edsd/public_rpms/f18/ ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
On Mon, Mar 10, 2014 at 12:38 PM, Sebastian Silva sebast...@fuentelibre.org wrote: Thanks for the notes, do you maybe have a link to the patches that need applying? No, but the src rpms are in the same directory, and have the patches. The changes were explained by dsd in a old mail in this mailing list, as you can expect, is not easy stuff :) Maybe you have a newer version of the library than us (in F18) just check if the changes are already included. Gonzalo Sebastian El 10/03/14 10:30, Gonzalo Odiard escribió: A quick note, be sure you have the last packages gobject-introspection and pygobject3 from dsd repository [1] or the patches applied to your own packages. Gonzalo [1] http://dev.laptop.org/~dsd/public_rpms/f18/ -- Gonzalo Odiard SugarLabs - Learning Software for children ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
This is a initial report about the use of systemtap to profile sugar http://wiki.sugarlabs.org/go/Profile/Systemtap Gonzalo On Mon, Mar 10, 2014 at 12:50 PM, Gonzalo Odiard godi...@sugarlabs.orgwrote: On Mon, Mar 10, 2014 at 12:38 PM, Sebastian Silva sebast...@fuentelibre.org wrote: Thanks for the notes, do you maybe have a link to the patches that need applying? No, but the src rpms are in the same directory, and have the patches. The changes were explained by dsd in a old mail in this mailing list, as you can expect, is not easy stuff :) Maybe you have a newer version of the library than us (in F18) just check if the changes are already included. Gonzalo Sebastian El 10/03/14 10:30, Gonzalo Odiard escribió: A quick note, be sure you have the last packages gobject-introspection and pygobject3 from dsd repository [1] or the patches applied to your own packages. Gonzalo [1] http://dev.laptop.org/~dsd/public_rpms/f18/ -- Gonzalo Odiard SugarLabs - Learning Software for children -- Gonzalo Odiard SugarLabs - Learning Software for children ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
On 9 March 2014 17:53, Sebastian Silva sebast...@fuentelibre.org wrote: Hi dear Sugar developers. We have participated in the deployment in Peru of Sugar 0.94 (classic) for XO1 and XO1.5. It will be ongoing in 2014 and hopefully we will tighten the feedback circle and work closer with upstream (master). Now we as a team are working in Colombia with XO1.75 and again, the issue of performance creeps in, and they are interested in downgrading to Sugar 0.94 (classic). that way madness lies, if we stay without updating, we break cohesion. It would be best if we could all just work on the same basis. For us to base our work on 0.101+ (new) Sugar, we have to make sure we have solved the performance issues plaguing (new) Sugar and/or OLPC/OS 13.x. Which OLPC version are you comparing with? In the rest of the answer I'll call that X.x :) For this I need your advice. *How do I setup an identical environment for (classic) 0.94 Sugar and (new) 0.101+ Sugar? Can I use sugar-build for this, or something else...?* On which hardware and on which distribution? Anyway it's probably not going to be trivial. So let me suggest an easier first step. You could test 0.94 activities on the top of OLPC 13.x. If they perform the same as OLPC X.x then we know the issue is the gtk3 toolkit (no change was made to the gtk2 toolkit). If they are bad as stock 13.x activities, then we will know it's something in the system. If it's something in the middle we will have to come up with a more complicated strategy. But I think the data we get from this initial testing will be useful to figure out that strategy. *How do I profile the session (CPU usage, memory consumption, timing)? * For memory I would try this https://github.com/pixelb/ps_mem For CPU top should be fine, but it depends what exactly you want to test. For timing I usually just print out time.time intervals from the code :) * Do we have some automated GUI testing? Can I make some?* See sugar-build/build/tests/shell.py, you could use something like that to measure startup time I suppose. Anyway you can use the same kind of code to click around in activities UI etc. ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] Sugar Performance testing
Something I forgot to mention is that when testing activity startup you really need to clean the kernel memory cache before each run, or you will get inconsistent data. sudo sh -c echo 3 /proc/sys/vm/drop_caches On 9 March 2014 18:46, Daniel Narvaez dwnarv...@gmail.com wrote: On 9 March 2014 17:53, Sebastian Silva sebast...@fuentelibre.org wrote: Hi dear Sugar developers. We have participated in the deployment in Peru of Sugar 0.94 (classic) for XO1 and XO1.5. It will be ongoing in 2014 and hopefully we will tighten the feedback circle and work closer with upstream (master). Now we as a team are working in Colombia with XO1.75 and again, the issue of performance creeps in, and they are interested in downgrading to Sugar 0.94 (classic). that way madness lies, if we stay without updating, we break cohesion. It would be best if we could all just work on the same basis. For us to base our work on 0.101+ (new) Sugar, we have to make sure we have solved the performance issues plaguing (new) Sugar and/or OLPC/OS 13.x. Which OLPC version are you comparing with? In the rest of the answer I'll call that X.x :) For this I need your advice. *How do I setup an identical environment for (classic) 0.94 Sugar and (new) 0.101+ Sugar? Can I use sugar-build for this, or something else...?* On which hardware and on which distribution? Anyway it's probably not going to be trivial. So let me suggest an easier first step. You could test 0.94 activities on the top of OLPC 13.x. If they perform the same as OLPC X.x then we know the issue is the gtk3 toolkit (no change was made to the gtk2 toolkit). If they are bad as stock 13.x activities, then we will know it's something in the system. If it's something in the middle we will have to come up with a more complicated strategy. But I think the data we get from this initial testing will be useful to figure out that strategy. *How do I profile the session (CPU usage, memory consumption, timing)? * For memory I would try this https://github.com/pixelb/ps_mem For CPU top should be fine, but it depends what exactly you want to test. For timing I usually just print out time.time intervals from the code :) * Do we have some automated GUI testing? Can I make some?* See sugar-build/build/tests/shell.py, you could use something like that to measure startup time I suppose. Anyway you can use the same kind of code to click around in activities UI etc. -- Daniel Narvaez ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
