Re: [Sugar-devel] Killing activities when memory gets short
2010/8/11 NoiseEHC noise...@freemail.hu: We used to do that, the problem is that we don't control our platform as Google controls Android and you need to make sure that resources that need to be specific of each child process aren't shared (dbus and X connections, etc). I'm personally more interested in reducing the amount of resources we need to start activities (which is quite insane right now), but sharing more of those resources sounds like a good idea to me. Since I spent quite a bit of time analyzing the Python runtime here is my conclusion, maybe it will make wasting all that time less painful. First, most of the memory consumed by an activity is the process heap. While the Python runtime and native libraries are shared among processes, the heap is not. Even if you fork processes it will not work because reference counting will dirty the shared pages This would hold true if a significant part of the stuff loaded during startup was actually ever referenced. This is not the case in Sugar, so you indeed can get quite some savings from the prefork trick. But of course, the right fix here is not sharing that memory, but not require it in the first place if we don't need it. One example is to stop compiling regular expressions eagerly, as mentioned in http://www.mail-archive.com/su...@lists.laptop.org/msg06957.html . There are several other situations that once fixed would give us wins in the order of hundreds of milliseconds with the corresponding wins in memory usage, such as importing modules that are never used. The move from PyGtk to introspection will mean we stop loading several dozens of classes that we never end up using, but there's lots to win in the Python std library as well. Regards, Tomeu and my instinct tells me that just loading a Python source file will reference almost all the shared pages because they are mostly used to store already loaded modules. What I finally did not do is to actually check the hypothesis that most of the heap is filled by strings which are the identifiers in the loaded Python modules. My goal was to somehow make identifiers constants and just readonly-mmap them into the process (replace the pathetic .pyc loading mechanism). Note that my plan was just a little subset of the .jar - .dex converter. Second, Python has a special memory manager which works with buckets so there are separate heaps for different object sizes. Somehow this special memory manager is turned off in release mode and it uses the standard C heap for some reason. Either it is a bug or just it turned out to be faster (more work was put into glibc) I do not know, but handling the linked free list can dirty shared pages as well or I am just mistaken... Third, the fact that Python is a dynamic language does not help any prefetching or memory sharing. I am not too convicted either that this dynamic nature is used at all in the Sugar codebase but you know I cannot program in Python and frankly I do not feel the need to learn another language. Just now, at my age of 34, I finally gave up and learned LISP (more specifically clojure) and I hope that it will be the last programming language I will have to learn (other than assembly languages of course)... :) Now this point is interesting because if you thought that the Dalvik VM could run the Sugar codebase via Jython then it just will not work. The Dalvik VM just cannot load .jar files and Jython just generates them on the fly because of the dynamic nature of Python. Fourth, moving Python (theoretically) to a GC environment (instead of reference counting) also would not work if the GC is compacting because it would also dirty the shared pages. So a mark and sweep nonmoving GC with separately stored visited bits is the only feasible solution. Now guess what the Dalvik VM does? For more info (45 min + questions): http://www.youtube.com/watch?v=ptjedOZEXPM So *my* conclusion is that solving this sharing problem is *very* hard. I am not sure that simply translating all activities from Python to Java would take more time. Another interesting thing is that meantime the unladen-swallow project progresses (just more slowly than planned). Their plan is to make it the default Python runtime so if it will happen (I cannot comment on that) then the Python VM will use even more memory (though it will be faster) so Sugar will be even less interesting on the myriad of low spec cheap ARM tablets which will flood the market soon. I think that is all I can say about this subject so I just finish it here. ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Dextrose] ip route in rc.sysinit
El Wed, 11-08-2010 a las 07:41 +1000, James Cameron escribió: I agree with Bernie. I've used /etc/rc.local on previous builds to achieve the same result, though with NetworkManager disabled. You do not *need* the IP default route to be able to do as you suggest, so perhaps there is another need for it that you have not described. I have found that a small delay (or polling) is required between some of the commands; iwconfig, ifconfig, dhclient. Without the delay the dhclient runs slower, because it has to retry. I've been using this script to bring up my network when NetworkManager is broken: http://people.sugarlabs.org/bernie/wlanup DHCP always get the IP at the first try with iwlagn (Intel) and ath9k (Atheros) cards. Overall, it's a lot faster than NetworkManager, which makes me wonder where the extra time is being spent... -- // Bernie Innocenti - http://codewiz.org/ \X/ Sugar Labs - http://sugarlabs.org/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Dextrose] ip route in rc.sysinit
2010/8/10 Bernie Innocenti ber...@codewiz.org El Tue, 10-08-2010 a las 09:17 -0300, Esteban Bordon escribió: The script try to connect to the Access Point that have the better signal using iwlist, iwconfig and dhclient. If dhclient return 0 it executes ip route to obtain the server IP and it download a few files. If you set all the access points in the school to the same ESSID, then clients will automatically connect to the AP with best signal and even switch to a different one if you move around (however, since background scanning is slow, you might loose association if you move around too quickly). Regarding the server ip, it would be cool if you could use the DNS. dhcp could configure one nameserver on the client and an automatic search domain. For example, in the school where I'm sitting now I get this in my /etc/resolv.conf: I know, we have in mind. The problem is that we have to impact that in 2500 servers and this may take a long time... This script must to run before user takes control of the OS. My idea is add the script at initrd but later. If the script runs before NetworkManager is started, you might manually configure the network like this: ifconfig eth0 up iwconfig eth0 essid $AP_ESSID key off dhclient eth0 If everything goes well, when dhclient exits you should have a valid, IP, route and DNS. The script does that, and set some delay after each command but I haven't IP and route. I understand why doesn't work if it worked in older versions. The script does something like this: ifconfig eth0 up iwconfig eth0 mode managed sleep 1 iwconfig eth0 channel $AP_CHANNEL sleep 3 iwconfig eth0 essid $AP_ESSID killall dhclient /dev/null 21 sleep 3 dhclient eth0 -T 20 /dev/null 21 -- // Bernie Innocenti - http://codewiz.org/ \X/ Sugar Labs - http://sugarlabs.org/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Installing XO1.5
First, if I am posting to the wrong list, can someone let me know which list I should post to? There were dozens of them and I thought this might be the best one. I am trying to install XO1.5 (the latest stable) into Vmware. Is this the latest 1.5 release: http://download.laptop.org/xo-1.5/os/official/latest/4GB/ ? Is there any documentation available to get XO installed into vmware? When I tried booting up an ISO, it couldn't find a bootloader, so I must be missing something... John ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Dextrose] ip route in rc.sysinit
2010/8/10 Bernie Innocenti ber...@codewiz.org El Tue, 10-08-2010 a las 09:17 -0300, Esteban Bordon escribió: The script try to connect to the Access Point that have the better signal using iwlist, iwconfig and dhclient. If dhclient return 0 it executes ip route to obtain the server IP and it download a few files. If you set all the access points in the school to the same ESSID, then clients will automatically connect to the AP with best signal and even switch to a different one if you move around (however, since background scanning is slow, you might loose association if you move around too quickly). Regarding the server ip, it would be cool if you could use the DNS. dhcp could configure one nameserver on the client and an automatic search domain. For example, in the school where I'm sitting now I get this in my /etc/resolv.conf: domain escuela485.caacupe.paraguayeduca.org search escuela485.caacupe.paraguayeduca.org nameserver 172.18.0.1 The DNS managed by the schoolserver has a zone called escuela485.caacupe.paraguayeduca.org, in which there's a host schoolserver. With all this infrastructure in place, all you need to do to download a file is: wget http://schoolserver/file I'm trying something like this. The problem is that dhclient isn't working correctly. Have you got any form to debug that command? ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] Killing activities when memory gets short
Hi James, On 11 Aug 2010, at 04:05, James Cameron wrote: On Tue, Aug 10, 2010 at 05:48:00PM -0700, John Gilmore wrote: ... and was also unsuccessful in convincing OLPC to prelink the shared libraries before shipping a release, thus allowing read-only pages to not get dirtied with shared library linkage relocations. 10.1.2 release candidate os851 has prelink before shipping, it's done in the builder. Out of curiosity, just been testing two XO-1s side by side one os850 and one os851. Rough observations (repeated about 12 times): - resuming/starting a sequence of activities after another reliably gets to 9 simultaneous activities (Terminal, Calculate, Log, Pippy, Implode, Chat, Moon, Paint, Memorize) - the tenth activity I happened to be testing, Turtle Blocks, usually triggered an OOM hang of some kind - the OOM hang was usually Turtle Blocks failing to launch. Failed launch would consist of the pulsing startup window, then hanging at a mainly dark grey screen with some partial toolbar light grey fill, non-responsive UI for ~30 seconds to a number of minutes (possibly 5 or 10min), finally you'd be dropped back at the last activity you had successfully launched (Memorize in my test cases). - on three occasions os851 successfully started Turtle Blocks - on one occasion os850 successfully started Turtle Blocks, however on one occasion resuming Turtle Blocks os850 managed to trigger an OOM kill of the Sugar shell resulting in all resumed activities dying and being dropped back at the home fav ring view. - as Turtle Blocks seems a little more memory intensive than some other activities, I tried a few others as the tenth and onward test case. Write as no. ten usually was fine and Distance as eleven. Then trying Maze, or Speak as no. twelve would trigger OOM and the activity would be killed (after some delay, as noted above). - on one occasion trying to start Maze in os850 as activity no. twelve, managed to trigger an OOM kill of the Sugar shell. So, not significant results over just 12 cycles for each XO-1 (need finer grained testing rather than 'number of activities'). Both exhibited long UI lockup's when launching an activity while resources were already maxed out, usually resulting with the activity in question being killed; but os850 did trigger OOM to kill the Sugar shell twice, bringing down all activities with it, where as os851 didn't. --Gary P.S. of corse you'll now tell me os850 was also pre-linked (I couldn't see anything about it in the build notes for either os850 or os851), and I'll look silly for trying to test for a difference, confirming my results were non significant ;-) -- James Cameron http://quozl.linux.org.au/ ___ Sugar-devel mailing list sugar-de...@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] Killing activities when memory gets short
gary wrote: P.S. of corse you'll now tell me os850 was also pre-linked (I couldn't see anything about it in the build notes for either os850 or os851), and I'll look silly for trying to test for a difference, confirming my results were non significant ;-) that's exactly right. :-) pre-linking has been in the builds for some time, i believe. paul =- paul fox, p...@laptop.org ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Dextrose] ip route in rc.sysinit
I found the problem. I was using dhclient eth0 -T 20 but the timeout parameter now is -timeout timeout. Thanks for your help! cheers, Esteban. 2010/8/11 Esteban Bordon ebor...@plan.ceibal.edu.uy 2010/8/10 Bernie Innocenti ber...@codewiz.org El Tue, 10-08-2010 a las 09:17 -0300, Esteban Bordon escribió: The script try to connect to the Access Point that have the better signal using iwlist, iwconfig and dhclient. If dhclient return 0 it executes ip route to obtain the server IP and it download a few files. If you set all the access points in the school to the same ESSID, then clients will automatically connect to the AP with best signal and even switch to a different one if you move around (however, since background scanning is slow, you might loose association if you move around too quickly). Regarding the server ip, it would be cool if you could use the DNS. dhcp could configure one nameserver on the client and an automatic search domain. For example, in the school where I'm sitting now I get this in my /etc/resolv.conf: domain escuela485.caacupe.paraguayeduca.org search escuela485.caacupe.paraguayeduca.org nameserver 172.18.0.1 The DNS managed by the schoolserver has a zone called escuela485.caacupe.paraguayeduca.org, in which there's a host schoolserver. With all this infrastructure in place, all you need to do to download a file is: wget http://schoolserver/file I'm trying something like this. The problem is that dhclient isn't working correctly. Have you got any form to debug that command? ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Sugar-devel] Killing activities when memory gets short
On 11 Aug 2010, at 18:42, Paul Fox p...@laptop.org wrote: gary wrote: P.S. of corse you'll now tell me os850 was also pre-linked (I couldn't see anything about it in the build notes for either os850 or os851), and I'll look silly for trying to test for a difference, confirming my results were non significant ;-) that's exactly right. :-) pre-linking has been in the builds for some time, i believe. Lol! :-) At least it's nice to know that the usual OOM behaviour is that the just launched activity is the one that gets it in the neck, most of the time. Pity about the lockup that usually happens before the kill, is it possible to make the OOM happen at a higher free mem threshold? --Gary paul =- paul fox, p...@laptop.org ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Dextrose] ip route in rc.sysinit
On Wed, Aug 11, 2010 at 07:45:34AM -0400, Bernie Innocenti wrote: DHCP always get the IP at the first try with iwlagn (Intel) and ath9k (Atheros) cards. Overall, it's a lot faster than NetworkManager, which makes me wonder where the extra time is being spent... Indeed. If you would like to lift the hood on NetworkManager and find out, I'm sure many people would welcome it. See http://dev.laptop.org/ticket/10272 -- James Cameron http://quozl.linux.org.au/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Installing XO1.5
On Wed, Aug 11, 2010 at 05:37:45AM -0700, john espiro wrote: First, if I am posting to the wrong list, can someone let me know which list I should post to? There were dozens of them and I thought this might be the best one. This one seems appropriate for the time being. I am trying to install XO1.5 (the latest stable) into Vmware. Good. I don't think anybody has done this yet; nobody has spoken or written about it, and it might take a fair amount of work. Most people who want to use Sugar in a VMware guest would use one of the Sugar on a Stick builds. See http://sugarlabs.org/ for more details. There are Sugar mailing lists too. Activities can be developed in this environment for deployment to XO-1.5 laptops. The reason work would be required to get an XO-1.5 operating system working in VMware is that VMware does not provide the OpenFirmware booting environment expected by the XO-1.5 operating system. One would have to retrofit a bootloader into the filesystem of the operating system; a bootloader that works with VMware. Is this the latest 1.5 release: http://download.laptop.org/xo-1.5/os/official/latest/4GB/ ? Yes, at time of writing this is the latest official release (build os206 in release 10.1.1). For development builds that are leading up to the next official release, read the draft release notes: http://wiki.laptop.org/go/Release_notes/10.1.2 Is there any documentation available to get XO installed into vmware? There is old documentation that may not work on our Wiki. Such as http://wiki.laptop.org/go/VMware http://wiki.laptop.org/go/Emulating_the_XO/Help_and_tips http://wiki.laptop.org/go/User_Feedback_on_Images http://wiki.laptop.org/go/Emulating_the_XO/Quick_Start/Mac There is documentation about GNU GRUB which is a bootloader typically used for booting Linux on VMware. It is at gnu.org or on Wikipedia. When I tried booting up an ISO, it couldn't find a bootloader, so I must be missing something... GNU GRUB is intentionally removed during our build of the XO-1.5 operating system. This is handled by olpc-os-builder.git/modules/base/kspkglist.10.core.inc which contains the line -grub. -- James Cameron http://quozl.linux.org.au/ ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] Introduction / What can I do to help?
On Tue, Aug 10, 2010 at 11:08 PM, ncorr...@gmail.com wrote: I'll try to get in touch with Jerry to see if I can help him with packaging and such. He's right here on the list ;-) Do we plan to turn this into a Fedora Spin? Or will it be a standalone project based on Fedora? There are a few packages that aren't good packages from a Fedora POV, so no, unlikely we'll ever be 'pure' enough to be a Spin. Another idea I had was to try to add a school management software, like proyecto alba, or something like that. We use Moodle, and it is very central to the XS user experience and administration. Not sure if Alba integrates w Moodle, or what it can offer (or whether moodle can learn a few thigns from it). cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Introduction / What can I do to help?
On Tue, 2010-08-10 at 17:34 -0300, SgtPepper wrote: Hi Guys! I'm Nicolas Corrarello, from Argentina. I used to be a Red Hat Technical Support Engineer, now I'm working at Symantec. I met Walter Bender, and the XO a few years ago.. and since then I've been a believer. I'm a UNIX Sysadmin and Fedora Contributor and I'd love to help with the project in any way I can. If you need to contact me you can reach me via Google Talk (ncorr...@gmail.com) or M$N messenger (nicolascorrare...@fibertel.com.ar). I generally have my ipkall number transfered to my cellphone (+1 (253) 242 8786) so you can reach me in that number at any time. Welcome aboard. I've been checking the server distribution, you've done a great job. I'd love to help in new builds. Are you using a bugzilla or something like that? Think for the most part, the discussion goes on here, on the list. I've been thinking on maybe creating a firstboot module to perform the configuration that now is done via the /etc/sysconfig/olpc-scripts/domain_config script, or stuff like that. In any case.. let me know what can I do to help, and what tools are you using for the builds. Now that fits in really well with the proposed changes to xs-config, if your kickstart file doesn't turn off firstboot-xs, run it. The current install is more or less controlled by the xs-config rpm and the kickstart file, with the rpm providing a drop-in networking layout. We're going to split the configuration of the XS's services and networking into different packages, allowing us better use the stock OS configuration tools such as firstboot. That should allow us to present a more friendly installer. http://dev.laptop.org/git/projects/xs-config/ The tools used to spin up the installer are found at: http://dev.laptop.org/git/projects/xs-livecd/ My sandbox is at(ATM): http://24.79.179.72/pub/ Jerry ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Introduction / What can I do to help?
Jerry, Are you in the IRC? in what channel? 2010/8/11 Jerry Vonau jvo...@shaw.ca On Tue, 2010-08-10 at 17:34 -0300, SgtPepper wrote: Hi Guys! I'm Nicolas Corrarello, from Argentina. I used to be a Red Hat Technical Support Engineer, now I'm working at Symantec. I met Walter Bender, and the XO a few years ago.. and since then I've been a believer. I'm a UNIX Sysadmin and Fedora Contributor and I'd love to help with the project in any way I can. If you need to contact me you can reach me via Google Talk (ncorr...@gmail.com) or M$N messenger (nicolascorrare...@fibertel.com.ar). I generally have my ipkall number transfered to my cellphone (+1 (253) 242 8786) so you can reach me in that number at any time. Welcome aboard. I've been checking the server distribution, you've done a great job. I'd love to help in new builds. Are you using a bugzilla or something like that? Think for the most part, the discussion goes on here, on the list. I've been thinking on maybe creating a firstboot module to perform the configuration that now is done via the /etc/sysconfig/olpc-scripts/domain_config script, or stuff like that. In any case.. let me know what can I do to help, and what tools are you using for the builds. Now that fits in really well with the proposed changes to xs-config, if your kickstart file doesn't turn off firstboot-xs, run it. The current install is more or less controlled by the xs-config rpm and the kickstart file, with the rpm providing a drop-in networking layout. We're going to split the configuration of the XS's services and networking into different packages, allowing us better use the stock OS configuration tools such as firstboot. That should allow us to present a more friendly installer. http://dev.laptop.org/git/projects/xs-config/ The tools used to spin up the installer are found at: http://dev.laptop.org/git/projects/xs-livecd/ My sandbox is at(ATM): http://24.79.179.72/pub/ Jerry ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Introduction / What can I do to help?
Jerry, The xs-livecd kickstart for f9 (from the git repo), is the same that you're using for Fedora 11? Is there anything in particular I can start working with? or shall I just build the Image, test what I like / dislike, and go ahead and fix stuff?. 2010/8/11 SgtPepper ncorr...@gmail.com Jerry, Are you in the IRC? in what channel? 2010/8/11 Jerry Vonau jvo...@shaw.ca On Tue, 2010-08-10 at 17:34 -0300, SgtPepper wrote: Hi Guys! I'm Nicolas Corrarello, from Argentina. I used to be a Red Hat Technical Support Engineer, now I'm working at Symantec. I met Walter Bender, and the XO a few years ago.. and since then I've been a believer. I'm a UNIX Sysadmin and Fedora Contributor and I'd love to help with the project in any way I can. If you need to contact me you can reach me via Google Talk (ncorr...@gmail.com) or M$N messenger (nicolascorrare...@fibertel.com.ar). I generally have my ipkall number transfered to my cellphone (+1 (253) 242 8786) so you can reach me in that number at any time. Welcome aboard. I've been checking the server distribution, you've done a great job. I'd love to help in new builds. Are you using a bugzilla or something like that? Think for the most part, the discussion goes on here, on the list. I've been thinking on maybe creating a firstboot module to perform the configuration that now is done via the /etc/sysconfig/olpc-scripts/domain_config script, or stuff like that. In any case.. let me know what can I do to help, and what tools are you using for the builds. Now that fits in really well with the proposed changes to xs-config, if your kickstart file doesn't turn off firstboot-xs, run it. The current install is more or less controlled by the xs-config rpm and the kickstart file, with the rpm providing a drop-in networking layout. We're going to split the configuration of the XS's services and networking into different packages, allowing us better use the stock OS configuration tools such as firstboot. That should allow us to present a more friendly installer. http://dev.laptop.org/git/projects/xs-config/ The tools used to spin up the installer are found at: http://dev.laptop.org/git/projects/xs-livecd/ My sandbox is at(ATM): http://24.79.179.72/pub/ Jerry ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
