Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, Jun 30, 2008 at 06:03:55PM -0700, [EMAIL PROTECTED] wrote: On Mon, 30 Jun 2008, Erik Garrison wrote: There is another primary value-add, which is a different operating system or window manager. To enable this value-add we could be distributing a minimal image for each of the popular linuxes and then distributing packages to install sugar, activites, other window managers, etc. Such packaging would be most useful to deployments engaged in customization. We already know that countries want to be able to run more traditional desktop environments. this sort of thing is drastic enough that the package-based updaters would not help much. A lot of work, yes. 'Drastic...' I guess I don't really understand what you mean. Aside from a significant amount of work I'm not sure what stands between us and this pattern of distribution. soapbox unless you have maintained the software for an embedded system, or a very similar focused set of systems you don't understand the trade-offs as much as you think you do. When things get small and tight the overhead of normal distros becomes a huge factor. also the 'small' risk of an upgrade failing and jamming the box up becomes unacceptable becouse you don't have hands available to touch the systems (if you even have people in the right place to be able to touch the systems) In the embedded space it is very common to use the approach that OLPC is useing. they provide a smapshot of the running system, and have a provision to load a second snapshot ans switch to it. My Tivo has been doing the same thing for about the last decade, and I've never had to send it in becouse an upgrade has failed (I have had to re-apply my own local modifications quite frequently as the upgrades wipe them out, but their stuff has just worked) /soapbox I will not claim any great expertise in the area, but I spent the last year working on an 'embedded' linux system that drives a gene sequencing device. In the case of the gene sequencer, our concern was primarily that it function exactly as specified and be 100% reliable without continued maintenance, so our software distribution plans centered around a disk image which could be used to flash the machine into a completely known and well-tested state. We are discussing software update systems in terms of a tradeoff between flexibility and guarantees about reliability. In the case of an appliance such as a gene sequencer or digital video recorder, this tradeoff is heavily weighted toward reliability. The usage of the system is well, and narrowly, defined. Thus software can be provided by the manufacturer which meets all the users' needs. The damage cause by software failures can be enormous. I agree that a headless appliance should not risk burying itself during software upgrades. Lab time is valuable, and television shows may not repeat. The usage patterns of embedded systems strike me as fundamentally distinct from those of the XO. In embedded systems we expect prescribed, well-defined patterns to make up the vast majority of usage. In the case of our laptop we observe highly diverse usage patterns. Children take pictures, draw, write, read, browse the web, and possibly even write a few applications of their own. Provided we do not prevent them, they will do everything which you can do with any roughly equivalent hardware. I am going to go out on a limb and make the claim that the XO is a general purpose computer and not an embedded system per se. If it does everything a general purpose computer can do, and is expressly designed to behave in such a way, what reason do we have to label it as an embedded system and treat it as such? A package management system can be used to solve a variety of problems which we are solving locally. I note: security (via security updates), software upgrades (to shorten the time delta between our bugfixes and their distribution), and configurability (consistently using a package manager plays nicely with downstream reconfiguration). The package manager need not preclude usage of a snapshot-based approach to initially set up systems or pull them out of broken states. We already ship a package manager. Why are we not using it? Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
The current OLPC design is heavily weighted toward reliability and maintainability. That's all. There's really nothing more to be said: we all agree that a full-fledged package manager would give more flexibility at the expense of less reliability. Therefore we are not using one at this time. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, Jun 30, 2008 at 9:03 PM, [EMAIL PROTECTED] wrote: what I would really like to see is for OLPC to not just release the snapshots, but to have a way for developers to get the rest of the build environment, complete with either the scripts, or command logs of what is done to go from the fedora build to the OLPC build. (This may already be available and I just don't know where to look) http://wiki.laptop.org/go/Pilgrim http://wiki.laptop.org/go/Building_custom_images I would then like to see someone maintain another base-level distro that can run on the OLPC, but not be based on Sugar so that people who want a normal distro can use one, and also so that various performance and usability issues can be identified as being caused by the software vs being caused by the limited hardware. there have been a few people who have made single-shot builds, but AFAIK nobody has maintained/improved the image after the initial 'here, see, it boots' announcement Partly this is because, once you have a traditional system in place and booting, you can use the package manager on that system to keep it continually up-to-date, so (IMO) there's no much need for me to keep re-releasing http://wiki.laptop.org/go/Installing_Debian_as_an_upgrade (for example). Red Hat developers used to routinely run full Fedora on the machines, but the very limited NAND space make this a tricky proposition, unless you plan on running on an external SD card. I did create a full Edubuntu installation on an SD card, but the result wasn't what I'd call child-friendly, mostly due to lack of a good activity chooser. As Martin points out, what's really been missing is a dedicated maintainer. Perhaps this is because (a) it's pretty easy to get started put a new distro on, but (b) it's really really hard to make something which will satisfy all those who want a non-Sugar distro. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 01:24:36PM -0700, [EMAIL PROTECTED] wrote: On Fri, 27 Jun 2008, Erik Garrison wrote: What functionality do we certainly lose by using a package management system as our default software distribution system? it's not that we loose functionality by using a package-based approach, it's that we increase complexity and eat up scarce development resources. You see the fact that this may work better with custom packages as a big advantage, I think that people who do custom packages can deal with the complexities themselves, and that they are very much the exception rather then the rule. by far the most common situation is, and is going to continue to be, the case where the laptops are running a standard image with no additional packages (note that this 'standard image' may be defined by the country, not OLPC, and therefor may contain some packages not in the OLPC image). it's only a small subset of the G1G1 and development machines that will have custom packages on them. I agree that a package-based approach increases the complexity of our software distribution processes. I observe, as you do, that we are already managing a complex deployment environment in which most large-scale deployments have their own customizations. Individual deployments have specific needs. We offer them monolithic images and also assistance in creating deployment-specific images. This deployment-by-deployment effort increases almost linearly with the number of large deployments that we engage. I suggest that a more sophisticated packaging system becomes useful as the effort expended on custom image creation reaches a certain level. It is not clear what that level is, but I doubt it lies at a scale of deployment much greater than where we currently stand. Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, 30 Jun 2008, Erik Garrison wrote: On Fri, Jun 27, 2008 at 01:24:36PM -0700, [EMAIL PROTECTED] wrote: On Fri, 27 Jun 2008, Erik Garrison wrote: What functionality do we certainly lose by using a package management system as our default software distribution system? it's not that we loose functionality by using a package-based approach, it's that we increase complexity and eat up scarce development resources. You see the fact that this may work better with custom packages as a big advantage, I think that people who do custom packages can deal with the complexities themselves, and that they are very much the exception rather then the rule. by far the most common situation is, and is going to continue to be, the case where the laptops are running a standard image with no additional packages (note that this 'standard image' may be defined by the country, not OLPC, and therefor may contain some packages not in the OLPC image). it's only a small subset of the G1G1 and development machines that will have custom packages on them. I agree that a package-based approach increases the complexity of our software distribution processes. I observe, as you do, that we are already managing a complex deployment environment in which most large-scale deployments have their own customizations. Individual deployments have specific needs. We offer them monolithic images and also assistance in creating deployment-specific images. This deployment-by-deployment effort increases almost linearly with the number of large deployments that we engage. I suggest that a more sophisticated packaging system becomes useful as the effort expended on custom image creation reaches a certain level. It is not clear what that level is, but I doubt it lies at a scale of deployment much greater than where we currently stand. how many different deployment builds do you think are being supported at this time? I think it's still in the single digits. I also think that before the complexity of things gets to the point where it's better to deploy to the laptops in a package-based system the number of builds directly supported by OLPC probably needs to get in the 30-40 range (or if they are indirectly supported, probably in the 100+ range) remember that for downstream customizers, OLPC is able to provide their development image (complete with the upstream package management tools in place, and the scripts to strip them out), so that those downstream customizers are able to take full advantage of the package based tools for creating their customized images that can then be published via the existing snapshot based infrastructure. the disagreement here is over the question of if OLPC should be supporting the end-user customizing the laptop (other then by installing activities). those who think that this should be happening see an obvious need for package-based tools, those who think that this should not be happening (that the customizations are at the country level or so) see much less of a need to drop down to the package level for OS management. David Lang ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, Jun 30, 2008 at 6:50 PM, [EMAIL PROTECTED] wrote: how many different deployment builds do you think are being supported at this time? I think it's still in the single digits. I expect this to change quite drastically soon. ... customizers are able to take full advantage of the package based tools for creating their customized images that can then be published via the existing snapshot based infrastructure. There are bitfrost issues that need to be addressed there IIRC. the disagreement here is over the question of if OLPC should be supporting the end-user customizing the laptop (other then by installing activities). End user being local educational authorities - yes, I think we must. But I'm the XS guy so I'll talkwith authority about the XS and say: Hell yeah! I don't consider the (XS) product usable by those clients without long-term supportable means of customising it. (The XS is far from finished, so these are in the works however...) I won't claim this is an easy task though - XS or XO. cheers, m -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, Jun 30, 2008 at 6:58 PM, Martin Langhoff [EMAIL PROTECTED] wrote: On Mon, Jun 30, 2008 at 6:50 PM, [EMAIL PROTECTED] wrote: how many different deployment builds do you think are being supported at this time? I think it's still in the single digits. I expect this to change quite drastically soon. Let's not get ahead of ourselves. Someday we may be able to support lots of different configurations. Today, we will only be successful if we can limit the number of configurations in the field to a testable number (and then test them!). That's the whole point of the core OS / activities split. Do whatever you like on the activities side, because that's your primary value-add (you == countries). We can also technically ensure that one bad activity won't spoil the whole bunch. We will in turn provide you with a core OS which is as stable and functional as we know how. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, Jun 30, 2008 at 07:10:23PM -0400, C. Scott Ananian wrote: On Mon, Jun 30, 2008 at 6:58 PM, Martin Langhoff [EMAIL PROTECTED] wrote: On Mon, Jun 30, 2008 at 6:50 PM, [EMAIL PROTECTED] wrote: how many different deployment builds do you think are being supported at this time? I think it's still in the single digits. I expect this to change quite drastically soon. Let's not get ahead of ourselves. Someday we may be able to support lots of different configurations. Today, we will only be successful if we can limit the number of configurations in the field to a testable number (and then test them!). In your opinion what is a 'testable number'? That's the whole point of the core OS / activities split. Do whatever you like on the activities side, because that's your primary value-add (you == countries). We can also technically ensure that one bad activity won't spoil the whole bunch. We will in turn provide you with a core OS which is as stable and functional as we know how. There is another primary value-add, which is a different operating system or window manager. To enable this value-add we could be distributing a minimal image for each of the popular linuxes and then distributing packages to install sugar, activites, other window managers, etc. Such packaging would be most useful to deployments engaged in customization. We already know that countries want to be able to run more traditional desktop environments. Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, 30 Jun 2008, Erik Garrison wrote: On Mon, Jun 30, 2008 at 07:10:23PM -0400, C. Scott Ananian wrote: On Mon, Jun 30, 2008 at 6:58 PM, Martin Langhoff [EMAIL PROTECTED] wrote: On Mon, Jun 30, 2008 at 6:50 PM, [EMAIL PROTECTED] wrote: how many different deployment builds do you think are being supported at this time? I think it's still in the single digits. I expect this to change quite drastically soon. Let's not get ahead of ourselves. Someday we may be able to support lots of different configurations. Today, we will only be successful if we can limit the number of configurations in the field to a testable number (and then test them!). In your opinion what is a 'testable number'? this is a very squishable number, it's going to depend to a large degree on how different the builds are. frankly, based on what I've been seeing, for the past 6 months the 'testable number' has been 1 (several more have been deployed, but the resources have not been allocated to really test any of them) note that I am speaking for myself. That's the whole point of the core OS / activities split. Do whatever you like on the activities side, because that's your primary value-add (you == countries). We can also technically ensure that one bad activity won't spoil the whole bunch. We will in turn provide you with a core OS which is as stable and functional as we know how. There is another primary value-add, which is a different operating system or window manager. To enable this value-add we could be distributing a minimal image for each of the popular linuxes and then distributing packages to install sugar, activites, other window managers, etc. Such packaging would be most useful to deployments engaged in customization. We already know that countries want to be able to run more traditional desktop environments. this sort of thing is drastic enough that the package-based updaters would not help much. soapbox unless you have maintained the software for an embedded system, or a very similar focused set of systems you don't understand the trade-offs as much as you think you do. When things get small and tight the overhead of normal distros becomes a huge factor. also the 'small' risk of an upgrade failing and jamming the box up becomes unacceptable becouse you don't have hands available to touch the systems (if you even have people in the right place to be able to touch the systems) In the embedded space it is very common to use the approach that OLPC is useing. they provide a smapshot of the running system, and have a provision to load a second snapshot ans switch to it. My Tivo has been doing the same thing for about the last decade, and I've never had to send it in becouse an upgrade has failed (I have had to re-apply my own local modifications quite frequently as the upgrades wipe them out, but their stuff has just worked) /soapbox what I would really like to see is for OLPC to not just release the snapshots, but to have a way for developers to get the rest of the build environment, complete with either the scripts, or command logs of what is done to go from the fedora build to the OLPC build. (This may already be available and I just don't know where to look) I would then like to see someone maintain another base-level distro that can run on the OLPC, but not be based on Sugar so that people who want a normal distro can use one, and also so that various performance and usability issues can be identified as being caused by the software vs being caused by the limited hardware. there have been a few people who have made single-shot builds, but AFAIK nobody has maintained/improved the image after the initial 'here, see, it boots' announcement David Lang ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, Jun 30, 2008 at 9:03 PM, [EMAIL PROTECTED] wrote: what I would really like to see is for OLPC to not just release the (note: I think what you are asking for is available) ... I would then like to see someone maintain another base-level distro that Guys, it'd be great to run all the distros, all the WMs out there, and Hurd too. It is just a ton of work, which has not been interesting enough for anybody to get it done.Let's move quickly from rethoric to working code, or drop the conversation. cheers, m -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
We separated out the activities so that we could push the testing and localization of activities out to the country. How many activities can they test? As many as they have people and time for. It is in the deployment guide (and starting to get good discussion from sales/deployment people) that a country must take responsibility for choosing, testing, and localizing activities and content. We will make the combined OS+Activities image for them, but our testing is limited to ensuring that the signed image loads and we'll do the equivalent of 10 minutes of testing (this is not exact, but meant to give you the idea that we won't spend days or even hours testing each customized image -- ideally this testing is automated so we can do 30 different country images in a day or in parallel). The country has to have done the testing to enusre proper operation of the activities and the correct language, etc.. OLPC's testing needs to be limited or there is no scalability. In the same way, we have set a precedent with Uruguay, that if they country wants to make changes to the code base that they need to send a developer to 1CC to learn how to work with our processes, our developers, our repositories, etc. and to make sure their features and bug fixes get in releases. And they have to do their own testing. If they do all that, then we will sign their builds, do the same '10 minute' test and be able to support them when they have to make more changes in the future. We won't fix their code, but we will encourage them to contribute as we do other developers. Note: for the G1G1 program OLPC has to choose the activities, ensure that the testing gets done (hopefully with community help), and take some responsibility for the activities that we ship. Kim On Mon, Jun 30, 2008 at 8:23 PM, Erik Garrison [EMAIL PROTECTED] wrote: On Mon, Jun 30, 2008 at 07:10:23PM -0400, C. Scott Ananian wrote: On Mon, Jun 30, 2008 at 6:58 PM, Martin Langhoff [EMAIL PROTECTED] wrote: On Mon, Jun 30, 2008 at 6:50 PM, [EMAIL PROTECTED] wrote: how many different deployment builds do you think are being supported at this time? I think it's still in the single digits. I expect this to change quite drastically soon. Let's not get ahead of ourselves. Someday we may be able to support lots of different configurations. Today, we will only be successful if we can limit the number of configurations in the field to a testable number (and then test them!). In your opinion what is a 'testable number'? That's the whole point of the core OS / activities split. Do whatever you like on the activities side, because that's your primary value-add (you == countries). We can also technically ensure that one bad activity won't spoil the whole bunch. We will in turn provide you with a core OS which is as stable and functional as we know how. There is another primary value-add, which is a different operating system or window manager. To enable this value-add we could be distributing a minimal image for each of the popular linuxes and then distributing packages to install sugar, activites, other window managers, etc. Such packaging would be most useful to deployments engaged in customization. We already know that countries want to be able to run more traditional desktop environments. Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Mon, 30 Jun 2008, Martin Langhoff wrote: On Mon, Jun 30, 2008 at 9:03 PM, [EMAIL PROTECTED] wrote: what I would really like to see is for OLPC to not just release the (note: I think what you are asking for is available) I thought that it might me. ... I would then like to see someone maintain another base-level distro that Guys, it'd be great to run all the distros, all the WMs out there, and Hurd too. It is just a ton of work, which has not been interesting enough for anybody to get it done.Let's move quickly from rethoric to working code, or drop the conversation. please note that I was making the request for there to be one alternate, not requesting any in particular, and definatnly not asking for all of them. I was also trying to be clear that this is not something for OLPC to do, but that should be done by someone else. the fact that OLPC has now pushed the kernel modifications upstream will be a drastic help for this sort of thing. I may end up doing this later on, but right now I'm working on getting ready to go do fireworks for the rest of the week, so it will definantly not be me this weekd ;-) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 8:47 PM, Bert Freudenberg [EMAIL PROTECTED] wrote: Am 27.06.2008 um 20:50 schrieb Erik Garrison: We already have yum installed on the XO. Only in the devel builds. You might never have wondered what the devel_ prefix means in http://pilgrim.laptop.org/~pilgrim/olpc/streams/update.1/build708/devel_jffs2/ but there was a time when there was a user build and a developer build, and the user build did not include the non-essential tools like yum. Might be interesting to see how much flash could be saved nowadays if those tools were removed. It is a significant amount of storage -- the RPM database alone is around 16M -- but the consensus at 1cc seems to be that it is worth it, since it's hard to install RPM later if you don't already have RPM installed, etc. The current list of OLPC_DEVEL_PACKAGES at: http://dev.laptop.org/git?p=users/cscott/pilgrim;a=blob;f=streams.d/olpc-development.stream;hb=joyride#l169 includes: rpm yum yum-metadata-parser openssh-server wget xterm which file tree xorg-x11-twm gdb ltrace strace pciutils bzip2 unzip lrzsz xorg-x11-apps dbench Of these, only xorg-x11-twm, pciutils, dbench, xorg-x11-apps, and xterm strike me as obvious candidates for pruning. What do other folk feel? --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
c. scott ananian wrote: The current list of OLPC_DEVEL_PACKAGES at: http://dev.laptop.org/git?p=users/cscott/pilgrim;a=blob;f=streams.d/olpc-develop ment.stream;hb=joyride#l169 includes: rpm yum yum-metadata-parser openssh-server wget xterm which file tree xorg-x11-twm gdb ltrace strace pciutils bzip2 unzip lrzsz xorg-x11-apps dbench Of these, only xorg-x11-twm, pciutils, dbench, xorg-x11-apps, and xterm strike me as obvious candidates for pruning. What do other folk feel? which is redundant with type -a in the shell (or type -ap if you're being picky). tree is so small it's hardly worth discussing, but i'm not sure how it gives info not available in other ways (i.e. find, xargs, ls). (but i've not used tree much.) when/how would lrzsz be useful? paul =- paul fox, [EMAIL PROTECTED] ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 10:45 AM, [EMAIL PROTECTED] wrote: which is redundant with type -a in the shell (or type -ap if you're being picky). But is 'which' large enough to merit the effort? when/how would lrzsz be useful? Many folks using Windows as their primary home OS find themselves with ssh clients which don't support scp. (Judging from my girlfriend's use) they seem to use lrzsz as a replacement for scp, and the popular windows client (sorry, don't know the name) implements the other size of the zmodem transfer. Yes, it seems incredibly baroque to me -- why didn't the windows client just implement scp? who thought zmodem was a good idea? -- but that's the way things go. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
c. scott ananian wrote: On Sat, Jun 28, 2008 at 10:45 AM, [EMAIL PROTECTED] wrote: which is redundant with type -a in the shell (or type -ap if you're being picky). But is 'which' large enough to merit the effort? probably not. i was mainly being pedantic. :-) when/how would lrzsz be useful? Many folks using Windows as their primary home OS find themselves with ssh clients which don't support scp. (Judging from my girlfriend's use) they seem to use lrzsz as a replacement for scp, and the popular windows client (sorry, don't know the name) implements the other size of the zmodem transfer. okay. as long as there's a real use. paul =- paul fox, [EMAIL PROTECTED] ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 5:21 PM, Michael Stone [EMAIL PROTECTED] wrote: We chose a monolithic update solution because of several deficiencies, *for our primary use case*, of all package-based upgrade solutions with which we were familiar at the time. Package-based update solutions with which we were familiar: ... These were some of the considerations that led to the development of the olpc-update technology. All reasonable, and the snapshot based approach has certain key advantages for some uses. There is one thing that really bothers me, however, and makes me suspect that we cannot actually use the snapshot approach long term: that it completely bypasses rpm's preinst/postinst scripts. The F7 to F9 update surely has interesting and important pre/post inst scripts, some of which may affect user data. We are skipping them completely. I know this is unworkable for the server (packages routinely use post-inst to perform database format conversion and such). cheers, m -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 5:46 PM, Erik Garrison [EMAIL PROTECTED] wrote: Let's say we dist-upgrade our system. It's in an unbootable state. In our current situation we attempt to avoid: * can leave the system in an inconsistent or even unbootable state on failure. ... by holding around the most recent distribution snapshot. A feature is provided to select that older snapshot at boot time. Correct? I've done it several times in the month+ since I've been at OLPC. That's reasonably easy with cp -aln, but all the other actions you mention below involve a ton of work. This is a deficiency of package managers which, if solved by us and I don't think it's trivial to. We are already doing too much to reinvent unix/linux and that takes from our effort to provide an education platform. We have to be lazy on this front. It's not a laptop or a linux project ;-) (Of course, we have to do work on linux, and ship a laptop to achieve our education goals.) * do not adequately verify the integrity of the resulting filesystem. (We have actually detected two filesystem data corruption bugs as a result of carefully auditing our filesystem consistency via the update process.) I'm sure rpm has a debsums equivalent. cheers, m -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 11:26 AM, Martin Langhoff [EMAIL PROTECTED] wrote: On Fri, Jun 27, 2008 at 5:21 PM, Michael Stone [EMAIL PROTECTED] wrote: We chose a monolithic update solution because of several deficiencies, *for our primary use case*, of all package-based upgrade solutions with which we were familiar at the time. Package-based update solutions with which we were familiar: ... These were some of the considerations that led to the development of the olpc-update technology. All reasonable, and the snapshot based approach has certain key advantages for some uses. There is one thing that really bothers me, however, and makes me suspect that we cannot actually use the snapshot approach long term: that it completely bypasses rpm's preinst/postinst scripts. That is not the case. rpm's preinst/postinst script are run when the image is built. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 11:35 AM, Martin Langhoff [EMAIL PROTECTED] wrote: This is a deficiency of package managers which, if solved by us and I don't think it's trivial to. We are already doing too much to reinvent unix/linux and that takes from our effort to provide an education platform. We have to be lazy on this front. It's not a laptop or a linux project ;-) It is, however, a *logistics* project, whether we like it or not, and providing rock-solid upgrades with zero tolerance for failure is part of that mandate. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 11:38 AM, C. Scott Ananian [EMAIL PROTECTED] wrote: All reasonable, and the snapshot based approach has certain key advantages for some uses. There is one thing that really bothers me, however, and makes me suspect that we cannot actually use the snapshot approach long term: that it completely bypasses rpm's preinst/postinst scripts. That is not the case. rpm's preinst/postinst script are run when the image is built. Yes, but this happens in absense of real user data or configuration. Which means that all those fancy conditionals (are we upgrading from an earlier version? update the format in which we store the data!) are skipped. The need for those scripts is somewhat lower on our platform so far, but without them there is a whole lot of stuff we cannot deal with gracefully. cheers, m -- [EMAIL PROTECTED] [EMAIL PROTECTED] -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 11:59 AM, Martin Langhoff [EMAIL PROTECTED] wrote: On Sat, Jun 28, 2008 at 11:38 AM, C. Scott Ananian [EMAIL PROTECTED] wrote: All reasonable, and the snapshot based approach has certain key advantages for some uses. There is one thing that really bothers me, however, and makes me suspect that we cannot actually use the snapshot approach long term: that it completely bypasses rpm's preinst/postinst scripts. That is not the case. rpm's preinst/postinst script are run when the image is built. Yes, but this happens in absense of real user data or configuration. Which means that all those fancy conditionals (are we upgrading from an earlier version? update the format in which we store the data!) are skipped. The need for those scripts is somewhat lower on our platform so far, but without them there is a whole lot of stuff we cannot deal with gracefully. Yes, exactly: olpc-update has been designed so that the need for those scripts is *zero*. You get a clean install every time, guaranteed. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 01:09:58PM -0400, C. Scott Ananian wrote: Yes, exactly: olpc-update has been designed so that the need for those scripts is *zero*. You get a clean install every time, guaranteed. Care to explain the existence and functioning of olpc-configure? Michael ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 1:29 PM, Michael Stone [EMAIL PROTECTED] wrote: On Sat, Jun 28, 2008 at 01:09:58PM -0400, C. Scott Ananian wrote: Yes, exactly: olpc-update has been designed so that the need for those scripts is *zero*. You get a clean install every time, guaranteed. Care to explain the existence and functioning of olpc-configure? olpc-configure exists because /home/olpc is not managed by olpc-update, and to do things like http://dev.laptop.org/ticket/6432. (Also, in the case of /etc/sysconfig/i18n, as a means for bernie to make build image changes without patching pilgrim.) I fail to see the relevance. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 02:19:09PM -0400, C. Scott Ananian wrote: On Sat, Jun 28, 2008 at 1:29 PM, Michael Stone [EMAIL PROTECTED] wrote: Care to explain the existence and functioning of olpc-configure? olpc-configure exists because /home/olpc is not managed by olpc-update, and to do things like http://dev.laptop.org/ticket/6432. (Also, in the case of /etc/sysconfig/i18n, as a means for bernie to make build image changes without patching pilgrim.) I fail to see the relevance. I see olpc-configure as a poor replacement for post-installation / configuration hooks. Michael ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 2:34 PM, Michael Stone [EMAIL PROTECTED] wrote: On Sat, Jun 28, 2008 at 02:19:09PM -0400, C. Scott Ananian wrote: On Sat, Jun 28, 2008 at 1:29 PM, Michael Stone [EMAIL PROTECTED] wrote: Care to explain the existence and functioning of olpc-configure? olpc-configure exists because /home/olpc is not managed by olpc-update, and to do things like http://dev.laptop.org/ticket/6432. (Also, in the case of /etc/sysconfig/i18n, as a means for bernie to make build image changes without patching pilgrim.) I fail to see the relevance. I see olpc-configure as a poor replacement for post-installation / configuration hooks. And I see RPM post-installation hooks as an imperfect solution to a complex problem we can bypass altogether. Please compare the number of lines of code in olpc-configure vs. the summed number of lines of code in RPM post-installation scripts for all packages in our build. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC XO Opera browser as Sugar activity
Am 27.06.2008 um 04:05 schrieb Stevens: Hi Bert, all -- On Jun 25, 2008, at 11:37 PM, Bert Freudenberg wrote: Sure. Do not use the Python script. You don't want to run a Python activity but a native one - otherwise you get two windows, the empty one opened by Python and the real one by Opera. Instead, you only need a tiny shell script that preloads a little library, which may work with Opera: http://lists.laptop.org/pipermail/devel/2008-January/009387.html I tried a shell script and it works: #!/bin/sh while [ -n $2 ] ; do case $1 in -b | --bundle-id) export SUGAR_BUNDLE_ID=$2 ;; -a | --activity-id) export SUGAR_ACTIVITY_ID=$2 ;; -o | --object-id) export SUGAR_OBJECT_ID=$2 ;; -u | --uri) export SUGAR_URI=$2 ;; *) echo unknown argument $1 $2 ;; esac shift;shift done export LD_PRELOAD=$SUGAR_BUNDLE_PATH/lib/libsugarize.so export NET_WM_NAME=OperaBrowse exec opera -notrayicon -personaldir $SUGAR_ACTIVITY_ROOT/data This starts Opera from Sugar and it runs correctly. Thanks for the direction. -Peter See - it's not hard ;) Now, all of this was known before. Could you put links on the wiki or where-ever you unsuccessfully tried to get information from, so the next person will have an easier time? I am too involved to imagine where newcomers look for information, since making non-Python activities work is part of my day job ... - Bert - And since you're investigating this, what would be great is if you could re-package Opera as a real activity. That means, just move the Opera files that are normally installed in the system into the bundle itself, and setup the environment in the shell script so it works from that non-standard directory. Then zip up the activity directory, rename to .xo, and we have a real Opera bundle :) That way it would also survive a system upgrade which erases all manually installed RPMs. - Bert - ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
What's the logic for having updates erase all manually installed RPMs? A couple of Support-Gangers and myself were talking about ways to remedy this. We came up with the following: - alias rpm -i $FILE to rpm -i $FILE cp FILE $HOME/.rpms$/FILE with a script on update that runs rpm -i $HOME/.rpms/* - have a script that constantly monitors $HOME/.bash_history for yum install $PROGRAM formatted files, then echos the name of $PROGRAM to $HOME/.rpms/installed, but removes it from that list if/when it sees yum remove $PROGRAM On update, yum install $(cat $HOME/.rpms/installed) is run. - rpm -qa $HOME/.rpms/clean could be run on install - rpm -qa $HOME/.rpms/custom could be run before update - a simple file compare program (python or python-parsed diff output) would be used to generate a file with which yum install $(cat $FILE) could be used thoughts? -- Ian Daniher -- OLPC Support Volunteer OLPCinci Repair Center Coordinator -- [EMAIL PROTECTED] Skype : it.daniher irc.freenode.com: Ian_Daniher ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, 27 Jun 2008, Ian Daniher wrote: What's the logic for having updates erase all manually installed RPMs? the updates aren't package based, they are snapshot based. as a result when you apply an update it alters everything outside of /home. when you have a very standardized system image this can end up being far more efficiant to do then a package-based approach (much faster, using less bandwidth, less CPU, and less disk space) It can also handle more drastic changes as it doesn't care what was in place before. with the snapshot based approach you can upgrade from a OLPC build to a debian build to a ubuntu build using exactly the same steps (and taking the same time) as going from OLPC build 1 to OLPC build 1.0.1 David Lang A couple of Support-Gangers and myself were talking about ways to remedy this. We came up with the following: - alias rpm -i $FILE to rpm -i $FILE cp FILE $HOME/.rpms$/FILE with a script on update that runs rpm -i $HOME/.rpms/* - have a script that constantly monitors $HOME/.bash_history for yum install $PROGRAM formatted files, then echos the name of $PROGRAM to $HOME/.rpms/installed, but removes it from that list if/when it sees yum remove $PROGRAM On update, yum install $(cat $HOME/.rpms/installed) is run. - rpm -qa $HOME/.rpms/clean could be run on install - rpm -qa $HOME/.rpms/custom could be run before update - a simple file compare program (python or python-parsed diff output) would be used to generate a file with which yum install $(cat $FILE) could be used thoughts? ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 01:23:46PM -0400, Ian Daniher wrote: What's the logic for having updates erase all manually installed RPMs? A couple of Support-Gangers and myself were talking about ways to remedy this. We came up with the following: - alias rpm -i $FILE to rpm -i $FILE cp FILE $HOME/.rpms$/FILE with a script on update that runs rpm -i $HOME/.rpms/* - have a script that constantly monitors $HOME/.bash_history for yum install $PROGRAM formatted files, then echos the name of $PROGRAM to $HOME/.rpms/installed, but removes it from that list if/when it sees yum remove $PROGRAM On update, yum install $(cat $HOME/.rpms/installed) is run. - rpm -qa $HOME/.rpms/clean could be run on install - rpm -qa $HOME/.rpms/custom could be run before update - a simple file compare program (python or python-parsed diff output) would be used to generate a file with which yum install $(cat $FILE) could be used thoughts? We should move away from using olpc-update to upgrade systems. We should not implement this or any hack to preserve manually installed rpms through olpc-updates. Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? We already have yum installed on the XO. Why are we not using it to implement software update procedures? There are several reasons which occur to me: 1) OLPC software developers mostly use apt-based systems and have been slow to adopt rpm-based ones. 2) Many have expressed frustration with yum, the user-friendly package manager interface to rpm. Even simple operations (yum search) will download megabyte-order header files every time these headers change unless yum is instructed not to (with the '-C' flag). More problematically, rpm refers to dependencies on a file-by-file level, instead of package level, increasing the space and processing complexity of rpm package management operations relative to deb-based tools, which track dependencies on a (coarser) package level. 3) The tools we have created work well enough to not halt software development and deployment. Therefore there has been insufficient pressure to move away from them. I don't think any of these reasons outweighs the benefits of migrating to rpm/yum for software distribution. Objections? Thoughts? Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 12:20 AM, Erik Garrison [EMAIL PROTECTED] wrote: On Fri, Jun 27, 2008 at 01:23:46PM -0400, Ian Daniher wrote: What's the logic for having updates erase all manually installed RPMs? A couple of Support-Gangers and myself were talking about ways to remedy this. We came up with the following: - alias rpm -i $FILE to rpm -i $FILE cp FILE $HOME/.rpms$/FILE with a script on update that runs rpm -i $HOME/.rpms/* - have a script that constantly monitors $HOME/.bash_history for yum install $PROGRAM formatted files, then echos the name of $PROGRAM to $HOME/.rpms/installed, but removes it from that list if/when it sees yum remove $PROGRAM On update, yum install $(cat $HOME/.rpms/installed) is run. - rpm -qa $HOME/.rpms/clean could be run on install - rpm -qa $HOME/.rpms/custom could be run before update - a simple file compare program (python or python-parsed diff output) would be used to generate a file with which yum install $(cat $FILE) could be used thoughts? We should move away from using olpc-update to upgrade systems. We should not implement this or any hack to preserve manually installed rpms through olpc-updates. Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? We already have yum installed on the XO. Why are we not using it to implement software update procedures? There are several reasons which occur to me: 1) OLPC software developers mostly use apt-based systems and have been slow to adopt rpm-based ones. 2) Many have expressed frustration with yum, the user-friendly package manager interface to rpm. Even simple operations (yum search) will download megabyte-order header files every time these headers change unless yum is instructed not to (with the '-C' flag). More problematically, rpm refers to dependencies on a file-by-file level, instead of package level, increasing the space and processing complexity of rpm package management operations relative to deb-based tools, which track dependencies on a (coarser) package level. 3) The tools we have created work well enough to not halt software development and deployment. Therefore there has been insufficient pressure to move away from them. I don't think any of these reasons outweighs the benefits of migrating to rpm/yum for software distribution. Objections? Thoughts? Something that comes to my mind is the potential memory usage issues some people have been seeing while trying to use yum. A description of one such case is in http://www.nabble.com/Re%3A-Moving-to-metacity-with-composition-(was%3A-Preparing-for-the-feature-freeze)-p17621634.html Thanks, Sayamindu -- Sayamindu Dasgupta [http://sayamindu.randomink.org/ramblings] ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, 27 Jun 2008, Erik Garrison wrote: We should move away from using olpc-update to upgrade systems. We should not implement this or any hack to preserve manually installed rpms through olpc-updates. Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? every existing package management system makes assumptions about how large an upgrade you are making. even apt (historicly one of the best long-term systems) runs into significant problems if the upgrade that you are making is too large, frequently without being able to identify the problem ahead of time. with yum, can you (in one step) do an upgrade from Fedora 7 to Fedora 9? I don't think it can. the snapshot based approach has headaches, but the one huge advantage that it does have is the ability to do the upgrade no matter what the condition of the old system image is (including the possibility that the system image is corrupt) David Lang ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 12:17:57PM -0700, [EMAIL PROTECTED] wrote: On Fri, 27 Jun 2008, Erik Garrison wrote: We should move away from using olpc-update to upgrade systems. We should not implement this or any hack to preserve manually installed rpms through olpc-updates. Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? every existing package management system makes assumptions about how large an upgrade you are making. even apt (historicly one of the best long-term systems) runs into significant problems if the upgrade that you are making is too large, frequently without being able to identify the problem ahead of time. with yum, can you (in one step) do an upgrade from Fedora 7 to Fedora 9? I don't think it can. It can't be done in one step, but people use yum for distribution upgrades already: http://www.howtoforge.com/upgrading-fedora7-desktop-to-fedora8 Recently I have been completing my Ubuntu distribution-level upgrades in one step. So we should know that it is possible that such upgrades can happen within the scope of a package manager. the snapshot based approach has headaches, but the one huge advantage that it does have is the ability to do the upgrade no matter what the condition of the old system image is (including the possibility that the system image is corrupt) IMO, the snapshot approach has headaches for everything *but* distribution upgrades. How, for instance, do we issue security updates? How do we push small bugfixes? This is problematic for developers, users, and particularly the support staff which have to sheperd users through monolithic upgrade processes. Despite any apparent vitriol to the contrary, I am not suggesting we get rid of snapshot based upgrades. They can obviously coexist with a well-maintained package management system vector for upgrades. It will always remain a useful sledgehammer. However, if we wish to preserve custom packages across such upgrades I suggest we can use a system such as yum rather than hacking olpc-update to play nice with the packages. Evidence strongly suggests that this is possible. What functionality do we certainly lose by using a package management system as our default software distribution system? Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, 27 Jun 2008, Erik Garrison wrote: On Fri, Jun 27, 2008 at 12:17:57PM -0700, [EMAIL PROTECTED] wrote: On Fri, 27 Jun 2008, Erik Garrison wrote: We should move away from using olpc-update to upgrade systems. We should not implement this or any hack to preserve manually installed rpms through olpc-updates. Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? every existing package management system makes assumptions about how large an upgrade you are making. even apt (historicly one of the best long-term systems) runs into significant problems if the upgrade that you are making is too large, frequently without being able to identify the problem ahead of time. with yum, can you (in one step) do an upgrade from Fedora 7 to Fedora 9? I don't think it can. It can't be done in one step, but people use yum for distribution upgrades already: http://www.howtoforge.com/upgrading-fedora7-desktop-to-fedora8 Recently I have been completing my Ubuntu distribution-level upgrades in one step. So we should know that it is possible that such upgrades can happen within the scope of a package manager. ubuntu uses apt, doing an upgrade from one ubuntu/debian release to the next works well, but if you tried to skip a couple releases and then do the upgrade you may run into 'interesting' problems (not becouse it can't work, but becouse the debian/ubuntu developers don't take the time and effort to define all the gotchas for the multiple step upgrades to tell apt how to do them properly) OLPC has a much smaller developer base, and are already questioning how many releases they can support. the problem of how many upgrade scenerios to support, and what to do for the ones they don't support is an area they really don't have the manpower to tackle the snapshot based approach has headaches, but the one huge advantage that it does have is the ability to do the upgrade no matter what the condition of the old system image is (including the possibility that the system image is corrupt) IMO, the snapshot approach has headaches for everything *but* distribution upgrades. How, for instance, do we issue security updates? How do we push small bugfixes? that's easy, you treat them like distrubution upgrades. the only place this runs into problems is where people have added their own packages, and it's impossible to anticipate what will break those without extensive testing of all the combinations, so changing the upgrade mechanism isn't going to help. This is problematic for developers, users, and particularly the support staff which have to sheperd users through monolithic upgrade processes. it's a lot easier to shepherd people through the monolithic upgrade process that always works then it is to have to deal with multiple different upgrade processes, and teach people which one to use when. Despite any apparent vitriol to the contrary, I am not suggesting we get rid of snapshot based upgrades. They can obviously coexist with a well-maintained package management system vector for upgrades. It will always remain a useful sledgehammer. However, if we wish to preserve custom packages across such upgrades I suggest we can use a system such as yum rather than hacking olpc-update to play nice with the packages. Evidence strongly suggests that this is possible. What functionality do we certainly lose by using a package management system as our default software distribution system? it's not that we loose functionality by using a package-based approach, it's that we increase complexity and eat up scarce development resources. You see the fact that this may work better with custom packages as a big advantage, I think that people who do custom packages can deal with the complexities themselves, and that they are very much the exception rather then the rule. by far the most common situation is, and is going to continue to be, the case where the laptops are running a standard image with no additional packages (note that this 'standard image' may be defined by the country, not OLPC, and therefor may contain some packages not in the OLPC image). it's only a small subset of the G1G1 and development machines that will have custom packages on them. for the record, there are a couple packages that I install on my XOs, I just store the .rpm files on a SD card and have a script in my home directory that installs them. after each upgrade I open a terminal and run that script David Lang ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Sat, Jun 28, 2008 at 12:30:14AM +0530, Sayamindu Dasgupta wrote: On Sat, Jun 28, 2008 at 12:20 AM, Erik Garrison [EMAIL PROTECTED] wrote: We already have yum installed on the XO. Why are we not using it to implement software update procedures? There are several reasons which occur to me: 1) OLPC software developers mostly use apt-based systems and have been slow to adopt rpm-based ones. 2) Many have expressed frustration with yum, the user-friendly package manager interface to rpm. Even simple operations (yum search) will download megabyte-order header files every time these headers change unless yum is instructed not to (with the '-C' flag). More problematically, rpm refers to dependencies on a file-by-file level, instead of package level, increasing the space and processing complexity of rpm package management operations relative to deb-based tools, which track dependencies on a (coarser) package level. 3) The tools we have created work well enough to not halt software development and deployment. Therefore there has been insufficient pressure to move away from them. I don't think any of these reasons outweighs the benefits of migrating to rpm/yum for software distribution. Objections? Thoughts? Something that comes to my mind is the potential memory usage issues some people have been seeing while trying to use yum. A description of one such case is in http://www.nabble.com/Re%3A-Moving-to-metacity-with-composition-(was%3A-Preparing-for-the-feature-freeze)-p17621634.html I believe that this is somewhat related to (2), but because of our uniquely low system memory it is XO-specific. I have experienced similar problems (OOM) using apt on an XO running debian. I have also watched yum fail repeatedly when working on slightly unreliable network connections (and on restart repeat most of the work it had already done). These issues should be considered fixable bugs and not showstoppers. I mentioned this to Dennis Gilmore and he said that the OOM issue was known. Seth Vidal (yum author) has been provided an XO on which to test yum, so perhaps we could raise the memory pressure issue with him to see if we he can resolve it. Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 02:50:34PM -0400, Erik Garrison wrote: Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? We chose a monolithic update solution because of several deficiencies, *for our primary use case*, of all package-based upgrade solutions with which we were familiar at the time. Package-based update solutions with which we were familiar: * can leave the system in an inconsistent or even unbootable state on failure. * ask users questions during the update process. * are not adequately bandwidth efficient. (They make no use of bitpatterns which are already available on the target system.) * do not adequately verify the integrity of the resulting filesystem. (We have actually detected two filesystem data corruption bugs as a result of carefully auditing our filesystem consistency via the update process.) * do not innately offer the user a 'big undo' button that permits the user to try out large changes with confidence that they can easily return to previous system states. These were some of the considerations that led to the development of the olpc-update technology. Now, clearly, there are several technically adept users who would like to be able to use their favorite package management tools to consume our software. We _are_ interested in supporting this use case, for example by modifying our build procedures to produce package repos 'suitable for use in upgrade procedures'. The hard question is: what more can we do? Do you have other thoughts on either: 1) other ways that we could address the aforementioned inadequacies of package-based updaters for our use case? 2) other things we can do to integrate package-based updaters into our current system? Regards, Michael ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, 27 Jun 2008, Michael Stone wrote: On Fri, Jun 27, 2008 at 02:50:34PM -0400, Erik Garrison wrote: Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? We chose a monolithic update solution because of several deficiencies, *for our primary use case*, of all package-based upgrade solutions with which we were familiar at the time. Package-based update solutions with which we were familiar: * can leave the system in an inconsistent or even unbootable state on failure. * ask users questions during the update process. * are not adequately bandwidth efficient. (They make no use of bitpatterns which are already available on the target system.) * do not adequately verify the integrity of the resulting filesystem. (We have actually detected two filesystem data corruption bugs as a result of carefully auditing our filesystem consistency via the update process.) * do not innately offer the user a 'big undo' button that permits the user to try out large changes with confidence that they can easily return to previous system states. These were some of the considerations that led to the development of the olpc-update technology. Now, clearly, there are several technically adept users who would like to be able to use their favorite package management tools to consume our software. We _are_ interested in supporting this use case, for example by modifying our build procedures to produce package repos 'suitable for use in upgrade procedures'. The hard question is: what more can we do? Do you have other thoughts on either: 1) other ways that we could address the aforementioned inadequacies of package-based updaters for our use case? based on my knowledge of them I would not depend on them for your primary use-case 2) other things we can do to integrate package-based updaters into our current system? one thing that would be handy is if a machine with a developer key would look in a specific place in /home for a list of packages to install in addition to the image. It could then install these packages (either via the net, or via local .rpm files) I suspect that you are using them heavily to create the snapshot images for your current system. If this is the case, and you are willing to package all your small changes/tweaks to the system as a package (which could be a non-trivial amount of work. on my systems it's enough that I don't do it) you could then make the resulting config available to the net for users who wanted to take the risk to update from. I don't know rpm/yum (my main work is on apt based systems), so I don't know exactly what the terms are for the main repository, but it would need to provide the rpms and host the index of what's what so that the systems that query it would learn what packages are availabe, and be configured to install anything that's available from that source (it would need to not do so to an alternate repository, such as the RedHat one the build currently points at) David Lang ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
On Fri, Jun 27, 2008 at 05:21:49PM -0400, Michael Stone wrote: On Fri, Jun 27, 2008 at 02:50:34PM -0400, Erik Garrison wrote: Existing package managers (e.g. apt, rpm) do exactly what we want and more. Furthermore they are extensively tested and well documented. Why have we locally manufactured and promoted the square wheels of olpc-update and copy-nand? We chose a monolithic update solution because of several deficiencies, *for our primary use case*, of all package-based upgrade solutions with which we were familiar at the time. Package-based update solutions with which we were familiar: Let's say we dist-upgrade our system. It's in an unbootable state. In our current situation we attempt to avoid: * can leave the system in an inconsistent or even unbootable state on failure. ... by holding around the most recent distribution snapshot. A feature is provided to select that older snapshot at boot time. Correct? I've done it several times in the month+ since I've been at OLPC. * ask users questions during the update process. This is obviously problematic for our users. But, I would be extremely surprised if there weren't ways to specify defaults to use for these questions and also force the defaults to be used without querying the user. * are not adequately bandwidth efficient. (They make no use of bitpatterns which are already available on the target system.) This is a deficiency of package managers which, if solved by us and pushed upstream, would be useful to all users of said package managers, not just OLPC XO's. That would be a feather in our cap. We could probably find assistance in the shape of authors and maintainers of said package managers. * do not adequately verify the integrity of the resulting filesystem. (We have actually detected two filesystem data corruption bugs as a result of carefully auditing our filesystem consistency via the update process.) This check, frankly, has nothing at all to do with software distribution. We might as well periodically run a script to check for filesystem corruption. We could easily trigger it after dist-upgrades. * do not innately offer the user a 'big undo' button that permits the user to try out large changes with confidence that they can easily return to previous system states. As I mentioned above, I thought we already had that button in the form of the 'swap boot to older version' button. If this functionality is desired more generally, again, why don't we attempt to push it upstream so that all users of whatever software distribution system we use can utilize it? Now, clearly, there are several technically adept users who would like to be able to use their favorite package management tools to consume our software. We _are_ interested in supporting this use case, for example by modifying our build procedures to produce package repos 'suitable for use in upgrade procedures'. The hard question is: what more can we do? The technically adept users which you're referring to are: (certainly) Developers Testers Deployment staff (in smaller proportion) Teachers (who want to install specific non-OLPC software) Students and end users .. Do you have other thoughts on either: 1) other ways that we could address the aforementioned inadequacies of package-based updaters for our use case? I'm confused by what you mean. 2) other things we can do to integrate package-based updaters into our current system? What is meant by 'other'? Where are the showstoppers that imply it would be impossible to use yum/apt? The only way to properly integrate the package management system is to start using it and test it internally. Where we find problems let's find the developers of the package manager (for now we're talking about yum), and ask if fixes are possible. I have serious doubts that there isn't upstream interest in meeting our needs as far as package managers go. We are currently, and will probably be for some time, the largest deployment of fedora in the world. Upstream will listen, and if it doesn't we can change what we mean by 'up'. Erik ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
See also http://dev.laptop.org/ticket/6432, which actually has a design with limitations which seemed to satisfy folks. --scott -- ( http://cscott.net/ ) ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC-Update + RPMs WAS:Re: OLPC XO Opera browser as Sugar activity
Am 27.06.2008 um 20:50 schrieb Erik Garrison: We already have yum installed on the XO. Only in the devel builds. You might never have wondered what the devel_ prefix means in http://pilgrim.laptop.org/~pilgrim/olpc/streams/update.1/build708/devel_jffs2/ but there was a time when there was a user build and a developer build, and the user build did not include the non-essential tools like yum. Might be interesting to see how much flash could be saved nowadays if those tools were removed. - Bert - ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC XO Opera browser as Sugar activity
Am 26.06.2008 um 00:49 schrieb Stevens:
Hi Bert, all,
I changed the script (OperaActivity.py) posted on the wiki
(http://wiki.laptop.org/go/Opera
).
This is the old script:
import logging
from sugar.activity import activity
import sys, os
import gtk
class OperaActivity(activity.Activity):
def __init__(self,handle):
activity.Activity.__init__(self,handle)
self.set_title('Opera Activity')
os.system('opera -notrayicon ')
The above script resulted in the 'blank screen' activity. I added a
personaldir switch, shown below:
import logging
from sugar.activity import activity
import sys, os
import gtk
class OperaActivity(activity.Activity):
def __init__(self,handle):
activity.Activity.__init__(self,handle)
self.set_title('Opera Activity')
os.system('opera -notrayicon -personaldir $SUGAR_ACTIVITY_ROOT/data
')
(The only change is in the last line: added personaldir command line
switch.)
This resulted in progress. Now, in addition to the blank screen
activity, it starts Opera.
However, this may not have been the right thing to do, or I may need
to make some additional changes as there is still a bug: two glyphs
show up in the activity circle, one the Opera glyph which when
clicked on goes to a blank screen, and the other (a gray circle)
which when clicked on goes to Opera. (Before I made the change to
the script just the blank screen activity started.)
Is the change I made correct for this case? And, is there some other
change I should make to eliminate the 'blank screen' being started?
Sure. Do not use the Python script. You don't want to run a Python
activity but a native one - otherwise you get two windows, the empty
one opened by Python and the real one by Opera. Instead, you only need
a tiny shell script that preloads a little library, which may work
with Opera:
http://lists.laptop.org/pipermail/devel/2008-January/009387.html
And since you're investigating this, what would be great is if you
could re-package Opera as a real activity. That means, just move the
Opera files that are normally installed in the system into the bundle
itself, and setup the environment in the shell script so it works from
that non-standard directory. Then zip up the activity directory,
rename to .xo, and we have a real Opera bundle :) That way it would
also survive a system upgrade which erases all manually installed RPMs.
- Bert -
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel
Activity home dirs (was Re: OLPC XO Opera browser as Sugar activity)
Am 26.06.2008 um 01:22 schrieb John Gilmore: The activity start script should configure Opera to put its configuration file in $SUGAR_ACTIVITY_ROOT/data instead of $HOME/.opera. Also it should set umask to 0002 so the config file is group-writable (otherwise the next activity instance cannot overwrite). See http://wiki.laptop.org/go/Low-level_Activity_API#File_Access QSettings: error creating /home/olpc/isolation/1/uid_to_home_dir/ 1/.qt opera: Can not use personal directory: /home/olpc/isolation/1/ uid_to_home_dir/1/.opera This looks more like a bug in Rainbow than in Opera. Why would Sugar or Rainbow be setting $HOME to a rainbow-created directory that the activity can't make subdirectories in? (The universe of Unix programs isn't going to rewrite itself because OLPC decided that $SUGAR_ACTIVITY_ROOT is the right place to keep your files on Unix. $HOME has been that place for decades. Rainbow is already setting $HOME. It's just apparently setting it to something that doesn't work.) Also it should set umask to 0002 so the config file is group-writable (otherwise the next activity instance cannot overwrite). If Rainbow runs the same activity as many different UIDs that share a single group ID, then yes, Rainbow should be setting the umask so that files are created group-writeable by default. There should be no need to modify ordinary Unix programs for this. Agreed, but Peter's question was about build 708 so it might be fixed in the mean time. Indeed I remember discussion about that, although I can't find the Trac report. I recall that HOME is set to $SUGAR_ACTIVITY_ROOT/instance now, which should work at least, but I think is also wrong as it is not shared between activity instances. The right place would be $SUGAR_ACTIVITY_ROOT/data. And I think umask is set by Sugar nowadays. But that won't help machines in the field now so I gave a recipe that would work around that bug. - Bert - ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Activity home dirs (was Re: OLPC XO Opera browser as Sugar activity)
On Thu, Jun 26, 2008 at 08:53:47AM +0200, Bert Freudenberg wrote: Am 26.06.2008 um 01:22 schrieb John Gilmore: The activity start script should configure Opera to put its configuration file in $SUGAR_ACTIVITY_ROOT/data instead of $HOME/.opera. Also it should set umask to 0002 so the config file is group-writable (otherwise the next activity instance cannot overwrite). See http://wiki.laptop.org/go/Low-level_Activity_API#File_Access QSettings: error creating /home/olpc/isolation/1/uid_to_home_dir/ 1/.qt opera: Can not use personal directory: /home/olpc/isolation/1/ uid_to_home_dir/1/.opera This looks more like a bug in Rainbow than in Opera. It was considered to be a feature at the time it was introduced. Why would Sugar or Rainbow be setting $HOME to a rainbow-created directory that the activity can't make subdirectories in? Because the spec it was built to said that activities should be permitted to write to precisely three directories named 'tmp', 'data', and 'instance'. Furthermore, it was entirely unclear at the time which one $HOME should point to. (The universe of Unix programs isn't going to rewrite itself because OLPC decided that $SUGAR_ACTIVITY_ROOT is the right place to keep your files on Unix. $HOME has been that place for decades. Rainbow is already setting $HOME. It's just apparently setting it to something that doesn't work.) Also it should set umask to 0002 so the config file is group-writable (otherwise the next activity instance cannot overwrite). rainbow = 0.7.4 (available since Nov. 10, 2007) sets umask(0) before running the activity. However, we found that several important library calls like mkstemp, mkdtemp, and the equivalent file creation code used by xulrunner hardcode the use of modes like 0700 and 0600 for directories and files that they create. It would not surprise me if Opera behaved similarly. If Rainbow runs the same activity as many different UIDs that share a single group ID, then yes, Rainbow should be setting the umask so that files are created group-writeable by default. There should be no need to modify ordinary Unix programs for this. Agreed, but Peter's question was about build 708 so it might be fixed in the mean time. rainbow = 0.7.12 causes $HOME to be writable. This change has been available since April 10, 2008 in joyride and is expected to be included in our next major release. $SUGAR_ACTIVITY_ROOT/instance now, which should work at least, but I think is also wrong as it is not shared between activity instances. As a result of the fact that xulrunner hardcodes the use of modes like 0700 and 0600 in its file creation code, I decided that we should set $HOME == $SAR/instance by default so that programs would be less likely to encounter files they couldn't write. Activities which dislike this default are fully capable of changing themselves when they are executed. That being said, I'm open to arguments about what the default should be. Have you got some mechanism for setting $HOME to $SAR/data which would be safe in the face of programs like xulrunner? (For what it's worth, I happen think that the real defect is that uids and instance dirs are deleted on reboot and recreated on activity resume rather than being persistent and reused at activity resume. Unfortunately, though I intend to address this issue as soon as my other responsibilities permit, it will probably be a while before that happens. Interested onlookers should definitely take initiative here and then submit their results for discussion and possible merging.) But that won't help machines in the field now so I gave a recipe that would work around that bug. Thanks! Michael ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC XO Opera browser as Sugar activity
Hi Bert, all -- On Jun 25, 2008, at 11:37 PM, Bert Freudenberg wrote: Sure. Do not use the Python script. You don't want to run a Python activity but a native one - otherwise you get two windows, the empty one opened by Python and the real one by Opera. Instead, you only need a tiny shell script that preloads a little library, which may work with Opera: http://lists.laptop.org/pipermail/devel/2008-January/009387.html I tried a shell script and it works: #!/bin/sh while [ -n $2 ] ; do case $1 in -b | --bundle-id) export SUGAR_BUNDLE_ID=$2 ;; -a | --activity-id) export SUGAR_ACTIVITY_ID=$2 ;; -o | --object-id) export SUGAR_OBJECT_ID=$2 ;; -u | --uri) export SUGAR_URI=$2 ;; *) echo unknown argument $1 $2 ;; esac shift;shift done export LD_PRELOAD=$SUGAR_BUNDLE_PATH/lib/libsugarize.so export NET_WM_NAME=OperaBrowse exec opera -notrayicon -personaldir $SUGAR_ACTIVITY_ROOT/data This starts Opera from Sugar and it runs correctly. Thanks for the direction. -Peter And since you're investigating this, what would be great is if you could re-package Opera as a real activity. That means, just move the Opera files that are normally installed in the system into the bundle itself, and setup the environment in the shell script so it works from that non-standard directory. Then zip up the activity directory, rename to .xo, and we have a real Opera bundle :) That way it would also survive a system upgrade which erases all manually installed RPMs. - Bert - ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
OLPC XO Opera browser as Sugar activity
Hi All, I run Opera on the XO, but if started as an activity it doesn't run (it runs fine when started from terminal). There is a note on the Opera install page that there is an incompatibility between Rainbow and Opera: Note: There is at present an incompatibility between the Opera activity and the OLPC Rainbow security system on some builds. If when you launch the Opera activity, the screen goes blank and stays blank, you have likely encountered that incompatibility. http://wiki.laptop.org/go/Opera That is the symptom I encounter. Could one of you tell me what this incompatibility is, and how to remove it? I'm using build 703, the latest stable build. -Peter ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC XO Opera browser as Sugar activity
The activity start script should configure Opera to put its configuration file in $SUGAR_ACTIVITY_ROOT/data instead of $HOME/.opera. Also it should set umask to 0002 so the config file is group-writable (otherwise the next activity instance cannot overwrite). See http://wiki.laptop.org/go/Low-level_Activity_API#File_Access QSettings: error creating /home/olpc/isolation/1/uid_to_home_dir/ 1/.qt opera: Can not use personal directory: /home/olpc/isolation/1/ uid_to_home_dir/1/.opera This looks more like a bug in Rainbow than in Opera. Why would Sugar or Rainbow be setting $HOME to a rainbow-created directory that the activity can't make subdirectories in? (The universe of Unix programs isn't going to rewrite itself because OLPC decided that $SUGAR_ACTIVITY_ROOT is the right place to keep your files on Unix. $HOME has been that place for decades. Rainbow is already setting $HOME. It's just apparently setting it to something that doesn't work.) Also it should set umask to 0002 so the config file is group-writable (otherwise the next activity instance cannot overwrite). If Rainbow runs the same activity as many different UIDs that share a single group ID, then yes, Rainbow should be setting the umask so that files are created group-writeable by default. There should be no need to modify ordinary Unix programs for this. John ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
OLPC XO Opera browser as Sugar activity
Hi All, I run Opera on the XO, but if started as an activity it doesn't run (it runs fine when started from terminal). There is a note on the Opera install page that there is an incompatibility between Rainbow and Opera: Note: There is at present an incompatibility between the Opera activity and the OLPC Rainbow security system on some builds. If when you launch the Opera activity, the screen goes blank and stays blank, you have likely encountered that incompatibility. http://wiki.laptop.org/go/Opera That is the symptom I encounter. Could one of you tell me what this incompatibility is, and how to remove it? I'm using build 703, the latest stable build. -Peter ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: OLPC XO Opera browser as Sugar activity
I run Opera on the XO, but if started as an activity it doesn't run (it runs fine when started from terminal). There is a note on the Opera install page that there is an incompatibility between Rainbow and Opera: Note: There is at present an incompatibility between the Opera activity and the OLPC Rainbow security system on some builds. If when you launch the Opera activity, the screen goes blank and stays blank, you have likely encountered that incompatibility. http://wiki.laptop.org/go/Opera That is the symptom I encounter. Could one of you tell me what this incompatibility is, and how to remove it? I'm using build 703, the latest stable build. I haven't used Opera lately. If it was working pre-700, this is probably a case of isolation failing. Try: rm /etc/olpc-security followed by a reboot. To reenable isolation: touch /etc/olpc-security and reboot. Good luck, Bob ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
