Re: Deployment image customization
On Tue, Dec 23, 2008 at 2:19 PM, Greg Smith gregsmitho...@gmail.com wrote: Your suggestion that we allow addition of RPMs and get those built into a signed image via pilgrim or puritan is certainly valuable and part of the requirement. However, it doesn't cover a few added things (language settings was specifically requested by Mongolia and others): - Updated language packs (I believe we are trying to make this an RPM which may solve it) - Starting language - Date, time and timezone - Network settings Well, my suggestion of pilgrim and puritan was only for customising RPM packages. For the other things, I wrote: As for other customisations, the current method (customization key) works fine for the limited customisations that it allows, so that simply needs to be expanded. I could have probably worded that better. In other words, my opinion is that the existing customisation stick system should be expanded to also allow customisation of timezone, language settings, translation installation, etc. The biggest challenge I see is to find those things which you do not want to clone from the source XO. The only things that come to mind are Name and Color. We could even pre-fill them as long as those dialog boxes come up at start up. There is a lot more than that - it's things that are invisible to the user, technical details of the system, which are the bits we don't have a good answer for. For example (an easy one), keys are generated on first boot, but it is potentially bad news down the line if multiple XOs have the same keys. The hard part is tracking these things, which are not specified anywhere and there's no one place you can look to find them. I wish I could find a link to Michael's mail, where he investigated some of these things for an old build, and some of the findings were surprising even to us who hack on the system level all day... In short, I like your proposal but I still want a little more :-) If you're looking for initial high-level action items: 1. The discussions on lease delegation / allowing countries to sign their builds / providing customised builds for countries need to be finished. The outcome would be someone implementing whatever method allows country A to say we want OLPC build B with added RPMs C,D,E 2. Someone needs to implement the customization stick enhancements (and surrounding projects, such as xot bundles). Will require some modifications on the sugar-level too. Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
Hi Dan, Those sound like two good steps. I think we should make a design decision here to either: 1 - clone minus a list of configurations or 2 - Extend customization to include everything relevant for a deployment. Both have challenges. My preference is clone because I think its easier for the end user (create an XO the way you like it then click clone). However, we need to figure out the list of things that should not be cloned as you mention. Extending the customization key has two main challenges as I see it. The first is that we may miss some customization that people want. The second is that its a little cumbersome to collect all the files, put them on a USB stick, add an fs.zip etc. I believe it also requires a second boot of the XO and possibly 2 USB sticks. One to update OS and a second to add customizations. The final image must be deliverable as an upgrade or a clean install via and of the following: olpc-update to internet, via olpc-update to XS, via NAND Blaster, via USB stick. I updated the requirements to bullet those out more clearly. In terms of customization stick or clone, this is where the lead engineer or whoever does the work gets to make the final call. Just make sure you address all the requirements and that we get it done in time for a QA cycle in February and we'll be OK. If some requirement cannot be met (e.g. set the language to Mongolian) we can consider living without it. I prefer to nail them all but hitting the date and improving the product is more important than the perfect solution which never comes. I agree that its closely related to the lease delegation, signing, and building requirements (your next step #1 below). A well architected approach that addresses all the points (section 2 here: http://wiki.laptop.org/go/9.1.0#Top_Priority) would be great! Thanks, Greg S Daniel Drake wrote: On Tue, Dec 23, 2008 at 2:19 PM, Greg Smith gregsmitho...@gmail.com wrote: Your suggestion that we allow addition of RPMs and get those built into a signed image via pilgrim or puritan is certainly valuable and part of the requirement. However, it doesn't cover a few added things (language settings was specifically requested by Mongolia and others): - Updated language packs (I believe we are trying to make this an RPM which may solve it) - Starting language - Date, time and timezone - Network settings Well, my suggestion of pilgrim and puritan was only for customising RPM packages. For the other things, I wrote: As for other customisations, the current method (customization key) works fine for the limited customisations that it allows, so that simply needs to be expanded. I could have probably worded that better. In other words, my opinion is that the existing customisation stick system should be expanded to also allow customisation of timezone, language settings, translation installation, etc. The biggest challenge I see is to find those things which you do not want to clone from the source XO. The only things that come to mind are Name and Color. We could even pre-fill them as long as those dialog boxes come up at start up. There is a lot more than that - it's things that are invisible to the user, technical details of the system, which are the bits we don't have a good answer for. For example (an easy one), keys are generated on first boot, but it is potentially bad news down the line if multiple XOs have the same keys. The hard part is tracking these things, which are not specified anywhere and there's no one place you can look to find them. I wish I could find a link to Michael's mail, where he investigated some of these things for an old build, and some of the findings were surprising even to us who hack on the system level all day... In short, I like your proposal but I still want a little more :-) If you're looking for initial high-level action items: 1. The discussions on lease delegation / allowing countries to sign their builds / providing customised builds for countries need to be finished. The outcome would be someone implementing whatever method allows country A to say we want OLPC build B with added RPMs C,D,E 2. Someone needs to implement the customization stick enhancements (and surrounding projects, such as xot bundles). Will require some modifications on the sugar-level too. Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
On Tue, Dec 23, 2008 at 18:29, Daniel Drake d...@laptop.org wrote: On Tue, Dec 23, 2008 at 2:19 PM, Greg Smith gregsmitho...@gmail.com wrote: Your suggestion that we allow addition of RPMs and get those built into a signed image via pilgrim or puritan is certainly valuable and part of the requirement. However, it doesn't cover a few added things (language settings was specifically requested by Mongolia and others): - Updated language packs (I believe we are trying to make this an RPM which may solve it) - Starting language - Date, time and timezone - Network settings Well, my suggestion of pilgrim and puritan was only for customising RPM packages. For the other things, I wrote: As for other customisations, the current method (customization key) works fine for the limited customisations that it allows, so that simply needs to be expanded. I could have probably worded that better. In other words, my opinion is that the existing customisation stick system should be expanded to also allow customisation of timezone, language settings, translation installation, etc. The biggest challenge I see is to find those things which you do not want to clone from the source XO. The only things that come to mind are Name and Color. We could even pre-fill them as long as those dialog boxes come up at start up. There is a lot more than that - it's things that are invisible to the user, technical details of the system, which are the bits we don't have a good answer for. For example (an easy one), keys are generated on first boot, but it is potentially bad news down the line if multiple XOs have the same keys. The hard part is tracking these things, which are not specified anywhere and there's no one place you can look to find them. I wish I could find a link to Michael's mail, where he investigated some of these things for an old build, and some of the findings were surprising even to us who hack on the system level all day... I believe the mail from Michael that you were looking for is http://lists.laptop.org/pipermail/devel/2008-March/012200.html - and http://lists.laptop.org/pipermail/devel/2008-April/012957.html is probably also relevant. The keys generated in ~/.sugar/default/ are AFAIK not used for crypto, but are used to generate the unique Jabber ID (JID). If two XOs (or Sugar clients) have the same keys, Strange Bad Things happen to presence and collaboration. In short, I like your proposal but I still want a little more :-) If you're looking for initial high-level action items: 1. The discussions on lease delegation / allowing countries to sign their builds / providing customised builds for countries need to be finished. The outcome would be someone implementing whatever method allows country A to say we want OLPC build B with added RPMs C,D,E 2. Someone needs to implement the customization stick enhancements (and surrounding projects, such as xot bundles). Will require some modifications on the sugar-level too. Daniel Regards Morgan ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
On Tue, Dec 23, 2008 at 09:18:51PM +0200, Morgan Collett wrote: On Tue, Dec 23, 2008 at 18:29, Daniel Drake d...@laptop.org wrote: On Tue, Dec 23, 2008 at 2:19 PM, Greg Smith gregsmitho...@gmail.com wrote: Your suggestion that we allow addition of RPMs and get those built into a signed image via pilgrim or puritan is certainly valuable and part of the requirement. However, it doesn't cover a few added things (language settings was specifically requested by Mongolia and others): - Updated language packs (I believe we are trying to make this an RPM which may solve it) - Starting language - Date, time and timezone - Network settings Both puritan and pilgrim install many unpackaged hacks; that's actually the major reason why they exist. (Some special indirection needs to be taken if you want to deploy hacks to /home via olpc-update, since it doesn't touch /home, but for whole-NAND-reflash tasks, either is certainly adequate.) It's also possible to combine a compose-tool like puritan or pilgrim with our existing image-builder technology (which generates mfg-ready images from customization-stick data.) Michael ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
On Tue, Dec 23, 2008 at 11:26 AM, Michael Stone mich...@laptop.org wrote: On Tue, Dec 23, 2008 at 09:18:51PM +0200, Morgan Collett wrote: On Tue, Dec 23, 2008 at 18:29, Daniel Drake d...@laptop.org wrote: On Tue, Dec 23, 2008 at 2:19 PM, Greg Smith gregsmitho...@gmail.com wrote: Your suggestion that we allow addition of RPMs and get those built into a signed image via pilgrim or puritan is certainly valuable and part of the requirement. However, it doesn't cover a few added things (language settings was specifically requested by Mongolia and others): - Updated language packs (I believe we are trying to make this an RPM which may solve it) - Starting language - Date, time and timezone - Network settings Both puritan and pilgrim install many unpackaged hacks; that's actually the major reason why they exist. (Some special indirection needs to be taken if you want to deploy hacks to /home via olpc-update, since it doesn't touch /home, but for whole-NAND-reflash tasks, either is certainly adequate.) It's also possible to combine a compose-tool like puritan or pilgrim with our existing image-builder technology (which generates mfg-ready images from customization-stick data.) Would it be possible to apply these tools to creating LiveCDs and qemu-ready image files? I certainly don't want to add to the burdens of over-burdened staff, but can some of us volunteers do that part? I ask in large part because recent LiveCDs and image files don't work on my computer. Michael ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel -- Silent Thunder (默雷/धर्ममेघशब्दगर्ज/دھرممیگھشبدگر ج) is my name And Children are my nation. The Cosmos is my dwelling place, The Truth my destination. http://wiki.sugarlabs.org/go/User:Mokurai ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
On Tue, Dec 23, 2008 at 5:31 PM, Greg Smith gregsmitho...@gmail.com wrote: Both have challenges. My preference is clone because I think its easier for the end user (create an XO the way you like it then click clone). However, we need to figure out the list of things that should not be cloned as you mention. It's not just creating a list. You also have to figure out how the list can be made automatically. And then finally (the easy part, but may bring some unexpected challenges) is acting upon the list. The actual imaging may also present challenges, e.g. do you clean up before or after? Extending the customization key has two main challenges as I see it. The first is that we may miss some customization that people want. The second is that its a little cumbersome to collect all the files, put them on a USB stick, add an fs.zip etc. I believe it also requires a second boot of the XO and possibly 2 USB sticks. One to update OS and a second to add customizations. It can be done like that, or the deployments can do as they already do with the customization stick: they make the customization stick, test it, then turn it into an image (including OS). http://wiki.laptop.org/go/Image_builder then OLPC signs the image, as has been done for Peru builds. In terms of customization stick or clone, this is where the lead engineer or whoever does the work gets to make the final call. Just make sure you address all the requirements and that we get it done in time for a QA cycle in February and we'll be OK. OK, glad to hear that engineering will have input into the decision ;) I do not have time to work on this myself but may chip in here and there. Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
morgan wrote: On Tue, Dec 23, 2008 at 18:29, Daniel Drake d...@laptop.org wrote: On Tue, Dec 23, 2008 at 2:19 PM, Greg Smith gregsmitho...@gmail.com wrote: ... The biggest challenge I see is to find those things which you do not want to clone from the source XO. The only things that come to mind are Name and Color. We could even pre-fill them as long as those dialog boxes come up at start up. There is a lot more than that - it's things that are invisible to the user, technical details of the system, which are the bits we don't have a good answer for. For example (an easy one), keys are generated on first boot, but it is potentially bad news down the line if multiple XOs have the same keys. The hard part is tracking these things, which are not specified anywhere and there's no one place you ... I believe the mail from Michael that you were looking for is http://lists.laptop.org/pipermail/devel/2008-March/012200.html - and http://lists.laptop.org/pipermail/devel/2008-April/012957.html is probably also relevant. The keys generated in ~/.sugar/default/ are AFAIK not used for crypto, but are used to generate the unique Jabber ID (JID). If two XOs (or Sugar clients) have the same keys, Strange Bad Things happen to presence and collaboration. ssh host keys are probably generated on first boot as well. with partitioning support, it should be possible to have a r.o. root overlaid by a unionfs writeable mount, so machine-specific changes don't modify the released partition. this would make cloning quite a bit easier, i'd think. i have no idea what the performance hit of a unionfs setup would be, nor how such a partitioning would fit into the rest of the update strategy (e.g. olpc-update). paul =- paul fox, p...@laptop.org give one laptop, get one laptop --- http://www.laptop.com/xo ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
On Tue, Dec 23, 2008 at 8:08 PM, p...@laptop.org wrote: ssh host keys are probably generated on first boot as well. with partitioning support, it should be possible to have a r.o. root overlaid by a unionfs writeable mount, so machine-specific changes don't modify the released partition. this would make cloning quite a bit easier, i'd think. i have no idea what the performance hit of a unionfs setup would be, nor how such a partitioning would fit into the rest of the update strategy (e.g. olpc-update). unionfs isn't upstream and was quite unreliable last time I use it. And it adds the challenge of differentiating state that must be discarded for the cloned image, and state that must not be. For example, we would want to ssh keys generated during first boot to *not* be included in the clonable image, that's obvious. But if the user boots the OLPC image, goes into the control panel and sets a language, then we *do* want that language change to be included in the clonable image that is the output of the process. How would the system differentiate between those two? Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
daniel wrote: On Tue, Dec 23, 2008 at 8:08 PM, p...@laptop.org wrote: ssh host keys are probably generated on first boot as well. with partitioning support, it should be possible to have a r.o. root overlaid by a unionfs writeable mount, so machine-specific changes don't modify the released partition. this would make cloning quite a bit easier, i'd think. i have no idea what the performance hit of a unionfs setup would be, nor how such a partitioning would fit into the rest of the update strategy (e.g. olpc-update). unionfs isn't upstream and was quite unreliable last time I use it. And it adds the challenge of differentiating state that must be discarded for the cloned image, and state that must not be. For example, we would want to ssh keys generated during first boot to *not* be included in the clonable image, that's obvious. But if the user boots the OLPC image, goes into the control panel and sets a language, then we *do* want that language change to be included in the clonable image that is the output of the process. How would the system differentiate between those two? i dunno. i guess the lead engineer on the project would have to decide. :-) paul =- paul fox, p...@laptop.org give one laptop, get one laptop --- http://www.laptop.com/xo ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Deployment image customization
On Tue, Dec 23, 2008 at 03:29:08PM -0500, p...@laptop.org wrote: daniel wrote: On Tue, Dec 23, 2008 at 8:08 PM, p...@laptop.org wrote: ssh host keys are probably generated on first boot as well. with partitioning support, it should be possible to have a r.o. root overlaid by a unionfs writeable mount, so machine-specific changes don't modify the released partition. this would make cloning quite a bit easier, i'd think. i have no idea what the performance hit of a unionfs setup would be, nor how such a partitioning would fit into the rest of the update strategy (e.g. olpc-update). unionfs isn't upstream and was quite unreliable last time I use it. And it adds the challenge of differentiating state that must be discarded for the cloned image, and state that must not be. For example, we would want to ssh keys generated during first boot to *not* be included in the clonable image, that's obvious. But if the user boots the OLPC image, goes into the control panel and sets a language, then we *do* want that language change to be included in the clonable image that is the output of the process. How would the system differentiate between those two? i dunno. i guess the lead engineer on the project would have to decide. :-) In my opinion, the simplest way to approach this is to add a hard-reset script (perhaps named olpc-hard-reset) which cleans up the image and then prints out a diff from the starting image to the result for manual review. Michael ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
