Re: [Server-devel] XS-rsync: automatic .contents creation
On Mon, Feb 23, 2009 at 5:12 PM, Daniel Drake d...@laptop.org wrote: If the XS shipped olpc-contents (http://xs-dev.laptop.org/~cscott/repos/joyride/olpc-contents-2.5-1.i386.rpm) then it would be easy to make XS-rsync be able to generate the .contents file automatically from the .tar.bz2 tree file. Is it possible to include that RPM, and would such patches be considered? It would simplify updates that have been built with lightweight tools such as image-builder. I dropped off this conversation earlier in the year. In any case, olpc-contents will be in 0.6 :-) cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] XS-rsync: automatic .contents creation
2009/2/24 Martin Langhoff martin.langh...@gmail.com: On Wed, Feb 25, 2009 at 2:48 AM, Daniel Drake d...@laptop.org wrote: 2009/2/23 Martin Langhoff martin.langh...@gmail.com: Can you flesh out the use cases a bit more? As Ties pointed out, it's related to .toc contents files (which XS-rsync calls .contents) and not content bundles. Colour me confused. What's the use case where the XS builds a special local image? There's no use case there. That's not what I'm suggesting. I'm describing the process of transferring the image (built on another system) to the XS for serving to the XOs. The XS-rsync readme states that 4 files are required: 1 xyz_jffs2-tree.tar.bz2 # tar.bz2 build img 2 xyz_jffs2-tree.tar.bz2.md5 # md5 of the tarbz2 3 xyz_jffs2.contents # json-encoded manifest 4 xyz_jffs2.name # file containing the name This mail is about (3). H. The way you're describing it makes me think that .contents still belongs in the image-preparation stage. Maybe fold the relevant bits of code into your script? It's possible, but I already explained why it's a bit painful. I see where you're coming from though, just thought I would ask! Daniel ___ Server-devel mailing list server-de...@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] XS-rsync: automatic .contents creation
On Tue, Feb 24, 2009 at 8:06 AM, Martin Langhoff martin.langh...@gmail.com wrote: On Tue, Feb 24, 2009 at 4:12 AM, Daniel Drake d...@laptop.org wrote: If the XS shipped olpc-contents (http://xs-dev.laptop.org/~cscott/repos/joyride/olpc-contents-2.5-1.i386.rpm) then it would be easy to make XS-rsync be able to generate the .contents file automatically from the .tar.bz2 tree file. Interesting idea. Not sure I understand it fully. It sounds to me like it'd be useful to wrap up content created or aggregated online on the XS (using Moodle, for example) -- content that you want to bundle up for download to the XOs. Yes, I'd also like this included. Martin, my guess is you're confusing concepts. The .contents or .toc file is used when updating or flashing an XO image, for example by olpc-update to verify the stuff that's now on the XO is the stuff that's expected. Or perhaps you're forseeing other uses. The way to create a .contents file right now is damn right dirty. You basically chroot into the fs tree of an XO on the server and use the contents manifest builder which is present in every standard XO. That's what we're doing now in any case, and that's what Pilgrim does. If one can believe the Pilgrim inline documentation, this was done this way because Python 2.5 (often?) wasn't available on the servers at the time. And I'm just lazy. /Ties I'm not sure that it'd help with the backups/restore workflow. AFAIK, a content bundle will appear as one entry in your journal if it's able to unpack into separate entries in your Journal, then you're hit gold with your thinking. Is it possible to include that RPM, and would such patches be considered? Anything that is useful in deployments I'm happy to include :-) - just need to flesh out how it's useful to more/most rather than a bespoke trick. Can you flesh out the use cases a bit more? cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] XS-rsync: automatic .contents creation
On Wed, Feb 25, 2009 at 2:48 AM, Daniel Drake d...@laptop.org wrote: 2009/2/23 Martin Langhoff martin.langh...@gmail.com: Can you flesh out the use cases a bit more? As Ties pointed out, it's related to .toc contents files (which XS-rsync calls .contents) and not content bundles. Colour me confused. What's the use case where the XS builds a special local image? The use case for xs-rsync serving content for olpc-update is for OS upgrades, which are generated in various formats by the team that manages the XS/XO infra at a central location, tested, some of the formats are signed (for NAND-flash updates), and so on and so forth. The XS generating its own images _locally_ is not something I considered. It sounds risky - serving a botched image will... botch XOs!.. that's why the unpack the tar part of the xs-rsync scripts is so anally paranoid. Sorry to sound repetitive... what's the use case? In future, we would like to use the same tool to build an image suitable for the XS to distribute using XS-rsync. 2 changes are needed to the image builder process for this to happen: 1. it should optionally output a tarball (in addition to, or instead of a jffs2 image). trivial modification to the script. 2. we need a .contents file, because XS-rsync requires that Right now, image builder is a nice standalone script without any painful dependencies, but generating the .contents file for (2) is a bit tricky. It would require installation of olpc-contents on the local system, which really means packaging for various distributions etc. It would be nicer if the server could create the .contents file for itself, which would not be hard. It even seems to unpack the tarball already for other reasons. H. The way you're describing it makes me think that .contents still belongs in the image-preparation stage. Maybe fold the relevant bits of code into your script? cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] XS-rsync: automatic .contents creation
Hi Dan, As Ties pointed out, it's related to .toc contents files (which XS-rsync calls .contents) and not content bundles. The xs-upserv script knows how to create a contents file given an exploded directory tree -- is that what you're looking for? http://dev.laptop.org/git?p=users/cjb/xs-rsync;a=summary - Chris. -- Chris Ball c...@laptop.org ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
[Server-devel] XS-rsync: automatic .contents creation
If the XS shipped olpc-contents (http://xs-dev.laptop.org/~cscott/repos/joyride/olpc-contents-2.5-1.i386.rpm) then it would be easy to make XS-rsync be able to generate the .contents file automatically from the .tar.bz2 tree file. Is it possible to include that RPM, and would such patches be considered? It would simplify updates that have been built with lightweight tools such as image-builder. Thanks, Daniel ___ Server-devel mailing list server-de...@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] XS-rsync: automatic .contents creation
On Tue, Feb 24, 2009 at 4:12 AM, Daniel Drake d...@laptop.org wrote: If the XS shipped olpc-contents (http://xs-dev.laptop.org/~cscott/repos/joyride/olpc-contents-2.5-1.i386.rpm) then it would be easy to make XS-rsync be able to generate the .contents file automatically from the .tar.bz2 tree file. Interesting idea. Not sure I understand it fully. It sounds to me like it'd be useful to wrap up content created or aggregated online on the XS (using Moodle, for example) -- content that you want to bundle up for download to the XOs. I'm not sure that it'd help with the backups/restore workflow. AFAIK, a content bundle will appear as one entry in your journal if it's able to unpack into separate entries in your Journal, then you're hit gold with your thinking. Is it possible to include that RPM, and would such patches be considered? Anything that is useful in deployments I'm happy to include :-) - just need to flesh out how it's useful to more/most rather than a bespoke trick. Can you flesh out the use cases a bit more? cheers, m -- martin.langh...@gmail.com mar...@laptop.org -- School Server Architect - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
