Re: [Server-devel] [XSCE] Reflashing over a network
Hi, This is a very valuable discussion which I hope gets summarized on a wiki page. It probably also belongs in server-devel as well. I believe the 'boot net' would require only moments more per laptop than the 4-key approach. The real advantage of wireless 'flashing' could be that a roomful of laptops could be flashed concurrently as in NandBlast. I assume that NandBlast is using the broadcast capability of the network. This is getting beyond my understanding of networking, but wouldn't it be possible to implement the 'sender' side of NandBlast on the server and have it start broadcasting the image. Tony On 08/29/2013 10:19 AM, James Cameron wrote: No. It addresses the boot server identified by the DHCP server, or if no boot server is identified it attempts TFTP against the DHCP server. (Reminder: this is the manual boot net scenario, not the automatic four game key boot scenario). Could we please move technical discussions on OLPC XO firmware back to devel@? ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] [XSCE] A couple of thoughts about moving forward.
On Wed, Aug 28, 2013 at 11:37 PM, Anish Mangal an...@activitycentral.com wrote: Well, I was sort of hoping: You were sort of hoping what? -walter * We could start to have discussions and work around some/all of the topics as a community. Everyone here has way more expertise than me in many (if not all) of the topics I listed. We can build a much better server if we all can use our expertise in the relevant part of the server. This transcends the pure software-development aspect of XSCE. * As the 0.4 version of the XSCE is nearing release, it's a good time to start thinking about additions/changes for 0.5. One of the consistent efforts (and demands) has been to make the server code more manageable, and by extension, modular and scalable. * 0.2.1 was a drop-in replacement of the XS-0.6/7 * 0.3 involved a major reorganization to make the services more modular * 0.4 built upon that, by providing all the code in the same modular structure Within Activity Central, a team of developers (Santi, Miguel, Ajay, Anna) have been working on converting services available on the XSCE into ansible playbooks. The playbooks are written in a syntax which is *very easy to understand*, and the same playbook can be run on different platforms to produce the same effect. The playbooks can provide variables which may be integrated easily with other administration web-services (for example ajenti). I hope to share the code for the playbooks very soon, so anyone can have a look at and try them. We have been able to get a fully functional server up just by playbooks and reusing/restructuring the available XSCE (xs-config) code. As someone leading the Dextrose Server initiative, I would push for the inclusion of these playbooks in XSCE-0.5. There is long term value in learning a bit of ansible and being able to work at a higher abstraction level. Best, Anish On Wed, Aug 28, 2013 at 6:48 PM, Tim Moody t...@timmoody.com wrote: Thanks for making this public. What do you see as the next step? From: Anish Mangal Sent: Wednesday, August 28, 2013 6:16 PM To: xsce-de...@googlegroups.com ; server-devel ; Tim Moody Subject: Re: [XSCE] A couple of thoughts about moving forward. Hi Tim, et. al., Since it was requested that I share my conversations with various deployments over the summer yielded in form of potential requirements for the school server, I created this wiki page: https://sugardextrose.org/projects/xsce/wiki/Primary_considerations There's obviously more data available, but what you see is a filtered version of guidelines I think we should keep in mind while developing a school server. Do the points in there (summarized below) make sense? I intentionally created this page on the sugardextrose.org wiki. If it has greater acceptance community-wide, I'd be happy to move it to the main XSCE wiki. * Statistics * Content * Internet traffic shaping * Administration * Networking * Classroom and School management * Total Cost of Ownership * Power * Sneakernet - LAN - Internet * i18n Best, Anish -- Sig inserted by AutoHotkey ver. 1.1.11.01 (signature - first line) WLMail QuoteFix - http://www.dusko-lolic.from.hr/ (signature - second line) ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel -- Walter Bender Sugar Labs http://www.sugarlabs.org ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] [XSCE] Reflashing over a network
On 08/29/2013 04:45 AM, Tony Anderson wrote: The real advantage of wireless 'flashing' could be that a roomful of laptops could be flashed concurrently as in NandBlast. I assume that NandBlast is using the broadcast capability of the network. This is getting beyond my understanding of networking, but wouldn't it be possible to implement the 'sender' side of NandBlast on the server and have it start broadcasting the image. Nandblaster is much more complex than a simple broadcast. It does use broadcast but it does it in as low level and as raw of a format possible. The reason is that wireless is lossy and broadcast packets have no means of ack from the clients. To compensate nandblaster uses forward error correction where it sends extra information with each block(s) of data. You can sort of think of it as RAID 5 for wireless. The actual amount of data is configurable but and example would be that for every 2 blocks of data 1 extra block is sent that allows you to reconstruct the 2 blocks of data from any of the 3 blocks. In that case you could sustain a 33% data loss and still get 100% of the data. But you have also decreased your throughput by that same 33%. In order to make this scheme work you have to turn off a whole host of things that are stock with wireless. The wireless card in XO-1 allowed operation at a very low level. The cards in XO-1.5 and beyond had varying degrees of that low level functionality making it more difficult to make nandblaster TX work. Over time I think some of those limitations were dealt with. I've lost touch of the complete state of nandblaster. James can perhaps update with exactly what works and what doesn't. Getting a good nandblaster in a noisy wireless environment can be very tricky. Unless you get 100% data the client has to wait an entire cycle of the image to pick up the missing blocks. That can be a long time. But if you crank up the error correction so your 100% reception rate goes up the overall programming time still suffers because for 99% of the image data you sent error correction that was not needed. You can quickly end up sending double the original image size. When the image size increased in XO-1.5. Mitch Bradley and I spent a lot of time working on a wired version of nandblaster for the factory production line. (At the factory XO's were flashed via USB wired network adapters) While it worked it was a bit twitchy based on what sort of equipment was in between the server and the XO clients. Broadcast and multi-cast packets are not handled optimally by a lot of equipment. We were able to get a lower programming time by using fiber-optics to the programming rack and a server that could handle the IO requirements of many stock http: connections sending the same file. That eventually gave way to a gang programmer for XO-1.5 SD cards and then for XO-1.75 and XO-4 the factory now preps a bunch of USB flash drives and uses those. USB flash drives have the unique ability to scale perfectly with the the number of XOs you want to program at once. It actually turns out to be cheaper than a bunch high performance equipment too. Somewhere in the repositories we have the code (linux) that will generate a image file with the forward error correction and a server that will broadcast that image onto any network that can do multi-cast. I think that should work with any wireless setup as long as its configured to allow multi-cast ranges. The XO (OFW) client would night need some love as its not been used since 1.5. IIRC the changes to stock nandblaster were not that extensive. One could also make a tiny linux version of the client that you could netboot and do it that way although net booting in the face of massive multi-cast traffic might be problematic. Perhaps you could load the tiny-linux client via USB and then let it join the multi-cast network. If you have a lot of XOs to program and you don't require speed then a nandblaster type solution is very workable. Its slow but requires very few resources. Again I'm hazy on the current state of nandblaster so someone else will have to tell you how much effort is needed in the XO firmware to make that something that might be compatible with an XS broadcaster. -- Richard A. Smith rich...@laptop.org Former One Laptop per Child ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] Reflashing over a network
(another composite reply) On Thu, Aug 29, 2013 at 10:45:29AM +0200, Tony Anderson wrote: I believe the 'boot net' would require only moments more per laptop than the 4-key approach. Yes, about 10 seconds, but the laptop would have to be unlocked. Also an added benefit of testing the b, o, o, t, space, n, e, and enter keys on the laptop. The real advantage of wireless 'flashing' could be that a roomful of laptops could be flashed concurrently as in NandBlast. I assume that NandBlast is using the broadcast capability of the network. This is getting beyond my understanding of networking, but wouldn't it be possible to implement the 'sender' side of NandBlast on the server and have it start broadcasting the image. Yes. However, there's no protocol for asking for the broadcast, the broadcast has to be already happening. If you consume one wireless channel at the site for this broadcast, that's one less channel you can use for collaboration and other client services. On Thu, Aug 29, 2013 at 10:45:29AM -0400, Richard A. Smith wrote: Again I'm hazy on the current state of nandblaster so someone else will have to tell you how much effort is needed in the XO firmware to make that something that might be compatible with an XS broadcaster. Could probably be done for XO-1.5, XO-1.75 and XO-4. The code is in C, in the multicast-nand git repository. It would need a week or two. I really don't think it is a good idea, because I can't imagine wanting to consume that server connected radio spectrum for something that rare. (A NANDblaster party can happen away from the server connected access points, using one of the laptops being deployed.) On Thu, Aug 29, 2013 at 05:26:51AM -0500, Jerry Vonau wrote: On Thu, 2013-08-29 at 18:19 +1000, James Cameron wrote: On Wed, Aug 28, 2013 at 03:52:46PM +0200, Tony Anderson wrote: This is getting interesting but I am still not sure I understand the process. [...] There is an alternative via boot net which invokes a different firmware process that is working in all XO models. This process uses tftp. The server would respond to a tftp request from the XO by sending the fs.zip and .img files. Is tftp used just to retrieve the fs.zip file? No. For boot net, the firmware opens the network, then uses TFTP to GET a file named OLPC. This is a Forth script. It can do what it likes. I guess you might use a Forth script inside an fs.zip file, but I don't see why that would be necessary. Typically, the Forth script would then use HTTP, FTP, CIFS, or NFS for the next step. Is there a speed advantage over http with tftp? Given the data fetched with TFTP is tiny, the size of a script, I don't understand why you ask. Perhaps you thought the bulk of the data would be fetched using TFTP, but it would not necessarily be so. TFTP is faster than HTTP for small files, because the number of network packets exchanged is smaller. HTTP is faster than TFTP for large files, because there can be multiple packets in flight. TFTP might not saturate a shared network as much as HTTP. Open Firmware uses TFTP for boot net because of requirements prior to OLPC. No point, 'cause the time you waste getting a Forth program into the laptop is better spent getting the install data into the laptop. The old recipe called for NN PP tags to be added that to OFW, not needed if you use OLPCOFW now. I'm confused by the context. In the absence of a USB ethernet adapter, and the absence of NN and PP tags, the firmware uses the SSID of OLPCOFW. Otherwise it uses the SSID from the NN tag, and an optional WPA passphrase from the PP tag. Yes. It works here, I just tried it with Q4D34JE with the web server on 172.18.0.1 loaded with 32013o2.zd and fs2.zip. 172.18.0.1 would need to be available/aliased on the server for this as that is coded as the default for the http transfer? Yes. In try-fs-update. Either DHCP needs to hand out addresses in 172.18.x.x, or 172.18.0.1 needs to be routable by the host that owns the default route. I added an alias to the host that is my DHCP server. In contrast tftp would rely on the DHCP server for the server to contact and doesn't have this limitation. That's one way to put it. Another way is: Use of TFTP requires a DHCP server and a TFTP server. Using an HTTP server configured for 172.18.0.1 doesn't have this limitation. suppose if we craft an olpc.fth script to setup the wifi networking that could be used in place of the 4 button boot? I don't understand the question, sorry. I'm was just wondering if we can override what is defined in firmware by booting a usb flashdrive with the same commands that you would use for initializing the wireless for example flashing of firmware. I mean setting of the SSID(MM), wep/wpa(PP), that sort of thing. Much easier to just configure an open access point with OLPCOFW. Yes, you might boot a USB drive with an olpc.fth file on it which
Re: [Server-devel] [XSCE] A couple of thoughts about moving forward.
Hi Anish, I look forward to playing with the XSCE installed via Ansible. Will there be an install procedure, and cookbook, to try it out? George On Wed, Aug 28, 2013 at 11:37 PM, Anish Mangal an...@activitycentral.comwrote: Well, I was sort of hoping: * We could start to have discussions and work around some/all of the topics as a community. Everyone here has way more expertise than me in many (if not all) of the topics I listed. We can build a much better server if we all can use our expertise in the relevant part of the server. This transcends the pure software-development aspect of XSCE. * As the 0.4 version of the XSCE is nearing release, it's a good time to start thinking about additions/changes for 0.5. One of the consistent efforts (and demands) has been to make the server code more manageable, and by extension, modular and scalable. * 0.2.1 was a drop-in replacement of the XS-0.6/7 * 0.3 involved a major reorganization to make the services more modular * 0.4 built upon that, by providing all the code in the same modular structure Within Activity Central, a team of developers (Santi, Miguel, Ajay, Anna) have been working on converting services available on the XSCE into ansible playbooks. The playbooks are written in a syntax which is *very easy to understand*, and the same playbook can be run on different platforms to produce the same effect. The playbooks can provide variables which may be integrated easily with other administration web-services (for example ajenti http://ajenti.org/). I hope to share the code for the playbooks very soon, so anyone can have a look at and try them. We have been able to get a fully functional server up just by playbooks and reusing/restructuring the available XSCE (xs-config) code. As someone leading the Dextrose Server initiative, I would push for the inclusion of these playbooks in XSCE-0.5. There is long term value in learning a bit of ansible and being able to work at a higher abstraction level. Best, Anish On Wed, Aug 28, 2013 at 6:48 PM, Tim Moody t...@timmoody.com wrote: Thanks for making this public. What do you see as the next step? From: Anish Mangal Sent: Wednesday, August 28, 2013 6:16 PM To: xsce-de...@googlegroups.com ; server-devel ; Tim Moody Subject: Re: [XSCE] A couple of thoughts about moving forward. Hi Tim, et. al., Since it was requested that I share my conversations with various deployments over the summer yielded in form of potential requirements for the school server, I created this wiki page: https://sugardextrose.org/projects/xsce/wiki/Primary_considerations There's obviously more data available, but what you see is a filtered version of guidelines I think we should keep in mind while developing a school server. Do the points in there (summarized below) make sense? I intentionally created this page on the sugardextrose.org wiki. If it has greater acceptance community-wide, I'd be happy to move it to the main XSCE wiki. * Statistics * Content * Internet traffic shaping * Administration * Networking * Classroom and School management * Total Cost of Ownership * Power * Sneakernet - LAN - Internet * i18n Best, Anish -- Sig inserted by AutoHotkey ver. 1.1.11.01 (signature - first line) WLMail QuoteFix - http://www.dusko-lolic.from.hr/ (signature - second line) ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] [XSCE] A couple of thoughts about moving forward.
The code is moving to github as we speak/type :) On Thu, Aug 29, 2013 at 7:32 PM, George Hunt georgejh...@gmail.com wrote: Hi Anish, I look forward to playing with the XSCE installed via Ansible. Will there be an install procedure, and cookbook, to try it out? George On Wed, Aug 28, 2013 at 11:37 PM, Anish Mangal an...@activitycentral.com wrote: Well, I was sort of hoping: * We could start to have discussions and work around some/all of the topics as a community. Everyone here has way more expertise than me in many (if not all) of the topics I listed. We can build a much better server if we all can use our expertise in the relevant part of the server. This transcends the pure software-development aspect of XSCE. * As the 0.4 version of the XSCE is nearing release, it's a good time to start thinking about additions/changes for 0.5. One of the consistent efforts (and demands) has been to make the server code more manageable, and by extension, modular and scalable. * 0.2.1 was a drop-in replacement of the XS-0.6/7 * 0.3 involved a major reorganization to make the services more modular * 0.4 built upon that, by providing all the code in the same modular structure Within Activity Central, a team of developers (Santi, Miguel, Ajay, Anna) have been working on converting services available on the XSCE into ansible playbooks. The playbooks are written in a syntax which is *very easy to understand*, and the same playbook can be run on different platforms to produce the same effect. The playbooks can provide variables which may be integrated easily with other administration web-services (for example ajenti). I hope to share the code for the playbooks very soon, so anyone can have a look at and try them. We have been able to get a fully functional server up just by playbooks and reusing/restructuring the available XSCE (xs-config) code. As someone leading the Dextrose Server initiative, I would push for the inclusion of these playbooks in XSCE-0.5. There is long term value in learning a bit of ansible and being able to work at a higher abstraction level. Best, Anish On Wed, Aug 28, 2013 at 6:48 PM, Tim Moody t...@timmoody.com wrote: Thanks for making this public. What do you see as the next step? From: Anish Mangal Sent: Wednesday, August 28, 2013 6:16 PM To: xsce-de...@googlegroups.com ; server-devel ; Tim Moody Subject: Re: [XSCE] A couple of thoughts about moving forward. Hi Tim, et. al., Since it was requested that I share my conversations with various deployments over the summer yielded in form of potential requirements for the school server, I created this wiki page: https://sugardextrose.org/projects/xsce/wiki/Primary_considerations There's obviously more data available, but what you see is a filtered version of guidelines I think we should keep in mind while developing a school server. Do the points in there (summarized below) make sense? I intentionally created this page on the sugardextrose.org wiki. If it has greater acceptance community-wide, I'd be happy to move it to the main XSCE wiki. * Statistics * Content * Internet traffic shaping * Administration * Networking * Classroom and School management * Total Cost of Ownership * Power * Sneakernet - LAN - Internet * i18n Best, Anish -- Sig inserted by AutoHotkey ver. 1.1.11.01 (signature - first line) WLMail QuoteFix - http://www.dusko-lolic.from.hr/ (signature - second line) -- David Farning Activity Central: http://www.activitycentral.com ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel