Re: [OpenWrt-Devel] rebooting from a kernel on external storage
On Wed, 2009-01-21 at 17:46 +0100, Harald Schiöberg wrote: here is the script we use to boot an Openwrt from a running Openwrt. Thanx! Make sure to have kexec-tools installed Indeed. kexec -l /mnt/openwrt-avila-zImage --append=rtc-ds1672.probe=0,0x68 root=$1 rootfstype=ext2 noinitrd console=ttyS0,115200 init=/etc/preinit Can I ask, just to be clear, and so there are no nasty accidents here, what *exactly* is /mnt/openwrt-avila-zImage? I mean I understand it's a kernel, but I want to be sure in what format. For example in my build_dir/linux-brcm47xx/ dir I have: -rwxr-xr-x 1 brian brian 2596986 2009-01-23 03:38 vmlinux -rwxr-xr-x 1 brian brian 2602200 2009-01-23 03:38 vmlinux.elf -rw-r--r-- 1 brian brian 778706 2009-01-23 03:38 vmlinux.lzma Is it by chance the vmlinux.lzma image, or one of the other two, or something else altogether? Thanx, b. signature.asc Description: This is a digitally signed message part ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
* Brian J. Murrell br...@interlinx.bc.ca [21.01.2009 16:15]: On Wed, 2009-01-21 at 15:41 +0100, Bastian Bittorf wrote: KISS! Heh. KISS would have been leaving the config in the NVRAM as it was intended. :-) only very few computersystems have NVRAM - thats why UCI was invented Maybe the list of configuration files can be queried from the package inventory database. That would depend on developers correctly flagging configuration files. What happens when the format of a configuration It think this is a clean way file changes from one release to another? You can't just roll a tarball down and overwrite newer formatted files with older formatted ones. This just feels like a fragile solution. anyway - if a config file changes you have always some fiddling by manual configuration. The most important thing is IMHO, that all network related stuff starts as usual. bye, Bastian signature.asc Description: Digital signature ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
On Thu, 2009-01-22 at 09:39 +0100, Bastian Bittorf wrote: only very few computersystems have NVRAM s/computersystems/[wireless] routers/ ? OpenWRT is targeted at [wireless] routers. thats why UCI was invented I would have suggested some kind of on disk nvram emulation for such non-nvram capable systems so that the nvram paradigm remains the same for nvram capable systems and is emulated for others that have, say, persistent disk. But given the advances made with sysupgrade, this might be getting moot. I guess only time will tell if my fears of synchro problems between config files and sysupgrade manifest themselves. That said, I wonder if sysupgrade has a user driven inventory list -- that is, a list of files to be included in the sysupgrade save set that the user can define. It think this is a clean way Agreed, with the above idea. anyway - if a config file changes you have always some fiddling by manual configuration. The most important thing is IMHO, that all network related stuff starts as usual. Agreed on the networking stuff. :-) b. signature.asc Description: This is a digitally signed message part ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
thats why UCI was invented I would have suggested some kind of on disk nvram emulation for such non-nvram capable systems so that the nvram paradigm remains the same for nvram capable systems and is emulated for others that have, say, persistent disk. Why emulate a 1 dimensional limited configuration system for all platforms, with a maximum capacity of 32kiB just because ONLY some old broadcom based routers use it? In this one dimension you cannot clearly create relationships between configuration options, you technically only have 1 datatype and you mess configurations together. I've heard that some other firmware projects use nvram-emulation but this is definitely very ugly. You will end up having lots and lots of abandoned configuration strings in your nvram if you use it overtime, install and uninstall packages etc. Clearly nvram has only disadvantages compared to UCI. But given the advances made with sysupgrade, this might be getting moot. I guess only time will tell if my fears of synchro problems between config files and sysupgrade manifest themselves. That is a problem that OpenWrt will face in the future when it comes to changes in some package/subsystem UCI format but you would and already had faced similar problems with nvram. That said, I wonder if sysupgrade has a user driven inventory list -- that is, a list of files to be included in the sysupgrade save set that the user can define. It has. It think this is a clean way Agreed, with the above idea. I disagree because that would create an unwanted relationship between ipkg and sysupgrade and also will result in having old configuration files if new versions of packages also have updated configuration files. anyway - if a config file changes you have always some fiddling by manual configuration. The most important thing is IMHO, that all network related stuff starts as usual. Probably networking stuff just just be kept backwards-compatible and/or include a conversion mechanism if a change is needed. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
I disagree because that would create an unwanted relationship between ipkg and sysupgrade and also will result in having old configuration files if new versions of packages also have updated configuration files. Why would you not want such a relationship. [io]pkg's conffiles are meant to solve the exact same problem of upgrading packages without messing up the package's configuration. So if the conffile info you get from [io]pkg isn't right for sysupgrade it won't be right for [io]pkg either (and vice versa). As for handling conflicts between user changes and package changes to conffiles, this problem is also faced by [io]pkg, and I see no reason why sysupgrade can't use the same solution(s). Stefan ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
On Thu, Jan 22, 2009 at 06:46:10PM +0100, Steven Barth wrote: Why emulate a 1 dimensional limited configuration system for all platforms, with a maximum capacity of 32kiB just because ONLY some old broadcom based routers use it? Right. NVRAM was usually a 32k block of consecutive name=value string pairs. - limitations on the size and character set of 'name' and 'value' - no ability to add comment or make the configuration user friendly and no ability to temporarily comment out an item - naming scheme for variables was nonstandard with each vendor using slight variations - certain variables were required at bootup before the firmware loaded, erasure of those variables or the inability of the loader to parse them rendered the device unbootable until nvram could be reprogrammed externally with a JTAG After running a few different firmwares you often had a multiple sets of variables, each conforming to whatever oem naming scheme that firmware used, making it difficult to tell which set of variables controlled the current configuration. The reset to defaults button found in many firmwares didn't attempt to clear out old variables, it simply reset the variables used by that firmware to default values. It was nearly impossible to clean out the unused variables for fear that the variable might be required by the boot loader of that particular device, rendering the device unbootable if that variable was removed or changed. Some devices included boot loaders that would repopulate NVRAM with a default set of variables if none were found, other devices had boot loaders which would attempt to repopulate NVRAM but would do so using variables/values that were incorrect for that type of device -- the result of the same boot loader code being used on multiple devices. We wanted to keep the unix style of human readable configuration files stored in /etc, but it was impractical to write a web interface when each utility used a slightly different config format -- you needed to be able to read in the entire configuration file, understand it, make one small change to the appropriate line in the file and then write out the new configuration file. UCI was created as a very user friendly confile file which could be easily parsed with a few lines of shell script. Of course, now that we'd chosen to remove NVRAM, and put the configuration data in the filesystem, how do you keep the filesystem across a firmware upgrade? Or, rephrased, why does the filesystem get nuked durring an upgrade? The problem really comes down to space -- 4M of flash isn't much space when you have to share it with bootloader, kernel, filesystem, nvram .. etc. The bootloader and NVRAM are pretty much fixed, but the firmware is allowed to do whatever it wants with the remaining space, so pack a kernel into the smallest possible size and then immediately after it, create a filesystem. Great. Now what happens if the next version of the kernel is a different size? Now you need to move that filesystem. Why can't we put the kernel in the filesystem so we'd never need to move the filesystem? Good idea, except how do you boot it? There needs to be some code at bootup that understands the filesystem and is able to extract the kernel, between the new bootup code and the overhead of the filesystem you've now used more space than just putting the kernel directly on flash where it could be booted by the exisitng bootloader. Similarly, we don't want to have free space between the kernel and filesystem to allow future kernel growth -- we don't exactly have a luxury of space on most devices. We came up with the idea of switching to a ramdisk and reflashing fairly early in the kamikaze development. The concept is pretty straight forward -- stop whatever you're doing that uses flash, copy the running system to a ramdisk, reflash, copy back -- the actual implementation isn't as simple, you can't just tell an application stop using flash, it's often impossible to get it to free file descriptors without killing it, so it's only with in the last few release canidates that we've had sysupgrade functionality working. I could probably go on for several more chapters, but that's enough of an explaination for now. - Mike. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
Hi, Brian J. Murrell wrote: I wonder what the feasibility is of instead of putting a linux kernel in the kernel portion of the flash image and essentially what's an initrd in the filesystem portion (because remember, all the / in the flash image does for me is mount USB storage on /), putting a simple boot loader that can load the kernel from the USB storage based filesystem. It's worth recognizing and mentioning that perhaps this boot loader could actually be a full linux kernel and a very small / that simply kexecs a new kernel from the USB storage once it's mounted at /. I wonder how portable kexec is amongst the processors Linux runs on. This is close to what we do for the PS3 game console (powerpc64). We use an initramfs so have a single (kernel + filesystem) image that is put into flash. We use petitboot for the UI. It can load kernels via nfs, http, tftp, usb mass storage, etc. It also understands how to use the info returned from the dhcp server to load a kernel. We have an ncurses petitboot UI also that can be used over telnet, but it is being re-worked currently and has not been committed to the OpenWrt repo yet. The whole of petitboot is being reworked and that rework will provide better support for other platforms. http://ozlabs.org/~jk/projects/petitboot/ http://git.infradead.org/ps3/petitboot-multiple-ui-patches.git -Geoff ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
* Brian J. Murrell br...@interlinx.bc.ca [21.01.2009 15:30]: restore process? This is _exactly_ the kind of rigmarole I am talking KISS! Just write an script which tar-gzip-uuencode's your prefered config-files into nvram-partition (if your hardware has one), which is (if existent) restored at firstboot (just mark with e.g. touch /www/firstboot_was_done) from nvram. Where is the problem? If you wait some weeks I will deliver a patch because we need exactly this functionality in our firmware - we plan to upgrade from whiterussian/freifunk to kamikaze trunk/b43wifi bye, Bastian. / weimar.freifunk.net signature.asc Description: Digital signature ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
On Wed, 2009-01-21 at 08:51 +0100, Markus Wigge wrote: Hello, Hi, Have you ever tried sysupgrade? No. A nice little tool to save your config, And how does it know where all of the various config is? Are you sure it's not missing any, now or in the future? flash a new image and restore the config? So, I've flashed my new image. Now what? Do I have to connect to the router using it's factory (or OpenWRT) default IP address? So now I have to reconfigure other equipment to get back into the router after I reflash it? And now I have to go through some kind of configuration restore process? This is _exactly_ the kind of rigmarole I am talking about. With the router's stock firmware, or OpenWRT's WhiteRussian, this is not necessary. You just reflash and when it comes back it's ready to go. Anyway, this particular annoyance in the whole upgrade thing was a minor point. b. signature.asc Description: This is a digitally signed message part ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
On Wed, 2009-01-21 at 16:39 +0100, Jo-Philipp Wich wrote: It has a builtin list of important configs like /etc/opkg.conf, /etc/dropbear etc. Right. But that's a list that will continually need updating as new packages are brought in. As I said in a previous message, having package maintainers identify config files within their packaging is still error-prone but probably less so than them having to know/remember to upgrade the sysupgrade list. It also backups the whole /etc/config/ directory which contains 100% of the uci config. Yes. That one seems quite obvious. Unfortunately, not all packages are uci driven [yet]. No, since the old network config is restored, you connect to the ip it previously had. So sysupgrade also takes care to restore the saved config, automatically? Does it do this on reboot? I might have to go investigate this sysupgrade more closely. I think the last time I looked at it though it didn't support much outside of x86 based systems though. Solved for the platforms which support sysupgrade which are currently the ones that have a unified image format (kernel + rootfs). So how do I know which ones specifically that is? Only if all firmwares agree on the same meaning for the same nvram variables. Yes, absolutely. They all should though, where there is common functionality. Dito for sysupgrade. Well, this is good news then. Maybe time to look at that again. Thanx for the clarification. b. signature.asc Description: This is a digitally signed message part ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
On Wed, 2009-01-21 at 17:01 +0100, Jo-Philipp Wich wrote: Yes. For squashfs + jffs based systems you could also investigate the contents of /jffs, which contains only files that where modified compared to the initial rom file system. Yes, that is an interesting approach for such systems. Wouldn't apply to me though as I mount / from a USB storage device and use it as a regular RW filesystem. It builds a temporary ramdisk, chroots to it, flashes the image from there and reinitalizes the jffs2 partition from within the yet running system. Interesting. It works pretty well now. Good news. So how do I know which ones specifically that is? Basically all platforms that utilize squashfs + jffs2 in only one single image file for flashing. Examples for that are bcm47xx, bcm63xx, x86 (not for ext2) and others. Ahhh. Cool. I might have to try playing with this on my WRT54GS. Yes, I'd suggest to try it out. If it supports your platform then it might be a suitable solution for you. Is it, or will it some day get Luci driven to operate as seamlessly as native firmware upgrades? b. signature.asc Description: This is a digitally signed message part ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I wonder what the feasibility is of instead of putting a linux kernel in the kernel portion of the flash image and essentially what's an initrd in the filesystem portion (because remember, all the / in the flash image does for me is mount USB storage on /), putting a simple boot loader that can load the kernel from the USB storage based filesystem. Given that model the kernel is coupled with the rest of the system on a single USB storage device which you just plug in and boot. Pull the device and replace with another and get the kernel that goes along with it. here is the script we use to boot an Openwrt from a running Openwrt. Make sure to have kexec-tools installed it assumes an ext2 partition on $1 (usually /dev/sdaX) which contains an OpenWRT *and* a kernel in / You may need to modify it a bit, since some filenames are hardcoded - --- snip launch-guest.sh --- #!/bin/sh if [ -z $1 ]; then echo You must specify a root device exit 1 fi dumpe2fs -h $1 | grep -e 'Filesystem state:.*not clean' ERROR=1 mount $1 /mnt | grep -e 'Device or resource busy' ERROR=1 if [ $ERROR -eq 1 ]; then echo Mount returned error, bailing out umount /mnt exit 1 fi # atheros cards freak out on warm-reboot otherwise: rmmod ath_pci kexec -l /mnt/openwrt-avila-zImage --append=rtc-ds1672.probe=0,0x68 root=$1 rootfstype=ext2 noinitrd console=ttyS0,115200 init=/etc/preinit umount /mnt sync kexec -e It's worth recognizing and mentioning that perhaps this boot loader could actually be a full linux kernel and a very small / that simply kexecs a new kernel from the USB storage once it's mounted at /. I wonder how portable kexec is amongst the processors Linux runs on. kexec runs on mips,arm,i386 - -- Harald Schiöberg Technische Universität Berlin | T-Laboratories | FG INET www: http://www.net.t-labs.tu-berlin.de Phone: +49-(0)30-8353-58476 | Fax: +49-(0)391 534 783 47 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJd1Fvy8wrZ9OvkU0RArpeAKDPFMmdWocJfvLgKlkO7pJfOVfZjwCfQ6bL hDzlVqTdTwpCF1wRm8/hyg4= =9V1D -END PGP SIGNATURE- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 | Is it, or will it some day get Luci driven to operate as seamlessly as | native firmware upgrades? LuCI's firmware upgrade pages already use sysupgrade (or better it's shell libraries) internally. ~ JoW -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJd3A4dputYINPTPMRAg8MAJ462d8fpEI96QldEObR7ioQp/DcLACfcXFJ CgV6KFsosZJI2c0jKcSUwqY= =8bo0 -END PGP SIGNATURE- ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
Is it, or will it some day get Luci driven to operate as seamlessly as native firmware upgrades? Is this seamless enough: http://dev.luci.freifunk-halle.net/sysupgrade.png ;-) Greetings Cyrus ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
On Wed, 2009-01-21 at 20:30 +0100, Steven Barth wrote: Is it, or will it some day get Luci driven to operate as seamlessly as native firmware upgrades? Is this seamless enough: http://dev.luci.freifunk-halle.net/sysupgrade.png Sure, as long as when it's done that, it comes back on the new firmware with all of my old settings. b. signature.asc Description: This is a digitally signed message part ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
[OpenWrt-Devel] rebooting from a kernel on external storage
One of the most frustrating tasks in regard to OpenWRT is needing to upgrade. There are several factors that instigate it further. Firstly, with any given router's traditional WAP/router firmware (as well as OpenWRT's White Russian), configuration settings were stored in NVRAM and survived up/downgrades. This was comforting in that you knew that no matter what you flashed (with caveats) you would not lose your configuration. This assurance is gone with Kamikaze. Upgrading is much more of PITA, ensuring that I've saved all my settings from /etc/[config] and having to reload them back, but only after I've gone through the gyrations of fixing up the default router address re-assignment [you all know the rigmarole], etc., etc. Along with all of this is the always uncertainty as to the stability of the upgraded firmware. Maybe it will be unsuitable for some reason, so now you have to downgrade again and go through the whole process a second time. Maybe your downgrade also does not roll everything back properly. Now where do you go? I mitigate some of this by keeping my / on USB storage. That way, I can to some degree, take comfort in upgrades by first duping my USB key to another key (or just some other storage) so that know that I can get back my previous / after I downgrade to my previous flashed image (IOW, kernel). But all this flashing to up/downgrade a kernel and essentially what is an initrd (given that I mount USB storage on /) seems silly. I wonder what the feasibility is of instead of putting a linux kernel in the kernel portion of the flash image and essentially what's an initrd in the filesystem portion (because remember, all the / in the flash image does for me is mount USB storage on /), putting a simple boot loader that can load the kernel from the USB storage based filesystem. Given that model the kernel is coupled with the rest of the system on a single USB storage device which you just plug in and boot. Pull the device and replace with another and get the kernel that goes along with it. It's worth recognizing and mentioning that perhaps this boot loader could actually be a full linux kernel and a very small / that simply kexecs a new kernel from the USB storage once it's mounted at /. I wonder how portable kexec is amongst the processors Linux runs on. Being able to operate like this would take a lot of the fear (and work) of upgrading away since you can upgrade simply by using a new USB storage device (which you might even have prepared on some machine other than the router) and keep the old one sitting around in case you need to quickly back out to plan B. FWIW, this is all in reference to my particular WAP which is an ASUS WL-500g Premium, but portability where USB storage devices can be used would be ideal. Thots? b. signature.asc Description: This is a digitally signed message part ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
Re: [OpenWrt-Devel] rebooting from a kernel on external storage
Hello, This assurance is gone with Kamikaze. Upgrading is much more of PITA, ensuring that I've saved all my settings from /etc/[config] and having to reload them back, but only after I've gone through the gyrations of fixing up the default router address re-assignment [you all know the rigmarole], etc., etc. Have you ever tried sysupgrade? A nice little tool to save your config, flash a new image and restore the config? At least it worked several times on my Asus premium... /Markus ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org http://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-devel
