Hi,
thank you all for your feedback, this is very useful.
Some specific points I'd like to respond to:
- Regarding disconnection issues that Stefan mentioned, these should not be
an issue in my case. The host system would normally be running 24/7 and the
use of directly connected USB flash disks eliminates issues with cabling or
flaky USB->SATA adapters.
- Regarding "stuttering", I guess this behaviour is heavily dependent on the
quality of the USB flash disk/FTL. In your experience, does this cause a
full system freeze or only block write operations to the device in question?
I'm assuming the latter?
- Regarding consistency/atomicity guarantees, would btrfs possibly be a
better choice? I don't have any experience with it, but the snapshot feature
would be useful if it could be used in a manner similar to what Jay
describes (continuous snapshots of /overlay with checksums).
- Regarding consumer hardware and SD flash, searching for "industrial grade
sd cards / usb flash" or similar terms comes up with quite a few vendors.
Some vendors offer industrial-grade USB flash disks [1], but many more offer
premium or industrial-grade SD cards [2] [3] and I've seen these in high
street shops so availability is not a problem. Smartphones have been using
eMMC storage for normal filesystem use for years, so SD flash (through the
use of an USB adapter) may be a better choice.
To give you a better idea of where I'm coming from, here are the
requirements I'm trying to satisfy for this project:
1) Reliable, in-place and automatic (unattended) updates of the software
running on the device, with the (probable) exception of the kernel.
Obviously there is no such thing as 100% reliability but I want to at least
minimize (or if possible provide a way to easily recover from) the end user
ending up with a bricked router.
AND
2) Implementing this on mainstream mid/high-end devices supported by OpenWRT
which do not have any kind of recovery mode in the bootloader. An example of
this category would be a TP-Link TL-WDR4300.
Based on my research to date this *should* be possible with a few tricks,
see below.
The options I have investigated so far:
1) Automated upgrades using the standard OpenWRT sysupgrade mechanism.
Ruled out on the basis that if it fails, you end up with a brick.
2) Manual updates using sysupgrade for those parts of the software which are
mostly static (kernel, core userland - busybox + libc) with the actual
application and network services that would require frequent updates living
on JFFS2 in internal flash. Automated updates would be done using opkg.
Ruled out on the basis that it still touches the internal flash too much,
and wear on the rootfs_data portion of the flash could cause upgrade
failures if the (kernel+rootfs) partitions grow and need to expand into
blocks used by rootfs_data. Plus, I've not tested how well sysupgrade would
handle several MB of data that would need to be preserved from rootfs_data
but I gather its not designed for that use-case.
3) As for 2), but use USB storage and standard OpenWRT extroot (pivot_root
or overlay).
A possible option. However, opkg does not do (and probably will not do)
package signing, and OpenWRT generally does not aim for supporting full
in-place upgrades of the system. The former could be worked around by
serving repositories exclusively via HTTPS, but the latter is part of the
general architecture of OpenWRT.
4) OpenWRT/Debian hybrid. The internal flash would only contain the kernel
and a minimal recovery root filesystem whose normal job would be to find the
USB device with the real rootfs on it and pivot into it. The recovery
filesystem could also contain code to boot a minimal OpenWRT with recovery
functionality, just enough to bring up the LAN-side network and put up a
webpage helping the user diagnose the problem ("Your USB stick is borked,
get a new one") and/or allow network boot or other useful diagnostics which
can be implemented using a UI of (WPS button + device LEDs) :-)
I have a first PoC of this working using OpenWRT trunk where I build a
custom package replacing base-files and procd. Testing on a TP-Link
TL-WDR4300 gives "OK" performance; package installation is obviously slow
but not unbearably so. RAM used on boot is surprisingly less (11MB Debian
wheezy vs 17MB OpenWRT trunk) however this is probably not a fair comparison
since wheezy is not running any services and also I'm only manually loading
enough kernel modules to see eth0 and the USB disk.
This also opens up other interesting possibilities, for example if the
platform has working kexec, then the internal flash on the router would only
ever need to be reflashed *ONCE*, and the real kernel could be loaded from
USB storage. Also, its a full Debian system, so gets to share in the
debian-security infrastructure (with the exception of the kernel) and
apt-get upgrade goodness.
This option does limit the class of devices I can implement it on
somewhat due to RAM and CPU speed requirements. However it looks like the
best trade-off so far to satisfy the requirements and devices which are
powerful enough are available today at reasonable price points.
I'd be interested in any feedback on these options from a technical
architecture point of view, or any other experiences others may have with
these kinds of setups. I'm obviously not the first person to try the
OpenWRT/Debian hybrid route [4] [5].
Regards,
Martin
[1] http://www.amtron.com/USB_flash_disk.htm and others
[2] http://www.sandisk.com/products/memory-cards/sd/, also see under OEM
[3] http://www.toshiba-memory.com/cms/en/products/sd-cards/
[4] http://www.debwrt.net/ (Project seems dormant)
[5] http://wiki.openwrt.org/toh/seagate/dockstar/alternatives
_______________________________________________
openwrt-users mailing list
[email protected]
https://lists.openwrt.org/cgi-bin/mailman/listinfo/openwrt-users
