Re: Removing duplication in /boot - affects kernel development
Hi,
I'd like to renew this thread, since we didn't get a great amount of
feedback last time. Also, one of the issues mentioned (but a little
downplayed) is the significance of removing the firmware updating code
from olpc.fth. Having stale code duplicated from the firmware in
olpc.fth bit us today with XO-4 development; the approach implemented
below would have avoided this problem. It would be nice if we could
land this before it bites again.
So, CCing various kernel developers, as this does affect kernel
development patterns a little. Will anyone be upset by the changes
proposed here?
On Mon, Mar 5, 2012 at 11:30 AM, Daniel Drake d...@laptop.org wrote:
Hi,
At the moment, /boot in our images looks like this:
Development builds - kernel is found at /boot/vmlinuz, initramfs at
/boot/initrd.img and /boot/actrd.img. Unsecure boot is always used,
meaning that olpc.fth is executed, which loads the kernel and
initrd.img from those locations.
Release builds - kernel is found unsigned at /boot/vmlinuz, unsigned
initramfs are found at /boot/initrd.img and /boot/actrd.img. Those
files are also signed and duplicated in actos.zip (and runos.zip which
is a symlink), actrd.zip and runrd.zip. For unsecure boot, olpc.fth is
used, loading the unsigned kernels. For secure boot, the signed zips
are loaded by the firmware and olpc.fth is unused.
The problem I'd like to solve here is that in our release builds, we
are duplicating the kernel and initramfs (both unsigned and signed
versions are both shipped).
As a side effect of the solution outlined below, another small problem
will be fixed: we currently duplicate the OFW code used to upgrade the
firmware in olpc.fth, so that firmware updates take effect even in
unsecure mode. We can remove this duplication too.
The solution I'm proposing, with help from Mitch, is the following:
Our builds will always ship the kernel and initramfs *only* in their
actos/runos/actrd/runrd zip forms. However, in development builds,
these zip files will not have a signature. This will be done by the
build system - the build system will take the
vmlinuz/initrd.img/actrd.img files installed by the kernel RPM, and
put them in zip files (optionally with signatures), deleting the
originals.
olpc.fth (attached), used only in unsecure mode, by default will
instruct the firmware to execute the system from the runos/runrd zip
files, as if it were booting in secure mode, but without verifying any
signatures. This also means that the standard secure boot code will
upgrade the firmware if a newer version is found in bootfw.zip, so we
can remove the duplicated firmware upgrade code from olpc.fth.
This solves the duplication and is quite straight forward.
The questions raised here relate to development. There are 2 common
development routines that will be affected by this:
1. Installing a new kernel RPM to test a new kernel/initramfs before
it gets shipped in a build (with rpm -Uvh kernel-foo.rpm).
This will continue to install the new kernel as /boot/vmlinuz and the
initramfs as /boot/{actrd,initrd}.img.
The olpc.fth shipped as part of this change will look for the presence
of /boot/vmlinuz, and, if found, will boot directly from that rather
than looking in the zip files and executing the secure boot code as
outlined above. So, this development routine is largely unchanged:
install a new kernel RPM, copy the files to the right place (e.g.
/bootpart, will probably try and improve on this step in future),
reboot.
The one side-effect that will happen here is that when you install a
kernel RPM in this fashion, the automatic firmware upgrade from
bootfw.zip will no longer happen. So, if you install a new kernel via
a RPM, and then a new firmware via a RPM, the firmware won't be
automatically flashed. I think we can live with this (and documented
on the wiki), what do others think?
2. Copying a new kernel into place by hand (e.g. compile on another
system, copy to /boot/vmlinuz on XO, reboot). This will no longer work
as-is, since the system will now try to boot from /boot/vmlinuz (OK)
and /boot/initrd.img (not found). The initramfs will have to be put in
place at the same time as installing a custom kernel.
I will add a olpc-dev-kernel script to olpc-utils which can simplify
this (it will put /boot/vmlinuz and /boot/initrd.img in place by
extracting them from the zip files). Run olpc-dev-kernel, *then* copy
your test kernel over, and then things will work as before.
(Similar considerations apply for initramfs development: if you drop a
new initramfs in place, it won't be used until you put a /boot/vmlinuz
kernel in place too. But, run olpc-dev-kernel first, and things will
work as before).
Are developers prepared to put up with these 2 minor changes in the
interest in solving this duplication problem?
Thanks,
Daniel
___
Devel mailing list
Devel@lists.laptop.org
Shipping bigger fonts by default for greater glyph coverage
Hi, At the moment we ship the Latin-Greek-Cyrillic stripped-down version of the DejaVu font set as our default font. This works well for many, but doesn't include glyphs(characters) that are important for some deployments who actively use recent OS images. e.g. Armenia, Canada (e.g. Inuktitut glyphs). So we're planning to ship the full DejaVu font set by default on all platforms starting from 13.1.0. This adds a lot more glyphs and satisifies those users, but the downside is that these fonts are bigger (in terms of disk space used, not in terms of visual size). This will increase the base uncompressed OS size by about 6mb. If they want to, deployments can revert to the old/smaller font packages with olpc-os-builder configuration, e.g.: [custom_packages] add_packages=dejavu-lgc-sans-fonts,dejavu-lgc-sans-mono-fonts,dejavu-lgc-serif-fonts del_packages=dejavu-sans-fonts,dejavu-sans-mono-fonts,dejavu-serif-fonts Any objections? If not we will go ahead and make this change soon. Thanks Daniel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Shipping bigger fonts by default for greater glyph coverage
On Wed, Sep 5, 2012 at 6:15 PM, Daniel Drake d...@laptop.org wrote: So we're planning to ship the full DejaVu font set by default on all Yes please! I made a half-hearted attempt at pushing for this in 12.1.0 cycle, and dropped the ball halfway. This will increase the base uncompressed OS size by about 6mb. IMHO, this is tiny compared to the tradeoff at hand. m -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Shipping bigger fonts by default for greater glyph coverage
On Wed, Sep 5, 2012 at 6:44 PM, Martin Langhoff martin.langh...@gmail.com wrote: This will increase the base uncompressed OS size by about 6mb. IMHO, this is tiny compared to the tradeoff at hand. I mean tiny compared to the benefit. The tradeoff is completely desirable... m -- martin.langh...@gmail.com mar...@laptop.org -- Software Architect - OLPC - ask interesting questions - don't get distracted with shiny stuff - working code first - http://wiki.laptop.org/go/User:Martinlanghoff ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] pushing activities to XOs
On Wed, Sep 5, 2012 at 9:28 AM, Daniel Drake d...@laptop.org wrote: On Wed, Sep 5, 2012 at 9:46 AM, Sameer Verma sve...@sfsu.edu wrote: Jerry, I had to manually add http://schoolserver/activities; to the Group under Software Update in the Control Panel. Then, when I run the update, it pings the XS and grabs new activities. Is this expected behavior? Yes. For a deployment you would use olpc-os-builder to preset that address. And yes, there is a missing link in that this currently must be user-invoked; theres no fully automated way of pushing activities yet. I hope to be a part of solving that in a future release cycle. OK. It seems the xs-rsync would be a way to do this, but from the docs it looks like its set up only to push the OS image. cheers, Sameer ___ Server-devel mailing list server-de...@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: Removing duplication in /boot - affects kernel development
On 09/05/2012 04:46 PM, Daniel Drake wrote:
Installing a new kernel RPM to test a new kernel/initramfs before
it gets shipped in a build (with rpm -Uvh kernel-foo.rpm).
This will continue to install the new kernel as /boot/vmlinuz and the
initramfs as/boot/{actrd,initrd}.img.
The olpc.fth shipped as part of this change will look for the presence
of /boot/vmlinuz, and, if found, will boot directly from that rather
than looking in the zip files and executing the secure boot code as
outlined above.
I'm a user, not a developer. As long as I can upgrade a kernel with
yum/rpm (plus copy), what you've proposed is fine with me.
mikus
___
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel
Re: Shipping bigger fonts by default for greater glyph coverage
Something to watch out for is the content of /usr/lib/locale. Having been caught several time in the past with C as the only language recognized, I now pay attention to the content of that directory. [For instance, allowing a Fedora upgrade of the package 'glibc-common' could add an extra 80 MB into that directory. mikus ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Shipping bigger fonts by default for greater glyph coverage
On Wed, Sep 5, 2012 at 6:45 PM, Martin Langhoff martin.langh...@gmail.com wrote: On Wed, Sep 5, 2012 at 6:44 PM, Martin Langhoff martin.langh...@gmail.com wrote: This will increase the base uncompressed OS size by about 6mb. IMHO, this is tiny compared to the tradeoff at hand. I mean tiny compared to the benefit. The tradeoff is completely desirable... I entirely agree, this is a good use of a small amount of space. I tend to view stock OLPC releases as the car you drive when you are shopping. Sure you could drive it off the lot as-is, but if you are really picky about the options packages (e.g. fonts and that 6 MB) you are going to wait for a custom ordered car from the factory (an OS Builder custom build). It is a very good thing if the OLPC stock releases be flexible enough to accommodate additional languages by just dropping in the right language pack. This makes them ideal technology demonstrators for assessment (with L10n and on locked XO laptops) before deciding to invest additional effort in a custom build and for many, just adding the language pack will be good enough (with the larger set of glyphs). This reminds me that I need to fiddle with the langpack generating script on Pootle so that we are cranking out 0.96 based langpacks going forward. cjl ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Shipping bigger fonts by default for greater glyph coverage
On Wed, Sep 5, 2012 at 6:58 PM, Mikus Grinbergs mi...@bga.com wrote: Something to watch out for is the content of /usr/lib/locale. Having been caught several time in the past with C as the only language recognized, I now pay attention to the content of that directory. [For instance, allowing a Fedora upgrade of the package 'glibc-common' could add an extra 80 MB into that directory. mikus That is an interesting and important point. I wonder if the langpack install could be adjusted to bring it's locale with it, if needed? cjl ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Filesystem sizes on XO-1.5 with 12.1.0
DISCLAIMER: I am not asking for help -- I'm sharing ny experiences I often perform 'yum upgrade' - just to pick up the latest fixes from Fedora. But with 12.1.0 on an XO-1.5, I am encountering 'cpio' errors when the upgrade tries to unpack certain packages (e.g., gnome-keyring-3.4.1-3). Since there is no sign of their owners rushing to fix such packages, I figure the problem lies at my end. One thing I have noticed with recent (on my XO-1.5) 12.1.0 builds is that filesystems /tmp and /var/tmp are absurdly small. I don't know if this is due to some customization I have applied, or if they come that way with 12.1.0. Anyway, I wonder if running out of temporary space might explain the 'cpio' failures (inside 'rpm') I'm seeing. mikus ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: Shipping bigger fonts by default for greater glyph coverage
IIRC its only the CJK fonts which really bloat the build. There's an old bug in trac which discussed fonts at length; it might be worth digging that up to ensure we're still covering all the languages we were covering then. --scott On Sep 5, 2012 6:58 PM, Chris Leonard cjlhomeaddr...@gmail.com wrote: On Wed, Sep 5, 2012 at 6:58 PM, Mikus Grinbergs mi...@bga.com wrote: Something to watch out for is the content of /usr/lib/locale. Having been caught several time in the past with C as the only language recognized, I now pay attention to the content of that directory. [For instance, allowing a Fedora upgrade of the package 'glibc-common' could add an extra 80 MB into that directory. mikus That is an interesting and important point. I wonder if the langpack install could be adjusted to bring it's locale with it, if needed? cjl ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] pushing activities to XOs
On Wed, 2012-09-05 at 10:28 -0600, Daniel Drake wrote: On Wed, Sep 5, 2012 at 9:46 AM, Sameer Verma sve...@sfsu.edu wrote: Jerry, I had to manually add http://schoolserver/activities; to the Group under Software Update in the Control Panel. Then, when I run the update, it pings the XS and grabs new activities. Is this expected behavior? Yes. For a deployment you would use olpc-os-builder to preset that address. And yes, there is a missing link in that this currently must be user-invoked; theres no fully automated way of pushing activities yet. I hope to be a part of solving that in a future release cycle. Daniel Hi All: I've cc'd other that may have some interest in this and may be able to chime in on the subject. I'd prefer to call Software Update in the Control Panel Activity Updater from OLPC. You're able to call Activity Updater from OLPC from the cli in a terminal with ./model.py and get a listing but need something the equivalent to view.py for use from the cli to be useful for blind installation. We've in OLPCAU have been using sugar-client[1] to run yum in the background, so I've been thinking that we could maybe extend sugar-client to run Activity Updater in the same way. Think the additions to sugar-client and sugar-update-control might be of some interest to those who use sugar-client to push updates to the XO's. This is also related to patches[2] sent to the mailing-list altering the behaviour in that only pre-existing activities would be selected for updates, here is the use case for improvements suggested. This has opened up an avenue in which we can offer bundles that are not installed on the image but can be user installed on demand[3]. This requires a minor change to OOB to use a different url[4] for the update. We have deployed this in the field and have received positive feed. Jerry 1. http://wiki.sugarlabs.org/go/Platform_Team/Server_Kit/sugar-client 2. http://lists.laptop.org/pipermail/devel/2011-December/034022.html 3. https://sugardextrose.org/issues/383 4. http://wiki.laptop.org/go/Activities/OLPCAU/11.3.1/Addons ___ Devel mailing list Devel@lists.laptop.org http://lists.laptop.org/listinfo/devel
Re: [Server-devel] pushing activities to XOs
On Mon, Sep 3, 2012 at 5:02 PM, Jerry Vonau jvo...@shaw.ca wrote: On Mon, 2012-09-03 at 14:51 -0700, Sameer Verma wrote: Is there a way to push activities (new or upgrades) or activity bundles to XOs seamlessly via the XS? Been looking at the documentation on the wiki, but no luck as yet... http://dev.laptop.org/git/users/dbagnall/xs-activity-server.git/tree/README Think sugar-update-control supports looking for http://schoolserver/activities by default. Jerry Thanks. This will push activities to the XS and make these available at http://schoolserver/activities I was wondering if its possible to actually *push* the activity from the XS to the XO seamlessly, so if we wanted to add new activities to the XO, we could push it from the XS. By sugar-update-control do you mean the Software Update in the control Panel (Settings)? cheers, Sameer ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] pushing activities to XOs
On Tue, Sep 4, 2012 at 11:47 PM, Sameer Verma sve...@sfsu.edu wrote: On Mon, Sep 3, 2012 at 5:02 PM, Jerry Vonau jvo...@shaw.ca wrote: On Mon, 2012-09-03 at 14:51 -0700, Sameer Verma wrote: Is there a way to push activities (new or upgrades) or activity bundles to XOs seamlessly via the XS? Been looking at the documentation on the wiki, but no luck as yet... http://dev.laptop.org/git/users/dbagnall/xs-activity-server.git/tree/README Think sugar-update-control supports looking for http://schoolserver/activities by default. Jerry Thanks. This will push activities to the XS and make these available at http://schoolserver/activities I was wondering if its possible to actually *push* the activity from the XS to the XO seamlessly, so if we wanted to add new activities to the XO, we could push it from the XS. By sugar-update-control do you mean the Software Update in the control Panel (Settings)? Jerry, I had to manually add http://schoolserver/activities; to the Group under Software Update in the Control Panel. Then, when I run the update, it pings the XS and grabs new activities. Is this expected behavior? ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] pushing activities to XOs
On Wed, Sep 5, 2012 at 9:46 AM, Sameer Verma sve...@sfsu.edu wrote: Jerry, I had to manually add http://schoolserver/activities; to the Group under Software Update in the Control Panel. Then, when I run the update, it pings the XS and grabs new activities. Is this expected behavior? Yes. For a deployment you would use olpc-os-builder to preset that address. And yes, there is a missing link in that this currently must be user-invoked; theres no fully automated way of pushing activities yet. I hope to be a part of solving that in a future release cycle. Daniel ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] pushing activities to XOs
On Wed, 2012-09-05 at 10:28 -0600, Daniel Drake wrote: On Wed, Sep 5, 2012 at 9:46 AM, Sameer Verma sve...@sfsu.edu wrote: Jerry, I had to manually add http://schoolserver/activities; to the Group under Software Update in the Control Panel. Then, when I run the update, it pings the XS and grabs new activities. Is this expected behavior? Yes. For a deployment you would use olpc-os-builder to preset that address. Think if you use OOB to install system-wide you disable the auto-detection of schoolserver in sugar-update-control. Jerry ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
Re: [Server-devel] pushing activities to XOs
On Wed, 2012-09-05 at 10:28 -0600, Daniel Drake wrote: On Wed, Sep 5, 2012 at 9:46 AM, Sameer Verma sve...@sfsu.edu wrote: Jerry, I had to manually add http://schoolserver/activities; to the Group under Software Update in the Control Panel. Then, when I run the update, it pings the XS and grabs new activities. Is this expected behavior? Yes. For a deployment you would use olpc-os-builder to preset that address. And yes, there is a missing link in that this currently must be user-invoked; theres no fully automated way of pushing activities yet. I hope to be a part of solving that in a future release cycle. Daniel Hi All: I've cc'd other that may have some interest in this and may be able to chime in on the subject. I'd prefer to call Software Update in the Control Panel Activity Updater from OLPC. You're able to call Activity Updater from OLPC from the cli in a terminal with ./model.py and get a listing but need something the equivalent to view.py for use from the cli to be useful for blind installation. We've in OLPCAU have been using sugar-client[1] to run yum in the background, so I've been thinking that we could maybe extend sugar-client to run Activity Updater in the same way. Think the additions to sugar-client and sugar-update-control might be of some interest to those who use sugar-client to push updates to the XO's. This is also related to patches[2] sent to the mailing-list altering the behaviour in that only pre-existing activities would be selected for updates, here is the use case for improvements suggested. This has opened up an avenue in which we can offer bundles that are not installed on the image but can be user installed on demand[3]. This requires a minor change to OOB to use a different url[4] for the update. We have deployed this in the field and have received positive feed. Jerry 1. http://wiki.sugarlabs.org/go/Platform_Team/Server_Kit/sugar-client 2. http://lists.laptop.org/pipermail/devel/2011-December/034022.html 3. https://sugardextrose.org/issues/383 4. http://wiki.laptop.org/go/Activities/OLPCAU/11.3.1/Addons ___ Server-devel mailing list Server-devel@lists.laptop.org http://lists.laptop.org/listinfo/server-devel
