Here is a high level functional spec for the split image feature. It include a
motivation, overview of design decisions, and a set of functional changes that
need to happen. I’d love to get feedback if you are interested.
Its not quite a complete design, but there are still a few unknowns to work
out. My though is to get feedback through this week and start the
implementation next week. I was thinking about starting with the Bootloader
Changes, then the Newt Changes, then writing the loader_app and new newtmgr
commands. Anyone up for helping?
Paul
Split Image Bootloader (SIB)
This document describes the split image bootloader design.
Status
This document is currently draft an anxiously awaiting review.
Motivation
The goal is to create an application in two pieces to fit into two image banks
such that one piece would contain the bluetooth stack and firmware update
application and the other would contain the customer application (that’s the
goal, but it would be defined generally to allow any split). These two are
linked together with a special property that the upgrade app (also called
loader_app below) could run without the customer app but not vice versa. To
give these names I call the independent one the AIIC (Application independent
image component) and the customer app the ADIC (Application Dependent Image
component).
Consider this example, two 112k flash banks which we want to store a bluetooth
application. Assume the following code sizes: Application size 32k, Bluetooth
stack size 64k, Upgrade code size 16k. With two independent app images, each
would be 112k filling the available sectors. However, if we split the image we
would have an upgrade image of 80k and an app image of 32k (since it uses
bluetooth and upgrade from the AIIC) leaving tons more space for sophisticated
applications and more space for the upgrade as well.
We decided to create this split image as a pair that are linked together during
the build process. There will be no dynamic bindings like SWI or function table
or anything like that. The goal is not to separate the OS/stack from the app,
but just to be a bit more efficient about duplicating the large components like
bluetooth. Thus, when up are updating one of the split images, you MUST update
the other. Splitting the image makes this “safe" as there will always be a bank
filled with an image that is capable of performing an upgrade.
You can see some discussion below from Contiki and NUT below about how they did
the same thing (although some were later replaced with dynamic loading)
High Level Operation
The basis principle is to create two image components that make up a single
image. One component would include an image loader and the components required
to load the image (e.g. bluetooth or serial). The other component would include
the remainder of the application. A small bootloader would verify the two
images and boot the one that was relevant.
There are three image components to the SIB:
1. A bootloader
2. An application independent image component (AIIC)
3. An application dependent image component (ADIC)
There are four flash regions required for the SIP
1. the boot image region (BIR)
2. the primary image region (PIR)
3. the secondary image region (SIR)
4. a scratch area region (SAR)
The bootloader is intended to be programmed at manufacturing and will not be
reprogrammable in the field. It is always loaded into the BIR.
The AIIC has the ability to upgrade itself and download new AIIC or ADICs into
the SIR. The AIIC always runs from the PIR, but can be loaded into the SIR and
copied to the PIR by the bootloader.
The ADIC always loads and runs in the SIR. The ADIC cannot upgrade itself (well
it may have code to do so, but has no-where to write a new image since it uses
the SIR and requires the PIR to run).
The AIIC and ADIC are not independent components. An upgrade must upgrade both
of these components. That is, they will be statically linked to run as a single
unit. However, the AIIC will be linked to be able to run without the ADIC in
order to perform an upgrade of the AIIC and ADIC.
The scratch area region (SAR) is used when swapping AIIC images to/from the SIR
to the PIR.
Requirements
The following lists the user requirements for the booting of split images.
1. Use the Nordic Soft Device as part of the AIIC
2. Update the ADIC with only an AIIC present
3. Update the AIIC with only an AIIC present
4. Update the ADIC with the AIIC and ADIC present
5. Must support protection of system to ensure that only approved images can
be loaded on device
Upgrade Procedure
An upgrade would work as follows.
1. An upgrade would trigger a reboot into AIIC-mode
2. The bootloader would detect AIIC mode and run the AIIC in independent mode
3. The AIIC would download a new AIIC into the SIR and restart
4. The bootloader would copy the SIR AIIC into the PIR and boot the PIR AIIC
5. This new AIIC will detect absense of a SIR image and download an ADIC
into the SIR.
6. The system will restart and the bootloader will run the ADIC.
Fallback Procedure
The following would list potential fallbacks during failed upgrade
1. If the AIIC in the PIR fails to download a new AIIC, it could always
download the old ADIC into the PIR to resstore behavior.
2. If the AIIC is able to download a new AIIC but the new AIIC doesn't run,
the bootloader could fallback to the old AIIC (since it swapped them) and the
restore the old ADIC.
3. if the new AIIC loads and runs, but is unable to download the new ADIC,
it could download the old AIIC and restore the old ADIC
Out of Scope
1. Use of the Nordic Bootloader. Current design will replace the nordic
bootloader.
2. Binary Bluetooth stack image to faciliate certification. This is a
separate issue and not considered.
3. More that two image regions. There may be use cases to have more than two
image regions. For example, there could be one AIIC region and two ADIC regions
with different applications liked against the same AIIC. An example would be
game1/game2. We will not support this today and do what our simple bootloader
does which is to look at two regions (PIR and SIR)
Current Design Discussion
Here are some tradeoffs considered during the design phase.
Versioning
How do we differentiate between the different pieces of a split image?
There are several places in the system that need to do this. To bootloader
simply needs to know whether an image is directly bootable or not.
The AIIC needs to determine whether it and the ADIC are a perfect match since
symbols are pre-linked between them
The user (via newtmgr) needs to be able to identify this package as a single
UniqueName.
The solution is desribed below.
Starting an Application Image
We have a design choice where the AIIC sometimes runs without the ADIC or the
ADIC and AIIC may neen to be run together. There is an architecture decision
about how this happens. Here are some choices
In all cases, the bootloader can read the preferences like it does now (from
FCB or NFFS). In all cases, the AIIC needs all the net_mgr code and bluetooth
(or serial code) to perform an image update of the SIR.
Split Image Aware Bootloader
In this case, the Bootloader looks at three possible image types (stand_alone,
split_indep, split_app) for each region and determines what it allows to copy
or run. If would need additional (from what we have now)logic to validate that
the split image components match and then run the appropriate one depending on
whether they are both present and valid.
For this model, the bootloader must understand the split image concept and must
perform the validation of the component parts. It would make sense here (I
think) for the bootloader to boot directly to the main of the AIIC or ADIC
depending on whether they were both valid and the status of the boot request
(whether the user wanted to boot the AIIC only).
This method has the advantage that we only perform the security checing in the
bootloader. It also means that we don't have to do any startup magic in the
AIIC to decide whether to run the AIIC main or look for and run the ADIC main.
Of course that same code would happen in the bootloader which is not-upgradable.
Pros: 1. no security code in the AIIC 2. simpler linking of AIIC and ADIC. In
this case, the AIIC would be a stand-alone linked-program and the ADIC would be
special linkage with 3. more rigid
Independent Bootloader
In this case the bootloader looks at two possible image types (runnable,
non-runnable) in each region and determines what it is allowed to copy and run.
It does not need logic to "match" the split image components.
In the stand-alone image case, the bootloader would look at the boot preference
and try to validate, copy and run the image specified. This is no different
than today, except that it would ensure that the image that it was asked to run
is "runnable".
In the split image case, the bootloader would look at the boot preference and
try to validate, copy and run the image specified, This is no different than
today, except that it would ensure that the image that it was asked to run is
"runnable".
This looks like the easy approach, as the bootloader would be identical for
split images and dual-images. However, it would mean that the AIIC image would
have to lauch the ADIC image as part of its startup. Depending on what
information is passed to it from the bootloader, it may have to re-compute the
security of the ADIC, and also understand the flashmap etc. Its likely that the
AIIC would link-in many of the same code as in the bootloader.
Pros: 1. Simple bootloader 2. Easily support dual and split image 3. more
flexibility in AIIC for dual-image behavior (i.e. this allows future models to
use syscall or linked or something else). it would also allow more flexibility
for compatibility between split images.
Bootloader Decision
The decision moving forward (for now) is to use the independent bootloader
model and have the AIIC validate the ADIC and run it. For now, we'll do this
through duplicating the authentication code and public key code from the
bootloader. This design will proceed with anindependent bootloader
Linker Magic
To build this split image, the following steps will be taken.
1. The AIIC image will be built and linked to run in the PIR. If this
succeeds (no unresolved externals and fits into the memory). This is a normal
linkage and guarantees that we have a stand-alone image with no unresolved
externals.
2. We will run a command on the AIIC.elf file to generate a list of external
symbols that are defined. For example, we may do nm -g --defined-only aiic.elf
which will generate somethimg that has a list of symbols with their addresses.
* 0000db90 T base64_decode
* 0000dac4 T base64_encode
* 0000db60 T base64_pad
* 00015000 T ble_att_clt_rx_error
* ...
3. We will remove a few key functions from that list depending on the
decision for the bootloader above that we need to repeat in the second image.
* main(), etc.
4. We will parse this output and generate linker command file split_app.ld
like this
* base64_decode = 0000db90 ;
* base64_encode = 0000dac4 ;
* ...
5. We will include this file as part of the linker script before the library
includes. INCLUDE(split_app.ld)
6. The linker will then use the existing symbols definitions from the AIIC
to build the ADIC.
This has the following benefits: 1. The AIIC will always be runnable (it will
always have exactly what it needs and no more) 2. The ADIC image will be
complete as well. It establishes a rule for building independent and dependent
images
Drawbacks: 1. There is limited control over the size of the AIIC. You would
have to tell the AIIC linker script to include stuff that was not used to
balance size between AIIC and ADIC. 2. Its not clear where unused global
variables will go. Will this force all global variables into the AIIC? 3.
Individual packages will be split between ADIC and AIIC.
Build Process
The question I am investigating is how to formulate this in the newt build
process. Here are two alternatives.
Single Target build
I was thinking that I would have a single target to produce both of these image
components. The target would still have one bsp, one build profile, but I would
define a new token for the target called loader=xxx. This would define the
loader AIIC that would go into the first image bank, and if “loader" was
defined, the app would be linked to run in the second image bank. Thus, if I
were building a bluetooth app called foo, I would set loader=AIIC (contains
upgrade and bluetooth) and then set app=foo. If “loader” was not defined, I
would ensure the newt tool would do a complete build of foo and link it into
the first image bank.
Separate targets
The alternative would be to have two separate targets. The ADIC target would
have to reference the AIIC target and know that it was being built to be
dependent on that bit. In this case, the AIIC target would be build as normal
with no changes to the build system. Then the ADIC target would have to have a
new tag that said that it was dependent on the other target. For example, in
the app:foo target descriptor it would say split_image_dep=AIIC. This would
cause newt to perform some special processing of the AIIC elf image, and ensure
that we link the foo ADIC app to run out of the second image bank and reference
common functions in the first image bank.
In either case, the application will be build as a split image and will contain
two elf files. One file will be the Application Independent AIIC) part, and
another will be the application dependent (ADIC) part. These will both get
converted to image files by newt create-image.
Target Decision
My thoughts are that IDEA1 is the better of these two. It would allow newt to
be split-image-aware during create-image and during download and debug. This
design will proceed with a single target build
Image Verification
We need a way to prove that the two images are indeed mates. Since the ADIC is
essentially statically linked to the AIIC, the AIIC must be bit for bit exactly
what the ADIC expects for code to run properly.
To accomplish this requires a slight modification to the current design. Today
each of the images has a SHA checksum that is used to validate that the image
is unmodified from build time.
The SHA in the form of a TLV that is added to the end of the image.
AIICs will not change. Its SHA will be computed over its image and appended as
it is now.
When ADICs are build, their SHA checksum will be computed over the AIIC and
ADIC image. This serves two purposes. First when the AIIC wants to run and ADIC
it can verify 100% that the ADIC was meant to run with the exact AIIC image.
Second, a bootloader that gets an ADIC image, will not run it since its SHA
will not match (computed only over its image)
Split Image Upgrade State
It may take several system boots to perform a split image upgrade. State needs
to be communicated between these boots. State will be communicated via a
sysconfig variable.
when newtmgr initiates and upgrade of a split image, it will set the sysconfig
split_state to AIIC and then reboot the system. Upon rebooting, the bootloader
will boot the AIIC which will see the split_state and run its internal
application (rather than booting ADIC). In this state, new_mgr can erase the
second image slot and download a new AIIC. Once complete successfully and the
new AIIC is verified (SHA), the current AIIC will set the split state to AIIC
and tell the bootloader to swap and run the alternate AIIC (in test mode) and
reboot. When it reboots into AIIC mode, the newtmgr can check to ensure the
right AIIC is loaded and, commit it (non-test-mode) then load the ADIC into the
bank 2. Once loaded, it will set the split_state to 'ADIC'.
Functional Spec
This section describes the functional specification for the Split Image
Bootloader. Its mean to describe the set of changes to each system component,
without getting into the design of each component.
Image Header
The image header will be augmented with another flag in the ih_flags field:
IMAGE_F_DEPENDENT
This will signify to the bootloader that this is not a runnable image and that
this image cannot be copied into the PIR of the flashmap.
This will signify to the AIIC image that this image may be its co-image (if its
version matches as well).
When building an AIIC, the SHA will be calcualted over the contents of the AIIC
image.
When building a ADIC, the SHA will be calculated over the contents of the AIIC
and ADIC.
Bootloader/Boot Util
The bootloader will be augmented to check the IMAGE_F_NON_BOOTABLE flag. It
will not copy a non-bootable image into the PIR and will not boot a
non-bootable image.
If a non-bootable image winds up in the PIR, and there is a valid bootable
iamge in the SIR, the bootloader will swap them (This should not happen)
The bootloader will NOT verify SHA or PKI on non-bootable images since it won't
boot them. It will only verify these on bootable images.
NOTE: These changes are minor and intended to allow the bootloader to be simple
and compatible with dual images or split images.
The bootloader will not know about the split image scheme, but only know about
two types of images, bootable and non-bootable.
The bootloader current stores version numbers in the sysconfig. We can still
use this method (but only boot images that also have BOOTABLE), but time
permitting, I will make the sysconfig in the bootloader rely only on the hash
values of the image TLV instead of the verison number. This would remove
dependency on the uniqueness of the version number.
boot Util Library
The bootutil library will be augmented to allow the following. These may be
used by the loader_app .
1) Ability to verify a combined AIIC and ADIC together. 2) Ability to boot into
a ADIC
Newt
Newt is the system build utility and package manager.
Target Descriptor
Newt will be augmented with a additional field in the target.
loader_app=@apache-mynewt-core/apps/loader
This will tell newt that it is to split the image loader from the application
and create a split image.
Newt Build
If loader_app is not defined in the target, the build will proceed as it does
today. If loader_app is defined, the following additional steps will be taken:
1. Before the build of the app, newt will build the the loader_app
application (with the same rules as the app). One exception is that the for
loader_app it will use the primary.ld linker script, which links to the first
flash bank for that target. The loader_app build components will be placed in
the bin/<target>/apps/<loader_app_name> directory.
2. Then, newt will generate an loader_app.ld file which contains the address
of all the symbols in loader_app_name except for a few key ones like main,
startup and maybe a few others which will be removed so they are duplicated in
the main app.
3. Now newt builds the app as before, but if loader_app is defined, uses the
secondary.ld linker script.
4. This linker script has an INCLUDE(loader_app.ld) directive to use the
addresses for the components already included in theloader_app. (NOTE: Not sure
about paths and how to get the app to find the path to the other directory.
Boot code will still be built with a special bootloader linker script.
Newt Clean
Will be expanded to ensure it cleans the loader and the the app both.
Newt Create-Image
This component needs to be modified to generate the appropriate SHA hashes on
both the loader_app and app.
Both the loader_app (optionally) and app will be built into images with the
version presented to the create-image command. For each individual image, the
SHA hash of the image will be appended as a TLV to the image.
The a build ID hash will be generated from the hash of each image component.
This hash will be appended to both images as aBUILD_ID_TLV.
This is done to ensure that either image can be identified as being part of the
build, and that the set of images can be identified as a perfect match before
running them.
The create image code will also be modified to create a donwload.json file that
contains a small set of rules for downloading these image components to the
target. For example, this may include something like: 1. {Boot PIR: AIIC} 1.
{Load SIR: { Image:loader_app.img, hash:23738941793274912749327497 }} 2. {Boot
SIR: AIIC} 3. {Load SIR: { Image:bletiny.img, hash:787658576876857687568765587
}} 4. {Boot PIR: ADIC}
This rules file defines the set of rules that the loader must use when loading
images into this device. This will be used by newt load to perform the load
operation.
For legacy dual images, the rules will be
1. {Load SIR { Image:loader_app.img, hash:23738941793274912749327497 }}
2. {Boot SIR: AIIC}
Newt Load
Newt Load will be modified to read the rules file and execute the rules to load
the target.
Newt Debug
This part is still unkown. TODO.. Its not clear how to get the symbols for both
images into GDB at the same time. I suspect we will have to loade the symbols
from the AIIC and ADIC separately.
Newt Run
This will just roll up the changes from the other new commands.
Newt Size
This must report the size of each bank image separately if there are two
Mynewt Loader (AIIC implementaiton)
A Loader application will be built in the apache-mynewt-core which will allow
upgrade over serial port newtmgr and bluetooth (via the Profile that Vipul is
writing). The loader will boot and make a first decision (possible after
starting mynewt) whether it will run as a stand alone applicaiton or whether it
will bypass its main routing and continue to boot a companion ADIC.
This will be a sample which can be used by mynewt users to create their own
loader_app or they can use this app as a stock split image loader.
It will contain the following existing components
1. The bluetooth stack as a periperhal
2. The newtmgr bluetooth profile
3. Optionally the newmgr serial protocol
4. imgmgr Library
5. sysconfig
The mynewt loader will have the following abilities:
1. Read the flash images and determine whether the SIR is matching or not
via SHA.
2. boot into the main of the second image depending on the split_state in
sysconfig
3. Handle the newtmgr commands to: inspect, program, and erase the SIR.
4. Pass image boot information to the booloader to activate and swap images.
5. Expose newtmgr interface via bluetooth and/or serial
NewtMgr
Newtmgr will be expanded to handle new image commands related to split image.
Specifically
1. Ability to tell the image, that upon boot, go into AIIC or ADIC mode.
TODO
These items are still TBD
1. How to performing this magic linking. Make sure to test before committing
to this development path. The trick is going to be to ensure that we don't get
duplicate symbols in the ADIC
2. How to debug a split image. See what we can get GDB to understand about
two images. Look up how to GDB with ROM images. Seeadd-symbol-file for a good
start.
Geek Notes
this links to a geeky paper on the topic
http://dunkels.com/adam/dunkels06runtime.pdf<http://dunkels.com/adam/dunkels06runtime.pdf>
NOTEs From Contiki
There are two aspects I'd like to share with you:
1. The only actual implementation I know of was done for 6502 based
platforms using the cc65 toolchain. I presume that this isn't the platform
your're interested in ;-) However for your reference here's the URL:
http://contiki.cvs.sourceforge.net/viewvc/checkout/contiki/contiki-cc65/make-labels
2. The approach works in general like this:
* Link the system core in the "usual" manner.
* Get yourself a list of all public symbols and their corresponding
addresses - obviously this only works for a non-relocatable binary.
* Write some script that converts this list to an assembler source file
that exports those symbols as aliases to their addresses in the system core.
* Create an object file from the assembler source.
* Put that object file on the linker cmdline when linking binary modules.
This approach has two effects:
1. The Contiki APIs implemented by the system core become available to the
modules making them work.
2. The C-library / compiler-runtime-library functions already present in the
system core don't get replicated but rather reused in the modules making them
much smaller. This is because linkers look at object files on the cmdline
before looking into archives when resolving symbols.
Notes From NUT
I did this for BTnut respectively Nut/OS (I sent Rick my diploma theses, but
not to the list as it is in German). You forgot that the address space needs to
be adjusted, as the code must not be linked into memory already used by the
core. Also I didn't take the detour using the object file, but used a
customized linkerscript, which also takes care of the some other adjustments
which have to be taken into account.
For those who are not able to read Moritz' superb thesis:
1. Build the kernel and link it
2. Extract the symbol table from the ELF file by using objdump -t
3. Convert the symbol table into the form SYMBOLNAME = ADDRESS
4. Add this file to the linkerscript by using the option -T in gnu-ld and
link the desired source files
5. You may want to write a custom LinkerScript to link the code to a custom
address space.
6. Tadaaa, you have a relocated executable module
