Hello all,
This will be a long email about changes to the way Mynewt performs system
initialization. Any and all feedback is greatly appreciated.
Proposal:
1. Move system initialization out of apps and into BSPs and
system-level packages.
2. Specify system settings in config file.
Motivations:
1. Simplify process of creating an app (no more boilerplate code).
2. Simplify system configuration (don't specify config within C
code).
3. Better code organization; cputime, msys, etc. "feel" more like
system-level modules than application-level, so put
initialization code where it belongs.
Examples of "system modules" are:
* mbufs / msys.
* Network stacks (e.g., NimBLE host, controller, and HCI transport).
* newtmgr
* console
* logging
* drivers
The biggest challenge is: ensuring the developer maintains precise
control over the system configuration. The current Mynewt scheme gives
control to the developer by pushing configuration up to the application
level where he is forced to deal with it.
### Proposal Summary
System initialization happens in two stages:
1. BSP init (bsp_init())
2. sysinit (sysinit())
(1) BSP init sets up everything that is BSP specific:
* UARTs
* Flash map
* Other hardware drivers
(2) Sysinit sets up all the libraries which aren't BSP or hardware
dependent (os, msys, network stacks, etc). Sysinit is implemented in
the sys/sysinit package.
The system knows what to initialize and how to initialize it based on
the contents of the following header file:
<target-path>/include/syscfg/syscfg.h
For example:
targets/bleprph-nrf52dk/include/syscfg/syscfg.h
This header file is generated by newt during the build process. Newt
arranges for all packages to have access to this header file.
The syscfg.h header file consists of three sections:
* Settings
* Indication of which packages are present
* Indication of which APIs are available
Here is an excerpt from a sample syscfg.h file (the actual file is much
larger):
/*** Settings */
#ifndef MYNEWT_CLOCK_FREQ
#define MYNEWT_CLOCK_FREQ (1000000)
#endif
#ifndef MYNEWT_MSYS_1_BLOCK_COUNT
#define MYNEWT_MSYS_1_BLOCK_COUNT (15)
#endif
#ifndef MYNEWT_MSYS_1_BLOCK_SIZE
#define MYNEWT_MSYS_1_BLOCK_SIZE (260)
#endif
/*** Packages */
#ifndef MYNEWT_PKG_LIBS_NEWTMGR
#define MYNEWT_PKG_LIBS_NEWTMGR (1)
#endif
#ifndef MYNEWT_PKG_LIBS_NEWTMGR_TRANSPORT_BLE
#define MYNEWT_PKG_LIBS_NEWTMGR_TRANSPORT_BLE (1)
#endif
#ifndef MYNEWT_PKG_SYS_CONFIG
#define MYNEWT_PKG_SYS_CONFIG (1)
#endif
/*** APIs */
#ifndef MYNEWT_API_BLE_DRIVER
#define MYNEWT_API_BLE_DRIVER (1)
#endif
#ifndef MYNEWT_API_BLE_TRANSPORT
#define MYNEWT_API_BLE_TRANSPORT (1)
#endif
#ifndef MYNEWT_API_CONSOLE
#define MYNEWT_API_CONSOLE (1)
#endif
### Syscfg header file generation
Newt generates the syscfg.h file by parsing a series of "syscfg.yml"
files. Any package may contain a syscfg.yml file in its directory.
A syscfg.yml file defining the three settings shown above might look
like this:
MYNEWT_CLOCK_FREQ: 1000000
MYNEWT_MSYS_1_BLOCK_COUNT: 12
MYNEWT_MSYS_1_BLOCK_SIZE: 260
If several packages define the same setting, the tie is broken according
to package priority. Packages are prioritized as follows (lowest
priority first, highest last):
1. Library
2. BSP
3. App
4. Target
The expectation is that libraries would define sensible defaults in
their syscfg.yml files, and BSPs, apps, and targets would override those
defaults.
To prevent unnecessary rebuilds, newt only overwrites the header file if
there are any configuration changes since the previous build.
### newt target config command
Newt would have a new command: newt target config <target-name>. This
command shows the following information about each setting for the
specified target:
* Default value and package which specifies it.
* Actual value and package which specifies it.
### System init C code
The system init code conditionally initializes modules in sequence.
Here is an example of how this might look:
#if MYNEWT_PKG_SYS_ID
id_init();
#endif
#if MYNEWT_API_CONSOLE
rc = console_init(NULL);
assert(rc == 0);
#endif
#ifdef MYNEWT_REBOOT_LOG_0_TYPE
reboot_init_handler(MYNEWT_REBOOT_LOG_0_TYPE,
MYNEWT_REBOOT_LOG_0_ENTRY_COUNT);
#endif
#if MYNEWT_PKG_LIBS_SHELL
shell_task_init();
#endif
Each module's init function (e.g., shell_task_init()) also reads the
syscfg.h header to know how to configure the module.
### Questions / issues
Here are some issues that I'm struggling with:
1. Settings lack structure.
Do the syscfg.yml files need more information? Ideally (in my opinion),
each setting would be defined by its package with the following
information:
* Name
* Default value
* Description
* Restrictions
By "restrictions", I mean prerequisites for enabling a particular
setting. For example, it is illegal to configure the config module to
use nffs if the fs/nffs package is not pulled into the build. It would
be nice if newt could catch this configuration error with a helpful
message rather than letting the compiler and linker generate
incomprehensible errors.
Thinking about this "richer" syscfg format is what got me worrying that
this feature may be getting way too complicated. Even though it might
be messy to change the syscfg format in the future, I don't think it is
a good idea to try to implement the super-complicated option for 1.0.
2. Default task priorities.
A package's default settings reside in the package's syscfg.yml file.
Among these settings is default task priority. I don't think it is
possible to assign meaningful default priorities. I see two options:
* Don't specify default priorities. Force the user to specify all
priorities in the bsp/app/target syscfg.yml file.
* Allow an "any" priority value to be specified. While generating
the syscfg.h file, newt would assign unused values to priorities
marked as "any". Tasks with strict timing requirements (e.g., the
nimble controller) would specify actual values. Other tasks
(e.g., shell) would use "any".
I think the second option is better, but requires more work.
3. General flash map problems.
Currently it is assumed that all parts of a BSP (C code, linker script,
and bash scripts) have the same flash map definition. It would be nice
if the flash map could be specified once in a single place.
Another problem concerns initialization of some flash-dependent
packages. For example, if the sys/config package is configured to use
FCB, something needs to indicate where in flash the FCB goes. With the
flash map definition living in C code, is not really feasible to specify
this in the system configuration yaml files.
Even though I see this as a problem, I am OK with keeping flash-specific
stuff in BSP C code for now.
Thanks,
Chris
