The new RTOS builds are close to producing packages as Seb has thankfully
updated the buildbot to run RT-PREEMPT and Xenomai kernels, so it is time to
turn what's coming next building on this work, and that is the unified binary
build. The code for that is mostly done, what remains is adapting the build
procedure to support several targets in one go. However, as an early branch is
now winding through the buildbot too it might make sense to get everybody on
the same page on what it provides, and how it is done.
The goal of this work is:
make linuxcnc build for any supported kernel or thread flavor in one go, and
have all that support in a single package; the actual type of threading used is
a startup option (subject to limitations of the running kernel, and what
platforms it was built for).
Intended side effects are:
make it easier to get Linuxcnc into major distributions
separate updating of the RT environment (modules) from the binary package
proper; that is, one will be able to just update modules, or install modules
for a new threading flavor without actually replacing or upgrading the binary
package.
selective updating for problems which are specific to either a flavor, a
kernel version, or both
Usage Examples - assuming the unified binary build is done:
1. running on RTAI: you can run either a 'realtime' flavor (RTAI kernel
modules) or 'sim' by specifying an environment variable: 'FLAVOR=rtai' or
'FLAVOR=posix'.
(not surprisingly 'FLAVOR=xenomai-user' will not work on an RTAI kernel)
2. running on Xenomai: you can in fact run four flavors: FLAVOR=xenomai-user
(supported, realtime) FLAVOR=posix (supported, 'sim'), FLAVOR=rt-preempt-user
(works, but not guaranteed to give good results on a Xenomai kernel) and
FLAVOR=xenomai-kernel (not supported, deprecated and just built for test
coverage
purposes; this flavor might be dropped from distribution)
3. running a stock linux kernel: the only option here is the default, which
is autodetected (FLAVOR=posix); other flavors will of course not work.
Instance support:
This means that - in principle - run several instances of LinuxCNC on the
same machine and they be 'ships the night' (modulo performance impact of
course).
This was a side effect of reworking the some of the singleton assumptions in
HAL/RTAPI and cleaning up here and there, for instance the hodge-podge
management of shared memory keys.
Usage example for multiple instances:
again, running RTAI - you can now start several instances like so:
INSTANCE=0 FLAVOR=rtai linuxcnc foo.ini # this will start the RTAI
kernel threads instance
INSTANCE=1 FLAVOR=posix linuxcnc bar.ini # this will start a posix (sim)
instance
INSTANCE=2 FLAVOR=posix linuxcnc baz.ini # this will start another posix
(sim) instance
Instance limitations:
At this point in time, the following restrictions apply:
- one kernel thread instance only per machine
- separate instance id's for all instances (provided through the
INSTANCE=<integer> environment variable)
- while HAL/RTAPI per se is multi-instance clean, using code might not be.
This in particular affects code using TCP port numbers (linuxcncsrv,
linuxcncrsh,
halrmt and probably others). While there is now a gross hack in place too
get this out of the way for at least libnml and some of the services, it is
nontrivial to remove such limiting assumptions from old code and some more
work here is needed.
- you cannot link HAL pins from one instance to another at this point in time.
Developer information - how this is done:
The major change was on the RTAPI layer, namely how RTAPI functions are called,
both in the realtime environment as well as userland applications like halcmd
or any other HAL-using applications.
Before, RTAPI code was compiled for a particular target flavor, either with
options for realtime or userland environemnts, and linked into the RT
components or the userland HAL library (liblinuxcnchal.so).
With the unified binary code, this changes as follows:
- ALL references to RTAPI functions (RT or userland) do not reference a
function directly through load-time symbol resolution anymore, but do so going
through a switch structure very similar to the struct device in the linuxc
kernel. There is a single 'hook' into RTAPI (RT or userland) to call such
functions, and that is a exported function which returns the address of the
switch structure. From there on, using code will call into RTAPI/ULAPI
through this switch reference.
- Consequently, there are loadable modules (rtapi.so and ulapi.so for userland
threads, and rtapi.ko and ulapi.so for kernel threads) which are
flavor-specific.
- flavor and kernel version resolution:
Before, there was a single location for RT modules, namely the 'rtlib'
directory.
With the unified binary build, the layout changes as follows:
Modules are stored in one of the following directories, which are searched
in turn when loading a module:
rtlib/<flavor>/<kernel version>
rtlib/<flavor>/
rtlib/
This means that any non-flavor specific module might be directly stored
under /rtlib, but it is always possible to override the default choice
by providing a more specific module (flavor, and/or
flavor-and-kernel-dependent) stored deeper in this tree.
As a consequence, it is possible to 'repair' an issue, say with a certain
flavor, by providing a more specific module in this scheme, without upgrading
the linuxcnc package proper.
- Autodetection:
Both the realtime script, as well as the HAL library, will honor the
INSTANCE and FLAVOR environment variables. However, not any flavor or instance
may result in a valid combination; for instance, there is the aforementioned
'one kernel threads' limitation, and also there are limitiations based
on the running kernel; for instance, the rt-preempt-user flavor will run on
RTAI, but likely not as expected.
Towards this end, both the realtime script, as well as the userland HAL
libarary, does some autodetection of the running kernel and checks for
compatibility with the desired flavor and instance.
- New concepts introduced:
There is a new data structure in a separate shared memory segment for all
flavors and instances. This data structure, really an instance descriptor, is
currently
called 'global_data'; I will likely rename this to 'instance_data' because
that covers the intent in a more exact form. This data structure is guaranteed
to
exist before any using code is executed.
Any entity - both userland and RT components, attach to this global segment
first thing before they do anything else (like running some HAL or RTAPI
function),
and share certain dynamic configuration values in this segment. Among them
are the RT and userland RTAPI message levels, which removes the old restrictions
that kernel thread systems and userland thread builds behave differently in
this respect.
Restrictions which come with the unified binary branch:
- it is not possible anymore to run a 'sim' instance without running
'realtime start' beforehand. I tried, and it just got too messy; in particular
a restriction which comes with kernel threads comes into play: a kernel
module cannot be assumed to be able attach to a shared memory segment created in
userland, it _has_ to be created in-kernel, thereafter userland processes
can attach to it. This restriction brings a whole rash of chicken-and-egg
problems which I have chosen to avoid.
Status:
The code for this branch is feature-complete although rough edges remain. The
major outstanding task is to coerce the current build system to build modules
(RT modules and ulapi.so) one-per-flavor into the directory structure described
above, and possible in one go or separate make invocations; this is TBD.
That is quite a task and John Morris has taken it on, for which I am very
grateful, and I wish he would get help with this work from folks which are
fluent with the build system.
I also want to thank John for the patience and diligent work as we have been
working through the issues with this branch.
For the curious, the branch is called dynload-rtapi in my repo, and it is
rebased ontop of rtos-integration-preview3. I do not recommend pulling and
building this branch except for folks intending to help with it.
- Michael
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Emc-developers mailing list
Emc-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/emc-developers
