Now that configure is only doing compiler and host setup, adjust the
relevant documentation. It is also possible to build emulators with
ninja directly if one is so inclined.
Signed-off-by: Paolo Bonzini <pbonz...@redhat.com>
---
docs/devel/build-system.rst | 275 +++++++++++++++++++++---------------
1 file changed, 164 insertions(+), 111 deletions(-)
diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 4a733fc0a747..a25070fcaac8 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -4,30 +4,14 @@ The QEMU build system architecture
This document aims to help developers understand the architecture of the
QEMU build system. As with projects using GNU autotools, the QEMU build
-system has two stages, first the developer runs the "configure" script
+system has two stages; first the developer runs the "configure" script
to determine the local build environment characteristics, then they run
-"make" to build the project. There is about where the similarities with
+"make" to build the project. This is about where the similarities with
GNU autotools end, so try to forget what you know about them.
+The two general ways to perform a build are as follows:
-Stage 1: configure
-==================
-
-The QEMU configure script is written directly in shell, and should be
-compatible with any POSIX shell, hence it uses #!/bin/sh. An important
-implication of this is that it is important to avoid using bash-isms on
-development platforms where bash is the primary host.
-
-In contrast to autoconf scripts, QEMU's configure is expected to be
-silent while it is checking for features. It will only display output
-when an error occurs, or to show the final feature enablement summary
-on completion.
-
-Because QEMU uses the Meson build system under the hood, only VPATH
-builds are supported. There are two general ways to invoke configure &
-perform a build:
-
- - VPATH, build artifacts outside of QEMU source tree entirely::
+ - build artifacts outside of QEMU source tree entirely::
cd ../
mkdir build
@@ -35,80 +19,114 @@ perform a build:
../qemu/configure
make
- - VPATH, build artifacts in a subdir of QEMU source tree::
+ - build artifacts in a subdir of QEMU source tree::
mkdir build
cd build
../configure
make
-The configure script automatically recognizes
-command line options for which a same-named Meson option exists;
-dashes in the command line are replaced with underscores.
+Most of the actual build process uses Meson under the hood, therefore
+build artifacts cannot be placed in the source tree itself.
-Many checks on the compilation environment are still found in configure
-rather than ``meson.build``, but new checks should be added directly to
-``meson.build``.
-Patches are also welcome to move existing checks from the configure
-phase to ``meson.build``. When doing so, ensure that ``meson.build`` does
-not use anymore the keys that you have removed from ``config-host.mak``.
-Typically these will be replaced in ``meson.build`` by boolean variables,
-``get_option('optname')`` invocations, or ``dep.found()`` expressions.
-In general, the remaining checks have little or no interdependencies,
-so they can be moved one by one.
+Stage 1: configure
+==================
-Helper functions
-----------------
+The configure script has four tasks:
-The configure script provides a variety of helper functions to assist
-developers in checking for system features:
+ - detect the host architecture
-``do_cc $ARGS...``
- Attempt to run the system C compiler passing it $ARGS...
+ - list the targets for which to build emulators; the list of
+ targets also affects which firmware binaries and tests to build
-``do_cxx $ARGS...``
- Attempt to run the system C++ compiler passing it $ARGS...
+ - find the compilers (native and cross) used to build executables,
+ firmware and tests. The results are written as either Makefile
+ fragments (``config-host.mak``) or a Meson machine file
+ (``config-meson.cross``)
-``compile_object $CFLAGS``
- Attempt to compile a test program with the system C compiler using
- $CFLAGS. The test program must have been previously written to a file
- called $TMPC. The replacement in Meson is the compiler object ``cc``,
- which has methods such as ``cc.compiles()``,
- ``cc.check_header()``, ``cc.has_function()``.
+ - invoke Meson to perform the actual configuration step for the
+ emulator build
-``compile_prog $CFLAGS $LDFLAGS``
- Attempt to compile a test program with the system C compiler using
- $CFLAGS and link it with the system linker using $LDFLAGS. The test
- program must have been previously written to a file called $TMPC.
- The replacement in Meson is ``cc.find_library()`` and ``cc.links()``.
+The configure script automatically recognizes command line options for
+which a same-named Meson option exists; dashes in the command line are
+replaced with underscores.
+
+Almost all QEMU developers that need to modify the build system will
+only be concerned with Meson, and therefore can skip the rest of this
+section.
+
+
+Modifying ``configure``
+-----------------------
+
+``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore
+should be compatible with any POSIX shell. It is important to avoid
+using bash-isms to avoid breaking development platforms where bash is
+the primary host.
+
+The configure script provides a variety of functions to help writing
+portable shell code and providing consistent behavior across architectures
+and operating systems:
+
+``error_exit $MESSAGE $MORE...``
+ Print $MESSAGE to stderr, followed by $MORE... and then exit from the
+ configure script with non-zero status.
``has $COMMAND``
Determine if $COMMAND exists in the current environment, either as a
shell builtin, or executable binary, returning 0 on success. The
replacement in Meson is ``find_program()``.
+``probe_target_compiler $TARGET``
+ Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g.,
+ ``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``). If a working
+ compiler is present, return success and set variables ``$target_cc``,
+ ``$target_ar``, etc. to non-empty values.
+
+``write_target_makefile``
+ Write a Makefile fragment to stdout, exposing the result of the most
+ ``probe_target_compiler`` call as the usual Make variables (``CC``,
+ ``AR``, ``LD``, etc.).
+
+
+Configure does not generally perform tests for compiler options beyond
+basic checks to detect the host platform and ensure the compiler is
+functioning. These are performed using a few more helper functions:
+
+``compile_object $CFLAGS``
+ Attempt to compile a test program with the system C compiler using
+ $CFLAGS. The test program must have been previously written to a file
+ called $TMPC.
+
+``compile_prog $CFLAGS $LDFLAGS``
+ Attempt to compile a test program with the system C compiler using
+ $CFLAGS and link it with the system linker using $LDFLAGS. The test
+ program must have been previously written to a file called $TMPC.
+
``check_define $NAME``
- Determine if the macro $NAME is defined by the system C compiler
+ Determine if the macro $NAME is defined by the system C compiler.
+
+``do_compiler $CC $ARGS...``
+ Attempt to run the C compiler $CC, passing it $ARGS... This function
+ does not use flags passed via options such as ``--extra-cflags``, and
+ therefore can be used to check for cross compilers. However, most
+ such checks are done at ``make`` time instead (see for example the
+ ``cc-option`` macro in ``pc-bios/option-rom/Makefile``).
``write_c_skeleton``
Write a minimal C program main() function to the temporary file
- indicated by $TMPC
-
-``error_exit $MESSAGE $MORE...``
- Print $MESSAGE to stderr, followed by $MORE... and then exit from the
- configure script with non-zero status
+ indicated by $TMPC.
Stage 2: Meson
==============
-The Meson build system is currently used to describe the build
-process for:
+The Meson build system describes the build and install process for:
1) executables, which include:
- - Tools - ``qemu-img``, ``qemu-nbd``, ``qga`` (guest agent), etc
+ - Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc
- System emulators - ``qemu-system-$ARCH``
@@ -118,7 +136,8 @@ process for:
2) documentation
-3) ROMs, which can be either installed as binary blobs or compiled
+3) ROMs, whether provided as binary blobs in the QEMU distributions
+ or cross compiled under the direction of the configure script
4) other data files, such as icons or desktop files
@@ -273,8 +292,7 @@ system/userspace emulation target
Adding checks
-------------
-New checks should be added to Meson. Compiler checks can be as simple as
-the following::
+Compiler checks can be as simple as the following::
config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h'))
@@ -351,22 +369,30 @@ script, which may point to something other than the first
python3
binary on the path.
-Stage 3: makefiles
-==================
+Stage 3: Make
+=============
-The use of GNU make is required with the QEMU build system.
+The next step in building QEMU is to invoke make. GNU Make is required
+to build QEMU, and may be installed as ``gmake`` on some hosts.
-The output of Meson is a build.ninja file, which is used with the Ninja
-build system. QEMU uses a different approach, where Makefile rules are
-synthesized from the build.ninja file. The main Makefile includes these
-rules and wraps them so that e.g. submodules are built before QEMU.
-The resulting build system is largely non-recursive in nature, in
-contrast to common practices seen with automake.
+The output of Meson is a ``build.ninja`` file, which is used with the
+Ninja build tool. However, QEMU's build comprises other components than
+just the emulators (namely firmware and the tests in ``tests/tcg``) which
+need different cross compilers. The QEMU Makefile wraps both Ninja and
+the smaller build systems for firmware and tests; it also takes care of
+running ``configure`` again when the script changes. Apart from invoking
+these sub-Makefiles, the resulting build is largely non-recursive.
-Tests are also ran by the Makefile with the traditional ``make check``
-phony target, while benchmarks are run with ``make bench``. Meson test
-suites such as ``unit`` can be ran with ``make check-unit`` too. It is also
-possible to run tests defined in meson.build with ``meson test``.
+Tests, whether defined in ``meson.build`` or not, are also ran by the
+Makefile with the traditional ``make check`` phony target, while benchmarks
+are run with ``make bench``. Meson test suites such as ``unit`` can be ran
+with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson"
+tests for all targets.
+
+If desired, it is also possible to use ``ninja`` and ``meson test``,
+respectively to build emulators and run tests defined in meson.build.
+The main difference is that ``make`` needs the ``-jN`` flag in order to
+enable parallel builds or tests.
Useful make targets
-------------------
@@ -378,6 +404,7 @@ Useful make targets
Print the value of the variable VAR. Useful for debugging the build
system.
+
Important files for the build system
====================================
@@ -391,8 +418,7 @@ number of dynamically created files listed later.
``Makefile``
The main entry point used when invoking make to build all the components
of QEMU. The default 'all' target will naturally result in the build of
- every component. Makefile takes care of recursively building submodules
- directly via a non-recursive set of rules.
+ every component.
``*/meson.build``
The meson.build file in the root directory is the main entry point for the
@@ -401,59 +427,86 @@ number of dynamically created files listed later.
other meson.build files spread throughout the QEMU source tree.
``tests/Makefile.include``
- Rules for external test harnesses. These include the TCG tests,
- ``qemu-iotests`` and the Avocado-based integration tests.
+ Rules for external test harnesses. These include the TCG tests
+ and the Avocado-based integration tests.
``tests/docker/Makefile.include``
- Rules for Docker tests. Like tests/Makefile, this file is included
- directly by the top level Makefile, anything defined in this file will
- influence the entire build system.
+ Rules for Docker tests. Like ``tests/Makefile.include``, this file is
+ included directly by the top level Makefile, anything defined in this
+ file will influence the entire build system.
``tests/vm/Makefile.include``
- Rules for VM-based tests. Like tests/Makefile, this file is included
- directly by the top level Makefile, anything defined in this file will
- influence the entire build system.
+ Rules for VM-based tests. Like ``tests/Makefile.include``, this file is
+ included directly by the top level Makefile, anything defined in this
+ file will influence the entire build system.
Dynamically created files
-------------------------
-The following files are generated dynamically by configure in order to
-control the behaviour of the statically defined makefiles. This avoids
-the need for QEMU makefiles to go through any pre-processing as seen
-with autotools, where Makefile.am generates Makefile.in which generates
-Makefile.
+The following files are generated at run-time in order to control the
+behaviour of the Makefiles. This avoids the need for QEMU makefiles to
+go through any pre-processing as seen with autotools, where configure
+generates ``Makefile`` from ``Makefile.in`.
Built by configure:
``config-host.mak``
When configure has determined the characteristics of the build host it
- will write a long list of variables to config-host.mak file. This
- provides the various install directories, compiler / linker flags and a
+ will write them to this file for use in ``Makefile`` and to a smaller
+ extent ``meson.build``. These include the paths to various tools and a
variety of ``CONFIG_*`` variables related to optionally enabled features.
- This is imported by the top level Makefile and meson.build in order to
- tailor the build output.
- config-host.mak is also used as a dependency checking mechanism. If make
+ ``config-host.mak`` is also used as a dependency checking mechanism. If make
sees that the modification timestamp on configure is newer than that on
- config-host.mak, then configure will be re-run.
+ ``config-host.mak``, then configure will be re-run.
- The variables defined here are those which are applicable to all QEMU
- build outputs. Variables which are potentially different for each
- emulator target are defined by the next file...
+ The variables defined here apply to all QEMU
+ build outputs.
+
+``config-meson.cross``
+
+ A Meson "cross file" (or native file) used to communicate the paths to
+ the toolchain and other configuration options.
+
+``config.status``
+
+ A small shell script that will invoke configure again with the same
+ environment variables that were set during the first run. It's used to
+ rerun configure after changes to the source code, but it can also be
+ inspected manually to check the contents of the environment.
+
+``Makefile.prereqs``
+
+ A set of Makefile dependencies that order the build and execution of
+ firmware and tests after the container images and emulators that they
+ need.
+
+``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``,
``tests/tcg/*/config-target.mak``
+
+ Configuration variables used to build the firmware and TCG tests,
+ including paths to cross compilation toolchains.
Built by Meson:
-``${TARGET-NAME}-config-devices.mak``
- TARGET-NAME is again the name of a system or userspace emulator. The
- config-devices.mak file is automatically generated by make using the
- scripts/make_device_config.sh program, feeding it the
- default-configs/$TARGET-NAME file as input.
+``config-host.h``
+ Used by C code to determine the properties of the build environment
+ and the set of enabled features for the entire build.
-``config-host.h``, ``$TARGET_NAME-config-target.h``,
``$TARGET_NAME-config-devices.h``
- These files are used by source code to determine what features are
- enabled. They are generated from the contents of the corresponding
- ``*.mak`` files using Meson's ``configure_file()`` function.
+``${TARGET-NAME}-config-devices.mak``
+ TARGET-NAME is the name of a system emulator. The file is
+ generated by Meson using files under ``configs/devices`` as input.
+
+``${TARGET-NAME}-config-target.mak``
+ TARGET-NAME is the name of a system or usermode emulator. The file is
+ generated by Meson using files under ``configs/targets`` as input.
+
+``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h``
+ Used by C code to determine the properties and enabled
+ features for each target. enabled. They are generated from
+ the contents of the corresponding ``*.mak`` files using Meson's
+ ``configure_file()`` function; each target can include them using
+ the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively.
``build.ninja``
The build rules.
--
2.38.1
