Re: [PATCH 8/8] configure: automatically parse command line for meson -D options
On 23/01/21 04:30, 罗勇刚(Yonggang Luo) wrote: > We could and I did it in the first version. However it's ugly that the user has to use --python on some setups in order to get a full help message. Yeap, but finally configure should gone, so I think --python are acceptable by user, just need make sure to be noticed when the default python are not python3 When configure will be gone, these 200 lines of Perl will disappear with it, so we don't have to think for future users and developers, but for the present. I don't expect configure or Makefile to go away completely, because for example things such as "git submodule" handling are easier to do in the shell. However, we also need to evaluate what's the road to a minimal configure (just like our Makefile is pretty minimal, only tests/tcg still has lots of build logic). In that respect, automatic parsing command line simplifies moving things from configure to meson, and that is true no matter what language the parser is written in. (BTW, I'm not such a Perl fan. I have only ever written like 3 or 4 programs in Perl). Paolo
Re: [PATCH 8/8] configure: automatically parse command line for meson -D options
On Sat, Jan 23, 2021 at 4:44 AM Paolo Bonzini wrote: > > > > Il ven 22 gen 2021, 09:00 罗勇刚(Yonggang Luo) ha scritto: >> >> Hi Paolo, as python and meson are required dependencies to building qemu now, >> can we detecting python/meson at the very begining of configure, >> even before the --help parameter. > > > We could and I did it in the first version. However it's ugly that the user has to use --python on some setups in order to get a full help message. Yeap, but finally configure should gone, so I think --python are acceptable by user, just need make sure to be noticed when the default python are not python3 > > Paolo > >> >> On Wed, Jan 13, 2021 at 6:08 AM Paolo Bonzini wrote: >> > >> > On 13/01/21 11:31, Daniel P. Berrangé wrote: >> > >> meson-buildoptions.json | 717 >> > > I'm not a fan of seeing this file introduced as it has significant >> > > overlap with meson_options.txt.I feel like the latter has enough >> > > information present to do an acceptable job for help output. After >> > > all that's sufficient if we were using meson directly. >> > >> > Sorry, I missed this remark. meson-buildoptions.json is not >> > hand-written. It is the result of Meson's own parsing meson_options.txt >> > exported as JSON. >> > >> > In the commit message "because we parse command-line options before >> > meson is available, the introspection output is stored in the source >> > tree. This is the reason for the unattractive diffstat; the number of >> > JSON lines added is higher than the number of configure lines removed. >> > Of course the latter are code that must be maintained manually and the >> > former is not". >> > >> > Paolo >> > >> > >> >> >> -- >> 此致 >> 礼 >> 罗勇刚 >> Yours >> sincerely, >> Yonggang Luo -- 此致 礼 罗勇刚 Yours sincerely, Yonggang Luo
Re: [PATCH 8/8] configure: automatically parse command line for meson -D options
Il ven 22 gen 2021, 09:00 罗勇刚(Yonggang Luo) ha scritto: > Hi Paolo, as python and meson are required dependencies to building qemu > now, > can we detecting python/meson at the very begining of configure, > even before the --help parameter. > We could and I did it in the first version. However it's ugly that the user has to use --python on some setups in order to get a full help message. Paolo > On Wed, Jan 13, 2021 at 6:08 AM Paolo Bonzini wrote: > > > > On 13/01/21 11:31, Daniel P. Berrangé wrote: > > >> meson-buildoptions.json | 717 > > > > I'm not a fan of seeing this file introduced as it has significant > > > overlap with meson_options.txt.I feel like the latter has enough > > > information present to do an acceptable job for help output. After > > > all that's sufficient if we were using meson directly. > > > > Sorry, I missed this remark. meson-buildoptions.json is not > > hand-written. It is the result of Meson's own parsing meson_options.txt > > exported as JSON. > > > > In the commit message "because we parse command-line options before > > meson is available, the introspection output is stored in the source > > tree. This is the reason for the unattractive diffstat; the number of > > JSON lines added is higher than the number of configure lines removed. > > Of course the latter are code that must be maintained manually and the > > former is not". > > > > Paolo > > > > > > > -- > 此致 > 礼 > 罗勇刚 > Yours > sincerely, > Yonggang Luo >
Re: [PATCH 8/8] configure: automatically parse command line for meson -D options
Hi Paolo, as python and meson are required dependencies to building qemu now, can we detecting python/meson at the very begining of configure, even before the --help parameter. On Wed, Jan 13, 2021 at 6:08 AM Paolo Bonzini wrote: > > On 13/01/21 11:31, Daniel P. Berrangé wrote: > >> meson-buildoptions.json | 717 > > I'm not a fan of seeing this file introduced as it has significant > > overlap with meson_options.txt.I feel like the latter has enough > > information present to do an acceptable job for help output. After > > all that's sufficient if we were using meson directly. > > Sorry, I missed this remark. meson-buildoptions.json is not > hand-written. It is the result of Meson's own parsing meson_options.txt > exported as JSON. > > In the commit message "because we parse command-line options before > meson is available, the introspection output is stored in the source > tree. This is the reason for the unattractive diffstat; the number of > JSON lines added is higher than the number of configure lines removed. > Of course the latter are code that must be maintained manually and the > former is not". > > Paolo > > -- 此致 礼 罗勇刚 Yours sincerely, Yonggang Luo
Re: [PATCH 8/8] configure: automatically parse command line for meson -D options
On 13/01/21 11:31, Daniel P. Berrangé wrote: meson-buildoptions.json | 717 I'm not a fan of seeing this file introduced as it has significant overlap with meson_options.txt.I feel like the latter has enough information present to do an acceptable job for help output. After all that's sufficient if we were using meson directly. Sorry, I missed this remark. meson-buildoptions.json is not hand-written. It is the result of Meson's own parsing meson_options.txt exported as JSON. In the commit message "because we parse command-line options before meson is available, the introspection output is stored in the source tree. This is the reason for the unattractive diffstat; the number of JSON lines added is higher than the number of configure lines removed. Of course the latter are code that must be maintained manually and the former is not". Paolo
Re: [PATCH 8/8] configure: automatically parse command line for meson -D options
On 13/01/21 11:31, Daniel P. Berrangé wrote: - because we need to run the script to generate the full help, we cannot rely on the user supplying the path to a Python interpreter with --python. For this reason, the script is written in Perl. Perl 5 is universally available as "/usr/bin/env perl", while (even ignoring the Python 2/3 difference) some systems do not have a "python" or "python3" binary on the path. Can't we just use "/usr/bin/env python3", and if that doesn't exist in $PATH, simply show truncated --help output, with a message requesting that they pass --python to see full help. We can and it's what the RFC did, but I found it to be ugly for the user. For example the rest of the build defaults not to "/usr/bin/env python3" but to whatever Python interpreter /usr/bin/meson uses. Not that Perl is pretty, :) but developer ugliness ranks a notch lower than user-visible ugliness. I cannot think of anything else that is already required to build QEMU and can parse JSON. I don't expect this script to see any change ever, except for occasionally updating SKIP_OPTIONS. Meson introspection data is backwards-compatible. Paolo
Re: [PATCH 8/8] configure: automatically parse command line for meson -D options
On Thu, Jan 07, 2021 at 03:00:39PM +0100, Paolo Bonzini wrote:
> Right now meson_options.txt lists almost 60 options. Each option
> needs code in configure to parse it and pass the option down to Meson as
> a -D command-line argument; in addition the default must be duplicated
> between configure and meson_options.txt.
>
> This series tries to remove the code duplication by passing unknown
> --enable and --disable options to a Perl program, and using Meson's
> introspection support to match those to meson_options.txt.
> About 80% of the options can be handled completely by the new
> mechanism. Five meson options are not of the --enable/--disable
> kind. Six more need to be parsed in configure for various reasons
> documented in the patch, but they still have their help automatically
> generated.
>
> The advantages are simple to explain, and are basically what you
> can expect from an introspection-based system:
>
> - there is obviously much less code in configure. About 1000 lines
> of the script deal with command line parsing, and the patch removes
> a quarter of them.
>
> - the script is higher quality than the repetitive code in configure.
> Help is generally more complete and useful, for example it consistently
> lists defaults as well as the possible choices if they are not just
> enabled/disabled/auto. Parsing is more consistent too, for example
> --enable-slirp and --enable-blobs were not supported.
>
> - new Meson options do not need any change to the configure script.
> This increases the attractiveness of converting options from
> hand-crafted parsing and config-host.mak to Meson.
>
> The disadvantages are:
>
> - a few options change name: --enable-tcmalloc and --enable-jemalloc
> become --enable-malloc={tcmalloc,jemalloc}; --disable-blobs becomes
> --disable-install-blobs.
>
> - because we need to run the script to generate the full help, we
> cannot rely on the user supplying the path to a Python interpreter
> with --python. For this reason, the script is written in Perl.
> Perl 5 is universally available as "/usr/bin/env perl", while
> (even ignoring the Python 2/3 difference) some systems do not
> have a "python" or "python3" binary on the path.
Can't we just use "/usr/bin/env python3", and if that doesn't
exist in $PATH, simply show truncated --help output, with a
message requesting that they pass --python to see full help.
>
> - because we parse command-line options before meson is available,
> the introspection output is stored in the source tree. This is
> the reason for the unattractive diffstat; the number of JSON lines
> added is higher than the number of configure lines removed.
> Of course the latter are code that must be maintained manually and
> the former is not.
>
> Note that the output of "meson introspect --buildoptions" is massaged
> slightly, in order to make it more stable and avoid horrible conflicts
> on every modification to meson_options.txt.
>
> Signed-off-by: Paolo Bonzini
> ---
> Makefile| 11 +
> configure | 371 ++--
> docs/devel/build-system.rst | 49 +-
> meson-buildoptions.json | 717
I'm not a fan of seeing this file introduced as it has significant
overlap with meson_options.txt.I feel like the latter has enough
information present to do an acceptable job for help output. After
all that's sufficient if we were using meson directly.
Regards,
Daniel
--
|: https://berrange.com -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
[PATCH 8/8] configure: automatically parse command line for meson -D options
Right now meson_options.txt lists almost 60 options. Each option
needs code in configure to parse it and pass the option down to Meson as
a -D command-line argument; in addition the default must be duplicated
between configure and meson_options.txt.
This series tries to remove the code duplication by passing unknown
--enable and --disable options to a Perl program, and using Meson's
introspection support to match those to meson_options.txt.
About 80% of the options can be handled completely by the new
mechanism. Five meson options are not of the --enable/--disable
kind. Six more need to be parsed in configure for various reasons
documented in the patch, but they still have their help automatically
generated.
The advantages are simple to explain, and are basically what you
can expect from an introspection-based system:
- there is obviously much less code in configure. About 1000 lines
of the script deal with command line parsing, and the patch removes
a quarter of them.
- the script is higher quality than the repetitive code in configure.
Help is generally more complete and useful, for example it consistently
lists defaults as well as the possible choices if they are not just
enabled/disabled/auto. Parsing is more consistent too, for example
--enable-slirp and --enable-blobs were not supported.
- new Meson options do not need any change to the configure script.
This increases the attractiveness of converting options from
hand-crafted parsing and config-host.mak to Meson.
The disadvantages are:
- a few options change name: --enable-tcmalloc and --enable-jemalloc
become --enable-malloc={tcmalloc,jemalloc}; --disable-blobs becomes
--disable-install-blobs.
- because we need to run the script to generate the full help, we
cannot rely on the user supplying the path to a Python interpreter
with --python. For this reason, the script is written in Perl.
Perl 5 is universally available as "/usr/bin/env perl", while
(even ignoring the Python 2/3 difference) some systems do not
have a "python" or "python3" binary on the path.
- because we parse command-line options before meson is available,
the introspection output is stored in the source tree. This is
the reason for the unattractive diffstat; the number of JSON lines
added is higher than the number of configure lines removed.
Of course the latter are code that must be maintained manually and
the former is not.
Note that the output of "meson introspect --buildoptions" is massaged
slightly, in order to make it more stable and avoid horrible conflicts
on every modification to meson_options.txt.
Signed-off-by: Paolo Bonzini
---
Makefile| 11 +
configure | 371 ++--
docs/devel/build-system.rst | 49 +-
meson-buildoptions.json | 717
meson_options.txt | 9 +-
scripts/configure-parse-buildoptions.pl | 205 +++
6 files changed, 993 insertions(+), 369 deletions(-)
create mode 100644 meson-buildoptions.json
create mode 100755 scripts/configure-parse-buildoptions.pl
diff --git a/Makefile b/Makefile
index fb9923ff22..eca42f08d3 100644
--- a/Makefile
+++ b/Makefile
@@ -142,6 +142,17 @@ ifneq ($(MESON),)
Makefile.mtest: build.ninja scripts/mtest2make.py
$(MESON) introspect --targets --tests --benchmarks | $(PYTHON)
scripts/mtest2make.py > $@
-include Makefile.mtest
+
+# python is used to keep the result stable
+.PHONY: update-buildoptions
+all update-buildoptions: $(SRC_PATH)/meson-buildoptions.json
+$(SRC_PATH)/meson-buildoptions.json: $(SRC_PATH)/meson_options.txt
+ $(MESON) introspect --buildoptions $(SRC_PATH)/meson.build | $(PYTHON)
-c \
+'import json, sys; print( \
+ json.dumps(sorted([x for x in json.loads(sys.stdin.read()) \
+ if x["section"] == "user"], \
+ key=lambda x: x["name"]), \
+ indent=2))' > $@
endif
# 4. Rules to bridge to other makefiles
diff --git a/configure b/configure
index 41866cc38e..64e19026f3 100755
--- a/configure
+++ b/configure
@@ -309,47 +309,19 @@ for opt do
esac
done
-brlapi="auto"
-curl="auto"
-iconv="auto"
-curses="auto"
-docs="auto"
-fdt="auto"
netmap="no"
-sdl="auto"
-sdl_image="auto"
-virtiofsd="auto"
-virtfs="auto"
-libudev="auto"
-mpath="auto"
-vnc="enabled"
-sparse="auto"
vde="$default_feature"
-vnc_sasl="auto"
-vnc_jpeg="auto"
-vnc_png="auto"
-xkbcommon="auto"
-xen="$default_feature"
xen_ctrl_version="$default_feature"
-xen_pci_passthrough="auto"
linux_aio="$default_feature"
linux_io_uring="$default_feature"
-cap_ng="auto"
-attr="auto"
xfs="$default_feature"
-tcg="enabled"
membarrier="$default_feature"
vhost_net="$default_feature"
vhost_crypto="$default_feature"
vhost_scsi="$default_feature"
vhost_vsock="$default_feature"
vhost_user="no"
