Re: [PATCH RFC] build: respect top-level .config also for out-of-tree hypervisor builds
On 22.05.2023 17:49, Anthony PERARD wrote: > On Mon, May 08, 2023 at 08:23:43AM +0200, Jan Beulich wrote: >> On 05.05.2023 18:08, Anthony PERARD wrote: >>> On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote: With in-tree builds Config.mk includes a .config file (if present) from the top-level directory. Similar functionality is wanted with out-of- tree builds. Yet the concept of "top-level directory" becomes fuzzy in that case, because there is not really a requirement to have identical top-level directory structure in the output tree; in fact there's no need for anything top-level-ish there. Look for such a .config, but only if the tree layout matches (read: if the directory we're building in is named "xen"). >>> >>> Well, as long as the "xen/" part of the repository is the only build >>> system to be able to build out-of-srctree, there isn't going to be a >>> top-level .config possible in the build tree, as such .config will be >>> outside of the build tree. Reading outside of the build tree or source >>> tree might be problematic. >>> >>> It's a possibility that some project might want to just build the >>> hypervisor, and they happened to name the build tree "xen", but they >>> also have a ".config" use for something else. Kconfig is using ".config" >>> for other reason for example, like we do to build Xen. >> >> Right, that's what my first RFC remark is about. >> >>> It might be better to have a different name instead of ".config", and >>> putting it in the build tree rather than the parent directory. Maybe >>> ".xenbuild-config" ? >> >> Using a less ambiguous name is clearly an option. Putting the file in >> the (Xen) build directory, otoh, imo isn't: In the hope that further >> sub-trees would be enabled to build out-of-tree (qemu for instance in >> principle can already, and I guess at least some of stubdom's sub- >> packages also can), the present functionality of the top-level >> .config (or whatever its new name) also controlling those builds would >> better be retained. > > I'm not sure what out-of-tree build for the whole tree will look like, > but it probably going to be `/path/to/configure && make`. After that, > Config.mk should know what kind of build it's doing, and probably only > load ".config" in the build tree. Except that the hypervisor build still isn't really connected to ./configure's results. > In the meantime, it feels out of place > for xen/Makefile or xen/Rules.mk to load a ".config" that would be > present in the parent directory of the build dir. Right, hence me looking for possible alternatives (and using this approach only for the apparent lack thereof). >>> I've been using the optional makefile named "xen-version" to adjust >>> variable of the build system, with content like: >>> >>> export XEN_TARGET_ARCH=arm64 >>> export CROSS_COMPILE=aarch64-linux-gnu- >>> >>> so that I can have multiple build tree for different arch, and not have >>> to do anything other than running make and do the expected build. Maybe >>> that's not possible because for some reason, the build system still >>> reconfigure some variable when entering a sub-directory, but that's a >>> start. >> >> Hmm, interesting idea. I could (ab)use this at least in the short >> term. Yet even then the file would further need including from >> Rules.mk. Requiring all variables defined there to be exported isn't >> a good idea, imo. Iirc not all make variables can usefully be >> exported. For example, I have a local extension to CROSS_COMPILE in >> place, which uses a variable with a dash in its name. >> Signed-off-by: Jan Beulich --- RFC: The directory name based heuristic of course isn't nice. But I couldn't think of anything better. Suggestions? >>> >>> I can only thing of looking at a file that is in the build-tree rather >>> than outside of it. A file named ".xenbuild-config" proposed early for >>> example. >>> RFC: There also being a .config in the top-level source dir would be a little problematic: It would be included _after_ the one in the object tree. Yet if such a scenario is to be expected/supported at all, it makes more sense the other way around. >>> >>> Well, that would mean teaching Config.mk about out-of-tree build that >>> part of the repository is capable of, or better would be to stop trying >>> to read ".config" from Config.mk, and handle the file in the different >>> sub-build systems. >> >> Hmm, is that a viable option? Or put differently: Wouldn't this mean doing >> away with ./Config.mk altogether? Which in turn would mean no central >> place anymore where XEN_TARGET_ARCH and friends could be overridden. (I >> view this as a capability that we want to retain, if nothing else then for >> the - see above - future option of allowing more than just xen/ to be >> built out-of-tree.) > > No, I'm not trying to get rid of Config.mk. There's a few thing in it > that I'd like to
Re: [PATCH RFC] build: respect top-level .config also for out-of-tree hypervisor builds
On Mon, May 08, 2023 at 08:23:43AM +0200, Jan Beulich wrote: > On 05.05.2023 18:08, Anthony PERARD wrote: > > On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote: > >> With in-tree builds Config.mk includes a .config file (if present) from > >> the top-level directory. Similar functionality is wanted with out-of- > >> tree builds. Yet the concept of "top-level directory" becomes fuzzy in > >> that case, because there is not really a requirement to have identical > >> top-level directory structure in the output tree; in fact there's no > >> need for anything top-level-ish there. Look for such a .config, but only > >> if the tree layout matches (read: if the directory we're building in is > >> named "xen"). > > > > Well, as long as the "xen/" part of the repository is the only build > > system to be able to build out-of-srctree, there isn't going to be a > > top-level .config possible in the build tree, as such .config will be > > outside of the build tree. Reading outside of the build tree or source > > tree might be problematic. > > > > It's a possibility that some project might want to just build the > > hypervisor, and they happened to name the build tree "xen", but they > > also have a ".config" use for something else. Kconfig is using ".config" > > for other reason for example, like we do to build Xen. > > Right, that's what my first RFC remark is about. > > > It might be better to have a different name instead of ".config", and > > putting it in the build tree rather than the parent directory. Maybe > > ".xenbuild-config" ? > > Using a less ambiguous name is clearly an option. Putting the file in > the (Xen) build directory, otoh, imo isn't: In the hope that further > sub-trees would be enabled to build out-of-tree (qemu for instance in > principle can already, and I guess at least some of stubdom's sub- > packages also can), the present functionality of the top-level > .config (or whatever its new name) also controlling those builds would > better be retained. I'm not sure what out-of-tree build for the whole tree will look like, but it probably going to be `/path/to/configure && make`. After that, Config.mk should know what kind of build it's doing, and probably only load ".config" in the build tree. In the meantime, it feels out of place for xen/Makefile or xen/Rules.mk to load a ".config" that would be present in the parent directory of the build dir. > > I've been using the optional makefile named "xen-version" to adjust > > variable of the build system, with content like: > > > > export XEN_TARGET_ARCH=arm64 > > export CROSS_COMPILE=aarch64-linux-gnu- > > > > so that I can have multiple build tree for different arch, and not have > > to do anything other than running make and do the expected build. Maybe > > that's not possible because for some reason, the build system still > > reconfigure some variable when entering a sub-directory, but that's a > > start. > > Hmm, interesting idea. I could (ab)use this at least in the short > term. Yet even then the file would further need including from > Rules.mk. Requiring all variables defined there to be exported isn't > a good idea, imo. Iirc not all make variables can usefully be > exported. For example, I have a local extension to CROSS_COMPILE in > place, which uses a variable with a dash in its name. > > >> Signed-off-by: Jan Beulich > >> --- > >> RFC: The directory name based heuristic of course isn't nice. But I > >> couldn't think of anything better. Suggestions? > > > > I can only thing of looking at a file that is in the build-tree rather > > than outside of it. A file named ".xenbuild-config" proposed early for > > example. > > > >> RFC: There also being a .config in the top-level source dir would be a > >> little problematic: It would be included _after_ the one in the > >> object tree. Yet if such a scenario is to be expected/supported at > >> all, it makes more sense the other way around. > > > > Well, that would mean teaching Config.mk about out-of-tree build that > > part of the repository is capable of, or better would be to stop trying > > to read ".config" from Config.mk, and handle the file in the different > > sub-build systems. > > Hmm, is that a viable option? Or put differently: Wouldn't this mean doing > away with ./Config.mk altogether? Which in turn would mean no central > place anymore where XEN_TARGET_ARCH and friends could be overridden. (I > view this as a capability that we want to retain, if nothing else then for > the - see above - future option of allowing more than just xen/ to be > built out-of-tree.) No, I'm not trying to get rid of Config.mk. There's a few thing in it that I'd like to remove from it, but not everything. I don't know how to deal with ".config" at the moment, but I guess that if Config.mk knew about out-of-tree build, it probably should only read one ".config", the one in the build tree. -- Anthony PERARD
Re: [PATCH RFC] build: respect top-level .config also for out-of-tree hypervisor builds
On 05.05.2023 18:08, Anthony PERARD wrote: > On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote: >> With in-tree builds Config.mk includes a .config file (if present) from >> the top-level directory. Similar functionality is wanted with out-of- >> tree builds. Yet the concept of "top-level directory" becomes fuzzy in >> that case, because there is not really a requirement to have identical >> top-level directory structure in the output tree; in fact there's no >> need for anything top-level-ish there. Look for such a .config, but only >> if the tree layout matches (read: if the directory we're building in is >> named "xen"). > > Well, as long as the "xen/" part of the repository is the only build > system to be able to build out-of-srctree, there isn't going to be a > top-level .config possible in the build tree, as such .config will be > outside of the build tree. Reading outside of the build tree or source > tree might be problematic. > > It's a possibility that some project might want to just build the > hypervisor, and they happened to name the build tree "xen", but they > also have a ".config" use for something else. Kconfig is using ".config" > for other reason for example, like we do to build Xen. Right, that's what my first RFC remark is about. > It might be better to have a different name instead of ".config", and > putting it in the build tree rather than the parent directory. Maybe > ".xenbuild-config" ? Using a less ambiguous name is clearly an option. Putting the file in the (Xen) build directory, otoh, imo isn't: In the hope that further sub-trees would be enabled to build out-of-tree (qemu for instance in principle can already, and I guess at least some of stubdom's sub- packages also can), the present functionality of the top-level .config (or whatever its new name) also controlling those builds would better be retained. > I've been using the optional makefile named "xen-version" to adjust > variable of the build system, with content like: > > export XEN_TARGET_ARCH=arm64 > export CROSS_COMPILE=aarch64-linux-gnu- > > so that I can have multiple build tree for different arch, and not have > to do anything other than running make and do the expected build. Maybe > that's not possible because for some reason, the build system still > reconfigure some variable when entering a sub-directory, but that's a > start. Hmm, interesting idea. I could (ab)use this at least in the short term. Yet even then the file would further need including from Rules.mk. Requiring all variables defined there to be exported isn't a good idea, imo. Iirc not all make variables can usefully be exported. For example, I have a local extension to CROSS_COMPILE in place, which uses a variable with a dash in its name. >> Signed-off-by: Jan Beulich >> --- >> RFC: The directory name based heuristic of course isn't nice. But I >> couldn't think of anything better. Suggestions? > > I can only thing of looking at a file that is in the build-tree rather > than outside of it. A file named ".xenbuild-config" proposed early for > example. > >> RFC: There also being a .config in the top-level source dir would be a >> little problematic: It would be included _after_ the one in the >> object tree. Yet if such a scenario is to be expected/supported at >> all, it makes more sense the other way around. > > Well, that would mean teaching Config.mk about out-of-tree build that > part of the repository is capable of, or better would be to stop trying > to read ".config" from Config.mk, and handle the file in the different > sub-build systems. Hmm, is that a viable option? Or put differently: Wouldn't this mean doing away with ./Config.mk altogether? Which in turn would mean no central place anymore where XEN_TARGET_ARCH and friends could be overridden. (I view this as a capability that we want to retain, if nothing else then for the - see above - future option of allowing more than just xen/ to be built out-of-tree.) > Or just let people writing ".config" files to handle > the cases in those .config files. I'm afraid I don't see what you mean here. >> --- a/xen/Makefile >> +++ b/xen/Makefile >> @@ -236,8 +236,17 @@ endif >> >> include scripts/Kbuild.include >> >> -# Don't break if the build process wasn't called from the top level >> -# we need XEN_TARGET_ARCH to generate the proper config >> +# Don't break if the build process wasn't called from the top level. We >> need >> +# XEN_TARGET_ARCH to generate the proper config. If building outside of the >> +# source tree also check whether we need to include a "top-level" .config: >> +# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree. >> +ifeq ($(building_out_of_srctree),1) >> +# Try to avoid including a random unrelated .config: Assume our parent dir >> +# is a "top-level" one only when the objtree is .../xen. >> +ifeq ($(patsubst %/xen,,$(abs_objtree)),) >> +-include ../.config >> +endif >> +endif >>
Re: [PATCH RFC] build: respect top-level .config also for out-of-tree hypervisor builds
On Wed, Mar 15, 2023 at 03:58:59PM +0100, Jan Beulich wrote: > With in-tree builds Config.mk includes a .config file (if present) from > the top-level directory. Similar functionality is wanted with out-of- > tree builds. Yet the concept of "top-level directory" becomes fuzzy in > that case, because there is not really a requirement to have identical > top-level directory structure in the output tree; in fact there's no > need for anything top-level-ish there. Look for such a .config, but only > if the tree layout matches (read: if the directory we're building in is > named "xen"). Well, as long as the "xen/" part of the repository is the only build system to be able to build out-of-srctree, there isn't going to be a top-level .config possible in the build tree, as such .config will be outside of the build tree. Reading outside of the build tree or source tree might be problematic. It's a possibility that some project might want to just build the hypervisor, and they happened to name the build tree "xen", but they also have a ".config" use for something else. Kconfig is using ".config" for other reason for example, like we do to build Xen. It might be better to have a different name instead of ".config", and putting it in the build tree rather than the parent directory. Maybe ".xenbuild-config" ? I've been using the optional makefile named "xen-version" to adjust variable of the build system, with content like: export XEN_TARGET_ARCH=arm64 export CROSS_COMPILE=aarch64-linux-gnu- so that I can have multiple build tree for different arch, and not have to do anything other than running make and do the expected build. Maybe that's not possible because for some reason, the build system still reconfigure some variable when entering a sub-directory, but that's a start. > Signed-off-by: Jan Beulich > --- > RFC: The directory name based heuristic of course isn't nice. But I > couldn't think of anything better. Suggestions? I can only thing of looking at a file that is in the build-tree rather than outside of it. A file named ".xenbuild-config" proposed early for example. > RFC: There also being a .config in the top-level source dir would be a > little problematic: It would be included _after_ the one in the > object tree. Yet if such a scenario is to be expected/supported at > all, it makes more sense the other way around. Well, that would mean teaching Config.mk about out-of-tree build that part of the repository is capable of, or better would be to stop trying to read ".config" from Config.mk, and handle the file in the different sub-build systems. Or just let people writing ".config" files to handle the cases in those .config files. > --- a/xen/Makefile > +++ b/xen/Makefile > @@ -236,8 +236,17 @@ endif > > include scripts/Kbuild.include > > -# Don't break if the build process wasn't called from the top level > -# we need XEN_TARGET_ARCH to generate the proper config > +# Don't break if the build process wasn't called from the top level. We need > +# XEN_TARGET_ARCH to generate the proper config. If building outside of the > +# source tree also check whether we need to include a "top-level" .config: > +# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree. > +ifeq ($(building_out_of_srctree),1) > +# Try to avoid including a random unrelated .config: Assume our parent dir > +# is a "top-level" one only when the objtree is .../xen. > +ifeq ($(patsubst %/xen,,$(abs_objtree)),) > +-include ../.config > +endif > +endif > include $(XEN_ROOT)/Config.mk > > # Set ARCH/SUBARCH appropriately. > --- a/xen/Rules.mk > +++ b/xen/Rules.mk > @@ -17,6 +17,13 @@ __build: > > -include $(objtree)/include/config/auto.conf > > +# See commentary around the similar contruct in Makefile. > +ifneq ($(abs_objtree),$(abs_srctree)) > +ifeq ($(patsubst %/xen,,$(abs_objtree)),) > +../.config: ; > +-include ../.config > +endif > +endif > include $(XEN_ROOT)/Config.mk > include $(srctree)/scripts/Kbuild.include There's another makefile, "scripts/Makefile.clean", doesn't this would need to be change as well? Cheers, -- Anthony PERARD
Ping: [PATCH RFC] build: respect top-level .config also for out-of-tree hypervisor builds
On 15.03.2023 15:58, Jan Beulich wrote: > With in-tree builds Config.mk includes a .config file (if present) from > the top-level directory. Similar functionality is wanted with out-of- > tree builds. Yet the concept of "top-level directory" becomes fuzzy in > that case, because there is not really a requirement to have identical > top-level directory structure in the output tree; in fact there's no > need for anything top-level-ish there. Look for such a .config, but only > if the tree layout matches (read: if the directory we're building in is > named "xen"). > > Signed-off-by: Jan Beulich > --- > RFC: The directory name based heuristic of course isn't nice. But I > couldn't think of anything better. Suggestions? > > RFC: There also being a .config in the top-level source dir would be a > little problematic: It would be included _after_ the one in the > object tree. Yet if such a scenario is to be expected/supported at > all, it makes more sense the other way around. Anyone? I'm certainly okay for my approach to be rejected, but I'd like to see out-of-tree builds to reach functional parity with in-tree ones. Jan > --- a/xen/Makefile > +++ b/xen/Makefile > @@ -236,8 +236,17 @@ endif > > include scripts/Kbuild.include > > -# Don't break if the build process wasn't called from the top level > -# we need XEN_TARGET_ARCH to generate the proper config > +# Don't break if the build process wasn't called from the top level. We need > +# XEN_TARGET_ARCH to generate the proper config. If building outside of the > +# source tree also check whether we need to include a "top-level" .config: > +# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree. > +ifeq ($(building_out_of_srctree),1) > +# Try to avoid including a random unrelated .config: Assume our parent dir > +# is a "top-level" one only when the objtree is .../xen. > +ifeq ($(patsubst %/xen,,$(abs_objtree)),) > +-include ../.config > +endif > +endif > include $(XEN_ROOT)/Config.mk > > # Set ARCH/SUBARCH appropriately. > --- a/xen/Rules.mk > +++ b/xen/Rules.mk > @@ -17,6 +17,13 @@ __build: > > -include $(objtree)/include/config/auto.conf > > +# See commentary around the similar contruct in Makefile. > +ifneq ($(abs_objtree),$(abs_srctree)) > +ifeq ($(patsubst %/xen,,$(abs_objtree)),) > +../.config: ; > +-include ../.config > +endif > +endif > include $(XEN_ROOT)/Config.mk > include $(srctree)/scripts/Kbuild.include > >
[PATCH RFC] build: respect top-level .config also for out-of-tree hypervisor builds
With in-tree builds Config.mk includes a .config file (if present) from the top-level directory. Similar functionality is wanted with out-of- tree builds. Yet the concept of "top-level directory" becomes fuzzy in that case, because there is not really a requirement to have identical top-level directory structure in the output tree; in fact there's no need for anything top-level-ish there. Look for such a .config, but only if the tree layout matches (read: if the directory we're building in is named "xen"). Signed-off-by: Jan Beulich --- RFC: The directory name based heuristic of course isn't nice. But I couldn't think of anything better. Suggestions? RFC: There also being a .config in the top-level source dir would be a little problematic: It would be included _after_ the one in the object tree. Yet if such a scenario is to be expected/supported at all, it makes more sense the other way around. --- a/xen/Makefile +++ b/xen/Makefile @@ -236,8 +236,17 @@ endif include scripts/Kbuild.include -# Don't break if the build process wasn't called from the top level -# we need XEN_TARGET_ARCH to generate the proper config +# Don't break if the build process wasn't called from the top level. We need +# XEN_TARGET_ARCH to generate the proper config. If building outside of the +# source tree also check whether we need to include a "top-level" .config: +# Config.mk, using $(XEN_ROOT)/.config, would look only in the source tree. +ifeq ($(building_out_of_srctree),1) +# Try to avoid including a random unrelated .config: Assume our parent dir +# is a "top-level" one only when the objtree is .../xen. +ifeq ($(patsubst %/xen,,$(abs_objtree)),) +-include ../.config +endif +endif include $(XEN_ROOT)/Config.mk # Set ARCH/SUBARCH appropriately. --- a/xen/Rules.mk +++ b/xen/Rules.mk @@ -17,6 +17,13 @@ __build: -include $(objtree)/include/config/auto.conf +# See commentary around the similar contruct in Makefile. +ifneq ($(abs_objtree),$(abs_srctree)) +ifeq ($(patsubst %/xen,,$(abs_objtree)),) +../.config: ; +-include ../.config +endif +endif include $(XEN_ROOT)/Config.mk include $(srctree)/scripts/Kbuild.include