Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
On Sat, 27 Aug 2022 at 19:00, Alexander Kanavin via lists.openembedded.org wrote: > > This is making the AB fail: > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/4140/steps/7/logs/stdio > > I believe this needs the patch for poky that moves the template to the > expected location, was it included? I just started a test with both > patches: > https://git.yoctoproject.org/poky-contrib/log/?h=akanavin/force-templateconf > > and it is able to proceed fine: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/4144 ...except the change does break eSDKs, so that needs to be fixed first. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169968): https://lists.openembedded.org/g/openembedded-core/message/169968 Mute This Topic: https://lists.openembedded.org/mt/93225500/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
On Sat, 27 Aug 2022 at 09:00, Luca Ceresoli wrote: > This is making the AB fail: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/4140/steps/7/logs/stdio I believe this needs the patch for poky that moves the template to the expected location, was it included? I just started a test with both patches: https://git.yoctoproject.org/poky-contrib/log/?h=akanavin/force-templateconf and it is able to proceed fine: https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/4144 Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169933): https://lists.openembedded.org/g/openembedded-core/message/169933 Mute This Topic: https://lists.openembedded.org/mt/93225500/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
Hi Alex,
On Wed, 24 Aug 2022 14:42:34 +0200
"Alexander Kanavin" wrote:
> specifically that ../../layer.conf exists, and that second-from-last
> component in the path is 'templates'.
>
> Signed-off-by: Alexander Kanavin
> ---
> scripts/oe-setup-builddir | 5 +
> 1 file changed, 5 insertions(+)
>
> diff --git a/scripts/oe-setup-builddir b/scripts/oe-setup-builddir
> index 975619789a..db26e3b138 100755
> --- a/scripts/oe-setup-builddir
> +++ b/scripts/oe-setup-builddir
> @@ -61,6 +61,11 @@ if [ -n "$TEMPLATECONF" ]; then
> echo >&2 "Error: TEMPLATECONF value points to nonexistent
> directory '$TEMPLATECONF'"
> exit 1
> fi
> +templatesdir=$(python3 -c "import sys;
> print(sys.argv[1].strip('/').split('/')[-2])" $TEMPLATECONF)
> +if [ ! -f "$TEMPLATECONF/../../layer.conf" -o $templatesdir !=
> "templates" ]; then
> +echo >&2 "Error: TEMPLATECONF value must point to
> meta-some-layer/conf/templates/template-name"
> +exit 1
> +fi
This is making the AB fail:
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/4140/steps/7/logs/stdio
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169927):
https://lists.openembedded.org/g/openembedded-core/message/169927
Mute This Topic: https://lists.openembedded.org/mt/93225500/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
> -Original Message- > From: Alexander Kanavin > Sent: den 25 augusti 2022 11:19 > To: Peter Kjellerstedt > Cc: openembedded-core@lists.openembedded.org; Alexander Kanavin > ; Richard Purdie > Subject: Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check > that TEMPLATECONF is valid > > On Thu, 25 Aug 2022 at 09:19, Peter Kjellerstedt > wrote: > > > > > We are trying to move towards standardizing build configuration > > > management. One step towards that goal is that config templates must > > > live in meta-some-layer/conf/templates, and aren't scattered around, > > > or generated on the fly. This rule only makes sense if there is some > > > way to enforce it, which is what this change does. > > > > Well, we gave up on the static templates almost immediately after we > > started using Yocto when we realized that they did not fit into our > > idea of being able to mix layers in different combinations in different > > builds. We rely on repo fetching the layers that are supposed to be part > > of the build, and then we generate a bblayers.conf.sample that matches > > the fetched layers. > > The static templates is only a starting point. The next step would be > to support prefabricated 'configuration fragments' stored inside > layers that can be dynamically added/removed into local.conf and/or a > distro definition, so ideas for the design and UI would be welcome. Well, the way we have solved that is to allow a layer to contain conf/local.conf.sample.XX files (and also conf/conf-notes.txt.XX files) where XX is any number between 00 and 99. We also assume the sample files from meta-poky have XX == 50. Then when we generate our templateconf directory, we concatenate all these files ordered by XX (regardless of which layer they come from). > > Alex //Peter -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169886): https://lists.openembedded.org/g/openembedded-core/message/169886 Mute This Topic: https://lists.openembedded.org/mt/93225500/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
On Thu, 25 Aug 2022 at 09:19, Peter Kjellerstedt wrote: > > > We are trying to move towards standardizing build configuration > > management. One step towards that goal is that config templates must > > live in meta-some-layer/conf/templates, and aren't scattered around, > > or generated on the fly. This rule only makes sense if there is some > > way to enforce it, which is what this change does. > > Well, we gave up on the static templates almost immediately after we > started using Yocto when we realized that they did not fit into our > idea of being able to mix layers in different combinations in different > builds. We rely on repo fetching the layers that are supposed to be part > of the build, and then we generate a bblayers.conf.sample that matches > the fetched layers. The static templates is only a starting point. The next step would be to support prefabricated 'configuration fragments' stored inside layers that can be dynamically added/removed into local.conf and/or a distro definition, so ideas for the design and UI would be welcome. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169839): https://lists.openembedded.org/g/openembedded-core/message/169839 Mute This Topic: https://lists.openembedded.org/mt/93225500/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
> -Original Message- > From: Alexander Kanavin > Sent: den 24 augusti 2022 16:53 > To: Peter Kjellerstedt > Cc: openembedded-core@lists.openembedded.org; Alexander Kanavin > > Subject: Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check > that TEMPLATECONF is valid > > On Wed, 24 Aug 2022 at 16:43, Peter Kjellerstedt > wrote: > > We create the sample files in a small wrapper for oe-init-build-env > > so that to the users, everything works just as described in the Yocto > > documentation. And since the OE scripts expect TEMPLATECONF to be set > > and contain sample files, that is what we create. > > Actually they don't expect that. You can simply create the configs > directly into the target build directory in your wrapper, and drop > TEMPLATECONF altogether. Then the OE scripts will detect that the > configs are already there and will skip the step of copying the > templates. Hmm, ok. I will have to look into that... > > Sure, that is fine. But the expectation I have is that TEMPLATECONF > > refers to the directory that has the sample files needed to set up > > the build environment. And that is it. As long as the sample files > > exist where it says they should be, why do you need the scripts to > > fail because the environment around the directory does not look > > like what you happen to expect? It is only the contents of the > > directory that matters. Or am I missing something? > > We are trying to move towards standardizing build configuration > management. One step towards that goal is that config templates must > live in meta-some-layer/conf/templates, and aren't scattered around, > or generated on the fly. This rule only makes sense if there is some > way to enforce it, which is what this change does. Well, we gave up on the static templates almost immediately after we started using Yocto when we realized that they did not fit into our idea of being able to mix layers in different combinations in different builds. We rely on repo fetching the layers that are supposed to be part of the build, and then we generate a bblayers.conf.sample that matches the fetched layers. > Alex //Peter -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169832): https://lists.openembedded.org/g/openembedded-core/message/169832 Mute This Topic: https://lists.openembedded.org/mt/93225500/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
On Wed, 24 Aug 2022 at 16:43, Peter Kjellerstedt wrote: > We create the sample files in a small wrapper for oe-init-build-env > so that to the users, everything works just as described in the Yocto > documentation. And since the OE scripts expect TEMPLATECONF to be set > and contain sample files, that is what we create. Actually they don't expect that. You can simply create the configs directly into the target build directory in your wrapper, and drop TEMPLATECONF altogether. Then the OE scripts will detect that the configs are already there and will skip the step of copying the templates. > Sure, that is fine. But the expectation I have is that TEMPLATECONF > refers to the directory that has the sample files needed to set up > the build environment. And that is it. As long as the sample files > exist where it says they should be, why do you need the scripts to > fail because the environment around the directory does not look > like what you happen to expect? It is only the contents of the > directory that matters. Or am I missing something? We are trying to move towards standardizing build configuration management. One step towards that goal is that config templates must live in meta-some-layer/conf/templates, and aren't scattered around, or generated on the fly. This rule only makes sense if there is some way to enforce it, which is what this change does. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169822): https://lists.openembedded.org/g/openembedded-core/message/169822 Mute This Topic: https://lists.openembedded.org/mt/93225500/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
> -Original Message-
> From: Alexander Kanavin
> Sent: den 24 augusti 2022 16:09
> To: Peter Kjellerstedt
> Cc: openembedded-core@lists.openembedded.org; Alexander Kanavin
>
> Subject: Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check
> that TEMPLATECONF is valid
>
> On Wed, 24 Aug 2022 at 15:58, Peter Kjellerstedt
> wrote:
> > Please no. This happens to match your expectations, but doesn't at
> > all match our use of TEMPLATECONF. In our .templateconf we set
> > TEMPLATECONF=${TEMPLATECONF:-templateconf} and create the templateconf
> > directory in runtime where we generate the sample files that match the
> > layers that repo has fetched for this particular build. It obviously
> > has no layer.conf file as it is in no layer, and the path doesn't
> > contain "templates/". I can of course change our tools to
> > create a couple of extra directories to match your expectations, but
> > at the same time it seems very unnecessary.
>
> If you create the template files on the fly, then you might as well
> create the actual configs directly into build/conf and skip the
> template step. No?
We create the sample files in a small wrapper for oe-init-build-env
so that to the users, everything works just as described in the Yocto
documentation. And since the OE scripts expect TEMPLATECONF to be set
and contain sample files, that is what we create.
> We do need to standardize the location for the templates for reasons
> of discoverability and consistency with machine and distro
> definitions, this has been discussed.
Sure, that is fine. But the expectation I have is that TEMPLATECONF
refers to the directory that has the sample files needed to set up
the build environment. And that is it. As long as the sample files
exist where it says they should be, why do you need the scripts to
fail because the environment around the directory does not look
like what you happen to expect? It is only the contents of the
directory that matters. Or am I missing something?
>
> Alex
//Peter
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169821):
https://lists.openembedded.org/g/openembedded-core/message/169821
Mute This Topic: https://lists.openembedded.org/mt/93225500/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
On Wed, 24 Aug 2022 at 15:58, Peter Kjellerstedt
wrote:
> Please no. This happens to match your expectations, but doesn't at
> all match our use of TEMPLATECONF. In our .templateconf we set
> TEMPLATECONF=${TEMPLATECONF:-templateconf} and create the templateconf
> directory in runtime where we generate the sample files that match the
> layers that repo has fetched for this particular build. It obviously
> has no layer.conf file as it is in no layer, and the path doesn't
> contain "templates/". I can of course change our tools to
> create a couple of extra directories to match your expectations, but
> at the same time it seems very unnecessary.
If you create the template files on the fly, then you might as well
create the actual configs directly into build/conf and skip the
template step. No?
We do need to standardize the location for the templates for reasons
of discoverability and consistency with machine and distro
definitions, this has been discussed.
Alex
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169819):
https://lists.openembedded.org/g/openembedded-core/message/169819
Mute This Topic: https://lists.openembedded.org/mt/93225500/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
> -Original Message-
> From: openembedded-core@lists.openembedded.org
> On Behalf Of Alexander Kanavin
> Sent: den 24 augusti 2022 14:43
> To: openembedded-core@lists.openembedded.org
> Cc: Alexander Kanavin
> Subject: [OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that
> TEMPLATECONF is valid
>
> specifically that ../../layer.conf exists, and that second-from-last
> component in the path is 'templates'.
Please no. This happens to match your expectations, but doesn't at
all match our use of TEMPLATECONF. In our .templateconf we set
TEMPLATECONF=${TEMPLATECONF:-templateconf} and create the templateconf
directory in runtime where we generate the sample files that match the
layers that repo has fetched for this particular build. It obviously
has no layer.conf file as it is in no layer, and the path doesn't
contain "templates/". I can of course change our tools to
create a couple of extra directories to match your expectations, but
at the same time it seems very unnecessary.
>
> Signed-off-by: Alexander Kanavin
> ---
> scripts/oe-setup-builddir | 5 +
> 1 file changed, 5 insertions(+)
>
> diff --git a/scripts/oe-setup-builddir b/scripts/oe-setup-builddir
> index 975619789a..db26e3b138 100755
> --- a/scripts/oe-setup-builddir
> +++ b/scripts/oe-setup-builddir
> @@ -61,6 +61,11 @@ if [ -n "$TEMPLATECONF" ]; then
> echo >&2 "Error: TEMPLATECONF value points to nonexistent
> directory '$TEMPLATECONF'"
> exit 1
> fi
> +templatesdir=$(python3 -c "import sys;
> print(sys.argv[1].strip('/').split('/')[-2])" $TEMPLATECONF)
> +if [ ! -f "$TEMPLATECONF/../../layer.conf" -o $templatesdir !=
> "templates" ]; then
> +echo >&2 "Error: TEMPLATECONF value must point to
> meta-some-layer/conf/templates/template-name"
> +exit 1
> +fi
> fi
> OECORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
> OECORELOCALCONF="$TEMPLATECONF/local.conf.sample"
> --
> 2.30.2
//Peter
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169818):
https://lists.openembedded.org/g/openembedded-core/message/169818
Mute This Topic: https://lists.openembedded.org/mt/93225500/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/2] scripts/oe-setup-builddir: add a check that TEMPLATECONF is valid
specifically that ../../layer.conf exists, and that second-from-last
component in the path is 'templates'.
Signed-off-by: Alexander Kanavin
---
scripts/oe-setup-builddir | 5 +
1 file changed, 5 insertions(+)
diff --git a/scripts/oe-setup-builddir b/scripts/oe-setup-builddir
index 975619789a..db26e3b138 100755
--- a/scripts/oe-setup-builddir
+++ b/scripts/oe-setup-builddir
@@ -61,6 +61,11 @@ if [ -n "$TEMPLATECONF" ]; then
echo >&2 "Error: TEMPLATECONF value points to nonexistent
directory '$TEMPLATECONF'"
exit 1
fi
+templatesdir=$(python3 -c "import sys;
print(sys.argv[1].strip('/').split('/')[-2])" $TEMPLATECONF)
+if [ ! -f "$TEMPLATECONF/../../layer.conf" -o $templatesdir !=
"templates" ]; then
+echo >&2 "Error: TEMPLATECONF value must point to
meta-some-layer/conf/templates/template-name"
+exit 1
+fi
fi
OECORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
OECORELOCALCONF="$TEMPLATECONF/local.conf.sample"
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169811):
https://lists.openembedded.org/g/openembedded-core/message/169811
Mute This Topic: https://lists.openembedded.org/mt/93225500/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
