Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
On Thu, 18 Jan 2024, Alexander Kanavin wrote: > On Thu, 18 Jan 2024 at 15:10, Robert P. J. Day wrote: > > ah, got it. that's still fine since as the content is almost > > exclusively binary (pre-built) content, redoing from scratch will take > > no time at all. thanks for clarifying that. > > Pre-built content is ok on its own, but if those items are in the > dependency tree of something that does take time to build, they will > subvert sstate and lengthen build times. i'm already going over the entire build to identify unnoticed dependencies that could cause problems. rday -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193989): https://lists.openembedded.org/g/openembedded-core/message/193989 Mute This Topic: https://lists.openembedded.org/mt/103807270/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
On Thu, 18 Jan 2024 at 15:10, Robert P. J. Day wrote: > ah, got it. that's still fine since as the content is almost > exclusively binary (pre-built) content, redoing from scratch will take > no time at all. thanks for clarifying that. Pre-built content is ok on its own, but if those items are in the dependency tree of something that does take time to build, they will subvert sstate and lengthen build times. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193988): https://lists.openembedded.org/g/openembedded-core/message/193988 Mute This Topic: https://lists.openembedded.org/mt/103807270/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
On Thu, 18 Jan 2024, Alexander Kanavin wrote:
> On Thu, 18 Jan 2024 at 15:04, Robert P. J. Day wrote:
> > i just tested, and the recipe does *not* rebuild unless i change
> > something in the ("externalsrc") source directory, at which point,
> > yes, it rebuilds. which is exactly what i'm after.
>
> It only doesn't rebuild because you already have a build/tmp directory.
>
> But as no sstate objects are created, if you wipe tmp/ or make a new
> build directory, it will rebuild again even without any changes at
> all. Try that!
ah, got it. that's still fine since as the content is almost
exclusively binary (pre-built) content, redoing from scratch will take
no time at all. thanks for clarifying that.
rday
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193987):
https://lists.openembedded.org/g/openembedded-core/message/193987
Mute This Topic: https://lists.openembedded.org/mt/103807270/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
This is reusing an existing build directory (Wanted 96 Local 96). You need to make a new one first. Alex On Thu, 18 Jan 2024 at 15:06, Robert P. J. Day wrote: > > On Thu, 18 Jan 2024, Alexander Kanavin wrote: > > > Mono-repo seemed like a good idea at the time, I'm sure. > > > > I need to point out that externalsrc disables sstate and always > > rebuilds from scratch. Is that okay? > > just tested a bitbake on recipe after changing nothing: > > Sstate summary: Wanted 96 Local 96 Mirrors 0 Missed 0 Current 100 > (100% match, 100% complete) | ETA: 0:00:00 > Initialising tasks: 100% > |#| > Time: 0:00:01 > NOTE: Executing Tasks > NOTE: Tasks Summary: Attempted 637 tasks of which 637 didn't need to > be rerun and all succeeded. > > then i made a trivial change to a single file in the source > directory and bitbake'd again: > > Sstate summary: Wanted 103 Local 96 Mirrors 0 Missed 7 Current 93 (93% > match, 96% complete)| ETA: 0:00:00 > Removing 6 stale sstate objects for arch allarch: 100% > |###| Time: 0:00:00 > Removing 1 stale sstate objects for arch qemux86_64: 100% > || Time: 0:00:00 > NOTE: Executing Tasks > NOTE: Tasks Summary: Attempted 637 tasks of which 627 didn't need to > be rerun and all succeeded. > NOTE: Build completion summary: > NOTE: do_populate_sysroot: 0.0% sstate reuse(0 setscene, 1 scratch) > NOTE: do_create_spdx: 0.0% sstate reuse(0 setscene, 1 scratch) > NOTE: do_create_runtime_spdx: 0.0% sstate reuse(0 setscene, 1 > scratch) > NOTE: do_package_qa: 0.0% sstate reuse(0 setscene, 1 scratch) > NOTE: do_package: 0.0% sstate reuse(0 setscene, 1 scratch) > NOTE: do_packagedata: 0.0% sstate reuse(0 setscene, 1 scratch) > NOTE: do_package_write_rpm: 0.0% sstate reuse(0 setscene, 1 scratch) > > seems to be honouring sstate-cache. > > rday -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193986): https://lists.openembedded.org/g/openembedded-core/message/193986 Mute This Topic: https://lists.openembedded.org/mt/103807270/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
On Thu, 18 Jan 2024, Alexander Kanavin wrote: > Mono-repo seemed like a good idea at the time, I'm sure. > > I need to point out that externalsrc disables sstate and always > rebuilds from scratch. Is that okay? just tested a bitbake on recipe after changing nothing: Sstate summary: Wanted 96 Local 96 Mirrors 0 Missed 0 Current 100 (100% match, 100% complete) | ETA: 0:00:00 Initialising tasks: 100% |#| Time: 0:00:01 NOTE: Executing Tasks NOTE: Tasks Summary: Attempted 637 tasks of which 637 didn't need to be rerun and all succeeded. then i made a trivial change to a single file in the source directory and bitbake'd again: Sstate summary: Wanted 103 Local 96 Mirrors 0 Missed 7 Current 93 (93% match, 96% complete)| ETA: 0:00:00 Removing 6 stale sstate objects for arch allarch: 100% |###| Time: 0:00:00 Removing 1 stale sstate objects for arch qemux86_64: 100% || Time: 0:00:00 NOTE: Executing Tasks NOTE: Tasks Summary: Attempted 637 tasks of which 627 didn't need to be rerun and all succeeded. NOTE: Build completion summary: NOTE: do_populate_sysroot: 0.0% sstate reuse(0 setscene, 1 scratch) NOTE: do_create_spdx: 0.0% sstate reuse(0 setscene, 1 scratch) NOTE: do_create_runtime_spdx: 0.0% sstate reuse(0 setscene, 1 scratch) NOTE: do_package_qa: 0.0% sstate reuse(0 setscene, 1 scratch) NOTE: do_package: 0.0% sstate reuse(0 setscene, 1 scratch) NOTE: do_packagedata: 0.0% sstate reuse(0 setscene, 1 scratch) NOTE: do_package_write_rpm: 0.0% sstate reuse(0 setscene, 1 scratch) seems to be honouring sstate-cache. rday -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193985): https://lists.openembedded.org/g/openembedded-core/message/193985 Mute This Topic: https://lists.openembedded.org/mt/103807270/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
On Thu, 18 Jan 2024 at 15:04, Robert P. J. Day wrote:
> i just tested, and the recipe does *not* rebuild unless i change
> something in the ("externalsrc") source directory, at which point,
> yes, it rebuilds. which is exactly what i'm after.
It only doesn't rebuild because you already have a build/tmp directory.
But as no sstate objects are created, if you wipe tmp/ or make a new
build directory, it will rebuild again even without any changes at
all. Try that!
Alex
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193984):
https://lists.openembedded.org/g/openembedded-core/message/193984
Mute This Topic: https://lists.openembedded.org/mt/103807270/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
On Thu, 18 Jan 2024, Alexander Kanavin wrote:
> Mono-repo seemed like a good idea at the time, I'm sure.
>
> I need to point out that externalsrc disables sstate and always
> rebuilds from scratch. Is that okay?
i just tested, and the recipe does *not* rebuild unless i change
something in the ("externalsrc") source directory, at which point,
yes, it rebuilds. which is exactly what i'm after.
rday
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193983):
https://lists.openembedded.org/g/openembedded-core/message/193983
Mute This Topic: https://lists.openembedded.org/mt/103807270/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
Mono-repo seemed like a good idea at the time, I'm sure. I need to point out that externalsrc disables sstate and always rebuilds from scratch. Is that okay? Alex On Thu, 18 Jan 2024 at 13:58, Robert P. J. Day wrote: > > > i'm pretty sure i know how to do this, just want to know if there's > an even easier/more elegant way. > > in current project, there is a *pile* of local content that is > identified by various recipes. so since that content is local, the > obvious starting point is for such recipes to "inherit externalsrc" to > properly identify the local source directory. > > in addition, many of those recipes represent packaging binary data > (imagine firmware files), so that there is really no meaningful > configure or compile tasks, so that further suggests inheriting > "bin_package" as well. > > finally, in some cases, there is a teeny bit of tweaking that might > be needed for the final installed content, which suggests adding: > > do_install:append() > > to the mix to handle whatever little adjustments are needed. > > so given that scenario, can i reasonably summarize the right > approach as: > > inherit externalsrc > inherit bin_package > do_install:append() [as needed] > > finally, it doesn't seem like many of those recipes are required as > build-time dependencies, so i can just make them either RDEPENDS as > appropriate, or just IMAGE_INSTALL them for the final image. > > sound reasonable? am i overthinking this? seems fairly > straightforward. > > rday > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193978): https://lists.openembedded.org/g/openembedded-core/message/193978 Mute This Topic: https://lists.openembedded.org/mt/103807270/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] want to confirm i'm using "externalsrc" and "bin_package" efficiently
i'm pretty sure i know how to do this, just want to know if there's an even easier/more elegant way. in current project, there is a *pile* of local content that is identified by various recipes. so since that content is local, the obvious starting point is for such recipes to "inherit externalsrc" to properly identify the local source directory. in addition, many of those recipes represent packaging binary data (imagine firmware files), so that there is really no meaningful configure or compile tasks, so that further suggests inheriting "bin_package" as well. finally, in some cases, there is a teeny bit of tweaking that might be needed for the final installed content, which suggests adding: do_install:append() to the mix to handle whatever little adjustments are needed. so given that scenario, can i reasonably summarize the right approach as: inherit externalsrc inherit bin_package do_install:append() [as needed] finally, it doesn't seem like many of those recipes are required as build-time dependencies, so i can just make them either RDEPENDS as appropriate, or just IMAGE_INSTALL them for the final image. sound reasonable? am i overthinking this? seems fairly straightforward. rday -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193977): https://lists.openembedded.org/g/openembedded-core/message/193977 Mute This Topic: https://lists.openembedded.org/mt/103807270/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
