Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2024-01-22 Thread Alexander Kanavin
On Thu, 14 Sept 2023 at 13:52, Alexander Kanavin wrote: > > On Tue, 12 Sept 2023 at 16:44, Stephen Jolley wrote: > > Alexander Kanavin will be working on the core workflow topic I thought I'd write a summary of where we are with these subjects, and make a plan for what needs to be done still:

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-10-30 Thread Alexander Kanavin
On Mon, 30 Oct 2023 at 15:07, Richard Purdie wrote: > > a) esdk sets a number of variables from its initialization script that > > aid with cross-compiling components directly (e.g. the core use case > > of SDKs). Normal mode doesn't do that, but recently added > > meta-ide-support will generate

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-10-30 Thread Richard Purdie
On Mon, 2023-10-30 at 14:50 +0100, Alexander Kanavin wrote: > On Thu, 14 Sept 2023 at 14:56, Richard Purdie > wrote: > > There are design elements to this work. We need to work out how we can > > make eSDK and "normal" builds more similar and less of an overhead to > > switch between one and the

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-10-30 Thread Alexander Kanavin
On Thu, 14 Sept 2023 at 14:56, Richard Purdie wrote: > There are design elements to this work. We need to work out how we can > make eSDK and "normal" builds more similar and less of an overhead to > switch between one and the other. A "bblock all" command does partly > get you to an eSDK

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-29 Thread Alexander Kanavin
On Fri, 29 Sept 2023 at 14:27, Richard Purdie wrote: > I'd prefer to see some dedicated bitbake API used even if we need to > create/add it. tinfoil and some of the bblock/unlock work shows we can > get stamp data, the question would be how to get it without > "disturbing" the existing build. >

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-29 Thread Richard Purdie
On Fri, 2023-09-29 at 14:06 +0200, Alexander Kanavin wrote: > On Thu, 28 Sept 2023 at 18:49, Richard Purdie > wrote: > > > I'm curious to see what you find with analysis of bitbake-whatchanged. > > I've taken a look a the script. It obtains the current location of > STAMPS_DIR, then runs this:

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-29 Thread Alexander Kanavin
On Thu, 28 Sept 2023 at 18:49, Richard Purdie wrote: > I'm curious to see what you find with analysis of bitbake-whatchanged. I've taken a look a the script. It obtains the current location of STAMPS_DIR, then runs this: # Generate the new stamps dir print("Generating the new

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-28 Thread Alexander Kanavin
On Thu, 28 Sept 2023 at 18:49, Richard Purdie wrote: > I've wondered if we should split bitbake -S printdiff into a separate > utility? It exists from a time before we had bitbake command APIs. It can also be a simple shell wrapper. I've separated -S lockedsigs into it's own option as well, so

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-28 Thread Richard Purdie
On Thu, 2023-09-28 at 18:43 +0200, Alexander Kanavin wrote: > On Fri, 22 Sept 2023 at 12:42, Richard Purdie > wrote: > > > Things which used to be problematic: > > > > a) changes involving changes to gcc-source since it uses a shared > > sources stamps which confused the tools (at least used

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-28 Thread Alexander Kanavin
On Fri, 22 Sept 2023 at 12:42, Richard Purdie wrote: > Things which used to be problematic: > > a) changes involving changes to gcc-source since it uses a shared > sources stamps which confused the tools (at least used to). That may > have been before gcc-source became a recipe? > b) changes to

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-22 Thread Richard Purdie
On Fri, 2023-09-22 at 11:17 +0200, Alexander Kanavin wrote: > On Thu, 21 Sept 2023 at 16:39, chris.lapla...@agilent.com > wrote: > > > That is very impressive and I'd also love to hear about what heuristics it > > uses. > > It's actually rather simple. It uses glob.glob on stamps in tmp/, then

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-22 Thread Alexander Kanavin
On Thu, 21 Sept 2023 at 16:39, chris.lapla...@agilent.com wrote: > That is very impressive and I'd also love to hear about what heuristics it > uses. It's actually rather simple. It uses glob.glob on stamps in tmp/, then on local sstate to find possible matches, then sorts them by mtime and

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-21 Thread Chris Laplante via lists.openembedded.org
> On Thu, 14 Sept 2023 at 21:54, Richard Purdie > wrote: > > The tools are already supposed to support doing this with local file > > sstate sources, they just do a bad job at getting the diffs right. One > > intent of this work item was to try and understand why they don't work > > and address

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-21 Thread Alexander Kanavin
On Thu, 14 Sept 2023 at 21:54, Richard Purdie wrote: > The tools are already supposed to support doing this with local file > sstate sources, they just do a bad job at getting the diffs right. One > intent of this work item was to try and understand why they don't work > and address that so at

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-20 Thread Julien Stephan
Unless I missed something, I think I fixed all comments from RP on the branch I force pushed :) Cheers Julien Le mer. 20 sept. 2023 à 16:31, Alexander Kanavin a écrit : > > That’s great to hear! When you have a new patch set please post it here. I > think there’s also a new reply from RP that

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-20 Thread Julien Stephan
Hi Alexander, About bblock tool: I just force pushed my branch on poky-contrib (https://git.yoctoproject.org/poky-contrib/log/?h=jstephan/bblock), trying to fix an autobuilder issue reported by Alexandre Belloni. I am still working on this, and I would be very happy to get this merged :) Cheers

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-20 Thread Alexander Kanavin
That’s great to hear! When you have a new patch set please post it here. I think there’s also a new reply from RP that you need to consider. Alex On Wed 20. Sep 2023 at 16.25, Julien Stephan wrote: > Hi Alexander, > About bblock tool: I just force pushed my branch on poky-contrib >

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-15 Thread Alexander Kanavin
On Thu, 14 Sept 2023 at 21:54, Richard Purdie wrote: > Effectively those "configs" are what the eSDK is, you're just proposing > a server side set of them rather than a built copy. In many ways that > could be useful way of possibly rethinking the way the eSDK works. > > So in that sense I like

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-14 Thread Richard Purdie
On Thu, 2023-09-14 at 20:51 +0200, Alexander Kanavin wrote: > On Thu, 14 Sept 2023 at 14:56, Richard Purdie > wrote: > > For the task signatures, we need to think about some questions. If I > > make a change locally, can I query how much will rebuild and how much > > will be reused? There is

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-14 Thread Alexander Kanavin
On Thu, 14 Sept 2023 at 14:56, Richard Purdie wrote: > For the task signatures, we need to think about some questions. If I > make a change locally, can I query how much will rebuild and how much > will be reused? There is bitbake --dry-run but perhaps it is time for a > an option (or dedicated

Re: [Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-14 Thread Richard Purdie
On Thu, 2023-09-14 at 13:52 +0200, Alexander Kanavin wrote: > On Tue, 12 Sept 2023 at 16:44, Stephen Jolley wrote: > > Alexander Kanavin will be working on the core workflow topic > > I am now ready to start doing this, but before I do, I'd like to > decompose the subject into manageable tasks

[Openembedded-architecture] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-09-14 Thread Alexander Kanavin
On Tue, 12 Sept 2023 at 16:44, Stephen Jolley wrote: > Alexander Kanavin will be working on the core workflow topic I am now ready to start doing this, but before I do, I'd like to decompose the subject into manageable tasks with a bit of help from RP and the community: