Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-24 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Richard and folks, > The only other way I could imagine this to work would require us to > keep the manifest file within the meta layer and do a somewhat > "hacky" > workaround: > > It is possible to run `repo init` on a local git folder. > As

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-15 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Will do! :) This only affects the other patch series though, the recipe itself should be fine now. It seems the only question left open is this: > > I think we can replace the patch 0001-Set-REPO_REV-to-v2.17.3.patch > > with a post function > >

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-15 Thread Alexander Kanavin
Thanks, maybe you should write a test for offline builds as well then :) Alex On Mon, 15 Nov 2021 at 13:59, Jasper Orschulko < jasper.orschu...@iris-sensing.com> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi Alexander, > > "[PATCH v3] fetch2/repo: Implement AUTOREV for repo

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-15 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Alexander, "[PATCH v3] fetch2/repo: Implement AUTOREV for repo fetcher" contains a fix for this issue. Reproducing a build offline with only the DL_DIR should work now. We are currently still lacking the appropriate tests for the fetcher. These

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-12 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Alex, thanks for your input. You are absolutely correct, this currently does not work. We'll look into this. - -- With best regards Jasper Orschulko DevOps Engineer Tel. +49 30 58 58 14 265 Fax +49 30 58 58 14 999

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-11 Thread Alexander Kanavin
On Thu, 11 Nov 2021 at 16:08, Jasper Orschulko < jasper.orschu...@iris-sensing.com> wrote: > So if you have done this initial fetch of your sources and stash your > working dir away, you can do an offline build. > But can you do an offline build without a prepopulated working dir? That's the

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-11 Thread Jasper Orschulko
...@lists.openembedded.org; kweihm...@outlook.com; Peter > > Kjellerstedt > > ; jas...@fancydomain.eu > > Cc: mar...@mko.dev; Daniel Baumgart > > ; > > bitbake-de...@lists.openembedded.org > > Subject: Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-11 Thread Peter Kjellerstedt
umgart ; > bitbake-de...@lists.openembedded.org > Subject: Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe > for repo 2.17.3 > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi Peter, > > > However, this means > > that only the sou

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-11 Thread Jasper Orschulko
439 Berlin > > > > https://iris-sensing.com/ > > > > > > > > > > > > On Wed, 2021-11-10 at 23:55 +, Peter Kjellerstedt wrote: > > > > -----Original Message- > > > > From: bitbake-de...@lists.openembedded.org  > &

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-11 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Richard, > Even ten recipes using this will show a degradation in parsing speed > and I do > get a lot of complaints when parsing slows down for any reason. The > user doesn't > expect this and it won't be visible what bitbake is doing (sitting

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-11 Thread Peter Kjellerstedt
umgart ; > bitbake-de...@lists.openembedded.org > Subject: Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe > for repo 2.17.3 > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi Peter, > > > How do you avoid the repo wrapper fetching

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-11 Thread Jasper Orschulko
on.org; openembedded- > > c...@lists.openembedded.org; kweihm...@outlook.com; > > jas...@fancydomain.eu > > Cc: mar...@mko.dev; daniel.baumg...@iris-sensing.net; bitbake- > > de...@lists.openembedded.org > > Subject: Re: [bitbake-devel] [oe-core][PATCH 1/2] devt

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-10 Thread Peter Kjellerstedt
.@outlook.com; jas...@fancydomain.eu > Cc: mar...@mko.dev; daniel.baumg...@iris-sensing.net; bitbake- > de...@lists.openembedded.org > Subject: Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe > for repo 2.17.3 > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > >

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-10 Thread Richard Purdie
On Wed, 2021-11-10 at 13:52 +, Jasper Orschulko wrote: > Hi Richard, > > > When you say "fixed refspec", will that be a definitive sha revision > > or a tag? > > We always force resolution of tags as they tend to cause problems and > > can change even if it is bad form. > > that's a good

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-10 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Richard, > When you say "fixed refspec", will that be a definitive sha revision > or a tag? > We always force resolution of tags as they tend to cause problems and > can change > even if it is bad form. that's a good point. Actually, Martin and

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-10 Thread Richard Purdie
On Tue, 2021-11-09 at 11:26 +, Jasper Orschulko wrote: > > > > e) fetcher output is deterministic > >  (i.e. if you fetch configuration XXX now it will match in future > > exactly in > >  a clean build with a new DL_DIR) > > check. When a fixed refspec is set within the recipe, the fetcher

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-09 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Richard, I think our implementation of the repo fetcher checks the (what I believe to be) most relevant parts of your checklist (thanks for the write-up). Martin, please corrent me if I'm missing something: > a) network access for sources is

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-08 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Richard and Peter, I've addressed your comments and will post the v3 patches momentarily. Quick comment regarding the new overrides syntax, or rather the documentation. I just noticed that my go-to page for the mega-manual is outdated

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-07 Thread Richard Purdie
On Fri, 2021-11-05 at 14:31 +0100, Jasper Orschulko via lists.openembedded.org wrote: > From: Jasper Orschulko > > Add a recipe for repo, prerequisite for the repo fetcher. > > Signed-off-by: Jasper Orschulko > --- > .../repo/files/0001-python3-shebang.patch | 21 >

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-06 Thread Richard Purdie
On Fri, 2021-11-05 at 14:09 +, Jasper Orschulko wrote: > Actually, I don't believe this to be an issue for the following > reasons: > > [...] > > 2. When using repo within the yocto build itself (with the repo > fetcher), the repo binary is only ever called during the do_fetch step, > which

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-06 Thread Alexander Kanavin
On Fri, 5 Nov 2021 at 21:32, Jasper Orschulko < jasper.orschu...@iris-sensing.com> wrote: > So yeah... as far as I can tell, there are multiple ways to approach > this issue, but none of them seem straight forward nor pretty. Bitbake > as it is is just fundamentally not good at handling highly

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Alex, we are getting a bit off topic here, but whatever... ;) > What I mean is that I would try harder to find a workable setup while > keeping the recipes for individual items that need to be built. the individual recipes don't really add

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Alexander Kanavin
On Fri, 5 Nov 2021 at 19:05, Jasper Orschulko wrote: > > But that is exactly what we are doing, by integrating the repo fetcher > into the yocto workflow, thus improving the yocto tooling :) > Why reinvent the wheel, when you can reuse whats already there? You > wouldn't reinvent git just for

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Jasper Orschulko via lists.openembedded.org
Hi Alex, > that you invented a custom, proprietary > workflow that you have to support entirely by > yourselves We are not though. We are integrating an established tool for multi-repository management into yocto. Google repo is not proprietary by the way, it is permissive licensed. It is

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Alexander Kanavin
On Fri, 5 Nov 2021 at 16:24, Jasper Orschulko < jasper.orschu...@iris-sensing.com> wrote: > > In our case some binary which is made up from some 50ish separately > versioned sub-components. We used to have separate recipes for each of > this components and static link them using DEPENDS. However,

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 SHA rev set in v2 of the patch series :) - -- With best regards Jasper Orschulko DevOps Engineer Tel. +49 30 58 58 14 265 Fax +49 30 58 58 14 999 jasper.orschu...@iris-sensing.com • • • • • • • • • • • • • • • • • • • • • • • • • • iris-GmbH

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Yeah. Unfortunately the current repo fetcher is pretty much broken, this patch is more or less a requirement for changes to the repo fetcher we proposed in the bitbake mailinglist. In our case some binary which is made up from some 50ish separately

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Alexander Kanavin
Ah I didn't notice it's something that already exists. Still, I'm curious for which items is it used, and why repo over other options? Alex On Fri, 5 Nov 2021 at 15:20, Alexander Kanavin wrote: > Wait, what is the use case for the repo fetcher? > > Alex > > On Fri, 5 Nov 2021 at 14:31, Jasper

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 > Okay fine that makes sense, even if it is still phoning home. True. Bad software design at best. But luckily not an issue from a build system perspective. :) > Another thing, could you please exchange "+REPO_REV = 'v2.17.3'" for > a > SHA

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Konrad Weihmann
On 05.11.21 15:09, Jasper Orschulko wrote: Actually, I don't believe this to be an issue for the following reasons: 1. The recipe does nothing more than taking the wrapper script as is and copying it to the sysroot. As the repo binary is never called when building, it does not have the

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Alexander Kanavin
Wait, what is the use case for the repo fetcher? Alex On Fri, 5 Nov 2021 at 14:31, Jasper Orschulko via lists.openembedded.org wrote: > From: Jasper Orschulko > > Add a recipe for repo, prerequisite for the repo fetcher. > > Signed-off-by: Jasper Orschulko > --- >

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Actually, I don't believe this to be an issue for the following reasons: 1. The recipe does nothing more than taking the wrapper script as is and copying it to the sysroot. As the repo binary is never called when building, it does not have the

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Jasper Orschulko
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi Konrad, that might be a good point. I patched the REPO_REV to a fixed refspec within the wrapper tool, as to stop repo from using the latest stable release. But you are right, and this might not be enough. Will do some more digging on this. -

Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

2021-11-05 Thread Konrad Weihmann
Is this implementation of repo being patched against phoning home to google everytime and trying to do that pesky auto-update of the tool? Otherwise this will become an issue when at one point in time an (then) outdated version of the tool is used in a BB_NO_NETWORK build. On 05.11.21 14:31,