Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Wed, Mar 9, 2022 at 07:43 PM, Bruce Ashfield wrote: > > On Wed, Mar 9, 2022 at 3:10 AM wrote: > >> On Thu, Mar 3, 2022 at 05:46 PM, Bruce Ashfield wrote: >> >> On Thu, Mar 3, 2022 at 10:13 AM wrote: >> >> On Thu, Mar 3, 2022 at 04:34 AM, Bruce Ashfield wrote: >> >> On Wed, Mar 2, 2022 at 4:57 PM Andrei Gherzan wrote: >> >> >> Mar 1, 2022 20:15:52 Bruce Ashfield : >> >> On Tue, Mar 1, 2022 at 10:54 AM wrote: >> >> On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: >> >> On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: >> >> >> On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: >> >> On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via >> lists.openembedded.org >> wrote: >> >> >> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: >> >> >> >> From: Andrei Gherzan >> >> Compile pulls in the go.mod list requiring network. Without this, do >> compile would fail with a similar error to the following: >> >> dial tcp: lookup proxy.golang.org: Temporary failure in name resolution >> >> This is something that needs to be carried in your own layers, IMHO it >> isn't appropriate for core. >> >> It isn't about the fetching, it is the entire gap in functionality >> that we are missing if go starts fetching dependencies during compile. >> >> A further thought is that if this is for go.mod issues, there is the >> go-mod.bbclass. >> >> Perhaps enabling it in that class and doing a bbwarn about go fetching >> dependencies would be appropriate ? >> >> Otherwise, someone may not know that this is happening and that a no >> network configuration has no chance of working. >> >> I reckon that is reasonable. I'll personally go down the recipe level to >> workaround this change but understanding and agreeing with the reasoning >> behind this change, I want to invest a bit into trying to find a proper >> solution in the core. Bruce, I know you invested a fair amount of time >> into this already. Would you be willing to sync up and see how we can work >> together in tackling this? >> >> Definitely, more ideas are good. In fact, I think there are probably >> several approaches that can co-exist, depending on what a >> recipe/developer needs. >> >> I'm in the Eastern time zone here, and will try and grab folks on IRC >> to have a level set >> >> Bruce >> >> Added Zyga to CC as he is also interested in this as part of his go >> development activities. >> >> Thanks, >> Andrei >> >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II >> >> The problem in allowing downloads during compile (e.g. by go) is, that it >> leads to non-reproducable builds. I'm currently facing the same issue and >> would like to have a reproducable go *offline* build. >> I would like to propose two ideas to workaround the go-compile fetching >> issue: >> >> First: >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by >> writing a seperate go fetcher or a wget-fetcher) and unpack the >> dependencies into go projects 'vendor' folder. This forces go to compile >> offline. However, one have to generate the 'modules.txt' file in the >> vendor folder 'manually' during unpack. This is error prone, as there is >> no official documentation how this format should look like. Anyway, I've >> tried this approach and it works for me. >> >> Second: >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by >> writing a seperate go fetcher) and unpack the dependencies into a local >> (workdir) go-path. This seemed a good solution for me as the go-path is >> well defined. But for some reason 'go' fetches the zip-files during >> compile into it's download-cache AGAIN, even if the source is already >> unpacked in the go-path. I'll assume this is required to verify the source >> files integrity?! With this approach one have to adapt 'go' to suppress >> this download behaviour. >> >> I've been doing offline builds using a constructed vendor/ directory >> and generated modules.txt. >> >> The only difference between what I have working and what you are >> suggesting (type 1), is that I've gone directly to the sources and >> constructed the vendor directory using the OE git fetcher. That allows >> all functionality to continue to work that is part of OEcore, and the >> build to continue. Switching out the git fetches for tarballs would >> be possible, I just wasn't sure how to use the proxied modules (and I >> wanted the history for debug). >> >> I've never had any issues with the modules.txt, as I generate it at >> the same time as the git fetch lines for the SRC_URI. I've also not >> been using information from the go.mod directly from go.proxy.org, it >> is information I've generated from a clone of the project and dumped >> via go mod. There's likely improvements I can do there, but with what >> I'm doing, I'm going directly to the source of the projects and doing >> clones, which
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Wed, Mar 9, 2022 at 3:10 AM wrote: > > On Thu, Mar 3, 2022 at 05:46 PM, Bruce Ashfield wrote: > > On Thu, Mar 3, 2022 at 10:13 AM wrote: > > On Thu, Mar 3, 2022 at 04:34 AM, Bruce Ashfield wrote: > > On Wed, Mar 2, 2022 at 4:57 PM Andrei Gherzan wrote: > > > Mar 1, 2022 20:15:52 Bruce Ashfield : > > On Tue, Mar 1, 2022 at 10:54 AM wrote: > > On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: > > On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: > > > On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: > > On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via > lists.openembedded.org > wrote: > > > On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: > > > From: Andrei Gherzan > > Compile pulls in the go.mod list requiring network. Without this, do > compile would fail with a similar error to the following: > > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution > > This is something that needs to be carried in your own layers, IMHO it > isn't appropriate for core. > > It isn't about the fetching, it is the entire gap in functionality > that we are missing if go starts fetching dependencies during compile. > > A further thought is that if this is for go.mod issues, there is the > go-mod.bbclass. > > Perhaps enabling it in that class and doing a bbwarn about go fetching > dependencies would be appropriate ? > > Otherwise, someone may not know that this is happening and that a no > network configuration has no chance of working. > > I reckon that is reasonable. I'll personally go down the recipe level to > workaround this change but understanding and agreeing with the reasoning > behind this change, I want to invest a bit into trying to find a proper > solution in the core. Bruce, I know you invested a fair amount of time into > this already. Would you be willing to sync up and see how we can work > together in tackling this? > > Definitely, more ideas are good. In fact, I think there are probably > several approaches that can co-exist, depending on what a > recipe/developer needs. > > I'm in the Eastern time zone here, and will try and grab folks on IRC > to have a level set > > Bruce > > Added Zyga to CC as he is also interested in this as part of his go > development activities. > > Thanks, > Andrei > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > The problem in allowing downloads during compile (e.g. by go) is, that it > leads to non-reproducable builds. I'm currently facing the same issue and > would like to have a reproducable go *offline* build. > I would like to propose two ideas to workaround the go-compile fetching issue: > > First: > - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > writing a seperate go fetcher or a wget-fetcher) and unpack the dependencies > into go projects 'vendor' folder. This forces go to compile offline. However, > one have to generate the 'modules.txt' file in the vendor folder 'manually' > during unpack. This is error prone, as there is no official documentation how > this format should look like. Anyway, I've tried this approach and it works > for me. > > Second: > - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > writing a seperate go fetcher) and unpack the dependencies into a local > (workdir) go-path. This seemed a good solution for me as the go-path is well > defined. But for some reason 'go' fetches the zip-files during compile into > it's download-cache AGAIN, even if the source is already unpacked in the > go-path. I'll assume this is required to verify the source files integrity?! > With this approach one have to adapt 'go' to suppress this download behaviour. > > I've been doing offline builds using a constructed vendor/ directory > and generated modules.txt. > > The only difference between what I have working and what you are > suggesting (type 1), is that I've gone directly to the sources and > constructed the vendor directory using the OE git fetcher. That allows > all functionality to continue to work that is part of OEcore, and the > build to continue. Switching out the git fetches for tarballs would > be possible, I just wasn't sure how to use the proxied modules (and I > wanted the history for debug). > > I've never had any issues with the modules.txt, as I generate it at > the same time as the git fetch lines for the SRC_URI. I've also not > been using information from the go.mod directly from go.proxy.org, it > is information I've generated from a clone of the project and dumped > via go mod. There's likely improvements I can do there, but with what > I'm doing, I'm going directly to the source of the projects and doing > clones, which keeps everything clear of the go infrastructure. > > I have a utility that I'm still cleaning up that generates the SRC_URI > lines, as well as the modules.txt, when I resolve a few nagging > issues, I'll
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Thu, Mar 3, 2022 at 05:46 PM, Bruce Ashfield wrote: > > On Thu, Mar 3, 2022 at 10:13 AM wrote: > >> On Thu, Mar 3, 2022 at 04:34 AM, Bruce Ashfield wrote: >> >> On Wed, Mar 2, 2022 at 4:57 PM Andrei Gherzan wrote: >> >> >> Mar 1, 2022 20:15:52 Bruce Ashfield : >> >> On Tue, Mar 1, 2022 at 10:54 AM wrote: >> >> On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: >> >> On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: >> >> >> On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: >> >> On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via >> lists.openembedded.org >> wrote: >> >> >> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: >> >> >> >> From: Andrei Gherzan >> >> Compile pulls in the go.mod list requiring network. Without this, do >> compile would fail with a similar error to the following: >> >> dial tcp: lookup proxy.golang.org: Temporary failure in name resolution >> >> This is something that needs to be carried in your own layers, IMHO it >> isn't appropriate for core. >> >> It isn't about the fetching, it is the entire gap in functionality >> that we are missing if go starts fetching dependencies during compile. >> >> A further thought is that if this is for go.mod issues, there is the >> go-mod.bbclass. >> >> Perhaps enabling it in that class and doing a bbwarn about go fetching >> dependencies would be appropriate ? >> >> Otherwise, someone may not know that this is happening and that a no >> network configuration has no chance of working. >> >> I reckon that is reasonable. I'll personally go down the recipe level to >> workaround this change but understanding and agreeing with the reasoning >> behind this change, I want to invest a bit into trying to find a proper >> solution in the core. Bruce, I know you invested a fair amount of time >> into this already. Would you be willing to sync up and see how we can work >> together in tackling this? >> >> Definitely, more ideas are good. In fact, I think there are probably >> several approaches that can co-exist, depending on what a >> recipe/developer needs. >> >> I'm in the Eastern time zone here, and will try and grab folks on IRC >> to have a level set >> >> Bruce >> >> Added Zyga to CC as he is also interested in this as part of his go >> development activities. >> >> Thanks, >> Andrei >> >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II >> >> The problem in allowing downloads during compile (e.g. by go) is, that it >> leads to non-reproducable builds. I'm currently facing the same issue and >> would like to have a reproducable go *offline* build. >> I would like to propose two ideas to workaround the go-compile fetching >> issue: >> >> First: >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by >> writing a seperate go fetcher or a wget-fetcher) and unpack the >> dependencies into go projects 'vendor' folder. This forces go to compile >> offline. However, one have to generate the 'modules.txt' file in the >> vendor folder 'manually' during unpack. This is error prone, as there is >> no official documentation how this format should look like. Anyway, I've >> tried this approach and it works for me. >> >> Second: >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by >> writing a seperate go fetcher) and unpack the dependencies into a local >> (workdir) go-path. This seemed a good solution for me as the go-path is >> well defined. But for some reason 'go' fetches the zip-files during >> compile into it's download-cache AGAIN, even if the source is already >> unpacked in the go-path. I'll assume this is required to verify the source >> files integrity?! With this approach one have to adapt 'go' to suppress >> this download behaviour. >> >> I've been doing offline builds using a constructed vendor/ directory >> and generated modules.txt. >> >> The only difference between what I have working and what you are >> suggesting (type 1), is that I've gone directly to the sources and >> constructed the vendor directory using the OE git fetcher. That allows >> all functionality to continue to work that is part of OEcore, and the >> build to continue. Switching out the git fetches for tarballs would >> be possible, I just wasn't sure how to use the proxied modules (and I >> wanted the history for debug). >> >> I've never had any issues with the modules.txt, as I generate it at >> the same time as the git fetch lines for the SRC_URI. I've also not >> been using information from the go.mod directly from go.proxy.org, it >> is information I've generated from a clone of the project and dumped >> via go mod. There's likely improvements I can do there, but with what >> I'm doing, I'm going directly to the source of the projects and doing >> clones, which keeps everything clear of the go infrastructure. >> >> I have a utility that I'm still cleaning up that
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Thu, Mar 3, 2022 at 10:13 AM wrote: > > On Thu, Mar 3, 2022 at 04:34 AM, Bruce Ashfield wrote: > > On Wed, Mar 2, 2022 at 4:57 PM Andrei Gherzan wrote: > > > Mar 1, 2022 20:15:52 Bruce Ashfield : > > On Tue, Mar 1, 2022 at 10:54 AM wrote: > > On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: > > On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: > > > On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: > > On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via > lists.openembedded.org > wrote: > > > On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: > > > From: Andrei Gherzan > > Compile pulls in the go.mod list requiring network. Without this, do > compile would fail with a similar error to the following: > > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution > > This is something that needs to be carried in your own layers, IMHO it > isn't appropriate for core. > > It isn't about the fetching, it is the entire gap in functionality > that we are missing if go starts fetching dependencies during compile. > > A further thought is that if this is for go.mod issues, there is the > go-mod.bbclass. > > Perhaps enabling it in that class and doing a bbwarn about go fetching > dependencies would be appropriate ? > > Otherwise, someone may not know that this is happening and that a no > network configuration has no chance of working. > > I reckon that is reasonable. I'll personally go down the recipe level to > workaround this change but understanding and agreeing with the reasoning > behind this change, I want to invest a bit into trying to find a proper > solution in the core. Bruce, I know you invested a fair amount of time into > this already. Would you be willing to sync up and see how we can work > together in tackling this? > > Definitely, more ideas are good. In fact, I think there are probably > several approaches that can co-exist, depending on what a > recipe/developer needs. > > I'm in the Eastern time zone here, and will try and grab folks on IRC > to have a level set > > Bruce > > Added Zyga to CC as he is also interested in this as part of his go > development activities. > > Thanks, > Andrei > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > The problem in allowing downloads during compile (e.g. by go) is, that it > leads to non-reproducable builds. I'm currently facing the same issue and > would like to have a reproducable go *offline* build. > I would like to propose two ideas to workaround the go-compile fetching issue: > > First: > - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > writing a seperate go fetcher or a wget-fetcher) and unpack the dependencies > into go projects 'vendor' folder. This forces go to compile offline. However, > one have to generate the 'modules.txt' file in the vendor folder 'manually' > during unpack. This is error prone, as there is no official documentation how > this format should look like. Anyway, I've tried this approach and it works > for me. > > Second: > - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > writing a seperate go fetcher) and unpack the dependencies into a local > (workdir) go-path. This seemed a good solution for me as the go-path is well > defined. But for some reason 'go' fetches the zip-files during compile into > it's download-cache AGAIN, even if the source is already unpacked in the > go-path. I'll assume this is required to verify the source files integrity?! > With this approach one have to adapt 'go' to suppress this download behaviour. > > I've been doing offline builds using a constructed vendor/ directory > and generated modules.txt. > > The only difference between what I have working and what you are > suggesting (type 1), is that I've gone directly to the sources and > constructed the vendor directory using the OE git fetcher. That allows > all functionality to continue to work that is part of OEcore, and the > build to continue. Switching out the git fetches for tarballs would > be possible, I just wasn't sure how to use the proxied modules (and I > wanted the history for debug). > > I've never had any issues with the modules.txt, as I generate it at > the same time as the git fetch lines for the SRC_URI. I've also not > been using information from the go.mod directly from go.proxy.org, it > is information I've generated from a clone of the project and dumped > via go mod. There's likely improvements I can do there, but with what > I'm doing, I'm going directly to the source of the projects and doing > clones, which keeps everything clear of the go infrastructure. > > I have a utility that I'm still cleaning up that generates the SRC_URI > lines, as well as the modules.txt, when I resolve a few nagging > issues, I'll make the WIP scripts available. > > Other projects (BSD, etc), have been doing different sorts of >
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Thu, Mar 3, 2022 at 04:34 AM, Bruce Ashfield wrote: > > On Wed, Mar 2, 2022 at 4:57 PM Andrei Gherzan wrote: > >> >> Mar 1, 2022 20:15:52 Bruce Ashfield : >> >> >>> On Tue, Mar 1, 2022 at 10:54 AM wrote: >>> On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via lists.openembedded.org wrote: On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: From: Andrei Gherzan Compile pulls in the go.mod list requiring network. Without this, do compile would fail with a similar error to the following: dial tcp: lookup proxy.golang.org: Temporary failure in name resolution This is something that needs to be carried in your own layers, IMHO it isn't appropriate for core. It isn't about the fetching, it is the entire gap in functionality that we are missing if go starts fetching dependencies during compile. A further thought is that if this is for go.mod issues, there is the go-mod.bbclass. Perhaps enabling it in that class and doing a bbwarn about go fetching dependencies would be appropriate ? Otherwise, someone may not know that this is happening and that a no network configuration has no chance of working. I reckon that is reasonable. I'll personally go down the recipe level to workaround this change but understanding and agreeing with the reasoning behind this change, I want to invest a bit into trying to find a proper solution in the core. Bruce, I know you invested a fair amount of time into this already. Would you be willing to sync up and see how we can work together in tackling this? Definitely, more ideas are good. In fact, I think there are probably several approaches that can co-exist, depending on what a recipe/developer needs. I'm in the Eastern time zone here, and will try and grab folks on IRC to have a level set Bruce Added Zyga to CC as he is also interested in this as part of his go development activities. Thanks, Andrei -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II The problem in allowing downloads during compile (e.g. by go) is, that it leads to non-reproducable builds. I'm currently facing the same issue and would like to have a reproducable go *offline* build. I would like to propose two ideas to workaround the go-compile fetching issue: First: - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by writing a seperate go fetcher or a wget-fetcher) and unpack the dependencies into go projects 'vendor' folder. This forces go to compile offline. However, one have to generate the 'modules.txt' file in the vendor folder 'manually' during unpack. This is error prone, as there is no official documentation how this format should look like. Anyway, I've tried this approach and it works for me. Second: - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by writing a seperate go fetcher) and unpack the dependencies into a local (workdir) go-path. This seemed a good solution for me as the go-path is well defined. But for some reason 'go' fetches the zip-files during compile into it's download-cache AGAIN, even if the source is already unpacked in the go-path. I'll assume this is required to verify the source files integrity?! With this approach one have to adapt 'go' to suppress this download behaviour. >>> >>> I've been doing offline builds using a constructed vendor/ directory >>> and generated modules.txt. >>> >>> The only difference between what I have working and what you are >>> suggesting (type 1), is that I've gone directly to the sources and >>> constructed the vendor directory using the OE git fetcher. That allows >>> all functionality to continue to work that is part of OEcore, and the >>> build to continue. Switching out the git fetches for tarballs would >>> be possible, I just wasn't sure how to use the proxied modules (and I >>> wanted the history for debug). >>> >>> I've never had any issues with the modules.txt, as I generate it at >>> the same time as the git fetch lines for the SRC_URI. I've also not >>> been using information from the go.mod directly from go.proxy.org, it >>> is information I've generated from a clone of the project and dumped >>> via go mod. There's likely improvements I can do there, but with what >>> I'm doing, I'm going directly to the source of the projects and doing >>> clones, which keeps
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Thu, 3 Mar 2022, at 03:34, Bruce Ashfield wrote: > On Wed, Mar 2, 2022 at 4:57 PM Andrei Gherzan wrote: >> >> >> Mar 1, 2022 20:15:52 Bruce Ashfield : >> >> > On Tue, Mar 1, 2022 at 10:54 AM wrote: >> >> >> >> On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: >> >> >> >> On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: >> >> >> >> >> >> On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: >> >> >> >> On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via >> >> lists.openembedded.org >> >> wrote: >> >> >> >> >> >> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: >> >> >> >> >> >> From: Andrei Gherzan >> >> >> >> Compile pulls in the go.mod list requiring network. Without this, do >> >> compile would fail with a similar error to the following: >> >> >> >> dial tcp: lookup proxy.golang.org: Temporary failure in name resolution >> >> >> >> This is something that needs to be carried in your own layers, IMHO it >> >> isn't appropriate for core. >> >> >> >> It isn't about the fetching, it is the entire gap in functionality >> >> that we are missing if go starts fetching dependencies during compile. >> >> >> >> A further thought is that if this is for go.mod issues, there is the >> >> go-mod.bbclass. >> >> >> >> Perhaps enabling it in that class and doing a bbwarn about go fetching >> >> dependencies would be appropriate ? >> >> >> >> Otherwise, someone may not know that this is happening and that a no >> >> network configuration has no chance of working. >> >> >> >> I reckon that is reasonable. I'll personally go down the recipe level to >> >> workaround this change but understanding and agreeing with the reasoning >> >> behind this change, I want to invest a bit into trying to find a proper >> >> solution in the core. Bruce, I know you invested a fair amount of time >> >> into this already. Would you be willing to sync up and see how we can >> >> work together in tackling this? >> >> >> >> Definitely, more ideas are good. In fact, I think there are probably >> >> several approaches that can co-exist, depending on what a >> >> recipe/developer needs. >> >> >> >> I'm in the Eastern time zone here, and will try and grab folks on IRC >> >> to have a level set >> >> >> >> Bruce >> >> >> >> Added Zyga to CC as he is also interested in this as part of his go >> >> development activities. >> >> >> >> Thanks, >> >> Andrei >> >> >> >> >> >> >> >> -- >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> >> thee at its end >> >> - "Use the force Harry" - Gandalf, Star Trek II >> >> >> >> The problem in allowing downloads during compile (e.g. by go) is, that it >> >> leads to non-reproducable builds. I'm currently facing the same issue and >> >> would like to have a reproducable go *offline* build. >> >> I would like to propose two ideas to workaround the go-compile fetching >> >> issue: >> >> >> >> First: >> >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. >> >> by writing a seperate go fetcher or a wget-fetcher) and unpack the >> >> dependencies into go projects 'vendor' folder. This forces go to compile >> >> offline. However, one have to generate the 'modules.txt' file in the >> >> vendor folder 'manually' during unpack. This is error prone, as there is >> >> no official documentation how this format should look like. Anyway, I've >> >> tried this approach and it works for me. >> >> >> >> Second: >> >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. >> >> by writing a seperate go fetcher) and unpack the dependencies into a >> >> local (workdir) go-path. This seemed a good solution for me as the >> >> go-path is well defined. But for some reason 'go' fetches the zip-files >> >> during compile into it's download-cache AGAIN, even if the source is >> >> already unpacked in the go-path. I'll assume this is required to verify >> >> the source files integrity?! With this approach one have to adapt 'go' to >> >> suppress this download behaviour. >> >> >> > >> > I've been doing offline builds using a constructed vendor/ directory >> > and generated modules.txt. >> > >> > The only difference between what I have working and what you are >> > suggesting (type 1), is that I've gone directly to the sources and >> > constructed the vendor directory using the OE git fetcher. That allows >> > all functionality to continue to work that is part of OEcore, and the >> > build to continue. Switching out the git fetches for tarballs would >> > be possible, I just wasn't sure how to use the proxied modules (and I >> > wanted the history for debug). >> > >> > I've never had any issues with the modules.txt, as I generate it at >> > the same time as the git fetch lines for the SRC_URI. I've also not >> > been using information from the go.mod directly from go.proxy.org, it >> > is information I've generated from a clone of the project and dumped >> > via go mod. There's likely improvements I can do there, but with what >> > I'm
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Wed, Mar 2, 2022 at 4:57 PM Andrei Gherzan wrote: > > > Mar 1, 2022 20:15:52 Bruce Ashfield : > > > On Tue, Mar 1, 2022 at 10:54 AM wrote: > >> > >> On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: > >> > >> On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: > >> > >> > >> On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: > >> > >> On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via > >> lists.openembedded.org > >> wrote: > >> > >> > >> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: > >> > >> > >> From: Andrei Gherzan > >> > >> Compile pulls in the go.mod list requiring network. Without this, do > >> compile would fail with a similar error to the following: > >> > >> dial tcp: lookup proxy.golang.org: Temporary failure in name resolution > >> > >> This is something that needs to be carried in your own layers, IMHO it > >> isn't appropriate for core. > >> > >> It isn't about the fetching, it is the entire gap in functionality > >> that we are missing if go starts fetching dependencies during compile. > >> > >> A further thought is that if this is for go.mod issues, there is the > >> go-mod.bbclass. > >> > >> Perhaps enabling it in that class and doing a bbwarn about go fetching > >> dependencies would be appropriate ? > >> > >> Otherwise, someone may not know that this is happening and that a no > >> network configuration has no chance of working. > >> > >> I reckon that is reasonable. I'll personally go down the recipe level to > >> workaround this change but understanding and agreeing with the reasoning > >> behind this change, I want to invest a bit into trying to find a proper > >> solution in the core. Bruce, I know you invested a fair amount of time > >> into this already. Would you be willing to sync up and see how we can work > >> together in tackling this? > >> > >> Definitely, more ideas are good. In fact, I think there are probably > >> several approaches that can co-exist, depending on what a > >> recipe/developer needs. > >> > >> I'm in the Eastern time zone here, and will try and grab folks on IRC > >> to have a level set > >> > >> Bruce > >> > >> Added Zyga to CC as he is also interested in this as part of his go > >> development activities. > >> > >> Thanks, > >> Andrei > >> > >> > >> > >> -- > >> - Thou shalt not follow the NULL pointer, for chaos and madness await > >> thee at its end > >> - "Use the force Harry" - Gandalf, Star Trek II > >> > >> The problem in allowing downloads during compile (e.g. by go) is, that it > >> leads to non-reproducable builds. I'm currently facing the same issue and > >> would like to have a reproducable go *offline* build. > >> I would like to propose two ideas to workaround the go-compile fetching > >> issue: > >> > >> First: > >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > >> writing a seperate go fetcher or a wget-fetcher) and unpack the > >> dependencies into go projects 'vendor' folder. This forces go to compile > >> offline. However, one have to generate the 'modules.txt' file in the > >> vendor folder 'manually' during unpack. This is error prone, as there is > >> no official documentation how this format should look like. Anyway, I've > >> tried this approach and it works for me. > >> > >> Second: > >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > >> writing a seperate go fetcher) and unpack the dependencies into a local > >> (workdir) go-path. This seemed a good solution for me as the go-path is > >> well defined. But for some reason 'go' fetches the zip-files during > >> compile into it's download-cache AGAIN, even if the source is already > >> unpacked in the go-path. I'll assume this is required to verify the source > >> files integrity?! With this approach one have to adapt 'go' to suppress > >> this download behaviour. > >> > > > > I've been doing offline builds using a constructed vendor/ directory > > and generated modules.txt. > > > > The only difference between what I have working and what you are > > suggesting (type 1), is that I've gone directly to the sources and > > constructed the vendor directory using the OE git fetcher. That allows > > all functionality to continue to work that is part of OEcore, and the > > build to continue. Switching out the git fetches for tarballs would > > be possible, I just wasn't sure how to use the proxied modules (and I > > wanted the history for debug). > > > > I've never had any issues with the modules.txt, as I generate it at > > the same time as the git fetch lines for the SRC_URI. I've also not > > been using information from the go.mod directly from go.proxy.org, it > > is information I've generated from a clone of the project and dumped > > via go mod. There's likely improvements I can do there, but with what > > I'm doing, I'm going directly to the source of the projects and doing > > clones, which keeps everything clear of the go infrastructure. > > > > I have a utility that I'm
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
Mar 1, 2022 20:15:52 Bruce Ashfield : > On Tue, Mar 1, 2022 at 10:54 AM wrote: >> >> On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: >> >> On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: >> >> >> On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: >> >> On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via >> lists.openembedded.org >> wrote: >> >> >> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: >> >> >> From: Andrei Gherzan >> >> Compile pulls in the go.mod list requiring network. Without this, do >> compile would fail with a similar error to the following: >> >> dial tcp: lookup proxy.golang.org: Temporary failure in name resolution >> >> This is something that needs to be carried in your own layers, IMHO it >> isn't appropriate for core. >> >> It isn't about the fetching, it is the entire gap in functionality >> that we are missing if go starts fetching dependencies during compile. >> >> A further thought is that if this is for go.mod issues, there is the >> go-mod.bbclass. >> >> Perhaps enabling it in that class and doing a bbwarn about go fetching >> dependencies would be appropriate ? >> >> Otherwise, someone may not know that this is happening and that a no >> network configuration has no chance of working. >> >> I reckon that is reasonable. I'll personally go down the recipe level to >> workaround this change but understanding and agreeing with the reasoning >> behind this change, I want to invest a bit into trying to find a proper >> solution in the core. Bruce, I know you invested a fair amount of time into >> this already. Would you be willing to sync up and see how we can work >> together in tackling this? >> >> Definitely, more ideas are good. In fact, I think there are probably >> several approaches that can co-exist, depending on what a >> recipe/developer needs. >> >> I'm in the Eastern time zone here, and will try and grab folks on IRC >> to have a level set >> >> Bruce >> >> Added Zyga to CC as he is also interested in this as part of his go >> development activities. >> >> Thanks, >> Andrei >> >> >> >> -- >> - Thou shalt not follow the NULL pointer, for chaos and madness await >> thee at its end >> - "Use the force Harry" - Gandalf, Star Trek II >> >> The problem in allowing downloads during compile (e.g. by go) is, that it >> leads to non-reproducable builds. I'm currently facing the same issue and >> would like to have a reproducable go *offline* build. >> I would like to propose two ideas to workaround the go-compile fetching >> issue: >> >> First: >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by >> writing a seperate go fetcher or a wget-fetcher) and unpack the dependencies >> into go projects 'vendor' folder. This forces go to compile offline. >> However, one have to generate the 'modules.txt' file in the vendor folder >> 'manually' during unpack. This is error prone, as there is no official >> documentation how this format should look like. Anyway, I've tried this >> approach and it works for me. >> >> Second: >> - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by >> writing a seperate go fetcher) and unpack the dependencies into a local >> (workdir) go-path. This seemed a good solution for me as the go-path is well >> defined. But for some reason 'go' fetches the zip-files during compile into >> it's download-cache AGAIN, even if the source is already unpacked in the >> go-path. I'll assume this is required to verify the source files integrity?! >> With this approach one have to adapt 'go' to suppress this download >> behaviour. >> > > I've been doing offline builds using a constructed vendor/ directory > and generated modules.txt. > > The only difference between what I have working and what you are > suggesting (type 1), is that I've gone directly to the sources and > constructed the vendor directory using the OE git fetcher. That allows > all functionality to continue to work that is part of OEcore, and the > build to continue. Switching out the git fetches for tarballs would > be possible, I just wasn't sure how to use the proxied modules (and I > wanted the history for debug). > > I've never had any issues with the modules.txt, as I generate it at > the same time as the git fetch lines for the SRC_URI. I've also not > been using information from the go.mod directly from go.proxy.org, it > is information I've generated from a clone of the project and dumped > via go mod. There's likely improvements I can do there, but with what > I'm doing, I'm going directly to the source of the projects and doing > clones, which keeps everything clear of the go infrastructure. > > I have a utility that I'm still cleaning up that generates the SRC_URI > lines, as well as the modules.txt, when I resolve a few nagging > issues, I'll make the WIP scripts available. > > Other projects (BSD, etc), have been doing different sorts of > constructed vendor directories, but they are similar in approach.
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Tue, Mar 1, 2022 at 10:54 AM wrote: > > On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: > > On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: > > > On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: > > On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via > lists.openembedded.org > wrote: > > > On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: > > > From: Andrei Gherzan > > Compile pulls in the go.mod list requiring network. Without this, do > compile would fail with a similar error to the following: > > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution > > This is something that needs to be carried in your own layers, IMHO it > isn't appropriate for core. > > It isn't about the fetching, it is the entire gap in functionality > that we are missing if go starts fetching dependencies during compile. > > A further thought is that if this is for go.mod issues, there is the > go-mod.bbclass. > > Perhaps enabling it in that class and doing a bbwarn about go fetching > dependencies would be appropriate ? > > Otherwise, someone may not know that this is happening and that a no > network configuration has no chance of working. > > I reckon that is reasonable. I'll personally go down the recipe level to > workaround this change but understanding and agreeing with the reasoning > behind this change, I want to invest a bit into trying to find a proper > solution in the core. Bruce, I know you invested a fair amount of time into > this already. Would you be willing to sync up and see how we can work > together in tackling this? > > Definitely, more ideas are good. In fact, I think there are probably > several approaches that can co-exist, depending on what a > recipe/developer needs. > > I'm in the Eastern time zone here, and will try and grab folks on IRC > to have a level set > > Bruce > > Added Zyga to CC as he is also interested in this as part of his go > development activities. > > Thanks, > Andrei > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > The problem in allowing downloads during compile (e.g. by go) is, that it > leads to non-reproducable builds. I'm currently facing the same issue and > would like to have a reproducable go *offline* build. > I would like to propose two ideas to workaround the go-compile fetching issue: > > First: > - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > writing a seperate go fetcher or a wget-fetcher) and unpack the dependencies > into go projects 'vendor' folder. This forces go to compile offline. However, > one have to generate the 'modules.txt' file in the vendor folder 'manually' > during unpack. This is error prone, as there is no official documentation how > this format should look like. Anyway, I've tried this approach and it works > for me. > > Second: > - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by > writing a seperate go fetcher) and unpack the dependencies into a local > (workdir) go-path. This seemed a good solution for me as the go-path is well > defined. But for some reason 'go' fetches the zip-files during compile into > it's download-cache AGAIN, even if the source is already unpacked in the > go-path. I'll assume this is required to verify the source files integrity?! > With this approach one have to adapt 'go' to suppress this download behaviour. > I've been doing offline builds using a constructed vendor/ directory and generated modules.txt. The only difference between what I have working and what you are suggesting (type 1), is that I've gone directly to the sources and constructed the vendor directory using the OE git fetcher. That allows all functionality to continue to work that is part of OEcore, and the build to continue. Switching out the git fetches for tarballs would be possible, I just wasn't sure how to use the proxied modules (and I wanted the history for debug). I've never had any issues with the modules.txt, as I generate it at the same time as the git fetch lines for the SRC_URI. I've also not been using information from the go.mod directly from go.proxy.org, it is information I've generated from a clone of the project and dumped via go mod. There's likely improvements I can do there, but with what I'm doing, I'm going directly to the source of the projects and doing clones, which keeps everything clear of the go infrastructure. I have a utility that I'm still cleaning up that generates the SRC_URI lines, as well as the modules.txt, when I resolve a few nagging issues, I'll make the WIP scripts available. Other projects (BSD, etc), have been doing different sorts of constructed vendor directories, but they are similar in approach. For the short term (i.e. the upcoming release), that is pretty much all we can do. There isn't enough time to implement a new go fetcher backend for bitbake. In the end, how we fetch
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On 3/1/22 7:54 AM, lukas.fu...@weidmueller.com wrote: On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via lists.openembedded.org wrote: On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: From: Andrei Gherzan Compile pulls in the go.mod list requiring network. Without this, do compile would fail with a similar error to the following: dial tcp: lookup proxy.golang.org: Temporary failure in name resolution This is something that needs to be carried in your own layers, IMHO it isn't appropriate for core. It isn't about the fetching, it is the entire gap in functionality that we are missing if go starts fetching dependencies during compile. A further thought is that if this is for go.mod issues, there is the go-mod.bbclass. Perhaps enabling it in that class and doing a bbwarn about go fetching dependencies would be appropriate ? Otherwise, someone may not know that this is happening and that a no network configuration has no chance of working. I reckon that is reasonable. I'll personally go down the recipe level to workaround this change but understanding and agreeing with the reasoning behind this change, I want to invest a bit into trying to find a proper solution in the core. Bruce, I know you invested a fair amount of time into this already. Would you be willing to sync up and see how we can work together in tackling this? Definitely, more ideas are good. In fact, I think there are probably several approaches that can co-exist, depending on what a recipe/developer needs. I'm in the Eastern time zone here, and will try and grab folks on IRC to have a level set Bruce Added Zyga to CC as he is also interested in this as part of his go development activities. Thanks, Andrei -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II The problem in allowing downloads during compile (e.g. by go) is, that it leads to non-reproducable builds. I'm currently facing the same issue and would like to have a reproducable go *offline* build. I would like to propose two ideas to workaround the go-compile fetching issue: First: - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by writing a seperate go fetcher or a wget-fetcher) and unpack the dependencies into go projects 'vendor' folder. This forces go to compile offline. However, one have to generate the 'modules.txt' file in the vendor folder 'manually' during unpack. This is error prone, as there is no official documentation how this format should look like. Anyway, I've tried this approach and it works for me. Second: - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by writing a seperate go fetcher) and unpack the dependencies into a local (workdir) go-path. This seemed a good solution for me as the go-path is well defined. But for some reason 'go' fetches the zip-files during compile into it's download-cache AGAIN, even if the source is already unpacked in the go-path. I'll assume this is required to verify the source files integrity?! With this approach one have to adapt 'go' to suppress this download behaviour. Please let me know your opinion on these two approaches. Second approach is more tenable, go community has larger plans for go modules, so it will be good for bitbake to delegate work to it rather than doing parallel duplicate work, that will mean we will have to keep up with go mod changes always. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162571): https://lists.openembedded.org/g/openembedded-core/message/162571 Mute This Topic: https://lists.openembedded.org/mt/89464905/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Tue, Mar 1, 2022 at 02:14 PM, Bruce Ashfield wrote: > > On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: > >> >> On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: >> >>> On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via >>> lists.openembedded.org >>> wrote: >>> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: > > From: Andrei Gherzan > > Compile pulls in the go.mod list requiring network. Without this, do > compile would fail with a similar error to the following: > > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution This is something that needs to be carried in your own layers, IMHO it isn't appropriate for core. It isn't about the fetching, it is the entire gap in functionality that we are missing if go starts fetching dependencies during compile. >>> >>> A further thought is that if this is for go.mod issues, there is the >>> go-mod.bbclass. >>> >>> Perhaps enabling it in that class and doing a bbwarn about go fetching >>> dependencies would be appropriate ? >>> >>> Otherwise, someone may not know that this is happening and that a no >>> network configuration has no chance of working. >> >> I reckon that is reasonable. I'll personally go down the recipe level to >> workaround this change but understanding and agreeing with the reasoning >> behind this change, I want to invest a bit into trying to find a proper >> solution in the core. Bruce, I know you invested a fair amount of time >> into this already. Would you be willing to sync up and see how we can work >> together in tackling this? > > Definitely, more ideas are good. In fact, I think there are probably > several approaches that can co-exist, depending on what a > recipe/developer needs. > > I'm in the Eastern time zone here, and will try and grab folks on IRC > to have a level set > > Bruce > > >> Added Zyga to CC as he is also interested in this as part of his go >> development activities. >> >> Thanks, >> Andrei > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II The problem in allowing downloads during compile (e.g. by go) is, that it leads to non-reproducable builds. I'm currently facing the same issue and would like to have a reproducable go *offline* build. I would like to propose two ideas to workaround the go-compile fetching issue: First: - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by writing a seperate go fetcher or a wget-fetcher) and unpack the dependencies into go projects 'vendor' folder. This forces go to compile offline. However, one have to generate the 'modules.txt' file in the vendor folder 'manually' during unpack. This is error prone, as there is no official documentation how this format should look like. Anyway, I've tried this approach and it works for me. Second: - Fetch go-dependencies using go.mod file from 'proxy.golang.org' (e.g. by writing a seperate go fetcher) and unpack the dependencies into a local (workdir) go-path. This seemed a good solution for me as the go-path is well defined. But for some reason 'go' fetches the zip-files during compile into it's download-cache AGAIN, even if the source is already unpacked in the go-path. I'll assume this is required to verify the source files integrity?! With this approach one have to adapt 'go' to suppress this download behaviour. Please let me know your opinion on these two approaches. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162562): https://lists.openembedded.org/g/openembedded-core/message/162562 Mute This Topic: https://lists.openembedded.org/mt/89464905/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Tue, Mar 1, 2022 at 6:42 AM Andrei Gherzan wrote: > > On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: > > On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via > > lists.openembedded.org > > wrote: > >> > >> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: > >> > > >> > From: Andrei Gherzan > >> > > >> > Compile pulls in the go.mod list requiring network. Without this, do > >> > compile would fail with a similar error to the following: > >> > > >> > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution > >> > >> This is something that needs to be carried in your own layers, IMHO it > >> isn't appropriate for core. > >> > >> It isn't about the fetching, it is the entire gap in functionality > >> that we are missing if go starts fetching dependencies during compile. > > > > A further thought is that if this is for go.mod issues, there is the > > go-mod.bbclass. > > > > Perhaps enabling it in that class and doing a bbwarn about go fetching > > dependencies would be appropriate ? > > > > Otherwise, someone may not know that this is happening and that a no > > network configuration has no chance of working. > > > > I reckon that is reasonable. I'll personally go down the recipe level to > workaround this change but understanding and agreeing with the reasoning > behind this change, I want to invest a bit into trying to find a proper > solution in the core. Bruce, I know you invested a fair amount of time into > this already. Would you be willing to sync up and see how we can work > together in tackling this? > Definitely, more ideas are good. In fact, I think there are probably several approaches that can co-exist, depending on what a recipe/developer needs. I'm in the Eastern time zone here, and will try and grab folks on IRC to have a level set Bruce > Added Zyga to CC as he is also interested in this as part of his go > development activities. > > Thanks, > Andrei -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162555): https://lists.openembedded.org/g/openembedded-core/message/162555 Mute This Topic: https://lists.openembedded.org/mt/89464905/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Tue, 2022-03-01 at 11:42 +, Andrei Gherzan wrote: > On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: > > On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via > > lists.openembedded.org > > wrote: > > > > > > On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: > > > > > > > > From: Andrei Gherzan > > > > > > > > Compile pulls in the go.mod list requiring network. Without this, do > > > > compile would fail with a similar error to the following: > > > > > > > > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution > > > > > > This is something that needs to be carried in your own layers, IMHO it > > > isn't appropriate for core. > > > > > > It isn't about the fetching, it is the entire gap in functionality > > > that we are missing if go starts fetching dependencies during compile. > > > > A further thought is that if this is for go.mod issues, there is the > > go-mod.bbclass. > > > > Perhaps enabling it in that class and doing a bbwarn about go fetching > > dependencies would be appropriate ? > > > > Otherwise, someone may not know that this is happening and that a no > > network configuration has no chance of working. > > > > I reckon that is reasonable. I'll personally go down the recipe level to > workaround > this change but understanding and agreeing with the reasoning behind this > change, > I want to invest a bit into trying to find a proper solution in the core. > Bruce, > I know you invested a fair amount of time into this already. Would you be > willing > to sync up and see how we can work together in tackling this? > > Added Zyga to CC as he is also interested in this as part of his go > development activities. I'd like to ask that anyone working around this does leave a bb.warn() in the code so that the issue is visible. We need to make sure there is awareness of the problem and hopefully that will encourage people not to forget about it. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162550): https://lists.openembedded.org/g/openembedded-core/message/162550 Mute This Topic: https://lists.openembedded.org/mt/89464905/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Tue, 1 Mar 2022, at 01:55, Bruce Ashfield wrote: > On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via > lists.openembedded.org > wrote: >> >> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote: >> > >> > From: Andrei Gherzan >> > >> > Compile pulls in the go.mod list requiring network. Without this, do >> > compile would fail with a similar error to the following: >> > >> > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution >> >> This is something that needs to be carried in your own layers, IMHO it >> isn't appropriate for core. >> >> It isn't about the fetching, it is the entire gap in functionality >> that we are missing if go starts fetching dependencies during compile. > > A further thought is that if this is for go.mod issues, there is the > go-mod.bbclass. > > Perhaps enabling it in that class and doing a bbwarn about go fetching > dependencies would be appropriate ? > > Otherwise, someone may not know that this is happening and that a no > network configuration has no chance of working. > I reckon that is reasonable. I'll personally go down the recipe level to workaround this change but understanding and agreeing with the reasoning behind this change, I want to invest a bit into trying to find a proper solution in the core. Bruce, I know you invested a fair amount of time into this already. Would you be willing to sync up and see how we can work together in tackling this? Added Zyga to CC as he is also interested in this as part of his go development activities. Thanks, Andrei -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162542): https://lists.openembedded.org/g/openembedded-core/message/162542 Mute This Topic: https://lists.openembedded.org/mt/89464905/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Mon, Feb 28, 2022 at 8:17 PM Bruce Ashfield via
lists.openembedded.org
wrote:
>
> On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote:
> >
> > From: Andrei Gherzan
> >
> > Compile pulls in the go.mod list requiring network. Without this, do
> > compile would fail with a similar error to the following:
> >
> > dial tcp: lookup proxy.golang.org: Temporary failure in name resolution
>
> This is something that needs to be carried in your own layers, IMHO it
> isn't appropriate for core.
>
> It isn't about the fetching, it is the entire gap in functionality
> that we are missing if go starts fetching dependencies during compile.
A further thought is that if this is for go.mod issues, there is the
go-mod.bbclass.
Perhaps enabling it in that class and doing a bbwarn about go fetching
dependencies would be appropriate ?
Otherwise, someone may not know that this is happening and that a no
network configuration has no chance of working.
Cheers,
Bruce
>
> Bruce
>
> >
> > Signed-off-by: Andrei Gherzan
> > ---
> > meta/classes/go.bbclass | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
> > index 9c4c92bffd..dc246c72cb 100644
> > --- a/meta/classes/go.bbclass
> > +++ b/meta/classes/go.bbclass
> > @@ -113,6 +113,7 @@ go_do_compile() {
> > }
> > do_compile[dirs] =+ "${GOTMPDIR}"
> > do_compile[cleandirs] = "${B}/bin ${B}/pkg"
> > +do_compile[network] = "1"
> >
> > go_do_install() {
> > install -d ${D}${libdir}/go/src/${GO_IMPORT}
> > --
> > 2.25.1
> >
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162522):
https://lists.openembedded.org/g/openembedded-core/message/162522
Mute This Topic: https://lists.openembedded.org/mt/89464905/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
On Mon, Feb 28, 2022 at 6:54 PM Andrei Gherzan wrote:
>
> From: Andrei Gherzan
>
> Compile pulls in the go.mod list requiring network. Without this, do
> compile would fail with a similar error to the following:
>
> dial tcp: lookup proxy.golang.org: Temporary failure in name resolution
This is something that needs to be carried in your own layers, IMHO it
isn't appropriate for core.
It isn't about the fetching, it is the entire gap in functionality
that we are missing if go starts fetching dependencies during compile.
Bruce
>
> Signed-off-by: Andrei Gherzan
> ---
> meta/classes/go.bbclass | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
> index 9c4c92bffd..dc246c72cb 100644
> --- a/meta/classes/go.bbclass
> +++ b/meta/classes/go.bbclass
> @@ -113,6 +113,7 @@ go_do_compile() {
> }
> do_compile[dirs] =+ "${GOTMPDIR}"
> do_compile[cleandirs] = "${B}/bin ${B}/pkg"
> +do_compile[network] = "1"
>
> go_do_install() {
> install -d ${D}${libdir}/go/src/${GO_IMPORT}
> --
> 2.25.1
>
>
>
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162519):
https://lists.openembedded.org/g/openembedded-core/message/162519
Mute This Topic: https://lists.openembedded.org/mt/89464905/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/1] go.bbclass: Allow network in do_compile
From: Andrei Gherzan
Compile pulls in the go.mod list requiring network. Without this, do
compile would fail with a similar error to the following:
dial tcp: lookup proxy.golang.org: Temporary failure in name resolution
Signed-off-by: Andrei Gherzan
---
meta/classes/go.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
index 9c4c92bffd..dc246c72cb 100644
--- a/meta/classes/go.bbclass
+++ b/meta/classes/go.bbclass
@@ -113,6 +113,7 @@ go_do_compile() {
}
do_compile[dirs] =+ "${GOTMPDIR}"
do_compile[cleandirs] = "${B}/bin ${B}/pkg"
+do_compile[network] = "1"
go_do_install() {
install -d ${D}${libdir}/go/src/${GO_IMPORT}
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162517):
https://lists.openembedded.org/g/openembedded-core/message/162517
Mute This Topic: https://lists.openembedded.org/mt/89464905/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
