Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
On Thu, Jun 23, 2022 at 9:27 PM Matthias Klose wrote:
>
> On 20.06.22 05:50, Michael Hudson-Doyle wrote:
> > On Mon, 20 Jun 2022 at 14:59, Shengjing Zhu wrote:
> >> diff --git a/lib/Debian/Debhelper/Buildsystem/golang.pm
> >> b/lib/Debian/Debhelper/Buildsystem/golang.pm
> >> index 60f725a..02f16fe 100644
> >> --- a/lib/Debian/Debhelper/Buildsystem/golang.pm
> >> +++ b/lib/Debian/Debhelper/Buildsystem/golang.pm
> >> @@ -369,6 +369,7 @@ sub _set_goproxy {
> >> sub _set_cgo_flags {
> >> my $bf = Dpkg::BuildFlags->new();
> >> $bf->load_config();
> >> +$bf->set_feature("optimize", "lto", 0)
> >>
> >> my @flags = ( "CFLAGS", "CPPFLAGS", "CXXFLAGS", "FFLAGS", "LDFLAGS"
> >> );
> >> foreach my $flag (@flags) {
> >>
This patch actually doesn't work. I just proposed a working patch for
dh-golang at
https://salsa.debian.org/go-team/packages/dh-golang/-/merge_requests/18
--
Shengjing Zhu
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
On 20.06.22 10:14, Shengjing Zhu wrote: On Mon, Jun 20, 2022 at 03:50:23PM +1200, Michael Hudson-Doyle wrote: On Mon, 20 Jun 2022 at 14:59, Shengjing Zhu wrote: On Mon, Jun 20, 2022 at 08:42:51AM +1200, Michael Hudson-Doyle wrote: Ah yes but that patch doesn't actually work in practice. I've been slack on this :( IIRC the problem with https://go-review.googlesource.com/c/go/+/339370 is that lto causes some of the references in the linked executable to disappear, meaning cgo can't do the analysis it needs to do. It's a shame because it's obviously a much cleaner patch... However CL281314 has its own problem that causes several packages FTBFS (#982701, #982714, #982720, #982724, #982734) I think that was a broken version of the patch -- the packages those bugs affect build fine in Ubuntu now afaics -- but I haven't checked explicitly. It seems there are still many golang* package in lto-disabled-list[1]. well, yes. but we won't introduce that list into Debian. [1] https://launchpad.net/ubuntu/+source/lto-disabled-list We should redo the archive-rebuild with the patch in golang-1.18, since the bugs are for golang-1.15, which is quite old now. that was just done recently. See http://qa-logs.debian.net/2022/06/09/ I'm about to file bug reports for the lto build failures, except those go packages.
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
On 20.06.22 05:50, Michael Hudson-Doyle wrote:
On Mon, 20 Jun 2022 at 14:59, Shengjing Zhu wrote:
On Mon, Jun 20, 2022 at 08:42:51AM +1200, Michael Hudson-Doyle wrote:
Ah yes but that patch doesn't actually work in practice. I've been slack
on
this :(
IIRC the problem with https://go-review.googlesource.com/c/go/+/339370
is
that lto causes some of the references in the linked executable to
disappear, meaning cgo can't do the analysis it needs to do. It's a shame
because it's obviously a much cleaner patch...
However CL281314 has its own problem that causes several packages FTBFS
(#982701, #982714, #982720, #982724, #982734)
I think that was a broken version of the patch -- the packages those bugs
affect build fine in Ubuntu now afaics -- but I haven't checked explicitly.
If CL339370 has problem too, can we try another routine? Like disable lto
in dh-golang?
If I read doko's message right, we are only going to enable lto by default
in dpkg-buildflags, not gcc itself. So hacking dh-golang looks more
sensible to me.
For example adding following line to dh-golang (I didn't test it though):
diff --git a/lib/Debian/Debhelper/Buildsystem/golang.pm
b/lib/Debian/Debhelper/Buildsystem/golang.pm
index 60f725a..02f16fe 100644
--- a/lib/Debian/Debhelper/Buildsystem/golang.pm
+++ b/lib/Debian/Debhelper/Buildsystem/golang.pm
@@ -369,6 +369,7 @@ sub _set_goproxy {
sub _set_cgo_flags {
my $bf = Dpkg::BuildFlags->new();
$bf->load_config();
+$bf->set_feature("optimize", "lto", 0)
my @flags = ( "CFLAGS", "CPPFLAGS", "CXXFLAGS", "FFLAGS", "LDFLAGS" );
foreach my $flag (@flags) {
If that works, it might be a better option for now indeed.
yes, that looks like a good solution.
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
On Mon, Jun 20, 2022 at 03:50:23PM +1200, Michael Hudson-Doyle wrote: > On Mon, 20 Jun 2022 at 14:59, Shengjing Zhu wrote: > > > On Mon, Jun 20, 2022 at 08:42:51AM +1200, Michael Hudson-Doyle wrote: > > > Ah yes but that patch doesn't actually work in practice. I've been slack > > on > > > this :( > > > > > > IIRC the problem with https://go-review.googlesource.com/c/go/+/339370 > > is > > > that lto causes some of the references in the linked executable to > > > disappear, meaning cgo can't do the analysis it needs to do. It's a shame > > > because it's obviously a much cleaner patch... > > > > > > > However CL281314 has its own problem that causes several packages FTBFS > > (#982701, #982714, #982720, #982724, #982734) > > > > I think that was a broken version of the patch -- the packages those bugs > affect build fine in Ubuntu now afaics -- but I haven't checked explicitly. > It seems there are still many golang* package in lto-disabled-list[1]. [1] https://launchpad.net/ubuntu/+source/lto-disabled-list We should redo the archive-rebuild with the patch in golang-1.18, since the bugs are for golang-1.15, which is quite old now.
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
On Mon, 20 Jun 2022 at 14:59, Shengjing Zhu wrote:
> On Mon, Jun 20, 2022 at 08:42:51AM +1200, Michael Hudson-Doyle wrote:
> > Ah yes but that patch doesn't actually work in practice. I've been slack
> on
> > this :(
> >
> > IIRC the problem with https://go-review.googlesource.com/c/go/+/339370
> is
> > that lto causes some of the references in the linked executable to
> > disappear, meaning cgo can't do the analysis it needs to do. It's a shame
> > because it's obviously a much cleaner patch...
> >
>
> However CL281314 has its own problem that causes several packages FTBFS
> (#982701, #982714, #982720, #982724, #982734)
>
I think that was a broken version of the patch -- the packages those bugs
affect build fine in Ubuntu now afaics -- but I haven't checked explicitly.
> If CL339370 has problem too, can we try another routine? Like disable lto
> in dh-golang?
>
> If I read doko's message right, we are only going to enable lto by default
> in dpkg-buildflags, not gcc itself. So hacking dh-golang looks more
> sensible to me.
>
> For example adding following line to dh-golang (I didn't test it though):
>
> diff --git a/lib/Debian/Debhelper/Buildsystem/golang.pm
> b/lib/Debian/Debhelper/Buildsystem/golang.pm
> index 60f725a..02f16fe 100644
> --- a/lib/Debian/Debhelper/Buildsystem/golang.pm
> +++ b/lib/Debian/Debhelper/Buildsystem/golang.pm
> @@ -369,6 +369,7 @@ sub _set_goproxy {
> sub _set_cgo_flags {
> my $bf = Dpkg::BuildFlags->new();
> $bf->load_config();
> +$bf->set_feature("optimize", "lto", 0)
>
> my @flags = ( "CFLAGS", "CPPFLAGS", "CXXFLAGS", "FFLAGS", "LDFLAGS" );
> foreach my $flag (@flags) {
>
If that works, it might be a better option for now indeed.
Cheers,
mwh
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
On Mon, Jun 20, 2022 at 08:42:51AM +1200, Michael Hudson-Doyle wrote:
> Ah yes but that patch doesn't actually work in practice. I've been slack on
> this :(
>
> IIRC the problem with https://go-review.googlesource.com/c/go/+/339370 is
> that lto causes some of the references in the linked executable to
> disappear, meaning cgo can't do the analysis it needs to do. It's a shame
> because it's obviously a much cleaner patch...
>
However CL281314 has its own problem that causes several packages FTBFS
(#982701, #982714, #982720, #982724, #982734)
If CL339370 has problem too, can we try another routine? Like disable lto
in dh-golang?
If I read doko's message right, we are only going to enable lto by default
in dpkg-buildflags, not gcc itself. So hacking dh-golang looks more
sensible to me.
For example adding following line to dh-golang (I didn't test it though):
diff --git a/lib/Debian/Debhelper/Buildsystem/golang.pm
b/lib/Debian/Debhelper/Buildsystem/golang.pm
index 60f725a..02f16fe 100644
--- a/lib/Debian/Debhelper/Buildsystem/golang.pm
+++ b/lib/Debian/Debhelper/Buildsystem/golang.pm
@@ -369,6 +369,7 @@ sub _set_goproxy {
sub _set_cgo_flags {
my $bf = Dpkg::BuildFlags->new();
$bf->load_config();
+$bf->set_feature("optimize", "lto", 0)
my @flags = ( "CFLAGS", "CPPFLAGS", "CXXFLAGS", "FFLAGS", "LDFLAGS" );
foreach my $flag (@flags) {
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
Ah yes but that patch doesn't actually work in practice. I've been slack on this :( IIRC the problem with https://go-review.googlesource.com/c/go/+/339370 is that lto causes some of the references in the linked executable to disappear, meaning cgo can't do the analysis it needs to do. It's a shame because it's obviously a much cleaner patch... Cheers, mwh On Sat, 18 Jun 2022 at 02:51, Shengjing Zhu wrote: > On Fri, Jun 17, 2022 at 3:36 PM Matthias Klose wrote: > > > > Package: src:golang-1.18 > > Version: 1.18.3-1 > > Severity: wishlist > > Tags: patch > > > > lto support now landed in dpkg in unstable, now evaluation the > possibility to > > turn on link time optimizations by default. Currently builds fail like: > > > > [...] > > loadelf: $WORK/b101/_pkg_.a(_x001.o): 151068: sym#11 > (g_cgo_export.c.d01d1947): > > ignoring symbol in section 4 (type 0) > > loadelf: $WORK/b101/_pkg_.a(_x002.o): 151072: sym#11 > > (cgo_linux.cgo2.c.e1a05304): ignoring symbol in section 4 (type 0) > > loadelf: $WORK/b101/_pkg_.a(_x003.o): 151077: sym#13 > > (cgo_resnew.cgo2.c.1f334c1b): ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b101/_pkg_.a(_x004.o): 151081: sym#11 > > (cgo_socknew.cgo2.c.00cb1c10): ignoring symbol in section 4 (type 0) > > loadelf: $WORK/b101/_pkg_.a(_x005.o): 151086: sym#13 > (cgo_unix.cgo2.c.81407354): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x001.o): 151068: sym#11 > (g_cgo_export.c.d01d1947): > > ignoring symbol in section 4 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x002.o): 151093: sym#11 > (cgo.cgo2.c.547524ab): > > ignoring symbol in section 4 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x003.o): 151098: sym#13 > (gcc_context.c.0c594a6b): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x004.o): 151105: sym#16 > (gcc_fatalf.c.104c99f5): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x005.o): 151116: sym#21 > (gcc_libinit.c.57049a80): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x006.o): 151126: sym#19 > > (gcc_linux_amd64.c.7bb88008): ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x007.o): 151133: sym#16 > (gcc_mmap.c.58726c34): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x008.o): 151138: sym#13 > (gcc_setenv.c.58922dfc): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x009.o): 151143: sym#14 > (gcc_sigaction.c.8c3bca0d): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x010.o): 151148: sym#13 > (gcc_traceback.c.2a322f93): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x011.o): 151156: sym#16 > (gcc_util.c.9bbd18fc): > > ignoring symbol in section 5 (type 0) > > loadelf: $WORK/b108/_pkg_.a(_x012.o): 151161: sym#13 > (linux_syscall.c.96145c1f): > > ignoring symbol in section 5 (type 0) > > _cgo_callers: relocation target x_cgo_callers not defined > > _cgo_init: relocation target x_cgo_init not defined > > _cgo_mmap: relocation target x_cgo_mmap not defined > > _cgo_munmap: relocation target x_cgo_munmap not defined > > /usr/lib/go-1.18/pkg/tool/linux_amd64/link: too many errors > > [...] > > > > > https://patches.ubuntu.com/g/golang-1.18/golang-1.18_1.18.3-1ubuntu1.patch > > > > so please disable these for now and/or forward the issue upstream. > > The patch says it has been forwarded to > https://go-review.googlesource.com/c/go/+/281314 > And on https://go-review.googlesource.com/c/go/+/281314, the comment > says it has been obsoleted by > https://go-review.googlesource.com/c/go/+/339370 > > -- > Shengjing Zhu > >
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
On Fri, Jun 17, 2022 at 3:36 PM Matthias Klose wrote: > > Package: src:golang-1.18 > Version: 1.18.3-1 > Severity: wishlist > Tags: patch > > lto support now landed in dpkg in unstable, now evaluation the possibility to > turn on link time optimizations by default. Currently builds fail like: > > [...] > loadelf: $WORK/b101/_pkg_.a(_x001.o): 151068: sym#11 > (g_cgo_export.c.d01d1947): > ignoring symbol in section 4 (type 0) > loadelf: $WORK/b101/_pkg_.a(_x002.o): 151072: sym#11 > (cgo_linux.cgo2.c.e1a05304): ignoring symbol in section 4 (type 0) > loadelf: $WORK/b101/_pkg_.a(_x003.o): 151077: sym#13 > (cgo_resnew.cgo2.c.1f334c1b): ignoring symbol in section 5 (type 0) > loadelf: $WORK/b101/_pkg_.a(_x004.o): 151081: sym#11 > (cgo_socknew.cgo2.c.00cb1c10): ignoring symbol in section 4 (type 0) > loadelf: $WORK/b101/_pkg_.a(_x005.o): 151086: sym#13 > (cgo_unix.cgo2.c.81407354): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x001.o): 151068: sym#11 > (g_cgo_export.c.d01d1947): > ignoring symbol in section 4 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x002.o): 151093: sym#11 (cgo.cgo2.c.547524ab): > ignoring symbol in section 4 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x003.o): 151098: sym#13 (gcc_context.c.0c594a6b): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x004.o): 151105: sym#16 (gcc_fatalf.c.104c99f5): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x005.o): 151116: sym#21 (gcc_libinit.c.57049a80): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x006.o): 151126: sym#19 > (gcc_linux_amd64.c.7bb88008): ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x007.o): 151133: sym#16 (gcc_mmap.c.58726c34): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x008.o): 151138: sym#13 (gcc_setenv.c.58922dfc): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x009.o): 151143: sym#14 > (gcc_sigaction.c.8c3bca0d): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x010.o): 151148: sym#13 > (gcc_traceback.c.2a322f93): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x011.o): 151156: sym#16 (gcc_util.c.9bbd18fc): > ignoring symbol in section 5 (type 0) > loadelf: $WORK/b108/_pkg_.a(_x012.o): 151161: sym#13 > (linux_syscall.c.96145c1f): > ignoring symbol in section 5 (type 0) > _cgo_callers: relocation target x_cgo_callers not defined > _cgo_init: relocation target x_cgo_init not defined > _cgo_mmap: relocation target x_cgo_mmap not defined > _cgo_munmap: relocation target x_cgo_munmap not defined > /usr/lib/go-1.18/pkg/tool/linux_amd64/link: too many errors > [...] > > https://patches.ubuntu.com/g/golang-1.18/golang-1.18_1.18.3-1ubuntu1.patch > > so please disable these for now and/or forward the issue upstream. The patch says it has been forwarded to https://go-review.googlesource.com/c/go/+/281314 And on https://go-review.googlesource.com/c/go/+/281314, the comment says it has been obsoleted by https://go-review.googlesource.com/c/go/+/339370 -- Shengjing Zhu
Bug#1013102: cmd/link: check CGO_CFLAGS for non -g/-I/-O options before,+ internal linking
Package: src:golang-1.18 Version: 1.18.3-1 Severity: wishlist Tags: patch lto support now landed in dpkg in unstable, now evaluation the possibility to turn on link time optimizations by default. Currently builds fail like: [...] loadelf: $WORK/b101/_pkg_.a(_x001.o): 151068: sym#11 (g_cgo_export.c.d01d1947): ignoring symbol in section 4 (type 0) loadelf: $WORK/b101/_pkg_.a(_x002.o): 151072: sym#11 (cgo_linux.cgo2.c.e1a05304): ignoring symbol in section 4 (type 0) loadelf: $WORK/b101/_pkg_.a(_x003.o): 151077: sym#13 (cgo_resnew.cgo2.c.1f334c1b): ignoring symbol in section 5 (type 0) loadelf: $WORK/b101/_pkg_.a(_x004.o): 151081: sym#11 (cgo_socknew.cgo2.c.00cb1c10): ignoring symbol in section 4 (type 0) loadelf: $WORK/b101/_pkg_.a(_x005.o): 151086: sym#13 (cgo_unix.cgo2.c.81407354): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x001.o): 151068: sym#11 (g_cgo_export.c.d01d1947): ignoring symbol in section 4 (type 0) loadelf: $WORK/b108/_pkg_.a(_x002.o): 151093: sym#11 (cgo.cgo2.c.547524ab): ignoring symbol in section 4 (type 0) loadelf: $WORK/b108/_pkg_.a(_x003.o): 151098: sym#13 (gcc_context.c.0c594a6b): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x004.o): 151105: sym#16 (gcc_fatalf.c.104c99f5): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x005.o): 151116: sym#21 (gcc_libinit.c.57049a80): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x006.o): 151126: sym#19 (gcc_linux_amd64.c.7bb88008): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x007.o): 151133: sym#16 (gcc_mmap.c.58726c34): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x008.o): 151138: sym#13 (gcc_setenv.c.58922dfc): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x009.o): 151143: sym#14 (gcc_sigaction.c.8c3bca0d): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x010.o): 151148: sym#13 (gcc_traceback.c.2a322f93): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x011.o): 151156: sym#16 (gcc_util.c.9bbd18fc): ignoring symbol in section 5 (type 0) loadelf: $WORK/b108/_pkg_.a(_x012.o): 151161: sym#13 (linux_syscall.c.96145c1f): ignoring symbol in section 5 (type 0) _cgo_callers: relocation target x_cgo_callers not defined _cgo_init: relocation target x_cgo_init not defined _cgo_mmap: relocation target x_cgo_mmap not defined _cgo_munmap: relocation target x_cgo_munmap not defined /usr/lib/go-1.18/pkg/tool/linux_amd64/link: too many errors [...] https://patches.ubuntu.com/g/golang-1.18/golang-1.18_1.18.3-1ubuntu1.patch so please disable these for now and/or forward the issue upstream.
