Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
Hi Matthias, On Wed, Sep 23, 2020 at 05:52:47PM +0200, Matthias Klose wrote: > On 9/23/20 1:26 PM, Helmut Grohne wrote: > > Hi Matthias, > > > > thank you for reviewing my patch! > > > > On Wed, Sep 23, 2020 at 12:48:10PM +0200, Matthias Klose wrote: > >> looking at this patch: > >> > >> - dh_installdocs -p$(p_gdc) > >> - dh_installchangelogs -p$(p_gdc) src/gcc/d/ChangeLog > >> + debian/dh_doclink -p$(p_gdc_n) $(p_xbase) > >> > >> doesn't work, replacing the doc directory with a symlink. > > > > Please elaborate. > > > > p_gdc_n is a new package. Nothing is replaced here. > > > > The calls for p_gdc are not removed. They're moved lower. The docs stay > > right there where they are now. > > > > The new p_gdc_n package uses a symlink for its doc directory. That > > should be fine. > > ... which then doesn't work with a separate gdc build. I failed to figure out how a separate gdc would build at all. gdc-10-multilib also uses dh_doclink targeting p_xbase. So either the separate gdc build is broken already or it works with the additional p_xbase link here. In neither case, a change to the patch seems required. Helmut
Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
On 9/23/20 1:26 PM, Helmut Grohne wrote: > Hi Matthias, > > thank you for reviewing my patch! > > On Wed, Sep 23, 2020 at 12:48:10PM +0200, Matthias Klose wrote: >> looking at this patch: >> >> -dh_installdocs -p$(p_gdc) >> -dh_installchangelogs -p$(p_gdc) src/gcc/d/ChangeLog >> +debian/dh_doclink -p$(p_gdc_n) $(p_xbase) >> >> doesn't work, replacing the doc directory with a symlink. > > Please elaborate. > > p_gdc_n is a new package. Nothing is replaced here. > > The calls for p_gdc are not removed. They're moved lower. The docs stay > right there where they are now. > > The new p_gdc_n package uses a symlink for its doc directory. That > should be fine. ... which then doesn't work with a separate gdc build.
Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
Hi Matthias, thank you for reviewing my patch! On Wed, Sep 23, 2020 at 12:48:10PM +0200, Matthias Klose wrote: > looking at this patch: > > - dh_installdocs -p$(p_gdc) > - dh_installchangelogs -p$(p_gdc) src/gcc/d/ChangeLog > + debian/dh_doclink -p$(p_gdc_n) $(p_xbase) > > doesn't work, replacing the doc directory with a symlink. Please elaborate. p_gdc_n is a new package. Nothing is replaced here. The calls for p_gdc are not removed. They're moved lower. The docs stay right there where they are now. The new p_gdc_n package uses a symlink for its doc directory. That should be fine. Helmut
Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
looking at this patch: - dh_installdocs -p$(p_gdc) - dh_installchangelogs -p$(p_gdc) src/gcc/d/ChangeLog + debian/dh_doclink -p$(p_gdc_n) $(p_xbase) doesn't work, replacing the doc directory with a symlink.
Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
Control: reassign -1 src:gcc-10
Hi Matthias,
On Sun, Sep 30, 2018 at 02:46:04PM +0200, Helmut Grohne wrote:
> > - please complete the patch to cover all frontends. while these
> >might not be necessary for your bootstrap effort, I'd like to
> >have all frontends be covered in an uniform way.
>
> I've worked on this. It now covers cpp, cxx, d, fortran, gcc, go and
> objc. Notably, it does not cover ada, brig or objc++. ada looks
> complex. brig depends on hsail-tools, which is not Multi-Arch: foreign.
> So we cannot do brig at this time. objc++ looks manageable.
I've completed objc++, which was indeed easy. The new gm2 frontend is
also missing.
I've also split up the big patch into individual patches to make them
easier to use/review/select. The updated patch stack for gcc-10 is
applied. Beware that the patch stack introduces Breaks and Replaces. The
versions in these Breaks and Replaces is always "0TODO" or "999:0TODO"
and needs to be replaced with the version used for applying.
> > - for testing:
I've performed a regular build and inspected the resulting lintian
warnings and package contents.
I've also attempted building cross compilers using the patch stack by
adding the stack to a fork of rebootstrap.
I've not attempted cross building gcc with the patch stack, because
cross builds are known broken:
* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92336
* https://gcc.gnu.org/legacy-ml/gcc-help/2015-01/msg00111.html
Both of these issues are reproducible on gcc-9 and gcc-10 in unstable.
Moving forward:
There is more urgency for this feature now, because we need to run 64bit
executables for building larger packages on 32bit architectures. There
are two routes:
* Moving this stack forward and going Multi-Arch.
* Introducing yet another pile of compiler executables for this purpose
(YunQiang Su's appraoch).
I believe that the utility for any of these -for-host variants is very
limited except for c, cxx and fortran. I recommend not applying the
other patches (d, go, objc, objcxx), but I have attached them for
reference.
How to apply the patch stack:
0001: Only makes sense if you apply one of 0002, 0003, 0004, 0005, 0006,
0007, 0009, 0010 or 0011.
0002: cpp-$v-$triplet. Requires 0001. Update Breaks+Replaces.
0003: gcc-$v-$triplet. Requires 0001 and 0002. Updates Breaks+Replaces.
0004: g++-$v-$triplet. Requires 0001, 0002, and 0003. Update
Breaks+Replaces.
0005: gfortran-$v-$triplet. Requires 0001. Update Breaks+Replaces.
0006: gdc-$v-$triplet. Requires 0001. Update Breaks+Replaces.
0007: gccgo-$v-$triplet: Requires 0001. Update Breaks+Replaces.
0008: Cleanup. Consider applying right away.
0009: Cleanup. Consider applying right away.
0010: gobjc-$v-$triplet: Requires 0001 and 0009. Update Breaks+Replaces.
0011: gobjc++-$v-$triplet. Requires 0001, 0010 and 0011. Update
Breaks+Replaces.
I hope that this allows us to move forward. I've also Cced
debian-cross@l.d.o to elicit more review and testing.
Helmut
>From 069536e9afe1082c29a3b2da9e6f6b156f44e9fc Mon Sep 17 00:00:00 2001
From: Helmut Grohne
Date: Fri, 27 Mar 2020 06:32:46 +0100
Subject: [PATCH 01/11] add support code for gcc-for-host
control.m4 macro for_each_arch iterates over all known architectures and
substitutes arch_deb (the Debian architecture name), arch_gnu (the GNU
triplet) and arch_gnusuffix (a package name suffix encoding the
architecture) variables.
The substitution variable target:suffix can be used to interpolate the
current target architecture suffix for dependencies from the regular
tool packages on architecture-qualified tool packages.
---
debian/control.m4 | 7 +++
debian/rules.conf | 2 ++
2 files changed, 9 insertions(+)
diff --git a/debian/control.m4 b/debian/control.m4
index bbc71a2..94a9741 100644
--- a/debian/control.m4
+++ b/debian/control.m4
@@ -39,6 +39,13 @@ define(`BUILT_USING', ifelse(add_built_using,yes,`Built-Using: ${Built-Using}
'))
define(`TARGET_PACKAGE',`X-DH-Build-For-Type: target
')
+define(`_for_each',`ifelse(`$3',`',`',`pushdef(`$1',`$3')$2`'popdef(`$1')`'$0(`$1',`$2',shift(shift(shift($@')')
+define(`for_each_arch',`_for_each(`_arch',`dnl
+pushdef(`arch_deb',`patsubst(_arch,`=.*',`')')dnl
+pushdef(`arch_gnu',`patsubst(_arch,`.*=',`')')dnl
+pushdef(`arch_gnusuffix',`-patsubst(arch_gnu,`_',`-')')dnl
+$1`'popdef(`arch_gnusuffix')popdef(`arch_gnu')popdef(`arch_deb')',dnl
+patsubst(ARCH_GNUTYPE_MAP,`\s+',`,'))')
divert`'dnl
dnl --
diff --git a/debian/rules.conf b/debian/rules.conf
index 441c0e8..6149073 100644
--- a/debian/rules.conf
+++ b/debian/rules.conf
@@ -686,6 +686,7 @@ ctrl_flags += \
-DPHOBOS_BUILD_DEP="$(PHOBOS_BUILD_DEP)" \
-DGM2_BUILD_DEP="$(GM2_BUILD_DEP)" \
-DOFFLOAD_BUILD_DEP="$(OFFLOAD_BUILD_DEP)" \
+ -DARCH_GNUTYPE_MAP="$(arch_gnutype_map)" \
-DMULTILIB_ARCHS="$(multilib_archs)" \
-DNEON_ARCHS="$(neon_archs)" \
-DTP=$(TP) \
@@ -1095,6 +1096,7 @@ substvars-
Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
On Sat, Apr 28, 2018 at 01:06:20PM +0200, Matthias Klose wrote:
> As addition to the feedback given on irc,
Thank you very much for spending time on this tricky feature and your
patience. I'm attaching an improved patch, but it is certainly not ready
for merging and I don't expect Matthias to look into it. This mail
merely constitutes a random snapshot of my work on the patch. Help from
others is appreciated.
> - please complete the patch to cover all frontends. while these
>might not be necessary for your bootstrap effort, I'd like to
>have all frontends be covered in an uniform way.
I've worked on this. It now covers cpp, cxx, d, fortran, gcc, go and
objc. Notably, it does not cover ada, brig or objc++. ada looks
complex. brig depends on hsail-tools, which is not Multi-Arch: foreign.
So we cannot do brig at this time. objc++ looks manageable.
> - as the first step, please don't build any -for-host and -for-build
>packages yet.
In the interest of keeping the diff manageable, this was the intention.
The addition of -for-host or -for-build packages will be covered in a
different bug report.
> - for testing:
Given that the patch is still very much incomplete, I've not performed
much testing beyond native builds and running lintian.
Helmut
diff -u gcc-8-8.2.0/debian/changelog gcc-8-8.2.0/debian/changelog
--- gcc-8-8.2.0/debian/changelog
+++ gcc-8-8.2.0/debian/changelog
@@ -1,3 +1,10 @@
+gcc-8 (8.2.0-7.1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ *
+
+ -- Helmut Grohne Sat, 29 Sep 2018 13:31:54 +0200
+
gcc-8 (8.2.0-7) unstable; urgency=medium
* Update to SVN 20180917 (r264370) from the gcc-8-branch.
diff -u gcc-8-8.2.0/debian/control.m4 gcc-8-8.2.0/debian/control.m4
--- gcc-8-8.2.0/debian/control.m4
+++ gcc-8-8.2.0/debian/control.m4
@@ -35,6 +35,13 @@
'))
define(`TARGET_PACKAGE',`X-DH-Build-For-Type: target
')
+define(`_for_each',`ifelse(`$3',`',`',`pushdef(`$1',`$3')$2`'popdef(`$1')`'$0(`$1',`$2',shift(shift(shift($@')')
+define(`for_each_arch',`_for_each(`_arch',`dnl
+pushdef(`arch_deb',`patsubst(_arch,`=.*',`')')dnl
+pushdef(`arch_gnu',`patsubst(_arch,`.*=',`')')dnl
+pushdef(`arch_gnusuffix',`-patsubst(arch_gnu,`_',`-')')dnl
+$1`'popdef(`arch_gnusuffix')popdef(`arch_gnu')popdef(`arch_deb')',dnl
+patsubst(ARCH_GNUTYPE_MAP,`\s+',`,'))')
divert`'dnl
dnl --
@@ -701,21 +708,22 @@
')`'dnl x32dev
')`'dnl cdev
-ifenabled(`cdev',`
-Package: gcc`'PV`'TS
-Architecture: any
-ifdef(`TARGET',`Multi-Arch: foreign
-')dnl
+ifenabled(`cdev',`dnl
+for_each_arch(`
+Package: gcc`'PV`'arch_gnusuffix
+Architecture: ifdef(`TARGET',`any',arch_deb)
+Multi-Arch: foreign
Section: devel
Priority: optional
-Depends: cpp`'PV`'TS (= ${gcc:Version}),ifenabled(`gccbase',` BASEDEP,')
+Depends: cpp`'PV`'arch_gnusuffix (= ${gcc:Version}),ifenabled(`gccbase',` BASEDEP,')
ifenabled(`gccxbase',` BASEDEP,')
${dep:libcc1},
- binutils`'TS (>= ${binutils:Version}),
+ binutils`'arch_gnusuffix (>= ${binutils:Version}),
${dep:libgccdev}, ${shlibs:Depends}, ${misc:Depends}
Recommends: ${dep:libcdev}
-Replaces: gccgo-8 (<< ${gcc:Version}), cpp`'PV`'TS (<< 7.1.1-8)
-Suggests: ${gcc:multilib}, gcc`'PV-doc (>= ${gcc:SoftVersion}),
+Replaces: gccgo-8 (<< ${gcc:Version}), cpp`'PV`'arch_gnusuffix (<< 7.1.1-8), gcc`'PV (<< 999:0TODO)
+Breaks: gcc`'PV (<< 0TODO)
+Suggests: gcc`'PV-doc (>= ${gcc:SoftVersion}),
gcc`'PV-locales (>= ${gcc:SoftVersion}),
libdbgdep(gcc`'GCC_SO-dbg,,>=,${libgcc:Version}),
libdbgdep(gomp`'GOMP_SO-dbg,),
@@ -730,15 +738,44 @@
')`'dnl
libdbgdep(mpx`'MPX_SO-dbg,),
libdbgdep(quadmath`'QMATH_SO-dbg,)
-Provides: c-compiler`'TS
+Provides: c-compiler`'arch_gnusuffix
ifdef(`TARGET',`Conflicts: gcc-multilib
')`'dnl
BUILT_USING`'dnl
-Description: GNU C compiler`'ifdef(`TARGET)',` (cross compiler for TARGET architecture)', `')
+Description: GNU C compiler for the arch_gnu architecture
+ This is the GNU C compiler for the arch_gnu architecture,
+ a fairly portable optimizing compiler for C.
+')`'dnl for_each_arch
+ifdef(`TARGET',`',`
+Package: gcc`'PV
+Architecture: any
+Section: devel
+Priority: optional
+Depends: gcc`'PV`'${target:suffix} (= ${gcc:Version}),ifenabled(`gccbase',` BASEDEP,')
+ ifenabled(`gccxbase',` BASEDEP,')
+ binutils (>= ${binutils:Version}),
+ ${misc:Depends}
+Recommends: ${dep:libcdev}
+Replaces: gccgo-8 (<< ${gcc:Version}), cpp`'PV (<< 7.1.1-8)
+Suggests: ${gcc:multilib}, gcc`'PV-doc (>= ${gcc:SoftVersion}),
+ gcc`'PV-locales (>= ${gcc:SoftVersion}),
+ libdbgdep(gcc`'GCC_SO-dbg,,>=,${libgcc:Version}),
+ libdbgdep(gomp`'GOMP_SO-dbg,),
+ libdbgdep(itm`'ITM_SO-dbg,),
+ libdbgdep(atomic`'ATOMIC_SO-dbg,),
+ libdbgdep(asan`'ASAN_SO-dbg,),
+ libdbgdep(lsan`'LSAN_SO-dbg,),
+ libdbgdep(tsan`'TSAN_SO-dbg,),
+ libdbgdep(ubsan`'UBSAN_SO-dbg,),
+ifenabled(`libvtv',`',`
+ libdbgdep(vtv`'VTV_SO-dbg,),
+')`'dnl
+ libdbgdep(mpx`'MPX_SO-dbg,),
+ libdbgdep(quadmath`'QMATH_SO-dbg,
Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
As addition to the feedback given on irc, - please complete the patch to cover all frontends. while these might not be necessary for your bootstrap effort, I'd like to have all frontends be covered in an uniform way. - as the first step, please don't build any -for-host and -for-build packages yet. - for testing: - the native build should succeed without disabling any packages. - I don't see much value in doing a non-dfsg build, just make sure that the new triplet packages still suggest the non-triplet doc packages. - please test a cross build, with CROSS_ARCHS set to amd64, one multilib arch, and one non-multilib arch (you can inject packaging changes into the gcc-8-cross package without generating a gcc-8-source first). - gcc-snapshot testing is not needed. at some point we might want to have the triplet prefixed binaries there as well. - Please make the testing public, e.g. https://launchpad.net/~gcc-for-host/+archive/ubuntu/ppa Any other build service is fine as well. Thanks, Matthias
Bug#666743: draft implementation for a Multi-Arch: foreign interface to gcc
Control: reopen -1
Control: reassign -1 src:gcc-8
Hi Matthias,
you requested that I send my unfinished work and here it goes. I've seen
that you have quickly applied my previous patches towards this matter
and would like to thank you. The attached patch was last tested on
8-20180402-1. I performed the following test methods thus far:
* native build (reduced using nolang=...)
* lintian
* piuparts
* looking at package contents
* manually installing and testing the resulting packages (thus
discovering #891289)
Caveats:
* The patch still contains temporary Breaks/Replaces versions. You'll
find them by searching for /0TODO/ in debian/control.m4. These need
to be updated before uploading (as we are moving files to new
packages).
* You'll go through NEW of course.
* I did not perform cross builds or cross compiler builds.
* I did not perform non-dfsg builds.
* I did not perform gcc-snapshot builds.
I had to make a number of bike coloring decision during the
implementation. If you want something different, I'm happy to change
that and will try to update the patch in a timely manner.
* "arch_gnutype_map" gets passed to control.m4 as "ARCH_GNUTYPE_MAP".
* The target suffix of the new packages is passed as a substvar called
"target:suffix".
* The make variables for these packages are called "p_${tool}n" and
"d_${tool}n".
* The new targets are called "$(binary_stamp)-$(tool)-nat".
I would like to thank James Clarke his advice during the planning stage
and for reviewing an earlier version of this patch.
I'd also be interested in learning which test build configurations are
particularly relevant to you.
I'v Cced d-cross@l.d.o to have more eyeballs on this difficult piece of
code.
Helmut
--- a/debian/control.m4
+++ b/debian/control.m4
@@ -35,6 +35,13 @@
'))
define(`TARGET_PACKAGE',`X-DH-Build-For-Type: target
')
+define(`_for_each',`ifelse(`$3',`',`',`pushdef(`$1',`$3')$2`'popdef(`$1')`'$0(`$1',`$2',shift(shift(shift($@')')
+define(`for_each_arch',`_for_each(`_arch',`dnl
+pushdef(`arch_deb',`patsubst(_arch,`=.*',`')')dnl
+pushdef(`arch_gnu',`patsubst(_arch,`.*=',`')')dnl
+pushdef(`arch_gnusuffix',`-patsubst(arch_gnu,`_',`-')')dnl
+$1`'popdef(`arch_gnusuffix')popdef(`arch_gnu')popdef(`arch_deb')',dnl
+patsubst(ARCH_GNUTYPE_MAP,`\s+',`,'))')
divert`'dnl
dnl --
@@ -700,21 +707,22 @@
')`'dnl x32dev
')`'dnl cdev
-ifenabled(`cdev',`
-Package: gcc`'PV`'TS
-Architecture: any
-ifdef(`TARGET',`Multi-Arch: foreign
-')dnl
+ifenabled(`cdev',`dnl
+for_each_arch(`
+Package: gcc`'PV`'arch_gnusuffix
+Architecture: ifdef(`TARGET',`any',arch_deb)
+Multi-Arch: foreign
Section: devel
Priority: optional
-Depends: cpp`'PV`'TS (= ${gcc:Version}),ifenabled(`gccbase',` BASEDEP,')
+Depends: cpp`'PV`'arch_gnusuffix (= ${gcc:Version}),ifenabled(`gccbase',` BASEDEP,')
ifenabled(`gccxbase',` BASEDEP,')
${dep:libcc1},
- binutils`'TS (>= ${binutils:Version}),
+ binutils`'arch_gnusuffix (>= ${binutils:Version}),
${dep:libgccdev}, ${shlibs:Depends}, ${misc:Depends}
Recommends: ${dep:libcdev}
-Replaces: gccgo-8 (<< ${gcc:Version}), cpp`'PV`'TS (<< 7.1.1-8)
-Suggests: ${gcc:multilib}, gcc`'PV-doc (>= ${gcc:SoftVersion}),
+Replaces: gccgo-8 (<< ${gcc:Version}), cpp`'PV`'arch_gnusuffix (<< 7.1.1-8), gcc`'PV (<< 999:0TODO)
+Breaks: cpp`'PV (<< 0TODO)
+Suggests: gcc`'PV-doc (>= ${gcc:SoftVersion}),
gcc`'PV-locales (>= ${gcc:SoftVersion}),
libdbgdep(gcc`'GCC_SO-dbg,,>=,${libgcc:Version}),
libdbgdep(gomp`'GOMP_SO-dbg,),
@@ -729,15 +737,44 @@
')`'dnl
libdbgdep(mpx`'MPX_SO-dbg,),
libdbgdep(quadmath`'QMATH_SO-dbg,)
-Provides: c-compiler`'TS
+Provides: c-compiler`'arch_gnusuffix
ifdef(`TARGET',`Conflicts: gcc-multilib
')`'dnl
BUILT_USING`'dnl
-Description: GNU C compiler`'ifdef(`TARGET)',` (cross compiler for TARGET architecture)', `')
+Description: GNU C compiler for the arch_gnu architecture
+ This is the GNU C compiler for the arch_gnu architecture, a fairly portable
+ optimizing compiler for C.
+')`'dnl for_each_arch
+ifdef(`TARGET',`',`
+Package: gcc`'PV
+Architecture: any
+Section: devel
+Priority: optional
+Depends: gcc`'PV`'${target:suffix} (= ${gcc:Version}),ifenabled(`gccbase',` BASEDEP,')
+ ifenabled(`gccxbase',` BASEDEP,')
+ binutils (>= ${binutils:Version}),
+ ${misc:Depends}
+Recommends: ${dep:libcdev}
+Replaces: gccgo-8 (<< ${gcc:Version}), cpp`'PV (<< 7.1.1-8)
+Suggests: ${gcc:multilib}, gcc`'PV-doc (>= ${gcc:SoftVersion}),
+ gcc`'PV-locales (>= ${gcc:SoftVersion}),
+ libdbgdep(gcc`'GCC_SO-dbg,,>=,${libgcc:Version}),
+ libdbgdep(gomp`'GOMP_SO-dbg,),
+ libdbgdep(itm`'ITM_SO-dbg,),
+ libdbgdep(atomic`'ATOMIC_SO-dbg,),
+ libdbgdep(asan`'ASAN_SO-dbg,),
+ libdbgdep(lsan`'LSAN_SO-dbg,),
+ libdbgdep(tsan`'TSAN_SO-dbg,),
+ libdbgdep(ubsan`'UBSAN_SO-dbg,),
+ifenabled(`libvtv',`',`
+ libdbgdep(vtv`'VTV_SO-dbg,),
+')`'dnl
+ libdbgdep(mpx`'MPX_SO-dbg,),
+ libdbgdep(quadmath`'QMATH_SO-dbg,)
+Provides: c-comp
