Bug#952701: mark golang-golang-x-text-dev Multi-Arch: foreign

2020-02-27 Thread Helmut Grohne
Hi,

On Fri, Feb 28, 2020 at 04:39:54AM +0800, Shengjing Zhu wrote:
> I think if we want to cross-build all Go packages.
> All the golang-*-dev packages which are arch:all can be marked as MA: foreign.

Unfortunately, no. That'd be harmful.

Once your Architecture: all golang-*-dev package has a runtime
dependency on another golang-*-dev package that happens to be
Architecture: any (even indirectly), the marking becomes wrong. So
simply marking all Architecture: all packages would cause more breakage.
Likewise, presence of maintainer scripts can (but doesn't have to) break
correctness of Multi-Arch: foreign.

> Currently there're 1k+ packages. Maybe some batch scripts to update them all.

Yes, automating this in some way at some point seems reasonable. I'm
unsure whether the time is now.

How would one automate this? I have little clue about the go repository
structure, so let me talk about the "what to change" part instead.

The multiarch hinter[1] analyzes the archive for opportunites to add
Multi-Arch annotations. The data is fed into tracker.d.o, but you can
also download the feed[2] directly. What you get is a yaml document
containing the hints. Excuse my use of Python:

for h in 
yaml.safe_load(requests.get("https://dedup.debian.net/static/multiarch-hints.yaml;).content)["hints"]:
if h["binary"].startswith("golang-") and h["binary"].endswith("-dev") and 
h["link"].endswith("#ma-foreign"):
print(h["binary"])

That gives you around 450 packages. Quite a bit less than 1k+. I'm not
sure that we need all of them marked either. I've therefore started
looking into where we'd benefit most from such markings and I guess it
really is few packages. Note that once you starting marking packages
Multi-Arch: foreign, the hinter will generate more hints for packages
depending on the marked packages.

The Debian Janitor[3] is also consuming these hints and can create salsa
pull requests to fix these (but it presently does not do that for
ma-foreign hints). We could ask Jelmer to opt golang into the fixes if
you like.

There was also an idea on having debhelper automatically compute the
Multi-Arch line for simple cases (and golang-golang-x-text-dev would
qualify here).

It seems that my report did exactly what I intended it to do: Rather
than having you blindly apply my patch, you've thought of the wider
consequences and extrapolated from the example. Thank you.

Helmut

[1] https://wiki.debian.org/MultiArch/Hints
[2] https://dedup.debian.net/static/multiarch-hints.yaml.xz (optionally
dropping the .xz)
[3] https://janitor.debian.net/



Bug#952701: mark golang-golang-x-text-dev Multi-Arch: foreign

2020-02-27 Thread Shengjing Zhu
On Fri, Feb 28, 2020 at 4:27 AM Helmut Grohne  wrote:
>
> Package: golang-golang-x-text-dev
> Version: 0.3.2-3
> Tags: patch
> User: debian-cr...@lists.debian.org
> Usertags: cross-satisfiability
> Control: affects -1 + src:abci src:acbuild src:aptly
>
> Lots of packages tranistively use golang-golang-x-text-dev in
> Build-Depends. I've given three examples above. Unfortunately, that
> means that all of these packages cannot satisfy cross Build-Depends,
> because Architecture: all packages cannot do that unless marked
> Multi-Arch: foreign or annotated :native. The foreign marking is
> reasonable here, because golang-golang-x-text-dev is a dependency-less
> and maintainer-script-less package containing only go source and
> compiled go modules (which still are architecture-independent).
> Accordingly the multiarch hinter flags it for Multi-Arch: foreign.
> Likely this is the first of a stream of bug reports asking for making
> go libraries Multi-Arch: foreign. Note that once there is
> architecture-dependent code involved (even as a dependency) the marking
> is not the correct solution. We can only use it in simple situations
> like golang-golang-x-text-dev. Please consider applying the attached
> patch.

I think if we want to cross-build all Go packages.
All the golang-*-dev packages which are arch:all can be marked as MA: foreign.

Currently there're 1k+ packages. Maybe some batch scripts to update them all.

-- 
Shengjing Zhu



Bug#952701: mark golang-golang-x-text-dev Multi-Arch: foreign

2020-02-27 Thread Helmut Grohne
Package: golang-golang-x-text-dev
Version: 0.3.2-3
Tags: patch
User: debian-cr...@lists.debian.org
Usertags: cross-satisfiability
Control: affects -1 + src:abci src:acbuild src:aptly

Lots of packages tranistively use golang-golang-x-text-dev in
Build-Depends. I've given three examples above. Unfortunately, that
means that all of these packages cannot satisfy cross Build-Depends,
because Architecture: all packages cannot do that unless marked
Multi-Arch: foreign or annotated :native. The foreign marking is
reasonable here, because golang-golang-x-text-dev is a dependency-less
and maintainer-script-less package containing only go source and
compiled go modules (which still are architecture-independent).
Accordingly the multiarch hinter flags it for Multi-Arch: foreign.
Likely this is the first of a stream of bug reports asking for making
go libraries Multi-Arch: foreign. Note that once there is
architecture-dependent code involved (even as a dependency) the marking
is not the correct solution. We can only use it in simple situations
like golang-golang-x-text-dev. Please consider applying the attached
patch.

Helmut
diff --minimal -Nru golang-golang-x-text-0.3.2/debian/changelog 
golang-golang-x-text-0.3.2/debian/changelog
--- golang-golang-x-text-0.3.2/debian/changelog 2019-12-28 17:00:29.0 
+0100
+++ golang-golang-x-text-0.3.2/debian/changelog 2020-02-27 21:07:10.0 
+0100
@@ -1,3 +1,10 @@
+golang-golang-x-text (0.3.2-3.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * Mark golang-golang-x-text-dev Multi-Arch: foreign. (Closes: #-1)
+
+ -- Helmut Grohne   Thu, 27 Feb 2020 21:07:10 +0100
+
 golang-golang-x-text (0.3.2-3) unstable; urgency=medium
 
   [ Utkarsh Gupta ]
diff --minimal -Nru golang-golang-x-text-0.3.2/debian/control 
golang-golang-x-text-0.3.2/debian/control
--- golang-golang-x-text-0.3.2/debian/control   2019-12-28 16:54:40.0 
+0100
+++ golang-golang-x-text-0.3.2/debian/control   2020-02-27 21:07:07.0 
+0100
@@ -17,6 +17,7 @@
 
 Package: golang-golang-x-text-dev
 Architecture: all
+Multi-Arch: foreign
 Depends: ${shlibs:Depends},
  ${misc:Depends}
 Description: Supplementary Go text-related libraries