Re: Architecture: all + M-A: foreign

[Wookey]

+++ Charles Plessy [2013-01-06 18:32 +0900]:
> Le Thu, Dec 06, 2012 at 02:05:13AM -0600, Peter Samuelson a écrit :
> > 
> > In bug #695229, I noted that an Architecture: all package really should
> > be Multi-Arch: foreign.  This led to an IRC discussion between Goswin,
> > Steve L. and me in which I formulated the proposal:
> > 
> > If a package is 'Architecture: all', and all its dependencies are
> > 'Multi-Arch: foreign' (including the case where there are no
> > dependencies), this package should implicitly be treated as
> > 'Multi-Arch: foreign' as well.
> 
> Hello everybody,
> 
> have there been progresses on that question ?
> 
> In light of the last messages in this thread, which suggest that more than 250
> packages should be converted from architecture-independant to
> architecture-dependant in order to solve the problem, I wonder if it is useful
> to start adding 'Multi-Arch: foreign' to 'Architecture: all' packages.

Yes it is. 695229 means that you won't _have_ to, but fixing the
metadata in the package is always correct.

> (I ask because it was suggested for mime-support in #695357)

There are tens of these pending, upload away!

Wookey
-- 
Principal hats:  Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/2013010612.gc5...@stoneboat.aleph1.co.uk

Re: Architecture: all + M-A: foreign

[Charles Plessy]

Le Thu, Dec 06, 2012 at 02:05:13AM -0600, Peter Samuelson a écrit :
> 
> In bug #695229, I noted that an Architecture: all package really should
> be Multi-Arch: foreign.  This led to an IRC discussion between Goswin,
> Steve L. and me in which I formulated the proposal:
> 
> If a package is 'Architecture: all', and all its dependencies are
> 'Multi-Arch: foreign' (including the case where there are no
> dependencies), this package should implicitly be treated as
> 'Multi-Arch: foreign' as well.

Hello everybody,

have there been progresses on that question ?

In light of the last messages in this thread, which suggest that more than 250
packages should be converted from architecture-independant to
architecture-dependant in order to solve the problem, I wonder if it is useful
to start adding 'Multi-Arch: foreign' to 'Architecture: all' packages.

(I ask because it was suggested for mime-support in #695357)

Cheers,

-- 
Charles Plessy
Tsurumi, Kanagawa, Japan


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20130106093218.gd13...@falafel.plessy.net

Re: Architecture: all + M-A: foreign

[Steve Langasek]

On Sun, Dec 09, 2012 at 12:14:30AM +0100, Jakub Wilk wrote:
> * Steve Langasek , 2012-12-08, 14:18:
> >it might be worth considering whether we could instead solve all
> >the real instances of A->B->C/D in the archive by converting all B
> >to Arch: any in wheezy, and then just allowing the package manager
> >to treat *all* Arch: all packages as implicitly satisfying
> >foreign-arch deps in jessie.

> If a Python module depends on an arch:any Python module, then it
> must not be treated as MA:foreign. Otherwise you would get a
> dependency chain like this:

> foo:amd64 (linked to libpython2.7:amd64) -> python-bar:all -> python-baz:i386

> So with your proposal, we would have to convert tons of arch:all
> Python modules to arch:any.

Good point.  I had a look, and there are upwards of 250 such Arch: all
python-* packages in sid.  So that's certainly not what I would call a
shortcut.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Architecture: all + M-A: foreign

[Jakub Wilk]

* Steve Langasek , 2012-12-08, 14:18:
it might be worth considering whether we could instead solve all the 
real instances of A->B->C/D in the archive by converting all B to Arch: 
any in wheezy, and then just allowing the package manager to treat 
*all* Arch: all packages as implicitly satisfying foreign-arch deps in 
jessie.


If a Python module depends on an arch:any Python module, then it must 
not be treated as MA:foreign. Otherwise you would get a dependency chain 
like this:


foo:amd64 (linked to libpython2.7:amd64) -> python-bar:all -> python-baz:i386

So with your proposal, we would have to convert tons of arch:all Python 
modules to arch:any.


--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20121208231430.ga4...@jwilk.net

Re: Architecture: all + M-A: foreign

[Steve Langasek]

On Thu, Dec 06, 2012 at 03:20:00AM -0600, Peter Samuelson wrote:

> [Helmut Grohne]
> > I ask you not to use this proposal for the following reasons:

> >  * Given a package it is now much harder to see whether it is tagged M-A
> >or not. Especially you can no longer determine the tagging by simple
> >examination of package lists.

> That's fair.  Though I imagine if apt knew about this mapping, it could
> just add 'M-A: foreign' to its package cache, so you would see it with
> 'apt-cache show' or 'apt-cache dumpavail'.  Anyway, it's a concern.

Implementing this via apt's internal cache would adversely impact dpkg's
availability to correctly resolve package dependencies at install time.  If
this were to be done at all, it should be done in a way that dpkg can
continue to DTRT in the absence of a higher-level package manager - so
either it needs to be something dpkg can calculate itself, or it needs to be
in the package's own control file.

-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org


signature.asc
Description: Digital signature

Re: Architecture: all + M-A: foreign

[Steve Langasek]

On Thu, Dec 06, 2012 at 02:05:13AM -0600, Peter Samuelson wrote:

> In bug #695229, I noted that an Architecture: all package really should
> be Multi-Arch: foreign.  This led to an IRC discussion between Goswin,
> Steve L. and me in which I formulated the proposal:

> If a package is 'Architecture: all', and all its dependencies are
> 'Multi-Arch: foreign' (including the case where there are no
> dependencies), this package should implicitly be treated as
> 'Multi-Arch: foreign' as well.

> Steve asked what the impact would be, given that Multi-Arch: foreign
> only matters if you have reverse dependencies that are not
> Architecture: all.  So I wrote a crude script to figure it out.  The
> numbers depend on whether you consider only Depends + Pre-Depends, or
> if you also consider Recommends and Suggests.

>Explicit M-A:foreign | Implicit M-A:foreign
>   +-+-
>   |With |With
>   | Total |   All  non-arch:all |   All  non-arch:all
> Rule variant  |   |  rev-dep(s) |  rev-dep(s)
> --+---+-+-
> Depends + Pre-Depends | 17339 |   580   145 |  5310   994
> Dep + P-D + Rec + Sug | 17339 |   580   197 |  2969  1151

> Now, whether to consider Recommends and Suggests in the calculations, I
> don't have a strong opinion.  It does change the mix of packages.  For
> example:

> - bash-completion could be implicitly M-A:foreign, but this only really
>   "matters" if we consider Recommends + Suggests.  Almost nothing
>   Depends on it, but several arch:any packages Recommend or Suggest it.

> - aptitude-common is implicitly M-A:foreign only if we do _not_
>   consider Recommends + Suggests, because while it has no dependencies,
>   it Recommends aptitude, which is not M-A:foreign.

> - alsa-base is implicitly M-A:foreign only if we do _not_ consider
>   Recommends, because while it only Depends on packages which are
>   M-A:foreign, it Recommends alsa-utils, which is not.

In practice, I think you actually want to distinguish between the
recommends/suggests of an arch: all package, and arch: all packages that are
recommended/suggested.  The scenario that led to arch: all packages *not*
being considered inherently Multi-Arch: foreign was:

  - package A requires functionality from package C or package D
  - package C/D must be of the same arch as package A
  - package B is architecture: all and depends on one of C or D, selecting
which implementation to expose
  - package A depends on B to express its requirement

(i.e.: package python-foo Depends: python Depends: python2.7)

When either relationship is weaker than a Depends, the first of these
conditions, "A requires functionality from package C or package D", cannot
be true, so it's not actually necessary to solve it.

Thus, when considering whether a package being implicitly M-A: foreign would
*break* the reverse dependencies by violating the expectation that A and C/D
will be of the same architecture, we only need to consider the depends and
pre-depends of package B.  But when considering whether it's *useful* to
promote B to be implicitly M-A: foreign, we should look at (at least) the
reverse-recommends as well, since a foreign-architecture package which
Recommends: B will have this recommendation ignored by the package manager
as unsatisfiable if B is not M-A: foreign, but will cause B to be installed
if it is M-A: foreign.

For the concrete examples you've given, it would be safe and (at least
potentially) useful for each of bash-completion, aptitude-common, and
alsa-base to be M-A: foreign, whether that's done explicitly or implicitly.

> Anyway, these numbers indicate to me that it may be worth patching
> dpkg, apt, aptitude and other deb + multi-arch aware tools, to
> implicitly mark all those Arch:all packages as Multi-Arch:foreign, so
> that this doesn't have to be done explicitly.

> (But not during the freeze.)  Thoughts?

Given that this package manager change won't happen it time for wheezy
(which I agree with), it might be worth considering whether we could instead
solve all the real instances of A->B->C/D in the archive by converting all B
to Arch: any in wheezy, and then just allowing the package manager to treat
*all* Arch: all packages as implicitly satisfying foreign-arch deps in
jessie.  Because the standard example here, python, does actually need to be
made Arch: any instead of Arch: all to satisfy certain constraints for
cross-compiling (a change which is already implemented in Ubuntu and AIUI is
planned for experimental soon), it's possible that we could make all this
complexity and confusion go away within the space of an upgrade cycle
without having to add non-local dependency traversal to dpkg.

-- 
Steve Langasek 

Re: Architecture: all + M-A: foreign

[Guillem Jover]

Hi!

On Thu, 2012-12-06 at 02:05:13 -0600, Peter Samuelson wrote:
> In bug #695229, I noted that an Architecture: all package really should
> be Multi-Arch: foreign.  This led to an IRC discussion between Goswin,
> Steve L. and me in which I formulated the proposal:
> 
> If a package is 'Architecture: all', and all its dependencies are
> 'Multi-Arch: foreign' (including the case where there are no
> dependencies), this package should implicitly be treated as
> 'Multi-Arch: foreign' as well.

The issue with arch:all packages goes deeper than that, while going
over the “multiarchification” (starting at [0] and continuing in [1],
broken due to new month archive), I realized that considering arch:all
packages equivalent to native architecture makes this impossible (?)
(I assume this to be the same for python).

 [0]: 
 [1]: 

Consider a perl script (arch:all) that depends on another perl module
(arch:all) and two XS modules (arch:any), if those need to be
co-installable, because they might be needed by a linked program
(arch:any through libperl), then that means the XS modules cannot be
marked M-A:foreign, as they would need to be M-A:same.

Considering arch:all packages == arch:native has the problem of making
some packages not cross-gradable. But making them able to depend or be
depended by any architecture, might imply in some conditions a broken
installation, like the aforementioned XS modules case (as the “wrong”
architecture might end up satisfying the dependency), *but* this is
not a problem with that package being arch:all! it's a problem of the
implicit relationship between the perl interpreter and the XS module,
so this alone still does not solve the “multiarchification” of
perl/python, but it would go a step further in making it possible.
Other things would be needed as I pointed out in [1].

If we'd change the satisfiability on one of the directions of arch:all
packages, I think it would make most sense, at least for consistency
sake, to change both.

> Anyway, these numbers indicate to me that it may be worth patching
> dpkg, apt, aptitude and other deb + multi-arch aware tools, to
> implicitly mark all those Arch:all packages as Multi-Arch:foreign, so
> that this doesn't have to be done explicitly.

Given the above I've been pondering (in the thread [0] and [1]) that
we do need to treat them as truly architecture independent, even on
dependencies (in any direction). That's actually something we already
had in mind when discussing the spec, but decided to leave out for
now because it could pose unforseen problems initially or present
“strange” dependency chains proposed by frontends, and it's something
that could always be loosened up later on.


But the thing is, if they are really architecture indendent and as
such their inbound and outbound direct interfaces are only through
architecture independent means (execve(2), shell command invocation,
script loading, etc); then the arch:all == arch:native restriction is
completely artificial, needing to mark all of them as M-A:foreign is
just pointless, «pkg:amd64 → pkg:all → pkg:i386» type of chains should
really be fine, and the frontends should just not propose such solutions
if not asked explicitly.

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20121206102257.ga27...@gaara.hadrons.org

Re: Architecture: all + M-A: foreign

[Peter Samuelson]

[Helmut Grohne]
> I ask you not to use this proposal for the following reasons:
> 
>  * Given a package it is now much harder to see whether it is tagged M-A
>or not. Especially you can no longer determine the tagging by simple
>examination of package lists.

That's fair.  Though I imagine if apt knew about this mapping, it could
just add 'M-A: foreign' to its package cache, so you would see it with
'apt-cache show' or 'apt-cache dumpavail'.  Anyway, it's a concern.

>  * Changing one package from Arch:all to Arch:any suddenly can break
>another package. An effect that one might not expect.

Well, today, changing one package from Arch:any to Arch:all can
suddenly break another package.  (Same problem: lack of M-A:foreign.)
Perhaps you think that is unexpected as well, but it is reality.

>  * If for some reason the package is actually not M-A:foreign there is
>no way to overrule the implicit decision besides turning the package
>to Arch:any or introducing a new artificial Arch:any dependency.

The only reason that I have seen that Arch:all is not _always_ treated
as M-A:foreign, relates to recursive dependency resolution: foo:i386 ->
foo-data:all -> bar:amd64, where foo:i386 and bar:amd64 cannot properly
interact.  My proposed rule accounts for that case.  Can you think of
any _other_ reasons not to set M-A:foreign on an Arch:all package?

> As a counter proposal I would like to ask whether such an implicit
> header could be added by debhelper (at a high enough compatibility
> level) by default.

Could be done - dh could do the same analysis I'm asking apt to do.  I
believe traditionally dh does not mess with debian/control beyond what
dpkg-gencontrol does with substvars and such, but I suppose that
doesn't mean it _couldn't_.

> Maybe the problem also solves itself after extending dh-make? ;-)

Heh - dh-make doesn't have enough context to know whether M-A:foreign
makes sense. (:


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20121206092000.gm4...@p12n.org

Re: Architecture: all + M-A: foreign

[Helmut Grohne]

On Thu, Dec 06, 2012 at 02:05:13AM -0600, Peter Samuelson wrote:
> 
> In bug #695229, I noted that an Architecture: all package really should
> be Multi-Arch: foreign.  This led to an IRC discussion between Goswin,
> Steve L. and me in which I formulated the proposal:
> 
> If a package is 'Architecture: all', and all its dependencies are
> 'Multi-Arch: foreign' (including the case where there are no
> dependencies), this package should implicitly be treated as
> 'Multi-Arch: foreign' as well.

I ask you not to use this proposal for the following reasons:

 * Given a package it is now much harder to see whether it is tagged M-A
   or not. Especially you can no longer determine the tagging by simple
   examination of package lists.
 * Changing one package from Arch:all to Arch:any suddenly can break
   another package. An effect that one might not expect.
 * If for some reason the package is actually not M-A:foreign there is
   no way to overrule the implicit decision besides turning the package
   to Arch:any or introducing a new artificial Arch:any dependency.

As a counter proposal I would like to ask whether such an implicit
header could be added by debhelper (at a high enough compatibility
level) by default. The main advantages I see that we don't have to
modify dpkg, apt, aptitude and probably more tools and that my above
arguments are invalidated as well. That said I have no implementation to
showcase.

Maybe the problem also solves itself after extending dh-make? ;-)

Helmut


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20121206082844.ga18...@alf.mars

Architecture: all + M-A: foreign

[Peter Samuelson]

In bug #695229, I noted that an Architecture: all package really should
be Multi-Arch: foreign.  This led to an IRC discussion between Goswin,
Steve L. and me in which I formulated the proposal:

If a package is 'Architecture: all', and all its dependencies are
'Multi-Arch: foreign' (including the case where there are no
dependencies), this package should implicitly be treated as
'Multi-Arch: foreign' as well.

Steve asked what the impact would be, given that Multi-Arch: foreign
only matters if you have reverse dependencies that are not
Architecture: all.  So I wrote a crude script to figure it out.  The
numbers depend on whether you consider only Depends + Pre-Depends, or
if you also consider Recommends and Suggests.

   Explicit M-A:foreign | Implicit M-A:foreign
  +-+-
  |With |With
  | Total |   All  non-arch:all |   All  non-arch:all
Rule variant  |   |  rev-dep(s) |  rev-dep(s)
--+---+-+-
Depends + Pre-Depends | 17339 |   580   145 |  5310   994
Dep + P-D + Rec + Sug | 17339 |   580   197 |  2969  1151

Now, whether to consider Recommends and Suggests in the calculations, I
don't have a strong opinion.  It does change the mix of packages.  For
example:

- bash-completion could be implicitly M-A:foreign, but this only really
  "matters" if we consider Recommends + Suggests.  Almost nothing
  Depends on it, but several arch:any packages Recommend or Suggest it.

- aptitude-common is implicitly M-A:foreign only if we do _not_
  consider Recommends + Suggests, because while it has no dependencies,
  it Recommends aptitude, which is not M-A:foreign.

- alsa-base is implicitly M-A:foreign only if we do _not_ consider
  Recommends, because while it only Depends on packages which are
  M-A:foreign, it Recommends alsa-utils, which is not.

Anyway, these numbers indicate to me that it may be worth patching
dpkg, apt, aptitude and other deb + multi-arch aware tools, to
implicitly mark all those Arch:all packages as Multi-Arch:foreign, so
that this doesn't have to be done explicitly.

(But not during the freeze.)  Thoughts?


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20121206080512.gl4...@p12n.org