Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-27 Thread Dmitry Shachnev
Control: reassign 818115 src:sphinx 1.4.9-2
Control: merge 818115 -1
Control: close -1

On Sat, Jan 26, 2019 at 05:50:13PM +0100, Helmut Grohne wrote:
> Hi Dmitry,
>
> On Sat, Jan 26, 2019 at 12:55:28AM +0300, Dmitry Shachnev wrote:
> > So I cannot promise that I will take care of half of those packages.
> > I will take care only of some of them, those that I am familiar with or
> > those that are easy to fix. Fortunately cmake matches these patterns.
>
> That was wishful thinking indeed. I don't expect you to save the world.
> ;)
>
> > P.S. Maybe I should merge this bug with #818115? They both discuss cross-
> > building issues.
>
> Given that dpkg now allows python3-sphinx:native, I'm inclined to not
> only merge them but also close them.

Thanks, doing so then.

--
Dmitry Shachnev


signature.asc
Description: PGP signature


Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-26 Thread Helmut Grohne
On Sat, Jan 26, 2019 at 11:40:55AM +0100, kaliko wrote:
> Indeed it did fix the issue :)
> I set "Build-Depends-Indep: python3-sphinx:native" and built with pbuilder on 
> testing.

Do note that :native does not make any sense in Build-Depends-Indep.
Whenever you move your dependency to Build-*-Indep, it becomes
irrelevant to cross building and thus :native becomes irrelevant.

> But I'm having other problems later on, not related to sphinx though (some of 
> them
> probably x-build related).

I invite you to share your cross problems with
debian-cr...@lists.debian.org or #debian-bootstrap on irc.oftc.net.

> I don't have enough time to go any further right now, but anyway the issue 
> regarding
> sphinx is actually fixed.
> 
> A final question, would you recommend sbuild rather than pbuilder?
> Especially when trying to cross-build packages ?

I recommend using one of pbuilder or sbuild, because both reportedly
work well. At least in unstable (and likely buster) they should be on
parity wrt. features.

Helmut



Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-26 Thread Helmut Grohne
Hi Dmitry,

On Sat, Jan 26, 2019 at 12:55:28AM +0300, Dmitry Shachnev wrote:
> So I cannot promise that I will take care of half of those packages.
> I will take care only of some of them, those that I am familiar with or
> those that are easy to fix. Fortunately cmake matches these patterns.

That was wishful thinking indeed. I don't expect you to save the world.
;)

> P.S. Maybe I should merge this bug with #818115? They both discuss cross-
> building issues.

Given that dpkg now allows python3-sphinx:native, I'm inclined to not
only merge them but also close them.

Helmut



Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-26 Thread kaliko
Dmitry, Helmut

First, thanks for this insightful thread, even though it goes beyond my 
expertise I
learn about cross-building then :)

On 25/01/2019 22:55, Dmitry Shachnev wrote:
> On Fri, Jan 25, 2019 at 05:41:56PM +0100, Helmut Grohne wrote:
>> On Fri, Jan 25, 2019 at 11:14:23AM +0300, Dmitry Shachnev wrote:
>>> Does this mean that packages that are not using autodoc (like ncmpc) can
>>> already build-depend on python3-sphinx:native to become cross-buildable?
>>
>> Yes, that is my understanding. My plan was postponing such patches to
>> simplify stretch-backports. I'm not opposed to them in general, I just
>> didn't want to cause unnecessary work.
> 
> @Kaliko: please test if it works for you.

Indeed it did fix the issue :)
I set "Build-Depends-Indep: python3-sphinx:native" and built with pbuilder on 
testing.
But I'm having other problems later on, not related to sphinx though (some of 
them
probably x-build related).

I don't have enough time to go any further right now, but anyway the issue 
regarding
sphinx is actually fixed.

A final question, would you recommend sbuild rather than pbuilder?
Especially when trying to cross-build packages ?

Thanks Dmitry and Helmut for your help :)
Cheers
k



signature.asc
Description: OpenPGP digital signature


Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-25 Thread Dmitry Shachnev
On Fri, Jan 25, 2019 at 05:41:56PM +0100, Helmut Grohne wrote:
> Hi Dmitry,
>
> On Fri, Jan 25, 2019 at 11:14:23AM +0300, Dmitry Shachnev wrote:
> > Does this mean that packages that are not using autodoc (like ncmpc) can
> > already build-depend on python3-sphinx:native to become cross-buildable?
>
> Yes, that is my understanding. My plan was postponing such patches to
> simplify stretch-backports. I'm not opposed to them in general, I just
> didn't want to cause unnecessary work.

@Kaliko: please test if it works for you.

> > Does cmake sound like a good start? It already has docs in a separate
> > package, just needs the B-D adjusted.
>
> Yes, cmake is on the list above precisely due to sphinx. It also is a
> key package, so it fits my description very well.
>
> I certainly appreciate you putting effort in resolving these issues. I
> also appreciate X-Debbugs-Cc (to record the bug numbers).
>
> Debian unstable currently has almost 3 source packages. Almost 14000
> of them build architecture-dependent binary packages. (The others are
> irrelevant to cross building). Around 7500 have cross-satisfiable
> dependencies. I didn't work that much on satisfiability problems lately,
> so this is only around 55%. Having you work here would be a boon. Of the
> 3900 packages that I tried building, 2900 do cross build. This is a much
> higher ratio of almost 75%. Getting those 55% up would be awesome. If
> you fix half of the sphinx users, we're 1% closer to perfection. :)

Sorry, I have quite little time for Debian at all, and from it very little
time I can devote to Sphinx (as I also have Qt and many other packages).
And most of that time comes into test-rebuilding reverse dependencies when
upstream releases a new version that breaks everything...

So I cannot promise that I will take care of half of those packages.
I will take care only of some of them, those that I am familiar with or
those that are easy to fix. Fortunately cmake matches these patterns.

P.S. Maybe I should merge this bug with #818115? They both discuss cross-
building issues.

--
Dmitry Shachnev


signature.asc
Description: PGP signature


Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-25 Thread Helmut Grohne
Hi Dmitry,

On Fri, Jan 25, 2019 at 11:14:23AM +0300, Dmitry Shachnev wrote:
> Does this mean that packages that are not using autodoc (like ncmpc) can
> already build-depend on python3-sphinx:native to become cross-buildable?

Yes, that is my understanding. My plan was postponing such patches to
simplify stretch-backports. I'm not opposed to them in general, I just
didn't want to cause unnecessary work.

> Does cmake sound like a good start? It already has docs in a separate
> package, just needs the B-D adjusted.

Yes, cmake is on the list above precisely due to sphinx. It also is a
key package, so it fits my description very well.

I certainly appreciate you putting effort in resolving these issues. I
also appreciate X-Debbugs-Cc (to record the bug numbers).

Debian unstable currently has almost 3 source packages. Almost 14000
of them build architecture-dependent binary packages. (The others are
irrelevant to cross building). Around 7500 have cross-satisfiable
dependencies. I didn't work that much on satisfiability problems lately,
so this is only around 55%. Having you work here would be a boon. Of the
3900 packages that I tried building, 2900 do cross build. This is a much
higher ratio of almost 75%. Getting those 55% up would be awesome. If
you fix half of the sphinx users, we're 1% closer to perfection. :)

Helmut



Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-25 Thread Dmitry Shachnev
On Sun, Jan 20, 2019 at 09:48:25PM +0100, Helmut Grohne wrote:
> Upon closer inspection, I'm growing doubts. The autodoc extension is not
> in some python3-sphinxcontrib.something package but in python3-sphinx
> proper. Therefore you want a way of using a particular architecture's
> sphinx anyway. Marking it M-A:foreign is likely going to cause trouble
> later on. So if you split out this sphinx package, you'd likely have
> python3-sphinx Depends: sphinx at least initially. Without that
> dependency, lots of packages will FTBFS. Then sphinx would of course
> Depends: python3-sphinx posing a circular dependency until we fix all
> those FTBFS. Then any package that refers to the scripts and uses
> autodoc must "Build-Depends: sphinx, python3-sphinx". This is confusing
> at the very least.
>
> Certainly not something we want to do before buster.

I agree, that would be confusing for people who are not experts in
cross-building and multiarch stuff.

> > Is it safe to add Multi-Arch: allowed to python-sphinx and python3-sphinx
> > right now (and Multi-Arch: foreign to sphinx-common)?
> 
> I overlooked one major issue with Multi-Arch: allowed. The value is not
> permitted on Architecture: all packages. So if you trying using it, you
> end up switching python3-sphinx to Architecture: any.
>
> Beyond that, the marking is certainly not doing any immediate harm.
> Until some other package uses "python3-sphinx:any" literally nothing
> changes. As soon as you get such a dependency however, reverting becomes
> difficult. Removing "Multi-Arch: allowed" will make all
> "python3-sphinx:any" dependencies immediately unsatisfiable (even
> natively).

Ack.

> For Build-Depends, you can use "python3-sphinx:native" now. Until very
> recently that wasn't possible as dpkg-checkbuilddeps would reject
> :native annotations on Architecture: all packages, but dpkg has adjusted
> behaviour to what apt and dose do now. Therefore such dependencies harm
> stretch-backports. Still it might be the best thing we have at the
> moment.

Oh, that's a very good piece of news!

Does this mean that packages that are not using autodoc (like ncmpc) can
already build-depend on python3-sphinx:native to become cross-buildable?

> Well, we do have some data on which packages are affected. Carefully
> open this huge html file: https://bootstrap.debian.net/cross_all.html.
> Then search for "python3-sphinx" and you get an overview of which
> packages are affected. It currently is the #10 cross unsatisfiable cause
> with 146 affected packages. Searching again will reveal the actual
> source packages. Note that the list already excludes Build-Depends-Indep
> as well as source packages that only build architecture-independent
> binary packages. Carefully pick valuable targets here. For instance
> large documentation trees, long build time, high popcon.

Does cmake sound like a good start? It already has docs in a separate
package, just needs the B-D adjusted.

> There is a reason why I have - for a long time - not actively worked on
> cross/sphinx: There is no obvious solution. I wish I could give more
> encouraging answers.

Ack.

--
Dmitry Shachnev


signature.asc
Description: PGP signature


Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-20 Thread Helmut Grohne
Hi Dmitry,

On Sun, Jan 20, 2019 at 06:21:27PM +0300, Dmitry Shachnev wrote:
> Now there is no binary package named “sphinx”. The executable files are
> provided in python-sphinx and python3-sphinx binary packages, and are
> managed by alternatives system (Sphinx 2.0 will drop Python 2 support, so
> only python3-sphinx will remain). Are you suggesting to split these files
> into a separate binary package?

Upon closer inspection, I'm growing doubts. The autodoc extension is not
in some python3-sphinxcontrib.something package but in python3-sphinx
proper. Therefore you want a way of using a particular architecture's
sphinx anyway. Marking it M-A:foreign is likely going to cause trouble
later on. So if you split out this sphinx package, you'd likely have
python3-sphinx Depends: sphinx at least initially. Without that
dependency, lots of packages will FTBFS. Then sphinx would of course
Depends: python3-sphinx posing a circular dependency until we fix all
those FTBFS. Then any package that refers to the scripts and uses
autodoc must "Build-Depends: sphinx, python3-sphinx". This is confusing
at the very least.

Certainly not something we want to do before buster.

> > A "weaker" alternative already employed by some other python extensions
> > is "Multi-Arch: allowed". It's essentially the same thing just renaming
> > "sphinx" to "python3-sphinx:any". It'll look more ugly in dependencies,
> > but it crucially is the same thing.
> 
> Is it safe to add Multi-Arch: allowed to python-sphinx and python3-sphinx
> right now (and Multi-Arch: foreign to sphinx-common)?

I overlooked one major issue with Multi-Arch: allowed. The value is not
permitted on Architecture: all packages. So if you trying using it, you
end up switching python3-sphinx to Architecture: any.

Beyond that, the marking is certainly not doing any immediate harm.
Until some other package uses "python3-sphinx:any" literally nothing
changes. As soon as you get such a dependency however, reverting becomes
difficult. Removing "Multi-Arch: allowed" will make all
"python3-sphinx:any" dependencies immediately unsatisfiable (even
natively).

For Build-Depends, you can use "python3-sphinx:native" now. Until very
recently that wasn't possible as dpkg-checkbuilddeps would reject
:native annotations on Architecture: all packages, but dpkg has adjusted
behaviour to what apt and dose do now. Therefore such dependencies harm
stretch-backports. Still it might be the best thing we have at the
moment.

> If you need help with this (e.g. filing bugs/patches to split -doc packages),
> please let me know.

Well, we do have some data on which packages are affected. Carefully
open this huge html file: https://bootstrap.debian.net/cross_all.html.
Then search for "python3-sphinx" and you get an overview of which
packages are affected. It currently is the #10 cross unsatisfiable cause
with 146 affected packages. Searching again will reveal the actual
source packages. Note that the list already excludes Build-Depends-Indep
as well as source packages that only build architecture-independent
binary packages. Carefully pick valuable targets here. For instance
large documentation trees, long build time, high popcon.

There is a reason why I have - for a long time - not actively worked on
cross/sphinx: There is no obvious solution. I wish I could give more
encouraging answers.

Helmut



Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-20 Thread Dmitry Shachnev
Hi Helmut!

On Sun, Jan 20, 2019 at 10:12:09AM +0100, Helmut Grohne wrote:
> Hi Dmitry,
>
> Thank you for contacting me about this issue. It is a very relevant one
> that I have neglected far too long, because I felt it was too difficult.
> That shouldn't stop us from working on it however and I'm really happy
> to help solve it. Beware that disruptive changes may be ahead though.

Thanks for responding quickly!

> > 2) After doing that, you can no longer use dh --with sphinxdoc, as dh will
> > not find sphinxdoc.pm. So you will need something like this instead:
>
> There actually is a way to use the addon.
>
> DH_ADDONS=
> %:
>   dh $@ $(DH_ADDONS)
> build binary %-indep: DH_ADDONS+=--with=sphinxdoc

The only thing sphinxdoc.pm does is calling dh_sphinxdoc after dh_installdocs,
so this is equivalent to my approach.

> > 3) Also maybe you will need to pass --binary-indep to pbuilder, as it does
> > not automatically add that option for cross builds.
>
> Since very recently, sbuild automatically adds --no-arch-all for cross
> builds. Not sure about the situation of pbuilder.

Ack.

> > > So maybe we can say that architectures don't matter for sphinx by
> > > marking it Multi-Arch: foreign? Unfortunately, no. python-sphinx
> > > transitively depends on python-markupsafe (via python-jinja2) and thus
> > > exposes it. Since python-markupsafe is an architecture dependent Python
> > > extension, python-sphinx exposes the architecture. This is the gist of
> > > the multiarch interpreter problem.
>
> Please consider this just as one example. I looked through the
> dependency stack and this was the first example found. There is likely
> more underneath.

I think the rest of Sphinx dependency stack is pure Python (not counting
some compiled parts of Python standard library).

> I'm quite sure that by now marking python3-sphinx M-A:foreign is wrong
> for precisely the reason you give: You don't know ahead of time whether
> you'll have to import architecture-dependent code.

Ack.

> These are two questions:
>
>  * If a package is M-A:foreign, do all dependencies have to be
>M-A:foreign?
>
>No! I often use dependencies not being M-A:foreign as an argument,
>but that is only an approximation. What really counts is the
>"interface" of a package. Unfortunately, "interface" is pretty
>loosely defined in Debian so a conservative approach is to treat all
>dependencies as part of the "interface". So having all dependencies
>M-A:foreign is part of a sufficient, but not necessary argument.
>
>  * Should there be a M-A:foreign sphinx package?
>
>Likely, that'd be a good idea. It must be done with care however. It
>must be made abundantly clear that combining sphinx (as opposed to
>python3-sphinx) with any sphinx extension is a serious bug. Using a
>sphinx with a sphinx extension is akin to a "missing dependency" and
>we usually call that an rc bug. (The missing dependency is
>python3-sphinx of course.) As a result, this approach is only useful
>for "simple" packages. It can be a partial solution and most cases
>can likely use either plain "sphinx" or split a -doc package.

Most Sphinx extensions are pure Python too. However, as I said, the autodoc
extension is pure Python itself but may import code from the source package
being built, which may be not pure Python.

Now there is no binary package named “sphinx”. The executable files are
provided in python-sphinx and python3-sphinx binary packages, and are
managed by alternatives system (Sphinx 2.0 will drop Python 2 support, so
only python3-sphinx will remain). Are you suggesting to split these files
into a separate binary package?

> A "weaker" alternative already employed by some other python extensions
> is "Multi-Arch: allowed". It's essentially the same thing just renaming
> "sphinx" to "python3-sphinx:any". It'll look more ugly in dependencies,
> but it crucially is the same thing.

Is it safe to add Multi-Arch: allowed to python-sphinx and python3-sphinx
right now (and Multi-Arch: foreign to sphinx-common)?

> The heart of the problem is the multiarch interpreter problem though.
> What you want to express here is that a number of dependencies must have
> a common architecture, but can differ from other packages. In a sense,
> we'd want group dependencies together and have the user select the
> architecture for each group. We discussed this extensively at least in
> Vaumarcus and concluded that the problem is too hard. Whenever you need
> such a thing, you have to create a "dependency package" depending on all
> packages in the group and making that dependency package Multi-Arch:
> foreign. Now creating such dependency packages for every subset of
> sphinx extensions is not gonna work obviously. I don't think this is
> entirely solvable. Therefore I propose concentrating on practical,
> individual, and relevant cases. Your suggestion to split -doc packages
> is a very good one in that respect.

If you need help

Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-20 Thread Helmut Grohne
Hi Dmitry,

Thank you for contacting me about this issue. It is a very relevant one
that I have neglected far too long, because I felt it was too difficult.
That shouldn't stop us from working on it however and I'm really happy
to help solve it. Beware that disruptive changes may be ahead though.

On Sun, Jan 20, 2019 at 12:45:36AM +0300, Dmitry Shachnev wrote:
> On Fri, Dec 07, 2018 at 11:42:09AM +0100, kaliko wrote:
> > here is a follow up of a discussion started on irc:debian-devel with
> > mitya57.
> >
> > I'm trying to cross build an "Architecture: any" package [0]
> > (commit:52326679) using the following:
> >
> > sudo pbuilder create --host-arch armhf
> > sudo pbuilder build --host-arch armhf ncmpc_0.33-1.dsc
> >
> > building failed on:
> >
> > "builddeps:/build/ncmpc_0.33-1.dsc:armhf : Depends: python3-sphinx:armhf but
> > it is not installable"
> 
> In other packages this is usually being fixed by splitting the documentation
> into a separate package that is Architecture: all.

This is actually the solution I have preferred thus far. For small
packages it is a poor trade-off though. At some point we'll be hitting
packages where this is undesirable.

> Then Sphinx is not needed when cross-building at all — there is no sense in
> cross-building Architecture: all packages, you can just build them natively.
> 
> To skip installing Sphinx on cross builds, you need to:
> 
> 1) Move python3-sphinx build-dependency to Build-Depends-Indep.
> 
> 2) After doing that, you can no longer use dh --with sphinxdoc, as dh will
> not find sphinxdoc.pm. So you will need something like this instead:

There actually is a way to use the addon.

DH_ADDONS=
%:
dh $@ $(DH_ADDONS)
build binary %-indep: DH_ADDONS+=--with=sphinxdoc

> 3) Also maybe you will need to pass --binary-indep to pbuilder, as it does
> not automatically add that option for cross builds.

Since very recently, sbuild automatically adds --no-arch-all for cross
builds. Not sure about the situation of pbuilder.

> This was a solution that does not need changing or using Sphinx at all.

Thank you for not stopping here.

> Now let’s talk about Sphinx. There is bug #818115 which discusses how to
> make it possible to install Sphinx in cross-build environments.

Yes, please.

> Last discussion on that bug was in 2016, so I checked whether some
> things described there are still actual. Found this thing:
> 
> > So maybe we can say that architectures don't matter for sphinx by
> > marking it Multi-Arch: foreign? Unfortunately, no. python-sphinx
> > transitively depends on python-markupsafe (via python-jinja2) and thus
> > exposes it. Since python-markupsafe is an architecture dependent Python
> > extension, python-sphinx exposes the architecture. This is the gist of
> > the multiarch interpreter problem.

Please consider this just as one example. I looked through the
dependency stack and this was the first example found. There is likely
more underneath.

> Maybe this means that python3-sphinx can be marked Multi-Arch: foreign.
> 
> However, I am not sure this is a safe thing to do. One potential issue may be
> that Sphinx autodoc extension [1] imports arbitrary Python code, and that
> code may be a compiled (for the host architecture) extension that will fail to
> import with the build architecture interpreter.

I'm quite sure that by now marking python3-sphinx M-A:foreign is wrong
for precisely the reason you give: You don't know ahead of time whether
you'll have to import architecture-dependent code.

> Also, I wonder if just making src:sphinx binary packages Multi-Arch: foreign
> will be enough, or we should also mark all its dependencies Multi-Arch:
> foreign too.

These are two questions:

 * If a package is M-A:foreign, do all dependencies have to be
   M-A:foreign?

   No! I often use dependencies not being M-A:foreign as an argument,
   but that is only an approximation. What really counts is the
   "interface" of a package. Unfortunately, "interface" is pretty
   loosely defined in Debian so a conservative approach is to treat all
   dependencies as part of the "interface". So having all dependencies
   M-A:foreign is part of a sufficient, but not necessary argument.

 * Should there be a M-A:foreign sphinx package?

   Likely, that'd be a good idea. It must be done with care however. It
   must be made abundantly clear that combining sphinx (as opposed to
   python3-sphinx) with any sphinx extension is a serious bug. Using a
   sphinx with a sphinx extension is akin to a "missing dependency" and
   we usually call that an rc bug. (The missing dependency is
   python3-sphinx of course.) As a result, this approach is only useful
   for "simple" packages. It can be a partial solution and most cases
   can likely use either plain "sphinx" or split a -doc package.

A "weaker" alternative already employed by some other python extensions
is "Multi-Arch: allowed". It's essentially the same thing just renaming
"sphinx" to "python3-sphinx:any". I

Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2019-01-19 Thread Dmitry Shachnev
Hi Kaliko!

Sorry that it took so long for me to reply. I was putting most of my Sphinx
time into preparing the Sphinx 1.8 transition for sid.

On Fri, Dec 07, 2018 at 11:42:09AM +0100, kaliko wrote:
> here is a follow up of a discussion started on irc:debian-devel with
> mitya57.
>
> I'm trying to cross build an "Architecture: any" package [0]
> (commit:52326679) using the following:
>
> sudo pbuilder create --host-arch armhf
> sudo pbuilder build --host-arch armhf ncmpc_0.33-1.dsc
>
> building failed on:
>
> "builddeps:/build/ncmpc_0.33-1.dsc:armhf : Depends: python3-sphinx:armhf but
> it is not installable"

In other packages this is usually being fixed by splitting the documentation
into a separate package that is Architecture: all.

Then Sphinx is not needed when cross-building at all — there is no sense in
cross-building Architecture: all packages, you can just build them natively.

To skip installing Sphinx on cross builds, you need to:

1) Move python3-sphinx build-dependency to Build-Depends-Indep.

2) After doing that, you can no longer use dh --with sphinxdoc, as dh will
not find sphinxdoc.pm. So you will need something like this instead:

override_dh_installdocs-indep:
dh_installdocs -i -A AUTHORS
dh_sphinxdoc

3) Also maybe you will need to pass --binary-indep to pbuilder, as it does
not automatically add that option for cross builds.


This was a solution that does not need changing or using Sphinx at all.
Now let’s talk about Sphinx. There is bug #818115 which discusses how to
make it possible to install Sphinx in cross-build environments.

Last discussion on that bug was in 2016, so I checked whether some
things described there are still actual. Found this thing:

> So maybe we can say that architectures don't matter for sphinx by
> marking it Multi-Arch: foreign? Unfortunately, no. python-sphinx
> transitively depends on python-markupsafe (via python-jinja2) and thus
> exposes it. Since python-markupsafe is an architecture dependent Python
> extension, python-sphinx exposes the architecture. This is the gist of
> the multiarch interpreter problem.

I looked at markupsafe code, and found out that the architecture-specific
part of it is optional:

https://github.com/pallets/markupsafe/blob/master/src/markupsafe/__init__.py#L320

So even if it is installed for a wrong architecture, it would still be
importable.

Maybe this means that python3-sphinx can be marked Multi-Arch: foreign.

However, I am not sure this is a safe thing to do. One potential issue may be
that Sphinx autodoc extension [1] imports arbitrary Python code, and that
code may be a compiled (for the host architecture) extension that will fail to
import with the build architecture interpreter.

Also, I wonder if just making src:sphinx binary packages Multi-Arch: foreign
will be enough, or we should also mark all its dependencies Multi-Arch:
foreign too.

I am Cc’ing Helmut Grohne who filed #818115 and who is a cross-building
expert. Dear Helmut, can you give me your opinion on this? Is my analysis
correct?

[1]: https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html

--
Dmitry Shachnev


signature.asc
Description: PGP signature


Bug#915856: sphinx: Failed cross building with Build-Depends on python3-sphinx

2018-12-07 Thread kaliko
Source: sphinx
Version: 1.4.9-2
Severity: normal

here is a follow up of a discussion started on irc:debian-devel with mitya57.

I'm trying to cross build an "Architecture: any" package [0] (commit:52326679) 
using the following:

sudo pbuilder create --host-arch armhf
sudo pbuilder build --host-arch armhf ncmpc_0.33-1.dsc

building failed on:

"builddeps:/build/ncmpc_0.33-1.dsc:armhf : Depends: python3-sphinx:armhf but it 
is not installable"

I'm using pbuilder-satisfydepends-apt as suggested when cross building [1].
Here is the only line I have in pbuilderrc:

PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-apt

I also tried moving python3-sphinx to Build-Depends-Indep, same error.

[0] https://salsa.debian.org/kaliko-guest/ncmpc/
[1] https://wiki.debian.org/CrossCompiling#Building_with_pbuilder

Cheers

-- System Information:
Debian Release: 9.6
  APT prefers stable
  APT policy: (990, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.18.0-0.bpo.1-amd64 (SMP w/8 CPU cores)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), 
LANGUAGE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
I: pbuilder: network access will be disabled during build
I: Current time: Fri Dec  7 11:14:30 CET 2018
I: pbuilder-time-stamp: 1544177670
I: Building the build Environment
I: extracting base tarball [/var/cache/pbuilder/base.tgz]
I: copying local configuration
I: mounting /proc filesystem
I: mounting /sys filesystem
I: creating /{dev,run}/shm
I: mounting /dev/pts filesystem
I: redirecting /dev/ptmx to /dev/pts/ptmx
I: policy-rc.d already exists
I: Doing a cross-architecture build
I: Build architecture: amd64
I: Host architecture: armhf
I: Setting up the environment for a cross build...
Hit:1 http://cdn-fastly.deb.debian.org/debian sid InRelease
Get:2 http://cdn-fastly.deb.debian.org/debian sid/main armhf Packages [8008 kB]
Fetched 8008 kB in 7s (1103 kB/s)
Reading package lists...
I: Obtaining the cached apt archive contents
I: Copying source file
I: copying [ncmpc_0.33-1.dsc]
I: copying [./ncmpc_0.33.orig.tar.xz]
I: copying [./ncmpc_0.33-1.debian.tar.xz]
I: Extracting source
dpkg-source: warning: extracting unsigned source package (ncmpc_0.33-1.dsc)
dpkg-source: info: extracting ncmpc in ncmpc-0.33
dpkg-source: info: unpacking ncmpc_0.33.orig.tar.xz
dpkg-source: info: unpacking ncmpc_0.33-1.debian.tar.xz
I: using fakeroot in build.
I: Installing the build-deps
I: -> Attempting to satisfy build-dependencies
Note, using file '/build/ncmpc_0.33-1.dsc' to get the build dependencies
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 builddeps:/build/ncmpc_0.33-1.dsc:armhf : Depends: python3-sphinx:armhf but it 
is not installable
E: Unable to correct problems, you have held broken packages.
E: pbuilder-satisfydepends failed.
I: Copying back the cached apt archive contents
I: unmounting dev/ptmx filesystem
I: unmounting dev/pts filesystem
I: unmounting dev/shm filesystem
I: unmounting proc filesystem
I: unmounting sys filesystem
I: cleaning the build env 
I: removing directory /var/cache/pbuilder/build/17140 and its subdirectories