Bug#838238: libclang1-3.6 shouldn't be Multi-Arch: same

2016-09-18 Thread Joe Crayne
Package: libclang1-3.6
Version: 1:3.6.2-3

This package installs the following problematic symlink:

/usr/lib/llvm-3.6/lib/libclang-3.6.so.1 ->
../../x86_64-linux-gnu/libclang-3.6.so.1

This is an architecture-specific file installed to an
architecture-independent path.  Either the installation paths should be
changed, or the Multi-Arch: same header should be replaced with Multi-Arch:
no.

Here is the result of trying to co-install i386 with amd64 variants of the
package:

# apt-get install libclang1-3.6:i386
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  libclang1-3.6:i386
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 0 B/4,174 kB of archives.
After this operation, 14.4 MB of additional disk space will be used.
(Reading database ... 633354 files and directories currently installed.)
Preparing to unpack .../libclang1-3.6_1%3a3.6.2-3_i386.deb ...
Unpacking libclang1-3.6:i386 (1:3.6.2-3) ...
dpkg: error processing archive
/var/cache/apt/archives/libclang1-3.6_1%3a3.6.2-3_i386.deb (--unpack):
 trying to overwrite shared '/usr/lib/llvm-3.6/lib/libclang-3.6.so.1',
which is different from other instances of package libclang1-3.6:i386
Errors were encountered while processing:
 /var/cache/apt/archives/libclang1-3.6_1%3a3.6.2-3_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)


Bug#770265: submitted patches

2016-09-18 Thread Joe Crayne
Control: reassign -1 dh-python

This is actually a bug in dh-python and python3-minimal.  I've submitted
fixes at bug #812228 and bug #838193.


Bug#812228: Multi-Arch support for dh-python

2016-09-18 Thread Joe Crayne
I've attached a patch that will make dh-python Multi-Arch aware by making
use of the environment variable DPKG_MAINTSCRIPT_ARCH which is provided in
the environment by dpkg.

Specifying the architecture this way will prevent a failure that requires
the human administrator to fix.  With version 3.5.1-4 of python3-minimal it
will unfortunately redundantly compile packages when multiple architecture
variants provide the same python modules.  However, I've submitted a patch
to python3-minimal that would correct that behavior.

See bug #838193 for the changes I propose to python3-minimal.
diff --git a/autoscripts/postinst-py3compile b/autoscripts/postinst-py3compile
index 95b..3d22fc7 100644
--- a/autoscripts/postinst-py3compile
+++ b/autoscripts/postinst-py3compile
@@ -1,3 +1,3 @@
 if which py3compile >/dev/null 2>&1; then
-	py3compile -p #PACKAGE# #ARGS#
+	py3compile -p #PACKAGE#:$DPKG_MAINTSCRIPT_ARCH #ARGS#
 fi
diff --git a/autoscripts/prerm-py3clean b/autoscripts/prerm-py3clean
index c0aadfa..45c8cf2 100644
--- a/autoscripts/prerm-py3clean
+++ b/autoscripts/prerm-py3clean
@@ -1,6 +1,6 @@
 if which py3clean >/dev/null 2>&1; then
-	py3clean -p #PACKAGE# #ARGS#
+	py3clean -p #PACKAGE#:$DPKG_MAINTSCRIPT_ARCH #ARGS#
 else
-	dpkg -L #PACKAGE# | perl -ne 's,/([^/]*)\.py$,/__pycache__/\1.*, or next; unlink $_ or die $! foreach glob($_)'
+	dpkg -L #PACKAGE#:$DPKG_MAINTSCRIPT_ARCH | perl -ne 's,/([^/]*)\.py$,/__pycache__/\1.*, or next; unlink $_ or die $! foreach glob($_)'
 	find /usr/lib/python3/dist-packages/ -type d -name __pycache__ -empty -print0 | xargs --null --no-run-if-empty rmdir
 fi
diff --git a/debian/changelog b/debian/changelog
index 253edae..a1dd124 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,10 +1,14 @@
-dh-python (2.201609XX) UNRELEASED; urgency=medium
+dh-python (2.201609XX+nmu1) UNRELEASED; urgency=medium
 
+  [ Piotr Ożarowski ]
   * pybuild.pm:
 - enable --test-{nose,pytest,tox} if appropriate pypy-* build
   dependency is set
 
- -- Piotr Ożarowski <pi...@debian.org>  Wed, 07 Sep 2016 22:39:27 +0200
+  [ Joe Crayne ]
+  * python3 autoscripts: support for Multi-Arch: same.
+
+ -- Joe Crayne <oh.hello@gmail.com>  Sun, 18 Sep 2016 03:11:04 -0400
 
 dh-python (2.20160818) unstable; urgency=medium
 


Bug#838193: Improved Multi-Arch support for python.

2016-09-18 Thread Joe Crayne
Package: python3-minimal
Version: 3.5.1-4

A package marked "Multi-Arch: same" that distributes python modules and
makes use of dh-python will lead to the following crash when multiple
architectures are installed together:

Setting up gir1.2-ibus-1.0:i386 (1.5.11-1) ...
dpkg-query: error: --listfiles needs a valid package name but
'gir1.2-ibus-1.0' is not: ambiguous package name 'gir1.2-ibus-1.0' with
more than one installed instance

Use --help for help about querying packages.
Traceback (most recent call last):
  File "/usr/bin/py3compile", line 290, in 
main()
  File "/usr/bin/py3compile", line 270, in main
options.force, options.optimize, e_patterns)
  File "/usr/bin/py3compile", line 154, in compile
for fn, versions_to_compile in filter_files(files, e_patterns,
versions):
  File "/usr/bin/py3compile", line 106, in filter_files
for fn in files:
  File "/usr/share/python3/debpython/files.py", line 71, in filter_public
for fn in files:
  File "/usr/share/python3/debpython/files.py", line 53, in from_package
raise Exception("cannot get content of %s" % package_name)
Exception: cannot get content of gir1.2-ibus-1.0
dpkg: error processing package gir1.2-ibus-1.0:i386 (--configure):
 subprocess installed post-installation script returned error exit status 1

This is a result of invoking py3compile or py3clean on an ambiguously
specified package.  dh-python can and should disambiguate the package by
specifying an architecture and I am submitting a patch to that effect.
However, I believe the correct behavior for py3clean and py3compile would
be to clean or compile only the packages uniquely provided by a given
instance of a package.  This would ensure the correct behavior for python
modules that are distributed exclusively for a specific architecture, as
well as modules that are common to all architectures.  I've attached a
patch that implements this behavior.

Additionally, in the case of an ambiguous package specification, this patch
will turn what is currently a severe failure requiring a human to repair
into
a minor redundancy that will go away over time as packages are built
against a newer dh-python that does not provide ambiguous package
specifications to
py3clean and py3compile.

This would solve or at least help alleviate the following bugs:

bug #770265
bug #810551
bug #812228


Please apply this patch.
=== modified file 'debian/changelog'
--- debian/changelog	2016-06-03 14:42:29 +
+++ debian/changelog	2016-09-18 07:15:37 +
@@ -1,3 +1,10 @@
+python3-defaults (3.5.1-4.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * Compiling/cleaning support for Multi-Arch:same.
+
+ -- Joe Crayne <oh.hello@gmail.com>  Sun, 18 Sep 2016 03:14:34 -0400
+
 python3-defaults (3.5.1-4) unstable; urgency=medium
 
   * Bump standards version.

=== modified file 'debpython/files.py'
--- debpython/files.py	2013-07-14 12:09:52 +
+++ debpython/files.py	2016-09-18 06:53:34 +
@@ -44,7 +44,11 @@
 
 
 def from_package(package_name, extensions=('.py',)):
-"""Generate *.py file names available in given package."""
+"""Generate *.py file names available in given package.
+
+This function is deprecated because it is not Multi-Arch aware.
+Use uniquely_from_package() instead.
+"""
 extensions = tuple(extensions)  # .endswith doesn't like list
 process = Popen("/usr/bin/dpkg -L %s" % package_name,
 shell=True, stdout=PIPE)
@@ -56,6 +60,56 @@
 if line.endswith(extensions):
 yield line
 
+def uniquely_from_package(package_name, extensions=('.py',)):
+"""Generate *.py file names uniquely provided by a given package.
+
+The given package should specify an architecture using dpkg's pkg:arch
+syntax.  If other instances of the package are installed, the contents
+of those instances will be excluded, most likely causing the resulting
+list to be empty.
+
+In the case that the package is not uniquely specified due to the
+architecture being omited, an architecture will be chosen arbitrarily
+and no files will be excluded on non-uniqueness grounds.  This behavior
+makes this function safe to call in the "don't care" scenario when all
+architecture varients provide the same *.py files and we don't care
+which one is selected.
+"""
+pkg,*arch = package_name.split(':')
+process = Popen("dpkg-query -f'${Architecture}\n' -W %s" % pkg,
+shell=True,stdout=PIPE)
+stdout, stderr = process.communicate()
+if process.returncode != 0:
+raise Exception("cannot determine installed architectures for %s" % pkg)
+stdout = str(stdout, 'utf-8')
+other_archs = list()
+for x in stdout.splitlines():
+if not x