Re: Bug #732703 and fixing ensurepip/pyvenv
On May 9, 2014, at 6:16 PM, Barry Warsaw wrote: > A follow up. > > I've done a fair bit of prototyping a solution, and with Donald's gracious > help and feedback, I think I have a plan that will work and should be > compliant with policy. I'm beginning to make changes to various DPMT packages > to build the whole stack, but I'm only uploading the simpler changes for now > until I have a working local testbed. > > The basic approach does involve building a bunch of wheels as explained > earlier. We have to build the pip and setuptools wheels from our devendorized > packages, and install those into the venv's site-packages directory. > Installation is done by ensurepip, and I have a branch of python3.4 that > re-enables this. > > Now that python3-wheel has cleared NEW, I need to modify the d/rules files to > build the wheels. I have this working for pip in svn (along with significant > packaging changes to use pybuild and update to 1.5.5), but I won't upload this > just yet. It installs the wheels into /usr/share/python-wheels. > > The tricky bit is what to do about the recursively vendorized packages. Here > I think the best course of action is as outlined previously - build wheels of > these from quilt-patched Debian source at package build time, and install > those too into /usr/share/python-wheels. When pyvenv is called, the > appropriate wheels will be copied into /usr/share/python-wheels and the > venv's pip will know to add those to sys.path. We cannot unpack these > dependent wheels into the venv's site-packages so this is to me the simplest > and best approach. I think Donald agrees. The one ugly bit is what to do > about the venv wheels when you `pip upgrade pip` since this will pull the PyPI > pip into the venv. Our unvendorized wheels won't get deleted from the venv, > but they also won't be used (because PyPI's pip will bundle, and won't contain > the patch to use the unvendored versions). IOW, our wheels will stick around > as unused trash. Oh well - this only happens if you `pip upgrade pip` in the > venv. (We can try to fix that later if we really care.) Yes I agree. For the record I don’t think getting pip to uninstall them during an upgrade will be very difficult but if you’re OK with them sticking around us unused turds then there’s no real harm to not doing it. > > Other than extensive testing , what needs to happen now is that I need > to modify the packaging of the dependencies to build the wheels. This is > fairly easy for html5lib, requests, python3-chardet, and urllib3, although > some of these need their setup.py patched to use setuptools instead of > distutils.core (because only the former supports the bdist_wheel command > provided by python3-wheel). These packages are all team maintained. > > It's trickier for distlib, colorama, and six because those are not team > maintained, so I will generate patches and pester the maintainers once I've > proven that the whole stack works. I've reached out to the colorama > maintainer to see if we can move that into DPMT, since it's only ever seen > NMUs since the initial upload. Colin, Matthias, keep an eye out for patches > to six and distlib respectively. > > I won't finish this up this weekend, but will continue working on it early > next week. > > Stay tuned, and best to ping me on IRC if you have questions. > > Cheers, > -Barry - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Bug #732703 and fixing ensurepip/pyvenv
A follow up. I've done a fair bit of prototyping a solution, and with Donald's gracious help and feedback, I think I have a plan that will work and should be compliant with policy. I'm beginning to make changes to various DPMT packages to build the whole stack, but I'm only uploading the simpler changes for now until I have a working local testbed. The basic approach does involve building a bunch of wheels as explained earlier. We have to build the pip and setuptools wheels from our devendorized packages, and install those into the venv's site-packages directory. Installation is done by ensurepip, and I have a branch of python3.4 that re-enables this. Now that python3-wheel has cleared NEW, I need to modify the d/rules files to build the wheels. I have this working for pip in svn (along with significant packaging changes to use pybuild and update to 1.5.5), but I won't upload this just yet. It installs the wheels into /usr/share/python-wheels. The tricky bit is what to do about the recursively vendorized packages. Here I think the best course of action is as outlined previously - build wheels of these from quilt-patched Debian source at package build time, and install those too into /usr/share/python-wheels. When pyvenv is called, the appropriate wheels will be copied into /usr/share/python-wheels and the venv's pip will know to add those to sys.path. We cannot unpack these dependent wheels into the venv's site-packages so this is to me the simplest and best approach. I think Donald agrees. The one ugly bit is what to do about the venv wheels when you `pip upgrade pip` since this will pull the PyPI pip into the venv. Our unvendorized wheels won't get deleted from the venv, but they also won't be used (because PyPI's pip will bundle, and won't contain the patch to use the unvendored versions). IOW, our wheels will stick around as unused trash. Oh well - this only happens if you `pip upgrade pip` in the venv. (We can try to fix that later if we really care.) Other than extensive testing , what needs to happen now is that I need to modify the packaging of the dependencies to build the wheels. This is fairly easy for html5lib, requests, python3-chardet, and urllib3, although some of these need their setup.py patched to use setuptools instead of distutils.core (because only the former supports the bdist_wheel command provided by python3-wheel). These packages are all team maintained. It's trickier for distlib, colorama, and six because those are not team maintained, so I will generate patches and pester the maintainers once I've proven that the whole stack works. I've reached out to the colorama maintainer to see if we can move that into DPMT, since it's only ever seen NMUs since the initial upload. Colin, Matthias, keep an eye out for patches to six and distlib respectively. I won't finish this up this weekend, but will continue working on it early next week. Stay tuned, and best to ping me on IRC if you have questions. Cheers, -Barry signature.asc Description: PGP signature
Re: Bug#747494: python3-biopython: Does not use C implementations of cpairwise2 functions
On Fri, 2014-05-09 at 13:32 +0200, Jakub Wilk wrote: > But in Python 3 imports as always absolute, unless explicitly > requested, so the import fails, and this code snippet is no-op. > Changing the import line to: > > from .cpairwise2 import rint, _make_score_matrix_fast > > should do the trick. Indeed. This was actually reported and fixed a couple of months ago upstream[1]; I foolishly didn't check the closed bugs there before submitting my report. Sorry for the noise. I'll make the import change locally. [1] https://github.com/biopython/biopython/pull/299 Jamie -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/1399670549.6390.3.ca...@atman.artefact.org.nz
Re: Packaging review
Ross Vandegrift, 2014-03-20 20:40:21 -0400 : > Hello, Hi again, I haven't heard much about the packaging of Paperwork and its dependencies recently… can I do anything to help? Roland. -- Roland Mas We reject: kings, presidents and voting. We believe in: rough consensus and running code. -- IETF -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/87k39vlywg@dragomir.placard.fr.eu.org
Switch to python3.4 as default python3 imminent
Based on discussions with the release team, I think we'll be able to do this next week (assuming the sip-api transition that just started goes well). blender will need an upload and python-astropy is broken and will remain broken until upstream fixes it. Are there any other issues people know of that need to be addressed? Scott K -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/27996095.uBLSf5hIGc@scott-latitude-e6320
Re: Bug#747494: python3-biopython: Does not use C implementations of cpairwise2 functions
* Andreas Tille , 2014-05-09, 13:15: thanks for your bug report. I think this should work out of the box but I personally not comfortable with cpython to know how this could be fixed. I keep the Debian Python list in CC - perhaps they might have some helpful advise. The relvant code in Bio/pairwaise2.py is: # Try and load C implementations of functions. If I can't, # then just ignore and use the pure python implementations. try: from cpairwise2 import rint, _make_score_matrix_fast except ImportError: pass But in Python 3 imports as always absolute, unless explicitly requested, so the import fails, and this code snippet is no-op. Changing the import line to: from .cpairwise2 import rint, _make_score_matrix_fast should do the trick. Before: $ python3 -c 'from Bio.pairwise2 import rint; print(rint.__module__)' Bio.pairwise2 After: $ python3 -c 'from Bio.pairwise2 import rint; print(rint.__module__)' Bio.cpairwise2 It seems this module is not affected by the test suite since this runs fine. Yeah, as the code comment says, if the import fails, everything(?) still works, just slower. -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140509113237.ga7...@jwilk.net
Re: Bug#747494: python3-biopython: Does not use C implementations of cpairwise2 functions
[Andreas Tille, 2014-05-09] > > >>> from cpairwise2 import rint > > Traceback (most recent call last): > > File "", line 1, in > > ImportError: No module named 'cpairwise2' > > > > But some (presumably relevant; I know nothing about how Python > > interfaces with C code) files are installed: > > > > $ locate pairwise2 > > /usr/lib/python3/dist-packages/Bio/cpairwise2.cpython-33m-x86_64-linux-gnu.so > > /usr/lib/python3/dist-packages/Bio/cpairwise2.cpython-34m-x86_64-linux-gnu.so Bio.cpairwise2 != cpairwise2 -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140509112105.gf19...@sts0.p1otr.com
Re: Bug#747494: python3-biopython: Does not use C implementations of cpairwise2 functions
Hi Jamie, thanks for your bug report. I think this should work out of the box but I personally not comfortable with cpython to know how this could be fixed. I keep the Debian Python list in CC - perhaps they might have some helpful advise. It seems this module is not affected by the test suite since this runs fine. If you would like to help making sure that this will be detected soon before we upload in the future it might make sense if you could provide a test case to the upstream developers who could include this test in their unit tests. Kind regards Andreas. On Fri, May 09, 2014 at 10:08:15PM +1200, Jamie Norrish wrote: > Package: python3-biopython > Version: 1.63+dfsg-4 > Severity: normal > > Dear Maintainer, > > After installing this package, the cpairwise2 C implementations of > functions are not available (see the end of Bio/pairwise2.py for the > attempted loading of these): > > Python 3.3.5 (default, Mar 22 2014, 13:24:53) > [GCC 4.8.2] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> from cpairwise2 import rint > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named 'cpairwise2' > > But some (presumably relevant; I know nothing about how Python > interfaces with C code) files are installed: > > $ locate pairwise2 > /usr/lib/python3/dist-packages/Bio/cpairwise2.cpython-33m-x86_64-linux-gnu.so > /usr/lib/python3/dist-packages/Bio/cpairwise2.cpython-34m-x86_64-linux-gnu.so > > Is this meant to work out of the box, or is there some action I can and > should take to be able to take advantage of the C implementation? > > Thanks, > > Jamie > > -- System Information: > Debian Release: jessie/sid > APT prefers unstable > APT policy: (500, 'unstable') > Architecture: amd64 (x86_64) > Foreign Architectures: i386 > > Kernel: Linux 3.14-1-amd64 (SMP w/4 CPU cores) > Locale: LANG=en_NZ.utf8, LC_CTYPE=en_NZ.utf8 (charmap=UTF-8) > Shell: /bin/sh linked to /bin/dash > > Versions of packages python3-biopython depends on: > ii libc6 2.18-5 > ii python3 3.3.4-1 > ii python3-numpy [python3-numpy-abi9] 1:1.8.1-1 > ii python3-reportlab 3.0-1 > pn python3:any > > Versions of packages python3-biopython recommends: > pn ncbi-blast+ > ii python-biopython-doc 1.63+dfsg-4 > > Versions of packages python3-biopython suggests: > pn bwa > pn clustalo > pn clustalw > pn dialign > pn dssp > pn emboss > pn fasttree > pn mafft > pn muscle > pn phyml > pn prank > pn probcons > ii python-matplotlib 1.3.1-1+b1 > ii python-mysqldb 1.2.3-2 > ii python-numpy1:1.8.1-1 > ii python-pil 2.3.0-2 > ii python-psycopg2 2.5.2-1 > pn python-rdflib > ii python-renderpm 3.0-1 > ii python-reportlab3.0-1 > ii python3-matplotlib 1.3.1-1+b1 > ii python3-pil 2.3.0-2 > ii python3-psycopg22.5.2-1 > pn python3-renderpm > ii python3-tk 3.3.4-1 > pn raxml > pn t-coffee > pn wise > > -- no debconf information > > ___ > Debian-med-packaging mailing list > debian-med-packag...@lists.alioth.debian.org > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debian-med-packaging > -- http://fam-tille.de -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140509111520.gg13...@an3as.eu
Re: Bug#747481: ITP: backports.ssl-match-hostname -- Backport of the Python 3.2 SSL hostname checking function
* Nicolas Dandrimont [2014-05-09 10:34:09 +0200]: > Package: wnpp > Severity: wishlist > Owner: Nicolas Dandrimont > > * Package name: backports.ssl-match-hostname > Version : 3.4.0.2 > Upstream Author : Brandon Craig Rhodes > * URL : https://bitbucket.org/brandon/backports.ssl_match_hostname > * License : Python > Programming Lang: Python > Description : Backport of the Python 3.2 SSL hostname checking function > > The Secure Sockets layer is only actually secure if you check the > hostname in the certificate returned by the server to which you are > connecting, and verify that it matches to hostname that you are trying > to reach. > . > But the matching logic, defined in RFC2818, can be a bit tricky to > implement on your own. So the ssl package in the Standard Library of > Python 3.2 and greater now includes a match_hostname() function for > performing this check instead of requiring every application to > implement the check separately. > . > This package contains a backport of the ssl.match_hostname function for > Python 2.4 and above. On IRC, Jakub kindly pointed me at #626539 and its resolution. As a recent update of a package I maintain (websocket-client) actually needs this backport, and I'll need to use it on wheezy (and therefore have to backport the backport), I'll go ahead and package that anyway. Thanks, -- Nicolas Dandrimont Microsoft is not the answer. Microsoft is the question. NO (or Linux) is the answer. (Taken from a .signature from someone from the UK, source unknown) signature.asc Description: Digital signature
Bug#747481: ITP: backports.ssl-match-hostname -- Backport of the Python 3.2 SSL hostname checking function
Package: wnpp Severity: wishlist Owner: Nicolas Dandrimont * Package name: backports.ssl-match-hostname Version : 3.4.0.2 Upstream Author : Brandon Craig Rhodes * URL : https://bitbucket.org/brandon/backports.ssl_match_hostname * License : Python Programming Lang: Python Description : Backport of the Python 3.2 SSL hostname checking function The Secure Sockets layer is only actually secure if you check the hostname in the certificate returned by the server to which you are connecting, and verify that it matches to hostname that you are trying to reach. . But the matching logic, defined in RFC2818, can be a bit tricky to implement on your own. So the ssl package in the Standard Library of Python 3.2 and greater now includes a match_hostname() function for performing this check instead of requiring every application to implement the check separately. . This package contains a backport of the ssl.match_hostname function for Python 2.4 and above. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140509083409.9292.16377.report...@darboux.home.olasd.eu
