Possible MBF for SyntaxWarning with Python 3.8

2019-12-05 Thread Stuart Prescott
Hi all,

Python 3.8 generates a SyntaxWarning when it sees code that is most probably 
incorrect such as

if foo is 0:
 ...

if bar is not 'quux':
 ...

if baz is ():
 ...

When the Debian package is installed, py3compile sees such code and causes a 
pile of SyntaxWarning messages to be spewed in the middle of the apt run. 
Since code like the above is wrong and errors are unwelcome for the user (and 
are likely to generate quite a lot of support requests), I propose an MBF on 
the issue.

Looking through piuparts installation logs for packages in sid, I've so far 
identified 75 binary packages that emit SyntaxWarning on installation, but that 
is only the subset of python packages that happen to have been installed by 
piuparts over the last 60 days. I guess we'll need to continue to look at 
piuparts logs and that we will find more.

Any thoughts?

cheers
Stuart



raw entries from piuparts logs: https://people.debian.org/~stuart/swlog

data for bug reports: https://people.debian.org/~stuart/swbugs

ddlist: https://people.debian.org/~stuart/swddlist

bug template:

Subject: {package} generates SyntaxWarning at install time

Package: {package}
Version: {version}
Severity: important
Tags: sid bullseye
User: debian-python@lists.debian.org
Usertags: py38syntaxwarning

Dear maintainer,

Python 3.8 introduced a series of checks that look for code that is highly
likely to be buggy. When the code is processed by Python 3.8, it emits a
SyntaxWarning complaining about the code. For Debian packages, these
warnings will be visible at package install time.

This package generates SyntaxWarning output during package install:

{output}

Fixing this class of bug is generally straightforward and the upstream
developers may have already done so.

If there are questions, please ask for help on IRC #debian-python, or
the debian-python@lists.debian.org mailing list.




-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7




Joining DPMT

2019-10-07 Thread Stuart Prescott
Hi folks

Could you please add me to DPMT.

I've been looking at upgrading plastex (which is team maintained but with no 
active maintainer) to a newer upstream version; upgrading it to a newer 
upstream version gets rid of another Python 2-only package.

My salsa login is 'stuart'.

I've read the DPMT policy and agree to it.

regards
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7




Bug#906949: Clarify documentation location in a Python2-less distribution

2018-08-22 Thread Stuart Prescott
Package: debian-policy
X-Debbugs-CC: debhelper-de...@lists.alioth.debian.org, 
debian-python@lists.debian.org

Hi!

[ Promised follow up to the Python BoF at DebConf ]

For historical reasons, Python 2 modules are packaged as python-foo while 
Python 3 modules are python3-foo. Documentation was originally only for 
Python 2 modules (but in any event is normally interpreter independent) and 
so is found in python-foo-doc packages.

Policy §12.3 (from version 3.9.7) recommends that documentation from the 
python-foo-doc package be installed in /usr/share/doc/python-foo/ and not in 
/usr/share/doc/python-foo-doc/. From debhelper 10.8, dh_installdocs now does 
this automatically. In buster, documentation in python-foo-doc is slowly 
drifting over to /usr/share/doc/python-foo as packages are updated to compat 
11.

After the release of buster, we can expect to see Python 2 modules being 
dropped from the archive. Policy §12.3 and debhelper's interpretation of 
"main package" will suddenly cause documentation move from 
/usr/share/doc/python-foo to /usr/share/doc/python3-foo. This seems somewhat 
perverse for the user, will require changes to packaging (doc-base, rules, 
links) and perhaps code patches to packages that can display their own 
documentation.

The BoF felt it made sense to adopt a standardised approach and to have this 
sorted prior to the buster release in case any tooling changes are required. 
This affects about 700 packages.

We seek some input from the policy and debhelper people in defining what this 
approach should be. The following options exist:

a) do nothing: allow all the documentation to drift slowly over to 
/usr/share/doc/python3-foo, probably releasing buster and bullseye with a bit 
of a mess of locations: /usr/share/doc/python-foo-doc, 
/usr/share/doc/python-foo, /usr/share/doc/python3-foo. This seems pretty bad 
for our users.

b) keep using /usr/share/doc/python-foo: do we stick with 
/usr/share/doc/python-foo as a version-independent path even though no such 
package exists once the Python 2 package is gone? (I think there are 
currently no /usr/share/doc/ paths that don't match /usr/share/doc/$package)

c) coordinated move: do we try to switch the "main package" to the Python 3 
variant now so that all documentation moves over to 
/usr/share/doc/python3-foo? Churning d/control on 700 packages doesn't seem 
fun, so perhaps debhelper can assist?

d) revert to /usr/share/doc/python-foo-doc: do we ignore policy's 
recommendation, overriding (or changing) dh_installdocs? While 
/usr/share/doc/main-package is only a recommendation in policy, 700 
python-foo-doc packages deliberately ignoring it seems wrong.

Thanks in advance for your suggestions!

Stuart
(on behalf of the Python BoF at DebConf18)

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Safest way to set python3 as default for `python`

2018-07-18 Thread Stuart Prescott
Ben Finney wrote:
> To have a command with custom behaviour, the recommendation is the
> general one: put an executable file at ‘/usr/local/bin/python’ with
> whatever behaviour you want.

eeek, please don't! 

If a user created an incompatible /usr/local/bin/rm (didn't delete files, 
didn't exit 0), that would break a lot of tools and we'd tell the user not 
to do that. Fortunately, users don't tend to. /usr/local/bin/python is no 
different but for some reason users feel they can meddle with it.

Do not put locally versions of system tools like 'python' in root's PATH (or 
indeed in any user's PATH if the user expects to be able to run packaged 
tools). Doing so ends up potentially breaking packaged modules (they can 
find the wrong versions of dependencies in /usr/local since that is also in 
sys.path), and packaged applications (broken modules plus perhaps now 
running with the wrong version of the interpreter which is either 
incompatible or doesn't have the necessary modules installed). There's a 
reasonable number of things in /usr/bin with "#!/usr/bin/env python" that 
would be unhappy with this configuration.

We have lots of experience of this sort of breakage in #debian, where we see 
the this breaking maintainer scripts or packaged tools. As soon as we see a 
python backtrace with /usr/local in it, we know that is going to be the 
problem and expunging the local python installation from /usr/local is what 
is needed.

Putting python in other places such as /opt or ~/.local would be fine; even 
better still is to use one of the many virtualenv approaches so you're not 
even leaking that incompatible interpreter into a user's default 
environment.

(see also: please don't run pip as root)

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Bug#902516: python-matplotlib-venn: FTBFS in buster/sid (failing tests)

2018-07-05 Thread Stuart Prescott
On Thursday, 5 July 2018 08:09:50 AEST Andreas Tille wrote:
> Control: tags -1 help
> 
> Hi,
> 
> I can confirm this problem in a pbuilder environment.  Unfortunately I
> have no idea about the cause of the issue nor how to fix it.  Any hint
> from the Debian Python team?

Looks a lot like at least some of the tests are failing with:

https://wiki.debian.org/ContinuousIntegration/TriagingTips/numpy-1.14-doctests

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: No clue yet for building snakemake since there are two dirs in .pybuild

2018-06-26 Thread Stuart Prescott
Hi Andreas,

> 
"/build/snakemake-5.1.5/.pybuild/cpython3_3.6_snakemake/build/tests/tests.py",
>   line 22, in 
> raise Exception("snakemake not in PATH. For testing, install snakemake
> with "
> Exception: snakemake not in PATH. For testing, install snakemake with 'pip
> install -e .'. You should do this in a separate environment (via conda or
> virtualenv).
> 
> --
> Ran 4 tests in 0.227s
> 
> FAILED (errors=1)
> E: pybuild pybuild:336: test: plugin distutils failed with: exit code=1:
> cd /build/snakemake-5.1.5/.pybuild/cpython3_3.6_snakemake/build; python3.6
> -m nose tests
> 
> 
> 
> I think the cause of the issue is that there are two dirs in .pybuild:
> 
> 
> # ls -l /build/snakemake-5.1.5/.pybuild
> total 8
> drwxr-xr-x 3 pbuilder pbuilder 4096 Jun 26 09:20 cpython3_3.6
> drwxr-xr-x 3 pbuilder pbuilder 4096 Jun 26 09:20 cpython3_3.6_snakemake
> 
> 
> and the latter contains all the installed files while the former is
> empty.  How can I prevent this?

Deleting PYBUILD_NAME from your debian/rules is sufficient to fix that. When 
set, that sets HOME to the above directory and the build system for some 
reason starts using that. 

And no, I can't explain why that is...

There are still 3 test failures; the one I glanced at was a missing build-
dep on python3-networkx (and I didn't look further).

cheers
Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead (Was: Bug#899205: python-cogent: Test suite fails with latest ma

2018-06-05 Thread Stuart Prescott
Hi Andreas,

> ==
> ERROR: test_consistent_gap_degen_handling
> (test_core.test_sequence.ModelSequenceTests) gap degen character should be
> treated consistently
> --
> Traceback (most recent call last):
>   File
>   "/tmp/autopkgtest-
lxc.5a99fnj6/downtmp/autopkgtest_tmp/tests/test_core/test_sequence.py",
>   line 660, in test_consistent_gap_degen_handling
> self.assertEqual(dna.stripBadAndGaps(), raw_ungapped)
>   File "/usr/lib/python2.7/dist-packages/cogent/core/sequence.py", line
>   1251, in stripBadAndGaps
> valid_indices -= self._data == i
> TypeError: numpy boolean subtract, the `-` operator, is deprecated, use
> the bitwise_xor, the `^` operator, or the logical_xor function instead.
> 
> ==

The error message basically describes the solution -- patch attached. Quite 
what the original meaning of "bool - bool" was supposed to be is hard to 
discern (and that is why numpy has deprecated it). You should get upstream 
to look at this carefully.

Note that there is more than one test that fails like this. There also may 
be other places depending on how good their test coverage is.

There are also failing doctests; a patch for them also attached, permitting 
the test to work with both old and new numpy.

These two patches only get the package to compile though. Lintian will then 
tell you about other RC bugs that you need to fix prior to upload.

regards
Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7Description: Use xor rather than subtraction on booleans
 Numpy no longer accepts the ambiguous construction of a-b when a and b are bools.
 Normally, this can be replaced with XORs.
Author: Stuart Prescott 
--- a/cogent/core/sequence.py
+++ b/cogent/core/sequence.py
@@ -1248,7 +1248,7 @@
 gap_indices = map(self.Alphabet.index, self.MolType.Gaps)
 valid_indices = self._data < len(self.Alphabet)
 for i in gap_indices:
-valid_indices -= self._data == i
+valid_indices ^= self._data == i
 result = compress(valid_indices, self._data)
 return self.__class__(result, Info=self.Info)
 
--- a/cogent/maths/distance_transform.py
+++ b/cogent/maths/distance_transform.py
@@ -670,9 +670,9 @@
 return zeros((0,0),'d')
 dists = zeros((numrows,numrows),'d')
 for i in range(numrows):
-r1 = datamtx[i] # cache here
+r1 = datamtx[i].astype(dists.dtype) # cache here
 for j in range(i):
-dists[i,j] = dists[j,i] = sum(abs(r1 - datamtx[j]))
+dists[i,j] = dists[j,i] = sum(abs(r1 - datamtx[j].astype(dists.dtype)))
 
 return dists
 

Description: Ensure doctests pass with different numpy versions
 The numpy output format changed with numpy 1.14 and this causes doctests to
 fail if raw numerical output is compared. See
 https://wiki.debian.org/ContinuousIntegration/TriagingTips/numpy-1.14-doctests
Author: Stuart Prescott 
--- a/tests/alltests.py
+++ b/tests/alltests.py
@@ -6,6 +6,14 @@
 import doctest, cogent.util.unit_test as unittest, sys, os
 from cogent.util.misc import app_path
 
+# Whitespace changes between numpy 1.13 and 1.14 will cause the doctests
+# to fail; when doctests are updated to 1.14 format, this can be removed.
+try:# CRUFT
+import numpy as np
+np.set_printoptions(legacy='1.13')
+except TypeError:
+pass
+
 __author__ = "Peter Maxwell and Gavin Huttley"
 __copyright__ = "Copyright 2007-2016, The Cogent Project"
 __credits__ = ["Peter Maxwell", "Gavin Huttley", "Rob Knight",



Re: Updated PEP 394 (python and python2 commands)

2018-05-18 Thread Stuart Prescott
Matthias Klose wrote:
> The distro should get
> out of the way of using the python symlink, and giving users the freedom /
> choice what to do about the link.

I think I understand your rationale to stop shipping /usr/bin/python and 
once the unversioned symlink disappears from use in Debian then at least 
that particular avenue to breaking one's system disappears. 

What I don't understand is why any changes to package names or dependencies 
are required to achieve that goal. 

It sounds like a reasonable amount of work in your proposal, but once we no 
longer have any Python 2 applications left at some stage in the bullseye 
cycle, isn't the following sufficient?

--- a/debian/rules
+++ b/debian/rules
@@ -247,12 +247,9 @@ binary-arch: build install stamp-doc
 
: # provide the python and python.1 defaults
mkdir -p debian/python-minimal/usr/bin
-   ln -sf python$(VER) debian/python-minimal/usr/bin/python
ln -sf python$(VER) debian/python-minimal/usr/bin/python2
mkdir -p debian/python-minimal/usr/share/man/man1
-   ln -sf python$(VER).1.gz \
-   debian/python-minimal/usr/share/man/man1/python.1.gz
ln -sf python$(VER).1.gz \
debian/python-minimal/usr/share/man/man1/python2.1.gz

and then either later in the bullseye or bookworm cycles, those python-
defaults simply go away along with all the other 'unversioned' python module 
and interpreter packages.

What have I (and others!) missed that would make a rather elaborate 
packaging dance preferable to this?

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: DPMT and git workflows

2018-01-19 Thread Stuart Prescott
IOhannes m zmölnig (Debian/GNU) wrote:
> On 01/19/2018 12:45 PM, Simon McVittie wrote:
>> Relatedly, Alioth is going to be shut down at some point, with git
>> repositories frozen and made read-only, so it would seem a good idea to
>> start migrating git packaging to salsa.debian.org before that happens.
>> python-modules-team and python-apps-team groups, perhaps? I can create
>> a python-modules-team group and migrate tap.py as a sample if people
>> would like to see an example package.
>> 
> 
> should we keep the structure of putting all packages into a separate
> subdir (aka "sub-group").

I don't think there is anything to be gained by that and there are some 
limitations on sub-groups in salsa such as not being able to have rendered 
websites (that feature is prototyped but not yet enabled).

 
> i was also thinking about creating a single python-team group with a
> PAPT and a DPMT subgroup, but apart from aesthetics i cannot think of
> any good reason to do so. it probably creates more trouble than it is
> worth.

I don't know the historical origins of this split and I have never seen the 
advantage in splitting the two groups. Maybe it's time to have PAPT do the 
svn→git transition and put everything in the same debian-python-team?

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Bug#866454: plastex: depends on obsolete python-imaging (replace with python3-pil or python-pil)

2018-01-18 Thread Stuart Prescott
[ Resending because I originally sent it to the ghost-town alioth list; this 
package is in DMPT but has no active human maintainers. Patches are at 
https://bugs.debian.org/866454 ]

On Friday, 19 January 2018 01:43:42 AEDT Stuart Prescott wrote:
> Dear maintainers of plastex,
> 
> On Monday, 8 January 2018 23:40:14 AEDT Stuart Prescott wrote:
> > Control: tags -1 patch
> > 
> > The attached patch does the minimum that is required to update the
> > package.
> > 
> > There's a lot more that could be done with this package (including a new
> > upstream release from a few years ago) but this is a start...
> 
> The attached is a patch series that I intend to include in an NMU shortly
> unless someone else beats me to it.
> 
> The patches sit on top of the current DMPT git and I would include the
> changes that are sitting there but never uploaded. The source in git does
> not currently build at all (the patches get applied twice which fails) and
> so this is also fixed.
> 
> The main objective is, of course, to fix bug #866454.
> 
> regards
> Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Python 2 removal strategy

2018-01-10 Thread Stuart Prescott
Hi Ole,

> I thought of a temporary package split: create a new source package
> "astropy" that inherits of the current python-astropy package, but only
> builds python3-astropy (and the utils + doc, which depend on
> python3-astropy), and update this to version 3.0. Then I would remove
> these binary packages from the python-astropy package. In parallel, I
> would file bugs (severity: important) to remove the reverse dependencies
> of the Python 2 packages (many of them are mine, but also may have
> reverse dependencies).

This is more or less what I ended up doing for the plotting module PyX and I 
think it's a reasonable path forward given that there will also be lots of 
legacy user scripts out there that are using pyx (and astropy) that don't 
need to be broken just yet.

The pyx source continues to build python-pyx and is stuck at the last 
version of pyx that supported Python 2. It's only used by users in their 
existing scripts for plotting their data and they will keep working for as 
long as we have a Python 2 interpreter. The only changes that happen in 
python-pyx are adjusting to new dependencies such as python-imaging → 
python-pil.

An additional source package pyx3 builds python3-pyx and is tracking 
upstream releases meaning new features get added etc. pyx3 isn't a great 
source name but then users never see it and certainly never need to guess 
it. That said, pyx3 is probably less confusing as a name than astropy vs 
python-astropy.

I've wondered about putting some noisy warnings into the python-pyx code 
that talks about the need to migrate to Python 3 if you want your scripts to 
keep working into the future. I've not done so because I assume that anyone 
using pyx probably pays attention to upstream documentation and knows this 
story already; at the risk of hijacking this thread, I'd be interested in 
the thoughts of others on this.

cheers
Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: doc-central

2017-10-12 Thread Stuart Prescott
Dear Diane,

> Pending a better repository my changes are currently at:
> https://github.com/detrout/doc-central

This is excellent news, thanks!

BTW when doc-central was first orphaned, there was a little discussion about 
bringing it under the debian-doc umbrella (but there is no reason not to use 
d-python either given that discussion never came to anything concrete). 
There was also a desire to look at rarian integration for tools like yelp 
and khelpcenter for ease of document discovery. Hopefully some of the people 
who were interested back then will re-emerge to contribute, now that you've 
laid the ground work. (I hope to do so…)

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Python 3 Statsmodels & Pandas

2017-09-18 Thread Stuart Prescott
Hi Diane,

On Sunday, 17 September 2017 22:14:18 AEST Diane Trout wrote:
> I just did it that way because it was the least disruptive change I
> could make that would let me build and test the package.

Sure, that's entirely sensible.

> In my experience I'm much more likely to to notice a build time test
> failure than one from the CI system. 

Absolutely agreed. I'm thinking that this is a rather exceptional situation 
because of the circular dependency and the fall-out that we regularly see from 
that.

> What do other people think? If there are autopkgtests, should you still
> let dh_auto_test run tests?

(I know I'm not the 'other people' from whom you solicit replies, I was just 
perhaps unclear in what I was musing on before so I want to be more clear now)

Like you, I would *normally* run all tests in both places: buildd tests catch 
severe problems right now; ci.d.n reruns the tests each time new versions of 
dependencies are uploaded too and later breakage is caught.

Perhaps this is not a normal situation, however. Either one of "circular 
dependencies" or "packages that often FTBFS on one or more architectures" 
would be unusual enough to justify doing something different. All I am 
wondering (from my position of ignorance!) if in this case, perhaps the tests 
that cause the circular dependency can be disabled or xfailed, with the 
remaining tests run as normal.

cheers
Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Python 3 Statsmodels & Pandas

2017-09-17 Thread Stuart Prescott
Hi Diane & other science+python people,

Thanks for spending time on pandas and statsmodels. They are two very valuable 
but very painful packages to work on.

I've not looked at pandas and statsmodels packages carefully beyond "why 
hasn't pandas migrated this time!?" investigations but the circular 
relationship between pandas and statsmodels and its associated problems has 
come up several times before. It seems to be an on-going point of pain that 
sucks in quite a lot of developer time.

On Saturday, 16 September 2017 14:18:10 AEST Diane Trout wrote:
> My solution was to use build-profiles to flag the test dependency with
> !nocheck

this is, of course, a very elegant solution and exactly what build profiles 
are for...

I wonder though, is it really sustainable given this problem keeps coming up?

For the maintainer upload, you can do this bootstrapping with build profiles. 
For other archs, however, I don't believe we can tell the buildd to use build 
profiles so we'd need it to built by hand on porter boxes for every arch and 
then uploaded.

Is it feaible to completely break this circular dependency? If it is only 
needed for tests, would be possible to disable the build-time tests and rely 
on the tests run on ci.d.n instead? If it's used for generating documentation 
too, then would creating a separate source package for the documentation that 
build-depends on statsmodels and pandas be possible such that there is now no 
circular dependency?

(thinking aloud)

cheers
Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Python 2 in the default installation -- progress made!

2016-12-30 Thread Stuart Prescott

Dear -boot & -cd,

In re-reading #758234, I noticed a request that -boot and -cd be kept up to 
speed with priority changes that might have an impact on the installer.

In the thread starting at

https://lists.debian.org/debian-python/2016/12/msg00070.html

we have been looking at whether Python 2 can finally be kicked out of the 
Standard task with only Python 3 being installed rather than both being 
installed as is currently the case.

The remaining binary (source) packages that need their priorities changed are:

* python (python-defaults)
* python2.7 (python2.7)
* python-apt (python-apt)
* python-gpg (gpgme1.0)

The relevant maintainers seem to be happy with these changes. (Upcoming 
python-defaults and python2.7 uploads will change the source packages; python-
apt and python-gpg already have 'optional' within the source package). Bugs 
asking the ftp team to change the overrides are ready to be filed.

Additionally, changed dependencies are needed in:

* logrotate (Recommends: mailx → bsd-mailx | mailx), #849743

Your comments on (and hopefully assent to) these changes would be welcome.

cheers
Stuart

(Please CC me in replies)


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Python 2 in the default installation -- progress made!

2016-12-30 Thread Stuart Prescott
On Fri, 30 Dec 2016 10:14:20 Julian Andres Klode wrote:
> On Fri, Dec 30, 2016 at 06:52:37AM +0100, Matthias Klose wrote:
> > I don't know the rationale for having the apt and gpg bindings as Priority
> > standard, but if these can be made optional, then the depending packages
> > can be made optional as well.
> 
> python-apt is standard because it was used by apt-listchanges which is
> standard, and nobody followed the transition to Python 3 in apt-listchanges
> by switching the priorities in python-apt (2 should become optional, 3
> standard).

While python-apt being Priority:standard is indeed a relic of the old apt-
listchanges dependency (I forgot to mention that relationship in my earlier 
mail, sorry), it is no longer useful or required for python3-apt to have its 
priority increased to match the new apt-listchanges. In the discussion in 
#758234, there is a consensus that it makes no sense to raise the priority of 
dependency packages since all the tools involved can already resolve the 
dependencies without requiring the priorities to be appropriately set.¹

Julian, as a python-apt maintainer, would you like to raise the appropriate 
bug against ftp.debian.org or are you happy for me to do this?

override: python-apt:python/optional

cheers
Stuart


①  What remains to have priority inversion permitted (encouraged!) by Policy 
is a suitable patch; since Policy documents standard practice and it is 
already standard practice to permit priority inversion for dependencies, 
there's no reason to raise the priority of python3-apt.

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Python 2 in the default installation -- progress made!

2016-12-29 Thread Stuart Prescott

Hi everyone,

one of the objectives for stretch was to reduce the number of Python 2 
packages that are installed in common scenarios, instead having the Python 3 
stack take over. The "standard task" within tasksel is a reasonable place to 
look.¹

Thanks to the recent work on reportbug and debianbts, this is now largely 
complete.

In a stretch chroot, install the standard task:²

# aptitude install ~prequired ~pimportant ~pstandard

and the only python packages installed are:

# aptitude search ~npython~i
i A dh-python
i A libpython-stdlib
i A libpython2.7
i A libpython2.7-minimal
i A libpython2.7-stdlib
i A libpython3-stdlib
i A libpython3.5-minimal
i A libpython3.5-stdlib
i   python
i   python-apt
i A python-apt-common
i   python-gpg
i   python-minimal
i   python2.7
i A python2.7-minimal
i A python3
i A python3-apt
i A python3-chardet
i A python3-debian
i A python3-debianbts
i   python3-gpg
i A python3-httplib2
i A python3-minimal
i A python3-pkg-resources
i A python3-pycurl
i A python3-pysimplesoap
i   python3-reportbug
i A python3-requests
i A python3-six
i A python3-urllib3
i A python3.5
i A python3.5-minimal

where the packages marked "A" (Automatically Installed) are there to satisfy 
dependencies only.

We see that:

python
python-apt
python-gpg
python-minimal
python2.7

are now only installed because they are marked as Priority: standard.


Question for discussion: Is Python 2 a tool that would be expected to be found 
on any linux machine and so should it be Priority: standard? Is Python 3? Is 
now the time to ask ftp-masters to change the overrides to move them to 
Priority: optional?

(My personal feeling is that none of the above packages should be Priority: 
standard, including Python 3)


The libpython2.7* packages are present because:

i   logrotate Recommends mailx
i A mailutils Provides   mailx
i A mailutils Dependslibpython2.7 (>= 2.7)

I don't know if mailutils can already use Python 3 instead of Python 2 or how 
it is actually using Python at all -- the build system will certainly need 
some work. Perhaps a keen Python 3 porter can investigate that as an opening 
move for stretch+1 (it's too late for stretch for that transition).

cheers
Stuart



① The standard task contains packages you'd expect to find on a linux machine 
such as `less`; yes, it also includes a mail-transport-agent and it probably 
shouldn't; no, let's not discuss that now.

② Unlike other tasks, the standard task is not defined as a list of packages 
but as this aptitude search pattern to find all packages that declare Priority: 
required, important or standard; aptitude will also then install packages of 
lower priority to satisfy the dependencies.

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7



Re: Proposed git migration plan

2014-08-27 Thread Stuart Prescott
Hi Sandro,

 I rarely need to use log, and I used to work on the svn repo on a 56k
[..]

that you rarely use log is (a) not relevant to the people who would use it 
and (b) quite possibly highly influenced by the fact that svn log is so slow 
and painful. Remember that the tool can shape the user.

 Cheap local branches which let you pill up work in progress patches /
 rewrite without having to keep several copy of the same svn repo.  The
 branches in git are just a name pointing to a commit in the tree.

 The stash, which let you save your uncommited changes and come back to
 them later (think of it as lightweight branches).  That is really nice
 when you have to interrupt your workflow, stash it, handle the
 interrupt, reapply your stash and resume work.

 Tags can be signed with gpg. They are a pointer to a commit just like
 branches, and hence don't force you to do a full copy to create a tag.

 Switching between branches is a breeze and does not need network access
 either.
 
 I was referring to attractive features of git in regarding to Debian
 packaging

these *are* things that are attractive for packaging work. 

* I use branches extensively when experimenting with changes to packaging. 
What gets pushed might not look like that was what happened, but the branch 
was there along the way.

* I use stash quite a lot to be able to better articulate the work I'm doing 
in commits and as testing smaller changes. I even used it the other day in 
my little case story on git-buildpackage.

* (signed tags are probably an inherently good thing, but are really only 
nice to have for us at this stage)

* Because I use branches a lot, I switch between them, rebase work and merge 
it quite a lot.

 well, let's first see the list of problems/features we have/want and
 then let's talk later of the solutions to them, not the other way
 around

This discussion has been had many times before over many years. Of late, the 
difficulty in switching VCS has been the blocker. I don't think there are 
many people who are unconvinced that switching makes sense from a technical 
or workflow perspective. I also don't think there are many people who are 
interested in rehashing the entire discussion yet again. It's almost as 
boring as systemd discussions.

Fundamentally, one of the problems is svn. Even if you don't want the thigns 
git offers, just use git as a new version of svn spelt differently, then 
you'll still be able to work effectively on packages and you only have two 
new commands you need to use (assuming you aren't already using debcheckout 
and debcommit, of course).

Subversion lacks features that make packaging work much easier to do. The 
tools shape what you do -- you said that you don't use branches or log much 
and that's going to be partly because they are so crap in svn. In the past, 
I didn't use these features either when I was only using svn. Now I know 
they are useful things, I use them quite a lot because they are so easy and 
efficient in git. When I go back to working on svn projects, I realise how 
much the tool is deficient and how much the deficiencies of the tool lead to 
deficiencies in process. 

Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/ltkaj1$gs8$1...@ger.gmane.org



Re: Proposed git migration plan

2014-08-26 Thread Stuart Prescott
Hi Barry,

thanks for this write-up.

An open question remains what to do about the current history that is in 
svn. My experience of doing this is as follows:

* For many repos, svn-all-fast-export does an excellent job of turning a 
complete svn history into something that looks identical to what git-
buildpackage would have made on its own.

* For some repos, (especially when there were missing tags) svn-all-fast-
export requires some additional grafts to be added and then the git history 
reworked. Of course, any conversion utility would require this -- it depends 
how ideally accurate you want your history to be, and given that the current 
history in svn can be incorrect, being overly picky is perhaps unwise.

* And then for some repos, the conversion with svn-all-fast-export is an 
utter trainwreck and I was not able to work out why. (I did quite a lot of 
poking around and was unable to determined whether it was because I was 
using a newer svn than on previous conversions or because retagging had 
happened in that repo). I eventually gave up and used git-import-dscs --
debsnap.

That said -- all the above experience was with full-source svn repos. I 
would anticipate that the debian/-only repos would be easier to migrate as 
they are much simpler, but then the new git repo would also be debian/-only. 
A minor git dance to import the next upstream source would be needed. I've 
not even contemplated how you'd go about adding historical upstream sources 
to the git repo.

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/ltjdto$aft$1...@ger.gmane.org



Re: Proposed git migration plan

2014-08-26 Thread Stuart Prescott
Hi Barry,

I eventually gave up and used git-import-dscs -- debsnap.

That said -- all the above experience was with full-source svn repos. I
would anticipate that the debian/-only repos would be easier to migrate as
they are much simpler, but then the new git repo would also be
debian/-only.
 
 I just tried this one of of my simple packages (cov-core, a fairly new
 package
 with only two upstream releases in Debian).  In svn it's a debian/-only
 package but after the g-i-d the git repo ended up full source (yay!). 
 Then I
 did a `git-import-orig --uscan` and it seemed to work just fine.  

sure -- git-import-dscs will make full source repos for you, but it is not 
converting svn repos to git repos which is what I was really talking about 
there... it's throwing away the svn repo and making a new git repo with some 
sparse, condensed history in it. If the real history needs to be preserved, 
then a helper like svn-all-fast-export, git-svn or similar is needed and 
that won't on its own make full source repos.

 I wish
 g-i-o would automatically run {git-,}dch with a d/changelog entry of New
 upstream release.

Yes, and that should be fairly easy to implement too. (Interestingly, I 
always passed --svn-noautodch to svn-bp as I found that particular feature 
exceedingly annoying but that is of course at build time not at upgrade 
time)

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/ltjhr3$kbg$1...@ger.gmane.org



Re: DebConf14 svn-git migration BOF notes

2014-08-26 Thread Stuart Prescott
Hi!

 svn history, do we keep it?  with git-svn, or with the kde
 git-svn-import, each tag becomes a branch which is a pain.  

BTW svn-all-fast-export is the package that contains the kde git-svn-import 
tool -- it does rather have a naming crisis. 

svn-all-fast-export accepts a mapping that takes an svn tag and turns it 
into a git tag. It also takes svn branches and map them into git branches. 
Further details are at:

https://wiki.debian.org/PackagingWithGit/Svn-
buildpackageConversion#Importing_using_svn-all-fast-export

and if you `debcheckout translate-toolkit` you will find a git repo that 
looks like a git repo for git-buildpackage should, even if some of the older 
commit messages are prefixed by '[svn-upgrade]'. The mapping rules on that 
wiki page are the ones I used for translate-toolkit and most packages would 
have the same rules with only s/mypackage/otherpackage/.

 - Workflow:
   * pristine-tar

While I currently use pristine-tar on all my packages and I think it's a 
reasonably important part of our infrastructure at this stage, I'd urge a 
little caution here given that it is orphaned both in Debian and upstream.

 http://bugs.debian.org/737871

[...]

 I've done some personal investigation since the BOF, and am preparing
 some really simple migration scripts, so we can get a feel for what it
 will look like. My scripts so far (very very simple)
 git://git.debian.org/users/stefanor/dpmt-migration.git

is there any reason to use a loop around git-import-dsc rather than git-
import-dscs --debsnap here?


thanks for reporting back from the BoF for us!

cheers
Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/ltjjb5$4i3$1...@ger.gmane.org



Re: Playing with git-buildpackage

2014-08-24 Thread Stuart Prescott
 I use with this:

~/.gitconfig:

[merge dpkg-mergechangelogs]
name = debian/changelog merge driver
driver = dpkg-mergechangelogs -m %O %A %B %A

~/.gbp.conf:

[DEFAULT]
keyid = 0x1396F2F7

debian/gbp.conf (in the source package -- applies to everyone)
[DEFAULT]
pristine-tar = True
sign-tags = True


.git/gbp.conf (in my working copy -- applies only to me or could actually go 
into ~/.gbp.conf):

[DEFAULT]
builder=sbuild 

[git-buildpackage]
export-dir=../build-area




-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7


--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/201408250136.21840.stu...@debian.org



Re: RFS: oct2py -- GNU Octave to Python bridge

2014-06-28 Thread Stuart Prescott

Hi Josué,

thanks for the follow-up!

 Sadly the current version released by upstream does not work with Octave
 3.8 which
 is the latests version of Octave in Sid.

interesting... I used your package on sid the other day with no issues in an 
ipython notebook. Maybe I was just lucky in the relatively small set of 
features that I was using.

BTW this means that the package isn't running the test suite at build time... 
but from the bug report you linked to, I assume that the test suite doesn't 
completely pass in any case. It would be good if you could run the test suite 
at build time, marking the known-failing tests as xfail so that they can fail 
gracefully but any further failures would be caught. Even better would be to 
also add some autopkgtest tests to the package so that the tests can be run on 
ci.debian.net every time a new octave version appears to make sure things 
don't break without someone noticing.

 It seems that upstream is working on that [0], I will have the package
 ready when
 the issue is closed.

the upstream bug is closed but I'm not sure how large the patch was to fix it 
or if it would be possible to apply it to 1.3.0. If you're keen, you could 
investigate that... but I perfectly understand waiting for upstream to release 
their next version.

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7


--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/201406290251.08066.stu...@debian.org



Re: RFS: oct2py -- GNU Octave to Python bridge

2014-06-15 Thread Stuart Prescott
Dear Josue,

Thanks for your work so far on oct2py -- Vincent raised a couple of points 
about the packages. I see that you addressed the dh-python dependency and the 
typo'd Suggests -- have you had a chance to address his other question about 
the orig tarball? It would be great to see this package in Debian.

cheers
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7


-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/201406161328.37432.stu...@debian.org



Re: building python debian packages and virtualenv

2014-05-01 Thread Stuart Prescott
Hi Brian,

 Fortunately, so far I have always noticed this before uploading the
 package, however think it is only a matter of time before I stuff up an
 upload.

While not directly addressing your question, perhaps this is also a good 
opportunity to add some autopkgtest tests to the packages -- even if there's 
not a full test suite from upstream, a (minimal) smoke test would catch 
this. The autopkgtest tests probe the installed package as opposed to the 
the just-built source so it catches wrong file paths, files that get missed 
out from the install step etc.

The autopkgtest specification is also known as DEP-8:

http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=blob_plain;f=doc/README.package-
tests;hb=HEAD

http://packaging.ubuntu.com/html/auto-pkg-test.html

and an example of a python module's test suite that runs upstream's test 
suite is:

http://sources.debian.net/src/translate-toolkit/1.11.0%2Bdfsg-1/debian/tests

(or look for other packages that declare Testsuite: autopkgtest in the 
Sources file)

To run the tests, there's sadt from devscripts which runs in your current 
environment (but would possibly still suffer from the virtualenv problem -- 
I don't think it sanitises this part of the environment). For better 
isolation, adt-run can run it in a chroot or virtual machine -- if you're 
already using schroot/sbuild to build things, then you can:

adt-run -B \
  --binary $DEBS \
  --source $DSC \
  --log-file $CHANGES.adt.log \
  --- adt-virt-schroot unstable-amd64-sbuild

cheers
Stuart

(this message was brought to you by the package testing fan club)


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/ljva27$lp5$1...@ger.gmane.org



Re: Recommending get-orig-source for packages ?

2013-12-03 Thread Stuart Prescott
Hi Paul,

 Policy recommends it, that should be enough?
 
 http://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules

tl;dr: is get-orig-source supposed to be a duplicate of uscan (d/watch) or 
apt-get source? And why do we want to reimplement either? And why do we want 
yet another location for the download URL in the packaging?


I'd personally say get-orig-source (optional) is quite different to get-
orig-source (recommended) but then the entire language of this part of 
policy is loose at best.

There's plenty of discussion about what policy actually means here: which 
version should be downloaded, exactly? -- the name orig seems to imply 
getting the current source tarball, policy suggests it should get the latest 
upstream release and many implementations differ from this. And of course 
the maintainer is supposed to have a suitable crystal ball to know how to 
repackage version N+1 when they package version N... because sites never 
change their download URL and upstreams never add or remove crap from their 
tarballs. 

This has reached the point where there's a feeling that it would better to 
remove this recommendation from policy altogether.

http://bugs.debian.org/466550

This bug isn't the first time it has been discussed, it's just the current 
discussion in the BTS against d-policy. It will have its 6th birthday soon. 
Given the lack of agreement about what get-orig-source should do, my feeling 
is that it is heading even further into the let's get rid of it zone.

Having uscan call debian/rules get-orig-source is quite difficult to do in 
a policy-compliant way (as already noted by Jakub) as the location for the 
munged tarball is different. Having uscan call a debian/repack from d/watch 
seems a little more sane only because there's no policy saying what d/repack 
must do; having uscan do the repacking itself with something like Files-
Excluded from d/copyright is even nicer and devscripts in git can do this.

Like so many things in Debian there is more than one way to do something 
that is truly simple and for which there probably should only be one way. It 
would be nice if we didn't have more than one way of doing something as 
simple as fetching an upstream source -- it's harder for automation, it's 
harder for QA, it's harder for new maintainers and it's harder for casual 
bug squashers. Without undertaking any sort of survey of packages, my 
feeling is that the project is centralising on d/watch + uscan instead of 
get-orig-source.

cheers
Stuart


-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprintBE65 FD1E F4EA 08F3 23D4 3C6D 9FE8 B8CD 71C5 D1A8




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/l7m4em$t54$1...@ger.gmane.org



Re: PEP 453 affects Debian packaging of Python packages

2013-09-18 Thread Stuart Prescott
Hi Paul,

 I don't understand the pip hate. Why don't you guys try and, you know,
 figure out *why* these tools were invented. It (for sure) is overly
 simplistic, but it's there for a reason.

It's pretty obvious why these tools were invented -- I think everyone 
appreciates the difficulties of installing and distributing software across 
different platforms, particularly legacy ones with no native concept of 
package management. I think it's also safe to assume that the python 
packaging list is probably acutely aware of API problems in python modules 
that would make it useful in virtualenvs too.

However, we're currently in a situation where users can very easily make an 
absolute mess of their set up without realising that's what they are going 
to do and it then becomes Debian's user support problem. We see this on a 
regular enough basis -- packages stop working and the python stackdumps show 
modules with incompatible APIs being loaded from /usr/local/. (#703874 is 
just one example of this that I have to hand -- I'm sure the BTS has many 
more and we see a few per week like this in #debian)

Who gets the blame for mess made? The Debian project and its developers. Who 
ends up helping users clean up the mess? The Debian project and its 
developers. Pip can as useful as it wants in virtualenvs but it already has 
a lot of historical baggage attached to it that is mostly associated copious 
amounts of time spent trying to fix the mess it makes on Debian machines. 

That's why you have pip hate.

For better or for worse, proposals to make further use of pip carry all this 
historical baggage with them. As suggested elsewhere in this thread changing 
pip so that it was difficult to do things system-wide would help with this.

cheers
Stuart



PS While not directly related to pip (but in a quite similar vein), in the 
last week, I've seen the exact problem below *twice*:

  I tried to install python2.7 on my squeeze box like this
   https://mail.python.org/pipermail/python-list/2011-May/603818.html
   and now I get:
update-alternatives: error: cannot stat
/usr/bin/python2.7/bin/python2.7: Not a directory
   Help!

tl;dr: trying to install jessie's python into squeeze doesn't work out so 
well and creating alternatives is harder than it looks when you're blindly 
copying instructions from the web. Suggestions on upstream mailing lists 
from people who either don't know what they're doing or don't fully explain 
the dangers aren't always the best things to follow. *sigh*


PPS Virtualenvs are another commonly cited use case for pip, but virtualenvs 
are mostly a short way of saying we don't care about API stability (your 
own story of the incompatibilities of django versions is an illustration of 
this); they're an interesting technical solution to a poor development model 
that focuses on developers getting high on finding the perfect solution at 
the expense of the users of the code. Every user of that module has to 
either: keep altering their own code to match new APIs, or try to insulate 
their code from API changes by using virtualenvs, or NIH it instead. So 
sure, virtualenv+pip is a valid use case for pip, but it should hardly be 
considered the pinnacle of python's achievements.

But I think I'll leave the rest of that rant (and the bit about what you 
shoudl do in a virtualenv when you need both modules A and B but module A 
needs C version 1.0 and module B needs C version 1.1) for another day.

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprintBE65 FD1E F4EA 08F3 23D4 3C6D 9FE8 B8CD 71C5 D1A8




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/l1dh0e$al9$1...@ger.gmane.org



Re: Inconsistency in source package naming for python modules

2013-07-11 Thread Stuart Prescott

 Oh, I need this pyX package... Let's download it.

You're using a python module name because you need to import it. If you want 
to import modules, you want the binary package name; if you want to work on 
the source package then you need *either* the binary package name or the 
source package name as others have already pointed out. Knowledge of source 
package names isn't required.

(I'll assume that your X in pyX was a placeholder... given that there is 
actually a pyx source package and a python-pyx binary package and debian 
package names are downcased, it's actually a rather confusing example to 
pick... using a policy-compliant lower case name apt-get source pyx or 
apt-get source python-pyx both work just fine.)

Binary package names for module packages already follow the python policy of 
being python-foo where you want to import foo. The thing that we do not 
easily get from this is the case when we import foo.bar and we need to 
work out if that's in the python-foo package or perhaps in a python-
foo.bar package. At present we have apt-file or dpkg -S to deal with this 
problem and in the future, as noted by Paul, DEP11 will give us a way of 
discovering this information much more easily (and this a common problem for 
perl and java and …). This module discovery is orthogonal to source package 
naming; note that exactly the same problem exists in trying to work out if 
foo and foo.bar were the same or different source packages no matter what 
naming scheme you want to come up with.

If your py$X in your example is not in a binary package called python-py$X 
and it is a public module-only package then it's already not policy 
compliant and wants fixing. Report a bug about the binary package name. Cite 
the appropriate part of python policy in it. If the module is in a policy-
compliant binary package then your problem is already solved as has been 
already pointed out so there's nothing to fix here. Knowledge of the source 
package name was not required.

Also note that your example of trying to find source package names by 
divination is still not going to work problem where a python module is 
provided as bindings its own C library. There are many sources like these -- 
let's look at one such example: xapian-bindings 

Package-List: 
 libxapian-ruby1.8 deb ruby optional
 libxapian-ruby1.9.1 deb ruby optional
 python-xapian deb python optional
 tclxapian deb libs optional

So that source package should be renamed to python-xapian because the python 
people want unified source package naming? Clearly not. And once you know 
you have a pile of exceptions to whatever naming scheme you want, it's not a 
useful naming scheme -- you'll be back to either (a) stumbling around trying 
things at random as per your examples or (b) using the tools we have that 
accept either binary or source package names or (c) using the available 
tools to look up the correct information to feed to git svn.

 And that's just an example on what can go wrong and be really annoying.
 It's even more annoying when you are trying to do a git svn clone
 which takes forever.

If you want to get the VCS data for the python-foo binary package:

$ apt-cache showsrc python-foo | grep Vcs

Knowledge of the source package name was not required. Like the other apt-
get commands shown by others, apt-cache showsrc takes binary package names 
as well as source package names. I guess it will take less than forever to 
write the appropriate 2 line shell script to solve this problem for you as 
your own replacement to the existing debcheckout. (Or perhaps you'd like to 
add a couple of lines to debcheckout to offer git-svn there? line 526 looks 
like the right place for you to start -- debcheckout is in the devscripts 
package for submitting patches to the bts.)

Making use of existing tools and extending them where necessary so that they 
always work seems a lot more useful than creating some sort of flawed naming 
scheme that will only ever sometimes work.

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprintBE65 FD1E F4EA 08F3 23D4 3C6D 9FE8 B8CD 71C5 D1A8




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/krmamp$p1j$1...@ger.gmane.org



Re: Inconsistency in source package naming for python modules

2013-07-11 Thread Stuart Prescott
Thomas Goirand wrote:

 On 07/11/2013 09:07 PM, Stuart Prescott wrote:
 
 Oh, I need this pyX package... Let's download it.
 
 You're using a python module name because you need to import it. If you
 want to import modules, you want the binary package name; if you want to
 work on the source package then you need *either* the binary package name
 or the source package name as others have already pointed out. Knowledge
 of source package names isn't required.
 
 But if you have pywhatever (let's say on pypi), then even the binary
 package isn't obvious. It could be:
 python-pywhatever
 
 or:
 python-whatever
 
 I've seen both cases in the archive!

and changing source package names isn't going to solve that either. There is 
no convention in python-land to only have one top-level module name per 
project in the namespace and to name your project after that top-level 
module. At that point, the unification of namespaces between pypi (or any of 
the other multitude of module repositories), source packages and binary 
packages becomes impossible. As Paul indicated, this is a difficult problem 
with a different solution.

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprintBE65 FD1E F4EA 08F3 23D4 3C6D 9FE8 B8CD 71C5 D1A8




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/krmtme$tun$1...@ger.gmane.org



Re: Inconsistency in source package naming for python modules

2013-07-10 Thread Stuart Prescott
Thomas Goirand wrote:
 On 07/08/2013 10:10 PM, Scott Kitterman wrote:
 There is no policy on this either way, so there's no mistake.
 
 Well, the mistake is precisely to have no rule, IMO.

Rules for packaging things are normally there to solve problems of 
interoperability and to assist QA efforts. Which of these is it going to 
help?
 
 Never the less, I think we should collectively decide what to do, rather
 than continuing the mess, with everyone having its own rule.

What mess? If there is a perceived mess, why is that a problem in any case? 
How does it help to make a new rule? Who does it help? What problem does 
this solve? Why is any intellectual energy being spent on this at all?

It looks exceedingly like a rule for the sake of having a rule. It will be 
an exceedingly complicated rule in that it will have to cover python 
modules, python applications and other libraries that offer python bindings 
all separately. It will have to be accompanied an explanation of why so many 
packages don't follow it because they were uploaded prior to the rule 
existing. Basically... unless we are going to force every existing source 
package to change name to comply with this rule there is no point in having 
it (and no-one has advocated renaming source packages as is useless work for 
everyone).

Rules like this look like yet another small barrier to entry to new 
contributors in the form of yet another thing to learn. Debian already has 
too many administrative hurdles and piles of little rules that scare away 
people. I'm yet to understand whether rules like this are created for 
benefit of people who like to have a policy with which to berate others or 
by people who like to impose order on the world around them.

2¢

Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprintBE65 FD1E F4EA 08F3 23D4 3C6D 9FE8 B8CD 71C5 D1A8




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/krjr6p$kmq$1...@ger.gmane.org



Re: python-aeidon package

2013-06-30 Thread Stuart Prescott
Hi Piotr,

 Upstream switched to Python 3.X and I don't want to port it back to 2.X.
 I can provide an older version of python-aeidon package (in a separate
 source package) as a temporary solution, but I'd rather not release it
 with Jessie. 

Ah. That's unfortunate. That's a pretty big break in compatibility.

Thanks for your analysis of how ttkit could be ported to python 3. 
Unfortunately, there are at least two fairly major reverse-dependencies from 
the same upstream developers that will make this very difficult: pootle¹ and 
virtaal. Both of these are python 2-only and import translate. They have 
dependencies on things like python-django and python-gtk2². (And yes, 
perhaps we should split the translate-toolkit package into python-translate 
and translate-toolkit packages where the latter only ships the files in 
/usr/bin and their manpages so that it is more obvious that there is both a 
module API and a command line interface to translate-toolkit. It's not 
obvious to me whether it's worth doing that split for the sake of a few 
small files in /usr/bin.)

I'll forward your analysis of this porting effort upstream, but I suspect 
that pootle and virtaal will prevent any porting of ttkit to python 3 for 
the time being; disabling subtitle support might be our only solution. I 
suspect (but have not checked) that the conversion to python 3 will also 
require some significant reworking of the translate-toolkit API to change 
what parts are done in bytes and what parts are in unicode/strings. ttkit is 
one of those places where python 3 will make the code much more robust and 
easier to work with but the conversion will be challenging.

cheers
Stuart


¹ I have now been trying to get pootle back into Debian for almost two years 
but I can never find a time where all its dependencies are actually in 
unstable and with compatible versions ☹. At present, it needs a newer 
python-django-south and an older python-django than what is available in 
sid. I'm starting to come to the conclusion that either pootle is very very 
unlucky or maintaining large python programs is impossible unless you 
minimise your exposure to modules that are outside your control and NIH as 
much as possible. Sadly, I think upstream is close to giving up on working 
with distributions and will encourage everyone to use virtualenv instead. If 
only API compatibility were more of a priority in python-land.

² I assume python-gtk2 is going to be a problem in itself some time soon.

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprintBE65 FD1E F4EA 08F3 23D4 3C6D 9FE8 B8CD 71C5 D1A8




-- 
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/kqpdtf$ega$1...@ger.gmane.org



python-aeidon package

2013-06-26 Thread Stuart Prescott

Hi Piotr,

translate-toolkit currently makes use of python-aeidon both at build-time and 
at runtime but I see that this package has now gone away. python3-aeidon is 
not a replacement for python-aeidon for python 2.x programs. Is there any 
chance of the python 2-compatible module coming back or was the 2→3 conversion 
done in a non-backwards compatible fashion?

thanks
Stuart

-- 
Stuart Prescotthttp://www.nanonanonano.net/   stu...@nanonanonano.net
Debian Developer   http://www.debian.org/ stu...@debian.org
GPG fingerprintBE65 FD1E F4EA 08F3 23D4 3C6D 9FE8 B8CD 71C5 D1A8
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7


--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/201306270313.19125.stu...@debian.org