dh-python piuparts problem?

[Barry Warsaw]

I like to run `sbuild --run-piuparts blah.dsc` before I upload a new package
version.  I've seen this failure on a number of packages now, but I haven't
had time to dig into it in detail.  I'm not sure if it's a problem in the
individual packages, or a bug in dh-python.  I should file a bug.  Has anybody
else seen this?

4m39.2s ERROR: FAIL: Package purging left files on system:
  /usr/share/dh-python/dhpython/__pycache__/ not owned
  /usr/share/dh-python/dhpython/build/__pycache__/   not owned

4m39.2s ERROR: FAIL: Installation and purging test.

For example, on distlib.

Cheers,
-Barry


pgpPSUIji2GPa.pgp
Description: OpenPGP digital signature

New pip/virtualenv stack

[Barry Warsaw]

I just uploaded virtualenv 14.0.5+ds-1

I'm hoping that this repairs the whole pip/virtualenv stack.  It does for me
in all my local testing, and all the DEP-8 tests pass (again, locally).  Some
headers for pip apparently got lost when I merged branches, so it's entirely
possible there are still some installability/upgradability problems, though
run-piuparts (as run by sbuild) seems to be happy for me locally.

I am now going to work on removing the individual -whl packages from DPMT
packages where they're no longer needed, and I'll file bugs on any others.

Once the new virtualenv lands, please give the whole stack a good workout and
let me know or report bugs for any problems you find.  I'll be working my way
through existing BTS bugs next to see if they're still relevant.

Apologies for breaking everything.  I know how much many of us rely on these
important tools.  Trust me, I feel the pain too. ;)

Cheers,
-Barry


pgpSGRSCyKlDk.pgp
Description: OpenPGP digital signature

Re: How to put experimental upload into git

[Barry Warsaw]

On Feb 01, 2016, at 08:10 AM, Nikolaus Rath wrote:

>Commit to a new branch that then becomes dangling, because the next upload to
>unstable descends from the most-recent unstable upload rather than from
>experimental.

I haven't done this specifically, but I have used branches w/git-dpm.  I can't
say if been particularly successful at it for more complex (long-running)
cases.  But your case sounds relatively simple so I'd probably do what you
suggest above.

Cheers,
-Barry

Re: New, updated pip coming

[Barry Warsaw]

On Feb 01, 2016, at 03:01 PM, Piotr Ożarowski wrote:

>Did you consider creating whl files at install time¹ rather than pip's
>build time? This way whl can be regenerated whenever one of needed
>packages is updated and there's no need to rebuild pip package.
>
>[¹] using dpkg (file) triggers (/usr/share/doc/dpkg-dev/triggers.txt.gz)

I did think about it, and maybe it's where we want to end up, but I didn't go
this way right now because of the added complexity.  Well, maybe "different"
complexity, since none of this stuff is simple. ;)

What we have now is better than what we had before, but once I get virtualenv
fixed and uploaded, I think I'll come back around to looking at what you
suggest.  (Certainly, patches welcome in the meantime.)  Fortunately, I
believe we have all the tools in place (e.g. dirtbike) to generate the wheels
at install time.  It does mean some extra run-time dependencies, but that's
probably fine.

Cheers,
-Barry

Re: New, updated pip coming

[Barry Warsaw]

On Jan 30, 2016, at 11:33 AM, Robert Collins wrote:

> If a new requests package is built, uploaded to the archive and
>apt-get installed on my machine, and I then do:
>virtualenv test
>. test/bin/activate
>pip install foobar
> ^--- what version of requests will be used by this in-virtualenv
>invocation of pip?
>
>>From your description I expect it to be the version of requests that was
>dpkg installed at the time the pip deb was built?

Correct.

On Jan 29, 2016, at 05:45 PM, Donald Stufft wrote:

>These wheel files that represent pip's dependencies are generated when
>python-pip is being built from the versions of the packages in Debian's
>repositories and the python-pip package has a Built-Using header to indicate
>which dependencies have been rebundled inside of pip through this
>process. This has essentially recreated static linking but for Python. It is
>my understanding that all of the wheels for pip's dependencies will exist
>inside of the python-pip-whl .deb.

Yep.

I don't actually know whether Built-Using *does* anything useful[*], but IWBNI
(maybe) some kind of notification or auto-rebuild happened.  I don't think it
does, but Donald's right.  It's the same problem that Go or statically linked
C/C++ has.

>On top of that, virtualenv and venv will both need wheels that it needs to
>install for pip, setuptools, and in the case of virtualenv, wheel. I'm not
>sure what the plan is for those.

I plan to use the same technique for virtualenv/pyvenv dependencies, although
they will probably Depend on python-pip-whl to just get those for free.  TBH,
I haven't spent a ton of time on those higher level tools, just because of how
much work it was to sort out pip.  But I'll look at those soon.

Cheers,
-Barry

[*] There are some plans to honor this field in Ubuntu, where Build-Depends
may not need to go through the same main-inclusion-report (MIR) process, but
Built-Using packages would.  It's all currently speculative, so I don't have
any more details right now.


pgpYCju0beL_D.pgp
Description: OpenPGP digital signature

New, updated pip coming

[Barry Warsaw]

TL;DR By the end of today, I expect to upload pip 8.0.2 to unstable, finally
bringing us up to the latest version.

It's been a long slog, with many people helping out.  I hope that all the hard
work done to get us here means it will be much, much easier to track new pip,
virtualenv, and pyvenv updates going forward.  And now, details!

pip requires the use of PEP 427 wheels, which are essentially zip files of
importable, pure-Python, bilingual code, with a little bit of metadata.  Put a
wheel on sys.path and Python can import from it.  pip and friends use wheels
because it ensures that they can't break themselves when their dependent
packages are updated.  E.g. if pip depends on requests, and you `pip install
--upgrade requests` it's possible the new version would be incompatible with
pip.  Then you'd have a broken pip on your hands.  Remember that you can use
pip to install pre-release packages too.

So by putting the wheels on sys.path *before* anything else, even if you pip
install an incompatible requests, it won't break pip itself.

pip "vendorizes" (i.e. bundles) the wheels of its dependencies with known good
versions, but this breaks Debian policy, so we have to unbundle these.
Several upstreams use this technique, with requests and urllib3 as probably
the most popular examples.  In all these cases, we have to remove the bundled
wheels and hope that the versions of their dependencies in Debian remain
compatible.  It's a chore, but that's our job as distro developers, and it's
the best option given the conflicting requirements.

Fortunately, we can create our own wheels and use them with pip and friends.
Until now, we had to build the wheels at the time that the dependent packages
where built, and we had to have a corresponding foo-whl binary package in the
archives, because we could only use setup.py's bdist_wheel command (which
comes with python{,3}-wheel) to create these.  With 15 run time dependencies
as of pip 7.1.2, this has become untenable, and was a huge reason why Debian's
pip has lagged so far behind upstream.  (That, and a few ITPs for new
dependencies. ;)

Along comes dirtbike.  https://github.com/paulproteus/dirtbike

While it has been much discussed and attempted over the years, it would be
really nice to have a tool that turned installed Python packages into wheels.
I think we discussed such a tool for Debian at last year's (two years ago?)
Debian BOF at Pycon.  Anyway, Asheesh Laroia finally took hold of the, ahem,
wheel, and wrote the initial implementation, which he called dirtbike.  Since
then I've been enhancing it to work with several oddball cases in the Debian
archive, and now it's good enough to turn pip's dependencies into wheels.

Yesterday I uploaded the last of the new pip dependencies (modulo any
surprises in 8.0.2 ;), and dirtbike so now all the pieces are in place.  What
pip now does during its own build process, is turn its installed dependencies
into .whl files and puts them in the policy defined location.  This allows us
to eliminate all those scattered -whl binary packages, and just
Build-Depend/Built-Using in pip to install the standard Python packages,
rewheel them, and put them in a python-pip-whl package.  Going forward, this
will make life so much easier, because only python-pip needs to know anything
about wheels, even if new dependencies are added.

I have everything working locally.  I need to update the master branch for
pip, which involves many curse words hurled at git-dpm (and that's a post for
another day!).  I also have to update my local branch to 8.0.2, but I'm
assuming there won't be any new surprises there.

Eventually, I'll be using the same techique for virtualenv and pyvenv, but not
right away.

Huge thanks also go to Donald Stufft, who always is so helpful with various
technical details, and works really hard to understand our needs and balance
them with upstream's.

Finally, we need to update Python policy to describe the new wheel policy.
Fortunately, this is much better because it removes constraints on other
packages, and reduces the number of -whl binary packages in the distro.  Yes,
I will update those packages that already ship -whls, or file bugs, but only
once I'm sure everything is working correctly.

Below is the policy diff, which I'll commit to the python-defaults bzr.

As always, once everything's uploaded, please do let me know via this list or
BTS, if you notice any problems with pip.

Cheers,
-Barry

=== modified file 'debian/python-policy.sgml'
--- debian/python-policy.sgml   2016-01-24 06:01:19 +
+++ debian/python-policy.sgml   2016-01-29 16:49:59 +
@@ -458,13 +458,12 @@
 
   http://legacy.python.org/dev/peps/pep-0427/;
name="PEP 427">
-  defines a built-package format called "wheels", which is a zip
-  format archive containing Python code and a "dist-info" metadata
+  defines a built-package format called "wheels", a zip
+  archive containing Python 

Re: mkdocs locale error building djangorestframework

[Barry Warsaw]

On Jan 26, 2016, at 01:12 AM, Ben Hutchings wrote:

>That's not the problem at all.  Read the error message again.  Read the
>source line it points to.  Now look at where rv comes from:
>
 import subprocess
 rv = subprocess.Popen(['locale', '-a'], stdout=subprocess.PIPE,  
>...   stderr=subprocess.PIPE).communicate()[0]
 type(rv)  
>

For Python 3, try adding `universal_newlines=True` to any subprocess call.
You'll get back a str.

Cheers,
-Barry


pgpDd84p51hjY.pgp
Description: OpenPGP digital signature

Re: Python Policy: Things to consider for Stretch

[Barry Warsaw]

Thanks for taking this on Ben,

On Jan 24, 2016, at 04:33 PM, Ben Finney wrote:

>I think you're right that this needs a general clean-up through the
>policy document, to consistently use:
>
>* “python2” to refer to that command only;
>
>* “python3” to refer to that command only;
>
>* “python” to refer to that command (and I'd suggest deprecating it
>  where feasible);
>
>* “Python 2” referring exclusively to that language version 2.x and no
>  other versions of that language;
>
>* “Python 3” referring exclusively to that language version 3.x and no
>  other versions of that language;
>
>* “Python” referring to the language implemented either as Python 2 or
>  Python 3.

I agree with all the above, though re: Scott's feedback on deprecation of
/usr/bin/python, here's my take (note, we've had these discussions many times
before ;).

Ultimately, we should be both guided by, and drive with our opinions, PEP 394
which should help keep the situation consistent across *nix distros.

https://www.python.org/dev/peps/pep-0394/

/usr/bin/python (on Debian) should never point to anything but the latest
Python 2 version *until* Python 2.7 is EOL'd upstream, meaning binary, source,
and security-only releases.  Bug fix releases (source only) are promised until
2020, and I'd wager we'll see supported security-only releases until 2025.

I would like to see scripts that are only compatible with Python 2 to be
shebanged `/usr/bin/python2` or `/usr/bin/python2.7`.  You can think of this
mostly as documentation at this point, but it does remove any possibility of
future ambiguity.

It also begins to open the door to the often discussed launcher idea someday
landing on /usr/bin/python, enabling bilingual scripts.

I think there's zero harm in this since shebang lines are generally not
user-evident.

Then there's no change to /usr/bin/python any time soon, or the fact that
invoking `python` at the shell starts the Python 2 interpreter.

Cheers,
-Barry

Re: Python Policy: Things to consider for Stretch

[Barry Warsaw]

On Jan 23, 2016, at 03:38 AM, Scott Kitterman wrote:

>Personally I seriously dislike the trend to call Python Python 2 (and I still
>thing approving a pep to invent /usr/bin/python2 because Arch went insane was
>a horrible idea).  There's an earlier spot in the document where it says that
>everything refers to python and python3 unless it's explicit.  I'll make this
>spot /usr/lib/python2.7/dist-packages and any risk of ambiguity is, I think,
>resolved.

I'll leave it to you, but my take on it is that "Python" is the generic term
for the language and its specification.  "Python 2" v "Python 3" provides
disambiguation when you're talking about specific major versions of the
language.  "Python 3.5" and such usually describe specific releases of the
CPython interpreter implementation (note how "CPython" is used to disambiguate
between alternative implementations of the language specification).  Finally,
python2.7, python3.4 and such are used to describe the executables that
provide the versions (e.g. mentally prepend them with /usr/bin).

All of this, except the last point perhaps, is orthogonal to the
/usr/bin/python2 issue you mention.

Back to the original point, to me saying "Python" and "Python 3" is confusing
or misleading, given the above definitions.  It's confusing because "Python 3"
*is* Python, so what's the difference?  It's misleading because it implies
that somehow "Python 3" isn't "Python".

>> B.2. dh_python2 and dh_python3
>> 
>> Again, I think here you want to say "Python2 and Python3" to disambiguate
>> between generic Python.  
>
>If I say Python and Python3, what version can the one that's not Python3
>possibly be?  I don't think it's any less confusing than starting to call
>what we've always called "Python" "Python 2".

See above, but to rephrase, "Python" is ambiguous in this context because you
could be talking about Python-the-language, not Python-some-release-version.

Cheers,
-Barry


pgpy0qrLvOIdq.pgp
Description: OpenPGP digital signature

Re: Amend Debian Python Proposal to Include More Python Metadata?

[Barry Warsaw]

Hey Donald, thanks for starting this conversation.  I for one am super
appreciative of all the consideration you give for Debian's little slice of
the world.

There's a lot to unpack in this thread, and I'm a little under the weather[1],
so hopefully this makes sense.

Big +1 for recording the files that get installed via the .egg-info/RECORD
file.  As you know, I've been working on dirtbike, which is a "rewheeling"
tool to turn an installed package into a .whl.  While Debian tightly controls
via policy the set of wheels we'll allow into the archive, dirtbike has code
for parsing the RECORD file.  Unfortunately this is never exercised in
practice because we don't have RECORD files - at least not for the packages we
care about[2].

Big +1 for using setuptools everywhere.  By my count, of the packages that I
happen to randomly have installed on my Ubuntu 16.04 system, I have 67
.egg-info files and 113 .egg-info directories.  I'd rather have .egg-info
directories everywhere.

+1 for a lintian warning if distutils is used.  I guess I'm +0 on forcing that
through pybuild because it'll be unobvious and mysterious, and kind of lets
upstreams off the hook.  I'd mildly prefer to patch packages that use
distutils because that's much more discoverable, but I can appreciate that
that's a lot of work we'd be imposing on maintainers, so I won't argue this
too much (other than to say that if pybuild forces it, let's definitely
document this in its manpage!).

On Jan 22, 2016, at 12:40 PM, Scott Kitterman wrote:

>Currently --record includes the .pyc files which is both unneeded and bad.
>Before this gets added either in setuptools or by us, this needed to be
>fixed.

+1 for (I think) another reason than has already been discussed.  We won't be
generating .egg-info directories on the end-user's machine, but instead the
machine the package is built on.  That could be a maintainer's own system or a
central build machine depending on various conditions.  But because the pycs
are generated on the end-user's machine, we actually don't know what pycs will
be generated when the debs are installed, so the egg-info/RECORD file *can't*
contain them, at least not accurately.

On Jan 22, 2016, at 11:54 AM, Donald Stufft wrote:

>Regardless of what happens in this thread, pip is going to stop mucking
>around in files that are owned by some other tool without some sort of
>opt-in/--force style flag *and* we're going to be restructuring things to try
>and guide people away from using pip outside of a virtual environment or
>through the user of --user as well.

Of course, I'd still like --user to be the default[3].  I think that's still
the eventual goal for pip, but isn't yet implemented because $priorities.

>A more controversial way that comes with possibly some extra benefits (which
>Debian may not care about) is to use ``pip`` itself as the build tool rather
>than directly invoking setup.py. In this pip would be responsible for mucking
>around with the distutils/setuptools mess instead of that needing to be
>handled by Debian specific tooling.

I'd like to better understand how this would work.  IIUC, the Fedora ecosystem
is making or already has made this switch, but I don't know how it works.
Obviously, we don't want to install wheels into /usr/lib/python3/dist-packages.

FTR, I am still working on updating Debian's pip.  I'm slowly shaving the yak,
but there are still a few things to do.  If you want to help, get in touch.

>[3] Import pkg_resources is not the fasest thing in the world, it has to scan
>every directory on sys.path looking for things to activate and it comes with
>a bunch of side effects. This happens implicitly for any project using
>console scripts.

Which frankly sucks.  It's also fragile.  Every once in a while a broken
package gets uploaded that breaks pkg_resources, and it's not easy to debug or
repair.  I really hope Brett can fix this when/if he builds this functionality
into the stdlib.

Cheers,
-Barry

[1] double meaning: I have a cold and we're in the early stages of an historic
snow storm!

[2] dirtbike has two fallbacks, both of which use `dpkg -L` to get the list of
installed files.  The first fallback uses `dpkg -S` to find the binary package
that contains the Python package's .egg-info file/directory (doesn't matter
for this purpose).  pkg_resources doesn't have one of those.  The second
fallback imports the Python package and looks for the binary package
containing the top-level directory.  It's all rather ugly, and I'd like to
just use the .egg-info/RECORD file, but I suspect I'll still need the import
fallback for pkg_resources.

[3] https://github.com/pypa/pip/issues/1668


pgpcYL6d9Pl7r.pgp
Description: OpenPGP digital signature

Re: Python Policy: Things to consider for Stretch

[Barry Warsaw]

On Jan 21, 2016, at 10:47 AM, Scott Kitterman wrote:

>I've taken a run through the current Python Policy to see where I think it
>needs to be updated for Stretch.

Thanks Scott for the badly needed update.

Some comments, apologies for the lack of good quoting, or if I've read the
diff incorrectly.

2.5 Module Path

"Public Python modules must be installed in the system Python modules
directory, /usr/lib/python./dist-packages.  Public Python 3 modules must
be installed in /usr/lib/python3/dist-packages."

I think this could use a bit of disambiguation, sice "system Python modules"
could mean either Python generically, in which case the second sentence is in
conflict.  Do you mean "system Python 2 modules"?  If so, let's say that.
Also, since Python 2.7 is all there will be from now on, why not say that
explicitly?

A good reason not to would be because Policy needs to cover older releases
where there can be multiple versions of Python 2.  But then later in your diff
you seem to mention python2.7 as being associated with
/usr/local/lib/python2./dist-packages.  So maybe say
/usr/local/lib/python2.7/dist-packages here too?  There really won't be any
other value for  than 7.

3.4 Specifying Supported Versions

"The optional `X-Python-Version (preferred) or `XS-Python-Version` field..."

It's a little confusing because we're now saying we prefer not to have either
field.  How about "(previously preferred)" or just drop the parenthetical?

3.6 Provides

s/substituation/substitution/

B.2. dh_python2 and dh_python3

Again, I think here you want to say "Python2 and Python3" to disambiguate
between generic Python.

Other than that, +1

Cheers,
-Barry


pgplZMuJy6II7.pgp
Description: OpenPGP digital signature

Re: Amend Debian Python Proposal to Include More Python Metadata?

[Barry Warsaw]

On Jan 22, 2016, at 05:50 PM, Donald Stufft wrote:

>Forget that pip can fetch files from PyPI and install them for a moment and
>consider the command ``pip install .``. Fundamentally this is similar to the
>command ``make install`` right?

Please remind me what the long term plan for this is.  Let's say it's 2021 and
you and your amazing team of dozens of distutil-sig developers have been
cranking away the whole time.

What tool will be used to build Python packages?

What tool will be used to upload (built/source?) Python packages to
WarehouseNG?

What tool will be used to download Python packages from WarehouseNG?

What tool will be used to install packages from WarehouseNG into an importable
location?

Cheers,
-Barry


pgpPQkjaWvhbD.pgp
Description: OpenPGP digital signature

Re: Updating python-jsmin

[Barry Warsaw]

On Jan 14, 2016, at 02:45 PM, gustavo panizzo (gfa) wrote:

>Usually I generate the orig.tar.gz from git tags myself (gbp will do it for
>you) and upload that.  if the pkg needs a -2 release I base on that, I never
>used pristine-tar

Team policy is to use PyPI tarballs and pristine-tar workflow with git-dpm.
If there's a really good reason not to do it (i.e. not just personal
preference), then you have to document both the rationale and maintenance
procedure in README.source.

https://wiki.debian.org/Python/GitPackaging

channeling-the-evil-general-ly y'rs,
-Barry

Re: Bug#810136: transition: python3-defaults (python3.5 as default python3)

[Barry Warsaw]

On Jan 06, 2016, at 01:34 PM, Nikolaus Rath wrote:

>If necessary, s3ql can also be build with cython instead of cython3.

I strongly suspect this is a regression due to
http://bugs.python.org/issue22995 which I've now reopened.  We're probably
just starting to see the unintended consequences of this change.

canary-in-the-coal-mine-ly y'rs,
-Barry

Re: RFS: twistar (Was: Request to join)

[Barry Warsaw]

On Jan 04, 2016, at 04:41 AM, Giovanni Pellerano wrote:

>i was reading https://lists.debian.org/debian-python/2015/10/msg00267.html
>while working on GlobaLeaks (https://github.com/globaleaks/GlobaLeaks)
>and evaluate it's adoption as replacement for python-storm that
>appears not to be an abandoned software :(

I don't know anything about twistar, but I do think Storm is effectively
abandoned.  I'm not sure twistar is in better shape, given that the last PyPI
release is 2013, although there are a few commits to upstream's GitHub in
2015.

I think SQLAlchemy or SQLObject are generally considered best-in-breed of the
Python ORMs.  (FWIW, GNU Mailman 3 core is Python 3 and was ported to SA last
year.)

>I would support the packaging of twistar for debian; is there stil
>lthe possibility if packaged readly to have it in the upcoming debian
>stretch/ubuntu xenial?

I can't say whether it's worth it or not, but yes, there is probably time to
package it up and get it through NEW for stretch and xenial, if you can find a
sponsor (I'm not volunteering ;).

Cheers,
-Barry

Re: Switching Default Python3 To Python3.5

[Barry Warsaw]

On Dec 31, 2015, at 04:44 AM, Scott Kitterman wrote:

>Does anyone object if I go ahead and ask the release team for a transition 
>slot?

+1 and thanks!
-Barry

Re: Help with pytest 2.8.5

[Barry Warsaw]

Thanks for the suggestions Tristan & Piotr,

On Dec 17, 2015, at 01:15 PM, Piotr Ożarowski wrote:

>> diff --git a/debian/rules b/debian/rules
>> index f473395..3c2f918 100755
>> --- a/debian/rules
>> +++ b/debian/rules
>> @@ -59,8 +59,9 @@ override_dh_clean:
>>  override_dh_auto_test:
>>  ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
>> set -ex; \
>> +   cd /tmp; \
>> for py in $(PYVERS) $(PY3VERS); do \
>> -   PYTHONPATH=$(CURDIR) python$$py -m pytest testing ; \
>> +   PYTHONPATH=$(CURDIR) python$$py -m pytest $(CURDIR)/testing 
>> ; \
>
>PYTHONPATH=$(CURDIR) part looks very suspicious (and error prone)
>
>> done
>>  endif
>
>I didn't test it but... why is override_dh_auto_test needed at all?
>Did you try with:
>
>  export PYBUILD_TEST_ARGS={dir}/testing
>
>?
>(only "test" or "tests" are copied to build dir by default, you need to
>pass path to directory with tests if they're in non standard location)

I did try various riffs on this theme, but am still unable to get
tests_genscript.py to pass.  FWIW, here's the current pybuild-based rule:

# 2015-12-16 ba...@debian.org: Because pytest does not clean up after itself,
# use a custom temporary directory (which is easier to clean up manually,
# e.g. in an sbuild).
override_dh_auto_test:
TMPDIR=`mktemp -t -d pytest.XX` \
PYBUILD_SYSTEM=custom \
PYTHONPATH=$$(CURDIR) \
PYBUILD_TEST_ARGS="{interpreter} -m pytest --lsof -rfsxX {dir}/testing" 
dh_auto_test

Don't worry about the style of the above unless it's material to the failure.
As I mentioned, I've tried lots of different variations.  None allow
test_genscript.py to pass.  (Setting or not PYTHONPATH also makes no
difference.)

I believe genscript is deprecated in pytest, so I'm seriously considering just
disabling this test for now and moving on.

Cheers,
-Barry

Help with pytest 2.8.5

[Barry Warsaw]

I've been working on an update to pytest 2.8.5, but I'm stuck and I'm hoping
someone here can help.  The DPMT git repo is up-to-date with my latest work if
you'd like to check the branches out and try to debug the builds.

I've made some significant changes to the packaging by switching it to
pybuild.  That's not the problem though. ;)  The problem is that I can't get
the test suite to run cleanly during package build.  I'm seeing two failures
in testing/test_genscript.py which causes the build to fail (or, if I ignore
that, prevents pybuild from running the test for Python 3.4 and 3.5).

I've tried running the tests via tox and directly invoking the tox.ini
command, as you'll see in the current d/rules file.  I'm at a loss, but then
pytest's output has always boggled me.  FWIW, when I run tox in the upstream
repo from the 2.8.5 tag, it works just fine.  There's something about the
build environment that's breaking the test suite, but I can't figure it out.

Note too that pytest does not seem to clean up properly after itself, thus the
mktemp in d/rules.  That at least makes it a little easier to clean up the /tmp
directories after an unsucessful sbuild.

I also added a DEP-8 test but haven't been able to get that working either.

Please, if you want to see pytest 2.8.5 in Debian, give it a look.  All help
and suggestions welcome.

Cheers,
-Barry


pgp3h6GSVOh4c.pgp
Description: OpenPGP digital signature

Re: Rebuild for packages with entry points?

[Barry Warsaw]

On Dec 08, 2015, at 08:48 AM, Nikolaus Rath wrote:

>Aeh, you know about a bug and you want to delay fixing it until someone
>has reporeds it for every affected package? This seems like a pretty
>inconsiderate waste of time for both users and maintainers.

There are always lots of bugs that affect people.  But hey, don't let me stop
you if you want to fix this particular one.

Cheers,
-Barry

Re: Rebuild for packages with entry points?

[Barry Warsaw]

On Dec 07, 2015, at 10:22 AM, Nikolaus Rath wrote:

>Would it make sense to do a no-change rebuild for all Python packages
>that use setuptool's entry point functionality?
>
>It'd be nice to have https://bitbucket.org/pypa/setuptools/issues/443/
>fixed in stretch. I believe most packages will see new releases anyway
>(and thus get the change), but I believe there are at least some
>packages that are rarely touched...

I'm also not sure how many packages it affects in practice.  We could also let
rebuilds be bug-driven.

Cheers,
-Barry

vim & Python 3 (was Re: vim-khuno: live Python checking within vim via flakes (in testing))

[Barry Warsaw]

On Dec 03, 2015, at 08:45 AM, Edward Betts wrote:

>I've packaged a vim plugin that provides live code checking for Python 2 and 3
>using pyflakes.

Speaking of vim, some work needs to be done along the port-to-Python-3 path.
I think vim still only builds against Python 2.  As part of this issue:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=729924

I submitted a (probably out-of-date) patch to enable Python 3, but the Debian
maintainer has some valid concerns.  I'm not a vim user so I really don't know
how people expect vim+Python to work, and whether it's actually feasible to
selectively enable Python 2 or Python 3 support.  Ideally (from a Python 3
p.o.v.) it would be possible to ship vim and not require any Python 2 to be
installed by default.

If you're a vim and Python fan, please take a look at the above issue and any
help you can provide to move this forward would be greatly appreciated.

Cheers,
-Barry

git-dpm tag possible gotcha

[Barry Warsaw]

I've occasionally seen this when I try to `git-dpm tag`:

git-dpm: ERROR: 'upstream' differs from recorded one!

In the past I've just done `git-dpm tag --refresh-upstream` but I think that's
not the right thing to do.  I think this means that you've `git pull`d while
in master and the upstream branch is still behind (i.e. it's been fetched but
not pulled).  Sure enough, when I switched to the upstream branch I saw:

Switched to branch 'upstream'
Your branch is behind 'origin/upstream' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)

I git pulled while in upstream, then checkout master and re-did the `git-dpm
tag`.  This time all was happy.

FYI I suppose.
-Barry


pgpfU1nLu4nDS.pgp
Description: OpenPGP digital signature

Re: How to maintain multiple branches (sid/bpo/exp etc)?

[Barry Warsaw]

On Nov 02, 2015, at 09:51 PM, Sandro Tosi wrote:

>with the new DPMT repo layout and tools, what is the right way to
>maintain multiple active branches for our packages? things like:
>
>1. unstable at v(ersion)3, bpo70 at v1 and bpo8 at v2
>2. unstable at v1, experimental at v2
>
>all of them active, so in the 2. case there should be a way to keep
>updating v1 and v2 releases (and so unstable and exp) as the come.

I'm kind of doing something like this for python-pip.  I'm (slowly) working on
updating the whole stack to the latest upstream, but I'm also maintaining
1.5.6.  I just created a release-1.5.6 branch and switch to that when I'm
doing stable maintenance, then switch back to master when I'm working on the
new upstream.

I have an upstream-release-1.5.6 and of course when I `git-dpm c-p` on the
release-1.5.6 branch, I end up in a patched-release-1.5.6 branch.  Really, I
don't do much special other than switch to the relevant branch; all other
workflows seem to continue to do the right thing.

Cheers,
-Barry

Re: How to maintain multiple branches (sid/bpo/exp etc)?

[Barry Warsaw]

On Nov 02, 2015, at 11:07 PM, Sandro Tosi wrote:

>I dont think I follow your description: looking at the debian repo, stable
>has 1.5.6-5 while testing and unstable has 1.5.6-7; looking at the git repo
>(as of http://anonscm.debian.org/cgit/python-modules/packages/python-pip.git)
>all the branches are from 5 months ago except for master which is 3 weeks
>old. so how do you make it work?

So I'll preface this by saying I suck.  I should really adopt the DEP-14
conventions so it's less confusing *and* document this in d/README.source.
release-1.5.6 is for uploads to unstable and the last push three months ago on
that branch was for 1.5.6-7.  master is for what will be the new upstream
whenever I get more time to work on it.  release-1.5.6 branch is at least
correct:

http://anonscm.debian.org/cgit/python-modules/packages/python-pip.git/tree/debian/changelog?h=release-1.5.6

So it's not completely analogous to your case; I just wanted to point out that
you can have whatever branches you want for other releases, and git-dpm should
continue to Just Work.

>say I have v1.0 packaged in master for unstable, and want to package
>v2.0 for experimental: should I created a release-2.0 (and a relative
>upstream-release-2.0 (what are the commands to obtain it, btw?)) and
>then doing the exp packaging there? how do I manage when I want to
>upload v2.0 to unstable? merge release-2.0 to master,
>upstream-release-2.0 to upstream? what if then I want to backport a
>previous revision of 1.0 to jessie?
>
>they might seem rare situations, but I face them everyday doing
>backports for work and wanting to upload them to debian as well.

Raphael can probably say more about this, and advocate for DEP-14, which in
your case probably makes more sense than I have:

http://dep.debian.net/deps/dep14/

It's probably worth some experimentation to see if DEP-14 is the right
approach for you, and how that might inform updates to DPMT policy at some
point.  Just remember to document it in README.source. ;)

trying-to-be-less-unhelpful-ly y'rs,
-Barry

DPMT policy update

[Barry Warsaw]

Since comments have stopped, I've updated DPMT policy:

http://python-modules.alioth.debian.org/policy.html

so now (g)it's official.

Cheers,
-Barry


pgpJPM1vPmxy6.pgp
Description: OpenPGP digital signature

Re: How to maintain multiple branches (sid/bpo/exp etc)?

[Barry Warsaw]

On Nov 03, 2015, at 12:03 AM, Sandro Tosi wrote:

>1. I have a simple package, which I keep maintaining with the usual
>'master' and 'upstream' branches, but then the testing freeze will
>come and I want to start uploading to experimental, at that point I
>need to change the layout of the repo (annoying, but I am the
>maintainer, I can do that I guess)

If you don't think you'll need to upload to unstable during the freeze, then I
think it's probably okay to put experimental changes on master.  I did this
during the last freeze and it worked out okay.  I guess it's a bit of a gamble
though.

>2. what if I am not the package maintainer tough? I cannot enforce a
>new layout to that repo. take this real-life example sqlalchemy: I
>asked Piotr for a jessie backport of 1.0.8, he agreed I can go ahead,
>I have a package ready and I dont know how to commit it to the repo.
>what should I do at this point?

I guess the maintainer has to give you permission to rename the branches, or
at least to add a debian/jessie branch for the backport.

Cheers,
-Barry

Re: dh-python (pybuild + dh_py*) documentation

[Barry Warsaw]

On Oct 28, 2015, at 05:59 PM, Piotr Ożarowski wrote:

>Each day I choose a victim and enable Piotr's evil mode.

Beware the Eye of Piotr.

:)

-Barry

Re: dh-python (pybuild + dh_py*) documentation

[Barry Warsaw]

On Oct 26, 2015, at 06:28 PM, Raphael Hertzog wrote:

>I for one lack a high level presentation of how the various bits work together
>and a clear description of the "magic" behind each tool.

I agree with this.  When you know where to look, the documentation is actually
quite good.  Maybe there are a few things here or there that could be filled
in or improved, but it's all generally there.

The problem is that when things don't build as expected, it's difficult to
understand which magical piece is misbehaving.

Cheers,
-Barry

Re: RFS: numpydoc 0.5-1

[Barry Warsaw]

On Oct 26, 2015, at 12:33 PM, Christoph Groth wrote:

>Denis Laxalde wrote:
>
>> I'd like to request sponsorship for numpydoc, which had a new > upstream 
>> release, the first supporting Python 3. It's available > in python-modules 
>> SVN repository (I can upload it on mentors if > that helps).
>
>Could you please reconsider sponsoring this upload?  It would close bug 
>801947.  The packaging is available in 
>http://anonscm.debian.org/cgit/python-modules/packages/numpydoc.git.

I'm looking at this now.

Cheers,
-Barry


pgpJxzF3csbVD.pgp
Description: OpenPGP digital signature

Re: pristine-tar (was: Git migration schedule)

[Barry Warsaw]

On Oct 22, 2015, at 01:04 PM, W. Martin Borgert wrote:

>So far, it just seems to work for (most of?) us.

Yep, pretty much.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 22, 2015, at 09:09 AM, Raphael Hertzog wrote:

>Yeah :( That makes another point that was missed in the evaluation of
>git-dpm vs git-buildpackage and its "gpb pq" command.

When we started down this road, `gbp pq` was pretty unusable and git-dpm was
much better (IMHO, others who did provide feedback).  It wouldn't be
surprising if it's come a long way since then.  However I echo Piotr's plea:
if you want to propose changing this, please do the work!  Until and unless
git-dpm breaks down, that won't be me.

Cheers,
-Barry

Re: Python Policy

[Barry Warsaw]

On Oct 22, 2015, at 11:14 AM, IOhannes m zmölnig (Debian/GNU) wrote:

>thanks for gender neutral wording.  however, you missed one "his" in the
>first sentence (probably more in other paragraphs).

Got it, thanks.
-Barry


pgpm4DkniheG1.pgp
Description: OpenPGP digital signature

Re: Python Policy

[Barry Warsaw]

On Oct 22, 2015, at 11:11 AM, IOhannes m zmölnig (Debian/GNU) wrote:

>something else i wonder whether we shouldn't drop it, as i don't quite
>understand why it has to be in the policy.
>
>i *think* it's supposed to urge DDs into becoming team members, even though
>they can ("are able to") already write to the repos without asking for
>permissions.  but in fact for me it conveys the complete opposite: DDs can
>("are welcome to") add stuff to the repo, and they should only "request" if
>they want to commit themselves to the team.
>
>...which is probably a (not so) good loophole for any DD who just want their
>python packages under the DMPT umbrella without getting too involved (e.g. no
>sponsoring, maybe not even obey the the policy,...) :- (

Ultimately it reflects reality given that we won't (can't?) prohibit DDs from
write access without joining the team.

For example, if an NMU gets uploaded, I'd like for the DD to be able to commit
the change to the repo directly rather than have ask us one of us to do it for
them.  But any DD who really cares about Python should join the team, IMHO.
It's a pretty low barrier.

Cheers,
-Barry


pgpn44xbQRleU.pgp
Description: OpenPGP digital signature

PyPI wheels (was Re: Python Policy)

[Barry Warsaw]

On Oct 21, 2015, at 08:47 PM, Brian May wrote:

>in one case this is because upstream have only supplied a *.whl
>file on Pypi.

I'm *really* hoping that the PyPA will prohibit binary wheel-only uploads.
There is talk about source wheels, and if that happens we'll probably have to
adjust our tools to handle them, but in theory they shouldn't be that much
different than being able to handle a different compression or encapsulation
format.

Cheers,
-Barry

Re: DPMT Policy

[Barry Warsaw]

On Oct 21, 2015, at 02:06 PM, Piotr Ożarowski wrote:

>Only version to version upstream changes should be kept in the repository -
>complete upstream git history should be avoided. In order to make it
>easier to cherry-pick upstream commits as patches, add remote repository
>(example below).
> 
>  git remote add upstream_repo git://example.com/foo.git
>  git fetch upstream_repo
>  git-dpm checkout-patched
>  git cherry-pick any_upstream_commit
>  git-dpm update-patches

How about this:

"""
DPMT requires a pristine-tar branch, and only upstream tarballs can be used to
advance the upstream branch.  Complete upstream git history should be avoided
in the upstream branch.
"""

I like the cherry picking advice, so I added that to the wiki.

>I simply suggest to:
>
>s/, either as a source package for use with ``pbuilder``, ``sbuild``, etc. or 
>as a binary package directory/ (it can be configured to use sbuild, pbuilder, 
>etc.)/
>
>or remove this part completely

I removed the sentence about gbp.  That's a local decision so it doesn't need
to go in policy.

>nothing against this paragraph, just wanted to state that wiki will not have
>to mention it if we have a tool that configures new package the way policy
>wants.

+1

>> >I'm wondering about something that bit me recently: `gbp pull` instead
>> >of `git pull` - should we put that into policy or is wiki warning enough?
>> 
>> I think wiki is enough. It is possible to operate with just straight `git
>> pull` because it will still fetch all commits, but when you switch to one of
>> the other branches, you'll see that its head is out of date, and git will
>> prompt you to pull in that branch to update it.  `gbp pull` is mostly
>> convenience.
>
>gbp buildpackage will fail if you never switched to pristine-tar (and
>git didn't warn you about new commits). It wasn't obvious to me why
>it fails when it hit me for the first time.

Yep.  Again though, since this is a local decision, it should go in the wiki.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 21, 2015, at 08:36 AM, Sandro Tosi wrote:

>I need to backport quite some packages, and use often experimental to
>stage big packages new releases (think of numpy and matplotlib) so it
>is not a rare situation at all and it should be considered now that we
>are at the beginning of a new era and changes are easier

Are you advocating for adopting DEP-14 in all cases?

Cheers,
-Barry

Re: DPMT Policy

[Barry Warsaw]

On Oct 22, 2015, at 08:32 AM, Ben Finney wrote:

>+1 
>

GOML
-Barry

:)

Re: Python Policy

[Barry Warsaw]

Thanks for the feedback Piotr.  I've made all the changes you suggest, except
one.  I'll discuss that below and include an updated diff against master.

On Oct 19, 2015, at 11:26 PM, Piotr Ożarowski wrote:

>I'm against this change. If we want all team packages to follow some
>rules, these rules need to be in policy document, not on the wiki page.
>I don't want this page to be removed, policy can even point to it, but I
>want it to be crystal clear that the binding document is the policy, not
>any other document in the internet (even if it's very helpful).
>
>What am I missing here is a set of branch/tag names and procedures
>(f.e. I didn't know that git pull is not enough and I should gbp pull or
>update pristine-tar branch by hand) and debian/patches related set of rules
>(do we require git-dpm or can one use plain quilt of some other rules
>are followed?)

Here's my concern: I don't want too much duplication of information in
multiple locations.  That's a sure recipe for bitrot, and I know no one wants
to have to edit information in more than one place.

Until now, the wiki has been the more convenient place to make these changes,
but I agree in principle with your opinion that some of that information must
be captured in policy.  So here's what I suggest.

For standards we must all adhere to, such as branch and tag names, source-full
branches, the use of git-dpm, and maybe a few other points, these should go in
policy.rst.

The wiki page points to the policy document but doesn't otherwise describe
those details.  That way, there's only one place to change when/if these
standards change.

The wiki then goes on to describe common workflows, how to create new
repositories, mr, etc.  These aren't specifically policy related but are
mostly there to help developers get started, or handle tricky situations.

If you're in agreement with that split, I'll update both the policy and wiki.

Cheers,
-Barry

diff --git a/policy.rst b/policy.rst
index c09f03a..aed57c0 100644
--- a/policy.rst
+++ b/policy.rst
@@ -1,19 +1,19 @@
-
- Python Modules Packaging Team - Policy
-
+=
+ Debian Python Modules Team - Policy
+=
 
-:Author: Gustavo Franco <stra...@debian.org>, Raphaël Hertzog 
<hert...@debian.org>
+:Author: Gustavo Franco <stra...@debian.org>, Raphaël Hertzog 
<hert...@debian.org>, Barry Warsaw <ba...@debian.org>
 :License: GNU GPL v2 or later
 
 :Introduction:
-  Python Modules Packaging Team aims to improve the python modules situation
-  in Debian, by packaging available modules that may be useful and providing
-  a central location for packages maintained by a team, hence improving
-  responsiveness, integration and standardization.
-
-  PMPT or just python-modules is hosted at alioth.debian.org, the Debian
-  GForge installation. We currently have a SVN repository and a mailing list
-  whose email address can be used in the Maintainer field on co-maintained
+  The Debian Python Modules Team (DPMT) aims to improve the Python modules
+  situation in Debian, by packaging available modules that may be useful and
+  providing a central location for packages maintained by a team, hence
+  improving responsiveness, integration, and standardization.
+
+  The DPMT is hosted at alioth.debian.org, the Debian GForge installation. We
+  currently have a git repository and a mailing list whose email address can
+  be used in the ``Maintainer`` or ``Uploaders`` fields on co-maintained
   packages.
 
   For more information send a message to: debian-python@lists.debian.org
@@ -24,16 +24,21 @@
 Joining the team
 
 
-The team is open to any python-related package maintainer. To be added on
-the team, please send your request on debian-python@lists.debian.org
-indicate why you want to join the team: maintain your current packages
-within the team, help maintain some specific packages, etc.
-Don't forget to indicate your Alioth login !
+The team is open to any Python-related package maintainer. To be added to
+the team, please send your request to debian-python@lists.debian.org.  Include
+the following in your request:
 
-Any Debian developer who wishes to integrate his packages in the team can do so
-without requesting access (as the repository is writable by all DD). If one
+* Why you want to join the team: e.g. maintain your current packages within
+  the team, help maintain some specific packages, etc.
+* Your Alioth login.
+* A statement that you have read
+  https://python-modules.alioth.debian.org/policy.html and that you accept it.
+
+Any Debian developer who wishes to integrate his packages in the team can do
+so without requesting access (as the repository is writable by all DD). If one
 wants to be more involved in the team, we still recommend requesting_ access
-so that he appears in 

Re: Python Policy

[Barry Warsaw]

On Oct 20, 2015, at 12:37 AM, Piotr Ożarowski wrote:

>should we also document that we're not OpenStack Packaging Team?

Or zope-packaging? .  Agreed that there are different teams here, but I
am hoping that we can do some consolidation.  E.g. I posted on the zope list
that I'd like to pull those packages into DPMT.  I heard *zero* responses, so
I'm honestly not sure there's still anybody who cares about that team.

(I'll post on the wider debian lists to follow up, but I will take silence as
assent at some point.)

>there is one HUGE difference, one is about packaging MODULES and the
>other one is packaging APPLICATIONS. One provides python-, python3-
>and/or pypy- packages, the other cannot do that.

Although there is often overlap.  Some packages intentionally provide both
libraries and applications.  These usually end up in DPMT, which I think is
fine.

I also think it would be fine to *eventually* merge the two teams.  I suspect
there isn't really much benefit to keeping them separate and a lot of
unnecessary cost.  Is there anybody on PAPT who doesn't want to be on DPMT?
Is there any reason why team policy couldn't be expanded to describe the few
differences between packaging libraries and pure-applications?

>> There are packages that do not provide public modules that are aimed at
>> developers. I imagine there are also packages that are end user
>> applications that do provide public modules, for end user
>> programming. These end user's may require the first group of packages
>> aimed at developers too.
>
>if something installs into dist-packages, it should (I'd make it a
>"must", but it's just me) provide python-/python3-/pypy- binary
>package. Python application should not (again, "must" is much better
>here IMO) pollute global Python namespace

I'm not sure I'd go as far as MUST, but aside from that, there's no inherent
reason IMHO why these two policies and procedures couldn't co-exist within the
same team.

Cheers,
-Barry

Re: Python Policy

[Barry Warsaw]

Latest diff against master.  If you're happy with this, I'll merge to master,
update the web page, and trim the wiki.

Cheers,
-Barry

diff --git a/policy.rst b/policy.rst
index c09f03a..123792c 100644
--- a/policy.rst
+++ b/policy.rst
@@ -1,39 +1,44 @@
-
- Python Modules Packaging Team - Policy
-
+=
+ Debian Python Modules Team - Policy
+=
 
-:Author: Gustavo Franco <stra...@debian.org>, Raphaël Hertzog 
<hert...@debian.org>
+:Author: Gustavo Franco <stra...@debian.org>, Raphaël Hertzog 
<hert...@debian.org>, Barry Warsaw <ba...@debian.org>
 :License: GNU GPL v2 or later
 
 :Introduction:
-  Python Modules Packaging Team aims to improve the python modules situation
-  in Debian, by packaging available modules that may be useful and providing
-  a central location for packages maintained by a team, hence improving
-  responsiveness, integration and standardization.
-
-  PMPT or just python-modules is hosted at alioth.debian.org, the Debian
-  GForge installation. We currently have a SVN repository and a mailing list
-  whose email address can be used in the Maintainer field on co-maintained
+  The Debian Python Modules Team (DPMT) aims to improve the Python modules
+  situation in Debian, by packaging available modules that may be useful and
+  providing a central location for packages maintained by a team, hence
+  improving responsiveness, integration, and standardization.
+
+  The DPMT is hosted at alioth.debian.org, the Debian GForge installation. We
+  currently have a git repository and a mailing list whose email address can
+  be used in the ``Maintainer`` or ``Uploaders`` fields on co-maintained
   packages.
 
   For more information send a message to: debian-python@lists.debian.org
 
 .. contents::
 
-
+
 Joining the team
-
+
 
-The team is open to any python-related package maintainer. To be added on
-the team, please send your request on debian-python@lists.debian.org
-indicate why you want to join the team: maintain your current packages
-within the team, help maintain some specific packages, etc.
-Don't forget to indicate your Alioth login !
+The team is open to any Python-related package maintainer. To be added to
+the team, please send your request to debian-python@lists.debian.org.  Include
+the following in your request:
 
-Any Debian developer who wishes to integrate his packages in the team can do so
-without requesting access (as the repository is writable by all DD). If one
+* Why you want to join the team: e.g. maintain your current packages within
+  the team, help maintain some specific packages, etc.
+* Your Alioth login.
+* A statement that you have read
+  https://python-modules.alioth.debian.org/policy.html and that you accept it.
+
+Any Debian developer who wishes to integrate his packages in the team can do
+so without requesting access (as the repository is writable by all DD). If one
 wants to be more involved in the team, we still recommend requesting_ access
-so that he appears in the public member list displayed on Alioth's project 
page.
+so that they appear in the public member list displayed on Alioth's project
+page.
 
 The team accepts all contributors and is not restricted to Debian developers.
 Several Debian developers of the team will gladly sponsor packages of non-DD
@@ -43,91 +48,129 @@ discussion list or on #debian-python IRC channel (OFTC 
network).
 All team members should of course follow the main discussion list:
 debian-python@lists.debian.org
 
---
+
 Maintainership
---
+==
 
 A package maintained within the team should have the name of the team either
-in the Maintainer field or in the Uploaders field.
+in the ``Maintainer`` field or in the ``Uploaders`` field.
 
 Maintainer: Debian Python Modules Team 
<python-modules-t...@lists.alioth.debian.org>
 
 This enables the team to have an overview of its packages on the DDPO_website_.
 
-* Team in Maintainers is a strong statement that fully collaborative
-  maintenance is preferred. Anyone can commit to the vcs and upload as
-  needed. A courtesy email to Uploaders can be nice but not required.
+* Team in ``Maintainers`` is a strong statement that fully collaborative
+  maintenance is preferred. Anyone can commit to the git repository and upload
+  as needed. A courtesy email to ``Uploaders`` can be nice but not required.
 
-* Team in Uploaders is a weak statement of collaboration. Help in maintaining
-  the package is appreciated, commits to vcs are freely welcomed, but before
-  uploading, please contact the Maintainer for the green light.
+* Team in ``Uploaders`` is a weak statement of collaboration. Help in
+  maintaining the package is appreciated, commits to the git repository are
+  freely welcomed, but before upload

Re: Python Policy

[Barry Warsaw]

On Oct 20, 2015, at 05:16 PM, Piotr Ożarowski wrote:

>I will leave this team the moment I have to read README.sources each day when
>I sponsor a package.

Nobody wants that! (either you leaving or having to read README.source for
every package).

Cheers,
-Barry

Re: Python Policy

[Barry Warsaw]

On Oct 21, 2015, at 11:17 AM, Ben Finney wrote:

>On the contrary, I think the Policy document should document the
>rationale for contingent decisions like this. When it is inevitably
>discussed again in the future, it is always better to know the intent of
>the authors.

+1

Cheers,
-Barry

Re: Python Policy

[Barry Warsaw]

On Oct 20, 2015, at 11:30 PM, Piotr Ożarowski wrote:

>I have few comments, but even if I didn't, please wait at least until after
>the weekend (or better: 7 days) so that others have time to review it and
>comment / propose changes.

Fair enough.  Of course, it's in a vcs so it's easy to change! :)

>I'd remove this paragraph. Releases can be made via `git archive` and I did
>that many times (assuming pristine-tar will still keep needed data to
>regenerate exact same tarball).  If you meant that we don't want to keep
>complete upstream git history, then I agree completely, but I'd made it a
>"should" rather than "must".

What I'm trying to express is the team decision (a couple of debconfs ago) for
pristine-tars rather than releasing from upstream git.  I do want to keep the
rationale in the policy doc; it's one sentence and it seems to come up often
enough.  Suggestions for better phrasing welcome!

>> +``gbp build-package`` is used to build the package, either as a source
>> package
>
>s/build-package/buildpackage/

Fixed, thanks.

>> +for use with ``pbuilder``, ``sbuild``, etc. or as a binary package 
>> directory.
>
>gbp can use sbuild/pbuilder, here's my ~/.gbp.conf:
>
>  [buildpackage]
>  builder=sbuild

This is the kind of thing that should go in the wiki.

>> +Use the following ``git-dpm`` tag formats for the three branches named 
>> above.
>> +Put these lines at the *end* of your ``debian/.git-dpm`` file::
>> +
>> +debianTag="debian/%e%v"
>> +patchedTag="patches/%e%v"
>> +upstreamTag="upstream/%e%u"
>
>I will update `py2dsp --profile dpmt ...` to do that out of the box, but
>even then, it's better to suggest that tool in the wiki only, I guess

I think so.  The tag format (and IMHO the mechanism to ensure it) should go in
policy though.

>> +All packages which have been automatically converted from the old
>> Subversion +repository should already have these lines present, but you
>> will need to add +them for any new packages.
>
>that's something for the wiki, not policy, IMO

Sure.  I reworded the policy docs a little bit here.

>other comment:
>I'm wondering about something that bit me recently: `gbp pull` instead
>of `git pull` - should we put that into policy or is wiki warning enough?

I think wiki is enough.  It is possible to operate with just straight `git
pull` because it will still fetch all commits, but when you switch to one of
the other branches, you'll see that its head is out of date, and git will
prompt you to pull in that branch to update it.  `gbp pull` is mostly
convenience.

Cheers,
-Barry

Re: git instead of svn in DPMT policy

[Barry Warsaw]

On Oct 17, 2015, at 08:49 PM, Piotr Ożarowski wrote:

>I moved policy.rst to python-modules.git repo (it was available to edit
>by every team member in SVN as well). It's not OK to edit it and push it
>without having a discussion here (on this mailing list) first, though.

I will make a pass through the file and push a non-master branch for review.

Cheers,
-Barry


pgpLjPoGAQPma.pgp
Description: OpenPGP digital signature

Python Policy

[Barry Warsaw]

So we currently have several places where we have team policy described.

* The Debian wiki
  https://wiki.debian.org/Python and subpages

* Another wiki page:
  https://wiki.debian.org/Teams/PythonModulesTeam

* https://www.debian.org/doc/packaging-manuals/python-policy/
  which comes from the python-defaults (*not* python3-defaults!) in the bzr
  repo at
  http://alioth.debian.org/anonscm/bzr/pkg-python/python-defaults-debian

* "PMPT" policy
  http://python-modules.alioth.debian.org/
  git+ssh://git.debian.org/git/python-modules/tools/python-modules.git
  except that that page specifically isn't represented in the git repo, only
  policy.rst

and maybe more.  This is crazy!  We really need to consolidate all this
information.

That said, I did a pass through policy.rst in the git repo above and pushed
the 'git-policy' branch for review.  `git diff master` to see the proposed
changes.  Here's a summary:

* PMPT -> DPMT - it seems like we mostly refer to ourselves as Debian Python
  Modules Team and rarely if ever Python Modules Packaging Team any more.
  Another opportunity for more consistency.

* Clarified slightly the wording in policy differences between
  team-in-Maintainer and team-in-Uploaders.

* Remove all the references to Subversion and add a short blurb about Git,
  pointing to the GitPackaging wiki page.

Cheers,
-Barry


pgpwsh5ZOfqhd.pgp
Description: OpenPGP digital signature

Re: Git migration schedule

[Barry Warsaw]

On Oct 18, 2015, at 07:19 PM, Jean-Michel Vourgère wrote:

>Git multiple remotes is a nice feature. We can plug right into upstream
>tree.

Currently, our git workflow is tarball-based, since we primarily package PyPI
releases, which are tarball-centric, and because orig.tar is required for
uploads.  We've had debates about whether to release from upstream git
revisions, but currently consensus is against that model.

We value consistency across our packages.  TOOWTDI.

My personal opinion is that we should live with the current git workflow
recommendations for a while and see how it goes.  If there are things we can
improve on (e.g. DEP-14 compatibility) then sure, let's discuss the pros and
cons, but let's not change things right now.  I'd like to see how it works in
practice for a while.

>You labelled the wiki as "official" and use terms like "must". I dislike
>that.

I think everyone will agree that we want consistency within the team.  You
want to be able to walk up to any package and just know how it is set up in
git.  That means following the team guidelines -- but there's an escape hatch
for when you really have to (not want to) diverge from the team.  You write a
debian/README.source explaining the rationale and difference.

>Also, for patches, I'm used to source format 3.0 (quilt). Is that bad now?

No.  In the master branch you will see quilt formatted patches.  It's just
that we're saying that the git repo must be compatible with git-dpm, which is
a patch management regime on top of git and gbp.  If you need to generate a
new patch against upstream, we recommend git-dpm to do so.

Now, in practice, it doesn't matter if you ignore git-dpm and just use quilt
*as long as the final state of the repo is compatible with git-dpm*.  Meaning,
in general, you can make whatever local decisions you want as long as they
don't force other team members to go outside of team recommendations.

Think of it like this: nobody cares if you use vim, emacs, or some other
editor because it doesn't affect anybody else.  Nobody cares if you use sbuild
or pbuilder or something else to build your packages for upload because it
doesn't affect anybody else.  But we all have to agree on tag names, branch
names, etc. because those *do* affect everyone.

So if you just use quilt, I don't care, as long as I can walk up to the same
repo and use git-dpm.  Local differences are fine, global differences are not.

>I'm not using git-buildpakage, but a simple debuild (tests) or sbuild
>(for final uploads). sbuild is the tool used by buildd on the official
>packages. I can't begin to understand why it would be unacceptable.

I use sbuild and pbuilder in different contexts.  Again, that's a local
decision so nobody cares.  (I personally always build to source package first
because that gives me maximum flexibility, e.g. to run autopkgtests or
whatever, but that's just me.)

>So basically, I'm out of line. Please soften the new rules.

Not yet please.  We've been using git for 10 days.  Let's give it time to
settle down.

>Can someone point me to some advantages of using gpb?

Again, much of this is a local decision.  Use whatever you want as long as it
doesn't affect other team members.  Where you have to create artifacts such as
tag names that are visible outside your local environment, they must conform
to team conventions, otherwise that defeats much of the purpose of being in
the team.

Cheers,
-Barry

Re: pybuild: passing {dir}/tests to nose/pytest by default duplicated binary

[Barry Warsaw]

On Oct 14, 2015, at 04:28 PM, Piotr Ożarowski wrote:

>[Barry Warsaw, 2015-10-14]
>> On Oct 14, 2015, at 01:26 PM, Piotr Ożarowski wrote:
>> 
>> >> export PYBUILD_TEST_ARGS={dir}/tests
>> >
>> >should I do that by default in pybuild if
>> >* "test" or "tests" directory is detected
>> >* PYBUILD_TEST_ARGS is not set
>> >* nose or pytest test suite is used
>> >
>> >?
>> 
>> Maybe just on the first two conditions, only if it's documented, and if
>
>does `{interpreter} setup.py test` or `{interpreter} -m unittest discover -v`
>(always? usually?) take dir as parameter?
>
>If not, then I'd keep it for nose/pytest only.

Oh sorry, I think I misunderstood.  The conditions above are ANDed not ORed. :)

Cheers,
-Barry

Re: pybuild: passing {dir}/tests to nose/pytest by default duplicated binary

[Barry Warsaw]

On Oct 14, 2015, at 01:26 PM, Piotr Ożarowski wrote:

>> export PYBUILD_TEST_ARGS={dir}/tests
>
>should I do that by default in pybuild if
>* "test" or "tests" directory is detected
>* PYBUILD_TEST_ARGS is not set
>* nose or pytest test suite is used
>
>?

Maybe just on the first two conditions, only if it's documented, and if
there's an easy way to disable it for corner cases.

Cheers,
-Barry

Re: git repo lint tool

[Barry Warsaw]

On Oct 13, 2015, at 04:30 PM, Stefano Rivera wrote:

>Many of the non-migrated git repos are a bit of a mess. I've written a
>tool that looks for common problems.
>
>It's on alioth:
>$ cd /git/python-modules
>$ ./check-repositories

Nice, thanks.

>codespeak-lib: Non-Canonical Vcs fields

I think the codespeak-lib source package is obsolete.

>python-pex: Non-Canonical Vcs fields

I guess this means that the Vcs-* headers are not of the format defined in
https://wiki.debian.org/Python/GitPackaging right?  I fixed python-pex (not
yet uploaded).

>Barry Warsaw <ba...@debian.org>
>   lazr.config
>   lazr.delegates
>   lazr.smtptest
>   pycurl
>   python-cachecontrol
>   python-future (U)
>   python-iso8601 (U)
>   python-pex
>   python-pip (U)
>   python-pluggy
>   tox

Barry is a bad person.

Cheers,
-Barry, not really evil

Re: Request to join

[Barry Warsaw]

On Oct 12, 2015, at 11:57 AM, Piotr Ożarowski wrote:

>transition is not over, our policy still mentions SVN only...

Unless I'm missing it, not in python-policy.sgml (Debian Python Policy) though.

Cheers,
-Barry

Re: warnings importing new upstream source

[Barry Warsaw]

On Oct 12, 2015, at 05:27 AM, Brian May wrote:

>As far as I know this actually did work despite the complaining about no
>parent commit... What does that mean?

No idea, but I've seen it before.  I don't think it has any negative effect.

>git-dpm import-new-upstream --ptc --rebase-patched
>../django-model-utils_2.3.1.orig.tar.gz
>There were no patches recorded, so merging the new state directly (no
>git-dpm rebase-patched needed).
>git-dpm: WARNING: No parent commit specified.
>This means the upstream branch was created from the .orig.tar with only
>theold upstream branch as parent (but not any possible upstream git
>history).
>Use 'git config dpm.importWithoutParent true' to no longer show this
>warning.
>or 'git config dpm.importWithoutParent false' to make this an hard error.
>calling pristine-tar commit '../django-model-utils_2.3.1.orig.tar.gz'
>'1b7b962b83568d87570e8865fb02b1d3331deda2'...
>pristine-tar: committed django-model-utils_2.3.1.orig.tar.gz.delta to
>branch pristine-tar

Cheers,
-Barry

Re: errors pushing tags to git

[Barry Warsaw]

On Oct 12, 2015, at 05:22 AM, Brian May wrote:

>git push --tags
>Total 0 (delta 0), reused 0 (delta 0)
>remote: Sending notification emails to:
>python-modules-comm...@lists.alioth.debian.org,
>python-django_...@packages.qa.debian.org
>remote: Sending notification emails to:
>python-modules-comm...@lists.alioth.debian.org,
>python-django_...@packages.qa.debian.org
>remote: Sending notification emails to:
>python-modules-comm...@lists.alioth.debian.org,
>python-django_...@packages.qa.debian.org
>remote: fatal: Invalid revision range
>..e2234da325c5962b54559e6c3d75dc4c116853f5
>remote: fatal: Invalid revision range
>..cdf0cc9f72f1847763290c018038990b3bcec4c1
>remote: fatal: Invalid revision range
>..e2f348a8f03d3f935e0233d7a747be5fd99c796f
>To git+ssh://git.debian.org/git/python-modules/packages/python-django.git
> * [new tag] debian/1.7.10-1 -> debian/1.7.10-1
> * [new tag] patched/1.7.10-1 -> patched/1.7.10-1
> * [new tag] upstream/1.7.10 -> upstream/1.7.10
>
>I think it worked...

Yep.  Again, I've seen this, but don't know the cause and don't think it
actually has any negative effect.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 10, 2015, at 12:18 PM, Ben Finney wrote:

>You need to specify which username to connect to over SSH. I have
>updated the Wiki page above to clarify this.

Hmm, I'm not sure about this recommendation.  I don't include my user name in
the url, and I'm pretty sure Mattia's suggestion to set this in your
~/.ssh/config is the right one.

I don't want to wiki-churn so what do you think about backing out this change
and suggesting the ssh config instead?

Cheers,
-Barry


pgpUgEtolbRC8.pgp
Description: OpenPGP digital signature

Re: python-django

[Barry Warsaw]

On Oct 10, 2015, at 09:13 AM, Raphael Hertzog wrote:

>Yes, except for the naming of branches where I would prefer to keep the
>DEP-14 naming scheme (we should just rename debian/sid into debian/master).
>
>http://dep.debian.net/deps/dep14/
>
>And I would suggest that we generalize the DEP-14 naming scheme as part of
>our git packaging policy...

I'm all for standardization, but I do like the shorter names for the common
case where you only need the unstable version.  Certainly if there are vendor
or series differences, the namespaces make a lot of sense.  But most of my
packages don't need it.

Would you be open to allowing such simplification in the common case in
DEP-14?

Cheers,
-Barry

Re: tagging releases in git

[Barry Warsaw]

On Oct 10, 2015, at 04:19 AM, Brian May wrote:

>According to the wiki I do this with the following command:
>
>brian@prune:~/tree/debian/python-modules/django-ajax-selects$ git-dpm tag
>git-dpm: ERROR: tag 'upstream/1.3.6' already exists and differs!
>
>This wasn't the response I was expecting. I am not sure why it is trying to
>change the upstream tag, or what it is trying to change it to. Should I
>just create the debian/1.3.6-4 tag manually?

What I've done, and it's probably horrible, is to just `git-dpm tag --refresh`
and then `git push --tags --force`.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 09, 2015, at 10:30 PM, Brian May wrote:

>Ok, I probably should create another thread to discuss this for Django then.
>
>Also, contrary to the rules we just agreed on, this sounds like one rare
>time when all uploaders should be contacted before moving any repositories
>around.

Probably so.  I have a checkout of python-django here and it seems to already
be in git.  I haven't looked at the converted svn->git repo yet, but I'm
wondering if what was converted was just old and out of date?  I'm pretty sure
it was in git to manage the 1.8 transition.  The debian/sid branch has 1.7.9-1
at its top while debian/experimental has UNRELEASED 1.8.3-2.

That's probably necessary given the complexity of python-django, but since it
deviates from GitPackaging, there should be a debian/README.source to explain
to other team members how the repo is supposed to be used.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 09, 2015, at 06:46 PM, Arthur de Jong wrote:

>Perhaps an email to d-d-announce would be in order.

Good idea, thanks.

-Barry


pgpt4EKqpYkKX.pgp
Description: OpenPGP digital signature

Re: Git migration schedule

[Barry Warsaw]

On Oct 09, 2015, at 10:01 AM, Stefano Rivera wrote:

>And it's done.

\o/

Thank you for all your amazing work on this Stefano!

>Things that need to be looked at:
>http://whiteboard.debian.net/dpmt-git-migration.wb
>
>Please mark them off if you've looked at them.

I've done an updating pass through the wiki page:

https://wiki.debian.org/Python/GitPackaging

and marked it "official".  Please pay special attention to the section on
post-migration clean ups that folks can help with:

https://wiki.debian.org/Python/GitPackaging#Post-migration_clean_up

I'd also like to send an email to debian-devel@ inviting people who may have
abandoned the DPMT because of our use of subversion, to come back to the team.

Cheers,
-Barry

Re: Python < 3.5 tests

[Barry Warsaw]

On Oct 08, 2015, at 11:15 AM, Brian May wrote:

>Maybe in this case I should file a bug report however.

I do this when the PyPI tarball is missing some important file or directory.
Fortunately the upstreams I've done this with have generally been very
responsive about fixing the problem and doing a new release.  On rare
occasions as a temporary workaround, I grab the missing file from their vcs
and arrange to have it installed properly during package build.

Cheers,
-Barry

Re: Python < 3.5 tests

[Barry Warsaw]

On Oct 08, 2015, at 11:53 PM, Piotr Ożarowski wrote:

>[Barry Warsaw, 2015-10-08]
>> For --buildsystem=pybuild, I've done this:
>> 
>> override_dh_auto_test:
>>  PYBUILD_SYSTEM=custom \
>>  PYBUILD_TEST_ARGS="{interpreter} -m nose2 -vv" \
>>  dh_auto_test
>
>override_dh_auto_test:
>   dh_auto_test -- --system=custom --test-args="{interpreter} -m nose2 -vv"
>
>if you want to keep it shorter

Thanks!  I cut-n-pasted that from an older packaging example.

Cheers,
-Barry

Re: Python < 3.5 tests

[Barry Warsaw]

On Oct 08, 2015, at 09:19 PM, Brian May wrote:

>What is the best way of calling unittest2 from debian/rules?

For --buildsystem=pybuild, I've done this:

override_dh_auto_test:
PYBUILD_SYSTEM=custom \
PYBUILD_TEST_ARGS="{interpreter} -m nose2 -vv" \
dh_auto_test

Should be easy to tweak for your case.

Cheers,
-Barry

Re: Python/LibraryStyleGuide: Executables and library packages dokonana przez BarryWarsaw

[Barry Warsaw]

Thanks for the feedback.  I did rewrite this a little bit, so hopefully it's
clearer.  I left some of the text in there because at least to me it reads
better and provides some rationale for why the rules are there.  But hey, it's
a wiki so please feel free to make further improvements!

Cheers,
-Barry

On Oct 07, 2015, at 11:34 PM, Piotr Ożarowski wrote:

>[Debian Wiki, 2015-10-07]
>> https://wiki.debian.org/Python/LibraryStyleGuide?action=diff=64=65
>> 
>>   
>>   == Gotchas ==
>>   
>> + === Executables and library packages ===
>> + 
>> + Let's say you have a Python package which results in Python 2 and 3 
>> libraries, and a Python 3 executable.  What is the best practices for naming 
>> and organizing your binary packages?
>> + 
>> + Clearly you want to have:
>> + 
>> +  * python-foo -- the Python 2 library
>> +  * python3-foo -- the Python 3 library
>
> * foo -- for the executable (and possible additional dependencies that 
> library doesn't need)
>
>maybe extent it to:
>
> * python-foo -- the Python 2 library (and Python 2.X scripts if they're 
> Python 2.X specific)
> * python3-foo -- the Python 3 library (and Python 3.X scripts if they're 
> Python 3.X specific)
>
>> + 
>> + but where should the `/usr/bin/foo` script go?  You could put it in 
>> `python3-foo` but you '''CANNOT''' put it in `python-foo` or for that matter 
>> any binary package that starts with the `python-` prefix.  `dh_python3` 
>> refuses to rewrite shebang lines for any executable in a binary package that 
>> starts with "python-" or "pypy-".  This means that something like 
>> `python-foo-cli` or `python-foo-bin` is unacceptable.
>
>I'd remove this part - it's not dh_python3 specific (dh_python2 and
>dh_pypy does similar things) and I don't think such corner case should
>be in style guide
>
>> + 
>> + Here are some recommendations.  We do not have a standard (though maybe we 
>> should):
>> + 
>> +  * `foo` - this is nice if it parallels the /usr/bin/foo name but it might 
>> collide with existing packages, and some people don't like to make such a 
>> claim on an unadorned top level package
>> +  * `python3-foo-cli` or `python3-foo-bin` - not as nicely discoverable, 
>> but `command-not-found` can help, and dh_python3 will work
>
>if someone creates python3-foo-cli binary just to ship /usr/bin/foo it
>might as well be foo (if there are no /usr/bin/foo name collisions,
>binary package name should be safe as well) so I'd remove it


pgpT040WQ7qgL.pgp
Description: OpenPGP digital signature

linux-sig

[Barry Warsaw]

In the upstream Python project, we recently created a new mailing list as a
focal point for cross-distro Linux-specific issues.  I invite all interested
folks to join and help make Python better on Linux.

https://mail.python.org/mailman/listinfo/linux-sig

Feel free to spread the announcement of this mailing list far and wide.

Cheers,
-Barry


pgpBR8zCUlgw5.pgp
Description: OpenPGP digital signature

Re: team vs individual as maintainer (was: radical changes)

[Barry Warsaw]

On Oct 07, 2015, at 02:33 PM, W. Martin Borgert wrote:

>Quoting Piotr Ożarowski :
>> I assume you all like other ideas, like no team in Maintainer, right?
>
>To me, "team maintenance" would mean, that the team appears in the
>"Maintainer" field. In "Uploaders" it doesn't make sense, so where
>would the team appear?
>
>Personally, I prefer to have the team in "Maintainer" to encourage
>others to intervent.
>
>How about leaving it as is? I.e. people in the team decide what is
>more appropriate for a package?

I agree.  I think the recently restated way of interpreting those two fields
made the most sense to me, and gives team members the choice of how to handle
those packages.

* Team in Maintainers is a strong statement that fully collaborative
  maintenance is preferred.  Anyone can commit to the vcs and upload as
  needed.  A courtesy email to Uploaders can be nice but not required.

* Team in Uploaders is a weak statement of collaboration.  Help in maintaining
  the package is appreciated, commits to vcs are freely welcomed, but before
  uploading, please contact the Maintainer for the green light.

Cheers,
-Barry

Re: team vs individual as maintainer

[Barry Warsaw]

On Oct 07, 2015, at 03:29 PM, Piotr Ożarowski wrote:

>[Barry Warsaw, 2015-10-07]
>> * Team in Maintainers is a strong statement that fully collaborative
>>   maintenance is preferred.  Anyone can commit to the vcs and upload as
>>   needed.  A courtesy email to Uploaders can be nice but not required.
>> 
>> * Team in Uploaders is a weak statement of collaboration.  Help in 
>> maintaining
>>   the package is appreciated, commits to vcs are freely welcomed, but before
>>   uploading, please contact the Maintainer for the green light.
>
>how about making it official and adding it to the policy?

I thought you had volunteered to do that (with native speaker review)?  I
previously mistakenly remembered Scott volunteering for that.

If not, I'm happy to do it.

Cheers,
-Barry

Re: linux-sig

[Barry Warsaw]

On Oct 08, 2015, at 05:34 AM, Ben Finney wrote:

>Is it already available at GMane?

It's been requested and acknowledged, and I resent a message to kick off
creation of the newsgroup, but I don't see it there yet and the gmane.org
website seems offline for me atm.

Cheers,
-Barry

Re: managing transitions (was: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental)

[Barry Warsaw]

On Oct 06, 2015, at 08:39 AM, Brian May wrote:

>On Tue, 6 Oct 2015 at 18:46 Thomas Goirand  wrote:
>
>> This IMO is the same topic as having a Gerrit review system (and not
>> just Git) which could do tests on each change of a package even before
>> having them committed to our git.
>>
>
>Sounds like an interesting thing to discuss/test after we move to git...

I don't intend to bikeshed this, nor do I have time to do the work, so in our
do-it-ocracy any online review system would be a fantastic addition.  I'll
just point to GitLab community edition as a nice open source option in this
space.

Cheers,
-Barry

Re: managing transitions

[Barry Warsaw]

On Oct 06, 2015, at 07:05 PM, Thomas Goirand wrote:

>Interesting. It's the first time I hear about it, I thought it was just
>closed source.

The instance at gitlab.com is the non-free Enterprise Edition (EE).  EE has
features we probably don't care about ayway.  The Community Edition (CE) is
MIT/Expat.

>Does it have test capabilities as well, so that we could
>run tests on each new patch, like Gerrit does? Can we plug zuul and
>nodepool to it?

I don't know about plugging in zuul and nodepool, but the CE does have CI
integration, which we use in the GNU Mailman project, and seems to work
great.

Cheers,
-Barry

Re: managing transitions

[Barry Warsaw]

On Oct 06, 2015, at 03:12 PM, Fred Drake wrote:

>What CI tools are you using with GitLab CE?

We don't run CE; we use the hosted EE at gitlab.com.  But anyway, we have a
custom VM on which we run the GitLab runner software in a docker image.  This
runs our test suite in all supported Python 3s against both SQLite and
PostgreSQL.  At the time we set things up they didn't have shared runners, but
now they do so I don't think you even need to provision a VM.

http://doc.gitlab.com/ci/

Cheers,
-Barry


pgp1V5NfV0r9R.pgp
Description: OpenPGP digital signature

Re: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental

[Barry Warsaw]

On Oct 05, 2015, at 02:51 PM, Thomas Goirand wrote:

>In other distributions (Red Hat and Ubuntu), everyone is aware of this
>kind of issue before uploading, and this kinds of things don't happen.

Ubuntu at least does have a technical solution that helps ameliorate
archive-wide breakages, and that is -proposed migration.  When you upload
e.g. to wily, it gets diverted to wily-proposed and to get promoted it has to
pass a number of tests.  The package and their reverses have to build.  DEP-8
tests have to pass, etc.  You can get a nice report about which -proposed
promotions are failing:

http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html

The downside is that you should probably be proactively checking this list
(poll vs ping) and it can sometimes be difficult to figure out why a promotion
fails or how to fix it.

But this does mean that the archive itself is very rarely broken, and it can
be a convenient way to stage package updates that may have effects in parts of
the archive you might not be aware of.

Cheers,
-Barry

Re: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental

[Barry Warsaw]

On Oct 05, 2015, at 10:57 AM, Scott Kitterman wrote:

>I agree that disabling package test suites doesn't improve their quality.  
>Were these bad tests?  Did you report these issues upstream?

Silently passing broken tests was one of a common pattern of issues I found
when making Python 3.5 supported in Ubuntu.  The tests were broken, and I
reported upstream or fixed the ones I found.  I was skeptical about this mock
change, and it did cause churn, but it was important for longer term
increasing the quality of the archive.

>Personally, even if the team was the maintainer of the package, I would never
>just upload something without giving a ping to anyone who was active as an
>uploader.  I think it's just polite, even if it goes beyond what the team
>strictly requires (note: I did this exact thing over the weekend for pyside,
>got a quick ping back and did a team upload - it's not that hard).
>
>If we can't get the social part of Debian right, the technical part gets very
>hard.  This is not a side issue.

Fully agreed, and I think it's a *good* thing we've been having this
discussion.  It makes me want to double check the assertions about
maintainership in the packages I touch, and it makes me be doubly conscience
of other maintainer's preferences here.

But let's be sure to capture these norms in Debian Python policy or the team
wiki pages.  I think Scott, you were going to propose some changes to policy
in this area?

Cheers,
-Barry

Re: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental

[Barry Warsaw]

On Oct 05, 2015, at 09:16 PM, Mattia Rizzolo wrote:

>Isn't this the whole point of unstable→testing?

I guess, although it seems a lot of people run unstable so breakages affect
more people.  I run unstable on most of my Debian machines.  I think almost
nobody actually runs -proposed.

Cheers,
-Barry


pgp2C7LN9TZaT.pgp
Description: OpenPGP digital signature

Re: [DPMT] radical changes: automation, carrot and stick

[Barry Warsaw]

On Sunday, October 04, 2015 11:54:18 PM Stefano Rivera wrote:

>There's a fundamental question to ask here. Do we want to welcome Python
>packages into the team, or do we want to put up barriers and require a
>level of commitment before packages can be brought into the team?

Thanks Stefano for stating this in such a clear way.  It is indeed a (*the*?)
fundamental question to ask about this team.

On Oct 05, 2015, at 07:23 AM, Scott Kitterman wrote:

>I think that you describe to reasonably accurate directions the team can go.  
>Personally, I prefer the "natural home for most Python packages in the 
>archive" vision.
>
>I think we should have a minimum of rules, but people should follow the ones 
>we do have.

+1 and +1

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 05, 2015, at 10:36 PM, Stefano Rivera wrote:

>How about: We move away existing repositories, and put the migrated ones
>in the /packages/ path. If people have existing repositories, that
>they'd prefer to use, they can move the migrated ones out the way, and
>theirs back. But they have to opt in to this.
>
>This means some (work done on pre-migration git packaging) history gets
>temporarily "lost". But ensures that everything is the same layout. And
>that any deviation was intentional, not accidental.

I think it would be okay.  It's only a minor inconvenience, although the git
remotes of the packages that get moved would also be temporarily incorrect.

Cheers,
-Barry

Re: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental

[Barry Warsaw]

On Oct 06, 2015, at 07:00 AM, Robert Collins wrote:

>The things you listed that I help maintain - mock, testtools, etc -
>are *not* OpenStack specific. They existed before OpenStack, and
>likely will exist after. They have other users, particularly mock
>which is very widely used.

I intensely dislike the separation between OS and DPMT, for exactly this
reason.  Too many packages of general use to Python developers is out of reach
of DPMT.  I thought it was mostly a vcs-induced separation, but now I think
it's clear that even after DPMT moves to git, this separation will continue.

The multi-version design of the archive does cause problems.  I outlined a big
thing that I think Ubuntu has that helps reduce the impact of this.  I'm not
sure if the same kind of this would help Debian, whether it would be feasible,
or even acceptable by the majority of Debian developers.

I really think we need to be finding ways to *reduce* the separation between
OS and DPMT.  One of the things I hope will happen after git migration is
subsuming as much as possible from zope-pkg into DPMT since again, there are a
lot of general interest packages in that namespace.

>Its entirely reasonable to say that known reverse dependencies should
>be considered when upgrading packages, but that is in no way OpenStack
>specific - and the release schedule of all of the things you listed is
>entirely independent of OpenStack.
>
>Its one of the defects of the single-version design of the archive
>that we have this uncontrolled use of new releases of software thats
>put into it, and - well, thats another discussion. We need to live
>with it though.

Right, and observe that it's not feasible to track down *all* reverse
dependencies when updating a single package.  That's part of why -proposed
migration is so nice, an automated system does that tracking for you.

>So I'm +1 on "Check reverse deps aren't significantly broken before
>uploading to unstable" as a general principle, not as an OpenStack
>specific thing.

+1 also, but it's always going to be a spot check for sanity so things will
fall through the cracks.  Until we get something like the promotion tests, we
just have to commit ourselves to being diligent within reason before
uploading, and responsible to help fix breakages after they're discovered.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 05, 2015, at 10:32 PM, Stefano Rivera wrote:

>Hi Barry (2015.10.05_17:51:41_+0200)
>> >and other 9, for a grand total of 109 packages that cannot be
>> >converted to git, 13.5% of DPMT (oh, what about PAPT?)
>> 
>> I've wondered about PAPT too.  I don't touch those nearly as often, but
>> eventually yes, they should come under the same vcs regime, IMHO.
>
>One step at a time. The same scripts should work, with minimal tweaks :)

Oh for sure!  There's no deadline on "eventually" :)

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 05, 2015, at 07:12 PM, Barry Warsaw wrote:

>I did an update (not uploaded) of webob from this migration and it worked
>perfectly.  But it's a simple package without patches.  I'll try a few more.

Similarly for ply 3.8.  The nice thing here is that there were several quilt
patches that got applied upstream and were no longer necessary.  git-dpm
recognized this and automatically deleted those patches.

For both packages `git-dpm tag` DTRT.

Cheers,
-Barry


pgpBmeBJPea0z.pgp
Description: OpenPGP digital signature

Re: Git migration schedule

[Barry Warsaw]

On Oct 03, 2015, at 08:52 PM, Stefano Rivera wrote:

>So, here is a migration at r34461:
>https://anonscm.debian.org/cgit/python-modules/svn-migration/

I did an update (not uploaded) of webob from this migration and it worked
perfectly.  But it's a simple package without patches.  I'll try a few more.

Cheers,
-Barry


pgpY7b4eQYxMv.pgp
Description: OpenPGP digital signature

Re: Git migration schedule

[Barry Warsaw]

On Oct 04, 2015, at 08:03 PM, Sandro Tosi wrote:

>am I the only one thinking it's quite a huge number to be handled by
>hand? and whose hands will be the ones converting these packages?
>yours or Barry's dont seem enough and others will need training/time.

I'm happy to pitch in if a maintainer needs help doing so.  I'd like to at
least capture the manual steps needed on the GitPackaging page.  I'm not sure
if there is enough commonality to make that worthwhile, but there might be.

>Additionally, now we are in the middle of the 3.5 transition, and so
>packages will need updating rather quickly: are we sure this is the
>best time to push full throttle with the migration? I'd rather wait a
>little bit longer and have a more automatic migration at a calmer
>moment for python modules.

Waiting longer isn't an option IMHO.  It's helping to add to the dysfunction
of the team.  I will also offer to help if the 3.5 transition gets stuck
because of the git conversion.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 04, 2015, at 08:31 PM, Sandro Tosi wrote:

>sorry, i forgot to ask another question: how will the packages already
>maintained in git be handled?

It should be easy.  Just push it to the team's vcs.  If it's not already in
git-dpm it's pretty easy to bootstrap.  Essentially just one call to git-dpm
init with the latest pristine tar.  It's possible that it will leave you with
limited pristine tar history, but most of the time YAGNI.

You might also have to futz with branch names to use the official names, and I
wouldn't worry if your older tag formats didn't match.

And we do have an "out" although I would caution to use it *very* sparingly
and only with a really good reason.  If your package *has to* diverge from
team standards, you must add a debian/README.source explaining why, and how to
interact with vcs for your package.  It should be needed only rarely.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 03, 2015, at 08:52 PM, Stefano Rivera wrote:

>No significant failures, but I wanted to setup an mr config, which I've done
>now:
>https://anonscm.debian.org/cgit/python-modules/svn-migration/python-modules.git/
>The pkg-perl team has fancier tools, but they require more bookkeeping, so I
>mostly cribbed from the pkg-ruby-extras team.

Is there some documentation on how to use these scripts, or set up mr?  Or
would that be obvious for anybody who's used mr before?

Can you add something about mr to https://wiki.debian.org/Python/GitPackaging
?

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 03, 2015, at 08:52 PM, Stefano Rivera wrote:

>So, here is a migration at r34461:
>https://anonscm.debian.org/cgit/python-modules/svn-migration/
>
>The errors:

Some of these may already be in git, and hopefully git-dpm so don't actually
need a conversion.  If it's in git but not git-dpm, it's fairly easy to
bootstrap, though perhaps with less history, but fully functional going
forward.

I've not done a full review of this list, but the ones I know about are (sorry
Piotr ;):

>Cannot "git-dpm init" package: pycurl
>Cannot "git-dpm init" package: python-pip

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 04, 2015, at 08:03 PM, Sandro Tosi wrote:

>and other 9, for a grand total of 109 packages that cannot be
>converted to git, 13.5% of DPMT (oh, what about PAPT?)

I've wondered about PAPT too.  I don't touch those nearly as often, but
eventually yes, they should come under the same vcs regime, IMHO.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 03, 2015, at 11:47 AM, Sandro Tosi wrote:

>what were the problems that -in the last period- delayed the conversion? were
>they just lack of time (understandable) or were they technical in the
>conversion process?

The very last patch I forwarded to Stefano involved correctly setting the tag
format so that you get that automatically, without having to munge a global
config.  Specifically, I had previously missed that the format can be
specified in the .git-dpm file (`man git-dpm` for details, and search!).

A good way to test that (when Stefano puts up the next test conversion) is to
make some change, then `git-dpm tag` and verify that the tag form is correct.
Be sure you don't have any settings in ~/.gitconfig though.

AFAIK, the conversion script is good to go now.

Cheers,
-Barry

Re: Git migration schedule

[Barry Warsaw]

On Oct 03, 2015, at 03:50 PM, Sandro Tosi wrote:

>if the last is a strong requirement, honestly it seems fragile: ~/.gitconfig
>applies to all the git repos I have (and I may want to specify a tag format
>there), while what is set in debian/.git-dpm (is this the file where the tag
>format is set right?) should override what's in ~/.gitconfig .

Oops.  I just meant that when you're testing that `git-dpm tag` DTRT, make
sure that you're testing that the migration's settings in .git-dpm are what's
producing the correct tags, not any possible settings in ~/.gitconfig.
Definitely test that .git-dpm overrides ~/.gitconfig, but I'm pretty sure that
it does.

Cheers,
-Barry


pgpZQAaD1D08F.pgp
Description: OpenPGP digital signature

Re: I've been removed from the Python team

[Barry Warsaw]

On Oct 01, 2015, at 07:47 PM, Vincent Bernat wrote:

>I am a bit worried that the team is handled behind closed walls.

I have no particular interest in either grabbing power nor in taking power
away from anybody, but I think there may be some value in making team
governance more transparent and democratic.  Two reasons come to mind:

No one person has to take the heat for uncomfortable decisions.  At some point
decisions have to be made for the good of the team, whether they're technical
or social.  What might be difficult for one person to decide can be made
easier when the burden of that decision can be shared among duly elected
representatives.

Team members can have more of a say --and more confidence in-- how the team is
run.  If you elect someone to a leadership role, you're giving your support to
them to make the tough decisions.  And you have the option of voting them out
at the next election.

I don't think any of that's controversial, given that the Debian project
itself is both transparent and democratic, and we always have those governance
rules to fall back on.  But that's a pretty heavyweight bureaucracy.

Does it make sense to have some lightweight rules for the team?  Is there
precedence within other Debian teams?

Cheers,
-Barry


pgpsDktJEK7ZG.pgp
Description: OpenPGP digital signature

Re: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental

[Barry Warsaw]

On Sep 29, 2015, at 12:26 PM, Sandro Tosi wrote:

>Once again, the python policy about Maintainer/Uploaders has been ignored
>
>either policy changes or this has to stop at some point.

A few observations.

The policy should perhaps be better promoted or more explicitly written.  The
links you provided are useful, but I wonder whether they are easily
overlooked, forgotten, or misinterpreted.

Policy doesn't really spell out the operational differences for when the team
is Maintainer vs. Uploader, and the wiki page explicitly says it's an
*unwritten* policy (despite the fact that putting it in the wiki probably
qualifies as "written" :).  How should the change be acknowledged by the
maintainer?  Should I Cc the mailing list when I contact the maintainer?  Is
it okay to commit to vcs but not upload?  How long do you wait for feedback
before you can do the upload?

(Aside: git!  I would love to be able to commit and push a branch and then ask
for the maintainer to review, merge, and upload - or give me the green light
to go ahead.)

Should we have some automated tools to help out here?  I'm not sure where to
hook in such a tool, but if for example when I build a source package, I got a
nice little lintian-like warning saying "The package is team uploadable but
not team maintained, did you give the maintainer time to respond to your
issue?"  Something like that would go a long way toward mitigating accidental
or careless toe-stepping.

Do all team members understand the implications when they set the two fields?
Some maintainers may not really care and may have been less conscientious
about setting the fields.

The wiki says that the general rule of thumb is to set the team as Maintainer,
to which I agree.  I may not have been as deliberate about my own packages, so
I plan on reviewing them, and will fix any that aren't "special".

Cheers,
-Barry

Re: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental

[Barry Warsaw]

On Sep 29, 2015, at 02:02 PM, Tristan Seligmann wrote:

>After reading this thread, I feel like I should go through all of my
>packages and remove the team from Maintainer for all of them. I try very
>hard to respond promptly to pings (bugs, email, IRC, ...) about my
>packages, even if it's just to say "sorry, I can't take care of that right
>now, feel free to upload"; but I'm not sure that having someone blindly
>upload my packages if they haven't worked on them before is a good idea.

I'm willing to deal with some honest mistakes with my packages in order to
foster a vibrant and active community.

I am happy to answer questions about my packages, but I feel spoiled by how
nicely things work as an upstream with code hosting sites like GitLab.  If we
had merge requests, online and email reviews, one-click auto-merge (and
uploads!) I think we'd have a better team collaboration workflow.  Even
post-commit/upload emailed diffs would at least let me review changes after
the fact, so I could repair things if I noticed a big problem.

Cheers,
-Barry

Re: Maintainer vs. Uploaders

[Barry Warsaw]

On Sep 29, 2015, at 04:30 PM, Piotr Ożarowski wrote:

>Maintainer vs Uploaders rules needs to be moved to policy. I will
>propose a patch to the policy soon (I'd prefer a native speaker to do
>it, though)

+1, and I'd be happy to review the specific text.

Cheers,
-Barry

Re: lintian and team uploads

[Barry Warsaw]

On Sep 29, 2015, at 09:46 PM, Piotr Ożarowski wrote:

>(and remember to remove DPMT from debian/control if it's not in SVN ;P)

Given that the final git migration is imminent (really! otherwise I'm going to
scream ;), can we not force people to go through this churn right now?

Yes, let's discourage any new packages from manually migrating or starting out
in git just until flag day.  But I'm not sure it's worth removing DPMT on
existing repos just to add it back, hopefully RSN.

Cheers,
-Barry

Re: Dealing with flit -- a simplified packaging of python modules

[Barry Warsaw]

On Sep 19, 2015, at 12:35 AM, Thomas Kluyver wrote:

>By the way, I am also upstream for flit, and I'm prepared to help build
>some tooling to use it for distro packaging. I know it will cause some
>inconvenience in the short term because there's infrastructure around
>setup.py packaging, but ultimately I think that having declarative
>metadata should be an advantage.

There have been countless attempts at moving the Python packaging
infrastructure to a declarative syntax over the years.  I remember talking to
Tarek at a Pycon *many* years ago about this.  Maybe this time it'll catch
on. :)

flit doesn't build sdists, so I guess the current toolchain which starts with
.orig.tar.gz won't work with flitted packages.  The README says:

"People may also want a traditional sdist for other reasons, such as Linux
distro packaging. I hope that these problems will diminsh over time."

I'm not sure which problem you hope will diminish!  People wanting traditional
sdists, the problem of Linux distro packaging  or needing sdists for
downstream consumers like Debian.

OTOH, as wheels get more popular for upstream package distributions, I do
think we need a good story for turning wheels into debs.  There's some work
ongoing to turn debs back into wheels:

https://github.com/paulproteus/dirtbike

Cheers,
-Barry

Re: About OpenStack, Mailman3 and python-falcon update

[Barry Warsaw]

On Sep 18, 2015, at 06:48 PM, Pierre-Elliott Bécue wrote:

>I'm currently trying to package mailman3 suite, I started with mailman3 core
>functionnalities, you can see the ITP here:
>
>https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799281

Thanks for working on this!

>
>But, mailman3-core would depend on falcon version >= 0.3.0rc1 which is not
>yet packaged in Debian.
>
>I filled a bug report here:
>
>https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799321
>
>Thomas Goirand (zigo) answered to me something I perfectly understand, that
>for now, zaqar, which is part of the python stuff used in OpenStack, depends
>on falcon < 0.2.0, and thus can no longer be packaged in debian if we update
>python-falcon.
>
>It seems he has contacted zaqar developers in hope to have them test zaqar
>with falcon 0.3, and, if needed, patch it.

The zaqar bug is this one, right?

https://bugs.launchpad.net/zaqar/+bug/1495402

If so, I've commented and subscribed to that bug.

Cheers,
-Barry

Re: Python 3.5 as a supported python3 version

[Barry Warsaw]

On Sep 14, 2015, at 04:26 PM, Scott Kitterman wrote:

>It seems very likely that we'll want to release Stretch with python3.5 as the
>default and only python3 version.

+1

>For packages that build fine with python3.5, there should be nothing required
>from a maintainer point of view.  Except pycxx this can all be done with
>binNMUs.  Ubuntu has already done this transition and has patches for a
>number of packages that may be of assistance when working through issues.

Wherever possible, I've been uploading fixes to the Debian packages, so most
of the Ubuntu transition should be good to go (modulo rebuilds) in Debian.
There's of course a long tail of packages that I haven't spent time on.  There
may be a couple that aren't in DPMT or otherwise were only uploaded to Ubuntu,
so when in doubt look there and/or in BTS.  For packages which needed upstream
fixes, I've either opened bugs or grabbed patches in upstream trackers or vcs.
There have been a handful of new upstream releases, and I expect many more now
that Python 3.5 final has been released.

Thanks for moving this forward Scott.

Cheers,
-Barry


pgpKFlPflHXRa.pgp
Description: OpenPGP digital signature

Re: Application libraries private, Distutils metadata available for console scripts and introspection

[Barry Warsaw]

On Sep 02, 2015, at 09:06 PM, Robert Collins wrote:

>So this is perhaps the disconnect: I did not say the metadata should
>move: it should be in the private directory - it has to be adjacent to
>the packages/modules its describing, since if it is present but the
>package/module is not that is at the very best confusing for tools
>like pip.
>
>That private directory must be on the python search path when running
>the application (or the modules can't be imported).

It may be a problem of code execution order.   Where does the sys.path
manipulation happen to add the private application path to sys.path?  If that
happens after the wrapper script tries to load the entry point, it will go
boom in the way Ben is describing.  sys.path has to be hacked before that, but
I don't think the entry point wrapper creation will do that automatically.

Cheers,
-Barry

Re: Application libraries private, Distutils metadata available for console scripts and introspection

[Barry Warsaw]

On Sep 02, 2015, at 11:19 AM, Karsten Hilbert wrote:

>> That private directory must be on the python search path
>
>Maybe I'm a bit dense but isn't that what OP very much
>DOESN'T want to _happen_ ?

Not on the default sys.path, sure.  But it has to get on sys.path at some
point before the application starts to import private modules.  How and where
does that happen?  (I.e. is it before or after the entry point tries to get
loaded?)

Cheers,
-Barry

Re: Python BoF at DebConf15 - summary

[Barry Warsaw]

Thanks for the summary Piotr!  I was really sorry I couldn't make Debconf this
year.

On Aug 24, 2015, at 11:08 PM, Piotr Ożarowski wrote:

Python 3.5 as supported
===

python3-defaults in experimental already has 3.5 as supported.  Ubuntu did
this change as well and reported back that it's not as problematic as many
previous transitions we had in the past. Some concerns were raised that many
upstreams didn't test their code with 3.5 as its final version is still not
released (and services like Travis do not enable it). In some places the
packaging assumes all python3 to be available, but the build dependency on
python3-all{,-dev} is missing.

I've made very good progress here, with one notable exception (see below),
although of course there is a long tail of packages which need fixing for
Python 3.5 support.  Ubuntu 15.10 will have 3.5 supported but not default.
Maybe by the time 16.04 opens, we'll have already made the switch in Stretch.
I'd certainly like the next Debian release, and the next Ubuntu LTS to be 3.5
by default, with 3.4 (possibly? probably?) dropped.

Wherever possible, I've tried to upload fixes to unstable, even though we
won't notice them until 3.5 is supported there too.  Anyone interested in
helping out should take a look at the ubuntu-devel@ mailing list threads, and
take a look at buildlogs from the Ubuntu PPA I am keeping up to date:

https://launchpad.net/~pythoneers/+archive/ubuntu/py35asdefault/+packages

Django is the notable exception.  I had hoped to be able to backport enough
Python 3.5 support to Django 1.7.9, but that just wasn't feasible.  We'll need
to get 1.8.4 out of experimental, but there are a bunch of packages that also
need updating or fixing, and I don't exactly know how compatible they all are
with both Python 3.5 and Django 1.8.

http://tinyurl.com/qz6rorh

If you can help with any of the Django packages, please get in touch.

Cheers,
-Barry


pgppMzPq3X85B.pgp
Description: OpenPGP digital signature

Re: Removing some python3-* packages

[Barry Warsaw]

Just a quick follow-up I've been meaning to send.

On Jul 02, 2015, at 03:55 PM, Barry Warsaw wrote:

As part of the 3.5 test rebuild I noticed an incompatibility with
python3-enum, which I reported upstream.  The response was: there's actually
no reason to have a Python 3 version of enum in any version = Python 3.4.
Since that's all we have now, maybe it makes more sense to just remove the
python3-enum package from Debian.

Which is what I'm going to do.  I've interacted with upstream and there's just
no reason for us to be shipping a Python 3 version of enum34, especially in
Stretch given our current and future supported Python 3 versions.  I'll be
uploading a new version of enum34 that deletes python3-enum34 asap.

py3porters, please keep this in mind and don't add it back.  It'll ftbfs
anyway and I'll close any related bug as won't fix.

In general, I think we'll just have to take similar situations on a
case-by-case basis.  For packages which provide additional functionality not
in stdlib Python 3, it may still make sense to provide a python3- version of
the package.

Cheers,
-Barry


pgp_2BLSpo3X4.pgp
Description: OpenPGP digital signature

Re: Removing some python3-* packages

[Barry Warsaw]

On Aug 19, 2015, at 06:41 PM, Matthias Klose wrote:

As a Debian developer you are duplicating code, and no, I don't think that
providing this code under a different name is different enough to rectify
distribution of this code in Debian.

In some cases however, the standalone library moves ahead of the stdlib,
because they have hugely different release cycles.  They may eventually catch
up every 18 months (when a new Python is released), but in the meantime the
divergence might be needed by dependent packages.

So I'm not sure a blanket policy is useful.  By all means, if there isn't any
significant difference between a standalone package and what's available in
the current supported Python 3 version, let's not ship unnecessary binary
packages.

Cheers,
-Barry

Re: Removing some python3-* packages

[Barry Warsaw]

On Aug 25, 2015, at 10:03 AM, Robert Collins wrote:

On 25 August 2015 at 09:57, Barry Warsaw ba...@debian.org wrote:
...
  By all means, if there isn't any
 significant difference between a standalone package and what's available in
 the current supported Python 3 version, let's not ship unnecessary binary
 packages.

Even at the cost of having to patch upstream projects?

Sorry, I'm not sure what the question means.

Cheers,
-Barry


pgpcXi3cKf4sQ.pgp
Description: OpenPGP digital signature

Re: Removing some python3-* packages

[Barry Warsaw]

On Aug 25, 2015, at 10:45 AM, Robert Collins wrote:

Lets take Ironic. While it supports Python 2.7+ and 3.4+ it will
depend on 'mock' for unit testing.

That's not unreasonable, and different upstreams will have different policies,
but if it was *my* upstream package, I'd probably do a conditional import so
that unittest.mock were used if available, falling back to mock if not.

If it's easy to patch upstream to do this, then it seems an acceptable
approach to take.  I wouldn't say it's *required* (which is why I'm not sure a
blanket policy is appropriate for us).

 - patch Ironic to use unittest.mock on Python 3.5

That would be my first preference, and I'd also submit the patch to upstream.

 - patch the stdlib to make 'mock' be an alias to unittest.mock

Nope.

 - include 'python3-mock' as a binary package

That's not unreasonable for Debian either.

I'll note another thing that perhaps makes the enum34 case different.  It
exports the package under the same name in both the standalone version and in
the stdlib, so there *really* is no difference between the two.  That's not
the case with unittest.mock/mock.

 - not run the Ironic unit tests.

Nope.

But anyway, that's MHO.

Cheers,
-Barry


pgpkdZ2ym1xdB.pgp
Description: OpenPGP digital signature