Re: MBF for deprecating Python2 usage

[Scott Kitterman]

On August 4, 2017 9:37:18 PM EDT, Barry Warsaw  wrote:
>Scott Kitterman wrote:
>
>> If the primary concern is what happens when a user types "python",
>then can we 
>> address that in command-not-found and leave /usr/bin/python out of
>it?
>
>I'm definitely willing to for now.  It's clearly not time to remove the
>link or point it elsewhere anyway.  I think it'll still be a good idea
>some day, but I'll leave it at that.
>
>Can we agree though to start using /usr/bin/python2 in the shebang line
>instead of /usr/bin/python, for system installed scripts?
>
>-Barry

I think that ship has already sailed, so sure, but I don't think it's anything 
more than a wishlist bug.

Scott K

Re: MBF for deprecating Python2 usage

[Barry Warsaw]

Scott Kitterman wrote:

> If the primary concern is what happens when a user types "python", then can 
> we 
> address that in command-not-found and leave /usr/bin/python out of it?

I'm definitely willing to for now.  It's clearly not time to remove the
link or point it elsewhere anyway.  I think it'll still be a good idea
some day, but I'll leave it at that.

Can we agree though to start using /usr/bin/python2 in the shebang line
instead of /usr/bin/python, for system installed scripts?

-Barry




signature.asc
Description: OpenPGP digital signature

Re: Uploading Python modules which drop support for Python 2?

[Scott Kitterman]

On August 4, 2017 8:00:27 PM EDT, "W. Martin Borgert"  
wrote:
>Hi team,
>
>pysolar upstream version 0.7 dropped support for Python 2, so I
>did not upload it for stretch. I'm considering upload for buster
>now. What do you think?
>
>Cheers

If there are no rdepends, I think it's fine to go ahead.

Scott K

Uploading Python modules which drop support for Python 2?

[W. Martin Borgert]

Hi team,

pysolar upstream version 0.7 dropped support for Python 2, so I
did not upload it for stretch. I'm considering upload for buster
now. What do you think?

Cheers


signature.asc
Description: PGP signature

Re: MBF for deprecating Python2 usage

[Scott Kitterman]

On Friday, August 04, 2017 10:13:00 AM ba...@debian.org wrote:
> On Aug 3, 2017, at 23:23, Scott Kitterman  wrote:
> > Read it.  I remain completely convinced that /usr/bin/python pointing at a
> > python3 version is utterly wrong and a disservice to our users.  Even
> > after we remove python2.7, people will be locally compiling it and using
> > it for a decade.
> I think it’s inevitable, and not doing so *eventually* will be a greater
> disservice.  Sure, not today, maybe not in 2020, but I don’t think we’ll be
> having this discussion in 2025.  If by then, 5 years after Python 2.7 EOL,
> users read on SO or some Learning Python site that they can just type
> ‘python' to get an interactive prompt, but it doesn’t work on Debian,
> they’ll think Debian is broken, not Python.  So I think it makes sense to
> consider how the transition will work.
> 
> You’re right that folks will still need Python 2 after 2020, and that their
> best option may be to build it themselves, but I don’t think they’ll be
> building and installing some old Debian package.  Instead they may build
> from source, which means they’ll be installing it into /usr/local/bin not
> /usr/bin, and they’ll have to change their shebang lines anyway.  And there
> will come a time when even Python 2.7 is unbuildable as toolchains and
> libraries evolve.  Upstream will stop tracking those changes so even the
> upstream git repo won’t give you buildable source.  There may be a bunch of
> third party forks, but I don’t think it’s our responsibility to ensure that
> Debian aligns with any of those hypotheticals.  Folks needing to stay on
> Python 2 will probably also just elect to not upgrade their OS, so what we
> do in future releases won’t matter to them.
> 
> Upstream and Debian already install a `python2` alias for `python`, and I
> think we need to join the chorus that in the future, the way to run Python
> 2 is via `python2`.  Yes, people will have legacy stuff around for a long
> time, but it’s better to begin the transition now rather than make a major
> break years from now.
> > Such a change would be actively user hostile.
> > 
> > When python2.7 goes away, /usr/bin/python should go too.
> 
> Maybe in the short term (as Matthias suggests), but I’m not so sure it’s
> best to disappear /usr/bin/python forever.  /usr/bin/python is not just the
> common shebang (and we should be actively transitioning to /usr/bin/python2
> for that), but it’s also the command line UI for people wanting to learn
> Python, or just wanting to get an interactive interpreter to play with.
> There will be a day when, if they get a command not found when typing
> `python` at a shell prompt, they will wonder why Debian is broken.
> 
> I think the transition should roughly be:
> 
> * Update policy and tooling so that any scripts that require Python 2 use
> /usr/bin/python2 in their shebang.
> 
> * Educate our users that they should be using `python2` for anything that
> has not been ported.
> 
> * Port as much as possible to Python 3 (eventually, everything maintained in
> Debian) and /usr/bin/python3 in their shebang.
> 
> * Plan for a date at which /usr/bin/python will point to Python 3.  I know
> that’s the most controversial bit, but I do think that as time goes on and
> we’re past 2020, it will be the choice that gives our users the best
> experience.
> 
> The discussion on linux-sig is a way to align the entire Python Linux
> ecosystem to the eventual goal, giving individual distros the leeway they
> need to time such transitions as they see fit to best serve their users. 
> I’m hoping more Debian Pythonistas will join that discussion, otherwise the
> outcome will be heavily weighted toward other Linux distros and Debian may
> find itself without a voice in the matter.

PEP-394 convinced me that upstream wasn't concerned about the same things 
Debian is concerned about.  Even though they did manage to say in the PEP:

> The main barrier to a distribution switching the python command from python2
> to python3 isn't breakage within the distribution, but instead breakage of
> private third party scripts developed by sysadmins and other users.
> Updating the python command to invoke python3 by default indicates that a
> distribution is willing to break such scripts with errors that are
> potentially quite confusing for users that aren't yet familiar with the
> backwards incompatible changes in Python 3.

they don't actually consider the implications of that statement.

The current discussion only seems to compound the error.  I don't see a lot of 
value in personally expending time on the upstream discussion.

If the primary concern is what happens when a user types "python", then can we 
address that in command-not-found and leave /usr/bin/python out of it?

Scott K

signature.asc
Description: This is a digitally signed message part.

Re: MBF for deprecating Python2 usage

[Barry Warsaw]

Ole Streicher wrote:

> It is very usual to use "#!/usr/bin/env python" as shebang, exactly for
> the case that python is not installed in /usr/bin.

Sure, but then all bets are off.  The script so shebanged can't assume
anything about $PATH so it gets whatever it gets.  Using /usr/bin/env in
system provided scripts has been known to break stuff, so it's a very
bad idea.

Plus, if that's what you use for your own scripts, then in the future
you'll be in luck because all you'd have to do is change your $PATH
to point to wherever your custom built and installed Python 2 lives
first, and then you'll get exactly the version you want.

> That is -- at least in my environmet -- "ipython", or (for Python 3)
> "ipython3".
>
> IMO it would be better to communicate that the best way for an
> interactive session is "ipython3" (or "python3", if you insist). I would
> wonder when todays tutorials (that cover Python 3) recommend to use
> plain "python".

But ipython doesn't come with upstream Python so that would require that
Something Extra be installed.  That's generally why Getting Started
guides and tutorials usually recommend the built-in interactive prompt.

>> * Port as much as possible to Python 3 (eventually, everything
>> maintained in Debian) and /usr/bin/python3 in their shebang.
> I disagree here, and I don't see an advantage over letting
> /usr/bin/python just die with Python 2.

/usr/bin/python didn't die with Python 2 (who else remembers 1.5.2 as an
effectively LTS release? :) and I don't believe it should with Python 3.
Maybe go on vacation, but eventually return home.

> I would also like to point PEP 394, which has a number of arguments why
> /usr/bin/python should remain as Python 2.
>
> https://www.python.org/dev/peps/pep-0394/

Right, but the ecosystem has evolved since then, so the discussions
going on in linux-sig and the *draft* PR at
https://github.com/python/peps/pull/315 are about updating the
recommendations now that we're somewhere less than three years away from
Python 2 EOL.

Fedora is moving forward with or without us.  I bet that other Linux
distros will too.  We can certainly do our own thing, but I think
that'll make Debian the odd distro out eventually.

Cheers,
-Barry

Re: MBF for deprecating Python2 usage

[Ole Streicher]

ba...@debian.org writes:
> On Aug 3, 2017, at 23:23, Scott Kitterman  wrote:
> You’re right that folks will still need Python 2 after 2020, and that
> their best option may be to build it themselves, but I don’t think
> they’ll be building and installing some old Debian package.  Instead
> they may build from source, which means they’ll be installing it into
> /usr/local/bin not /usr/bin, and they’ll have to change their shebang
> lines anyway.

It is very usual to use "#!/usr/bin/env python" as shebang, exactly for
the case that python is not installed in /usr/bin.

>> Such a change would be actively user hostile.
>> 
>> When python2.7 goes away, /usr/bin/python should go too.
>
> Maybe in the short term (as Matthias suggests), but I’m not so sure
> it’s best to disappear /usr/bin/python forever.  /usr/bin/python is
> not just the common shebang (and we should be actively transitioning
> to /usr/bin/python2 for that), but it’s also the command line UI for
> people wanting to learn Python, or just wanting to get an interactive
> interpreter to play with.

That is -- at least in my environmet -- "ipython", or (for Python 3)
"ipython3".

> There will be a day when, if they get a command not found when typing
> `python` at a shell prompt, they will wonder why Debian is broken.

IMO it would be better to communicate that the best way for an
interactive session is "ipython3" (or "python3", if you insist). I would
wonder when todays tutorials (that cover Python 3) recommend to use
plain "python".

> I think the transition should roughly be:
>
> * Update policy and tooling so that any scripts that require Python 2
> use /usr/bin/python2 in their shebang.
>
> * Educate our users that they should be using `python2` for anything
> that has not been ported.
>
> * Port as much as possible to Python 3 (eventually, everything
> maintained in Debian) and /usr/bin/python3 in their shebang.

I disagree here, and I don't see an advantage over letting
/usr/bin/python just die with Python 2.

I would also like to point PEP 394, which has a number of arguments why
/usr/bin/python should remain as Python 2.

https://www.python.org/dev/peps/pep-0394/

Best regards

Ole

Re: MBF for deprecating Python2 usage

[barry]

On Aug 3, 2017, at 23:23, Scott Kitterman  wrote:
> 
> Read it.  I remain completely convinced that /usr/bin/python pointing at a 
> python3 version is utterly wrong and a disservice to our users.  Even after 
> we remove python2.7, people will be locally compiling it and using it for a 
> decade.

I think it’s inevitable, and not doing so *eventually* will be a greater 
disservice.  Sure, not today, maybe not in 2020, but I don’t think we’ll be 
having this discussion in 2025.  If by then, 5 years after Python 2.7 EOL, 
users read on SO or some Learning Python site that they can just type ‘python' 
to get an interactive prompt, but it doesn’t work on Debian, they’ll think 
Debian is broken, not Python.  So I think it makes sense to consider how the 
transition will work.

You’re right that folks will still need Python 2 after 2020, and that their 
best option may be to build it themselves, but I don’t think they’ll be 
building and installing some old Debian package.  Instead they may build from 
source, which means they’ll be installing it into /usr/local/bin not /usr/bin, 
and they’ll have to change their shebang lines anyway.  And there will come a 
time when even Python 2.7 is unbuildable as toolchains and libraries evolve.  
Upstream will stop tracking those changes so even the upstream git repo won’t 
give you buildable source.  There may be a bunch of third party forks, but I 
don’t think it’s our responsibility to ensure that Debian aligns with any of 
those hypotheticals.  Folks needing to stay on Python 2 will probably also just 
elect to not upgrade their OS, so what we do in future releases won’t matter to 
them.

Upstream and Debian already install a `python2` alias for `python`, and I think 
we need to join the chorus that in the future, the way to run Python 2 is via 
`python2`.  Yes, people will have legacy stuff around for a long time, but it’s 
better to begin the transition now rather than make a major break years from 
now.

> Such a change would be actively user hostile.
> 
> When python2.7 goes away, /usr/bin/python should go too.

Maybe in the short term (as Matthias suggests), but I’m not so sure it’s best 
to disappear /usr/bin/python forever.  /usr/bin/python is not just the common 
shebang (and we should be actively transitioning to /usr/bin/python2 for that), 
but it’s also the command line UI for people wanting to learn Python, or just 
wanting to get an interactive interpreter to play with. There will be a day 
when, if they get a command not found when typing `python` at a shell prompt, 
they will wonder why Debian is broken.

I think the transition should roughly be:

* Update policy and tooling so that any scripts that require Python 2 use 
/usr/bin/python2 in their shebang.

* Educate our users that they should be using `python2` for anything that has 
not been ported.

* Port as much as possible to Python 3 (eventually, everything maintained in 
Debian) and /usr/bin/python3 in their shebang.

* Plan for a date at which /usr/bin/python will point to Python 3.  I know 
that’s the most controversial bit, but I do think that as time goes on and 
we’re past 2020, it will be the choice that gives our users the best experience.

The discussion on linux-sig is a way to align the entire Python Linux ecosystem 
to the eventual goal, giving individual distros the leeway they need to time 
such transitions as they see fit to best serve their users.  I’m hoping more 
Debian Pythonistas will join that discussion, otherwise the outcome will be 
heavily weighted toward other Linux distros and Debian may find itself without 
a voice in the matter.

Cheers,
-Barry




signature.asc
Description: Message signed with OpenPGP

Re: dpkg-buildpackage -B fails

[Guðjón Guðjónsson]

Thanks Andrey and Ghislain

>  dpkg-genbuildinfo --build=any

> > dpkg-genbuildinfo: error: binary build with no binary artifacts found;
> > .buildinfo is meaningless
> Why are you trying to build arch:any subpackages of a package that doesn't
> have arch:any subpackages?
>
> The packages are Architecture: all

This is quite tricky for silly me :)
Yesterday I changed the order of binary-indep and binary-arch but that
didn't fix the problem. What I missed is that
 dpgk-buildpackage -B is no longer the problem but that sbuild doesn't
download python.

I will have to find out why sbuild doesn't download the build dependencies
and preferably how to make it cache
the downloads.

Regards
Gudjon

Re: MBF for deprecating Python2 usage

[Scott Kitterman]

On August 4, 2017 6:49:23 AM EDT, Matthias Klose  wrote:
>On 03.08.2017 21:08, ba...@debian.org wrote:
>> On Aug 3, 2017, at 17:57, Matthias Klose  wrote:
>>>
>>> While at DebCamp, Stefano Rivera and I sat down to analyze what
>needs to be done
>>> to deprecate Python2 usage within the distribution.  It might not be
>possible to
>>> drop Python2 for the next release, but there are still too many
>issues with
>>> packages.  For now we identified some categories which need fixing.
>These are
>>> documented at https://wiki.debian.org/Sprints/2017/Python3Montreal,
>resulting in
>>> more than 3000 bug reports.  That's a high number, on the other hand
>we won't
>>> make much progress if the issues are not named.  My intent is to
>bring that up
>>> in the Python BoF next week at DebConf and then filing these bug
>reports with
>>> the user tags mentiond on the wiki page.
>> 
>> Great to hear that you guys talked about it.
>> 
>> Just a quick note that PEP 394 discussions have revived, lead by the
>Fedora folks.  Please do check out the new thread, especially if you
>have opinions about what /usr/bin/python should do once Python 2.7 is
>EOL.
>> 
>> https://mail.python.org/pipermail/linux-sig/2017-August/thread.html
>
>I replied to this thread.  I think there should be one release which is
>not
>shipping /usr/bin/python before /usr/bin/python should be reused and
>pointed at
>python (>> 2). This should be good enough to get all scripts actively
>converted
>which are not part of the distribution.  If that release is buster, we
>should
>require the use of python2 instead of python now, document that in
>policy and
>provide a lintian check for that.
>
>Matthias

I disagree.

There will be users whose reaction to removal of python2.7 will be to compile 
their own.

Reintroducing /usr/bin/python as a python3 version risks their systems for no 
benefit (since all python3 stuff points to /usr/bin/python3 and works fine).  
Just let it go and don't bring it back.

As far as when to drop it goes, I believe that the python-django maintainers 
are planning on releasing Buster with Django 1.11 as bilingual python2/python3. 
 Then they'll go to Django 2, which is python3 only, early in the Bullseye 
cycle.

For the chunk of the Python ecosystem that is built around Django, keeping 
python2.7 in for Buster would be a significant aid to transition.

Scott K

Re: MBF for deprecating Python2 usage

[Matthias Klose]

On 03.08.2017 21:08, ba...@debian.org wrote:
> On Aug 3, 2017, at 17:57, Matthias Klose  wrote:
>>
>> While at DebCamp, Stefano Rivera and I sat down to analyze what needs to be 
>> done
>> to deprecate Python2 usage within the distribution.  It might not be 
>> possible to
>> drop Python2 for the next release, but there are still too many issues with
>> packages.  For now we identified some categories which need fixing. These are
>> documented at https://wiki.debian.org/Sprints/2017/Python3Montreal, 
>> resulting in
>> more than 3000 bug reports.  That's a high number, on the other hand we won't
>> make much progress if the issues are not named.  My intent is to bring that 
>> up
>> in the Python BoF next week at DebConf and then filing these bug reports with
>> the user tags mentiond on the wiki page.
> 
> Great to hear that you guys talked about it.
> 
> Just a quick note that PEP 394 discussions have revived, lead by the Fedora 
> folks.  Please do check out the new thread, especially if you have opinions 
> about what /usr/bin/python should do once Python 2.7 is EOL.
> 
> https://mail.python.org/pipermail/linux-sig/2017-August/thread.html

I replied to this thread.  I think there should be one release which is not
shipping /usr/bin/python before /usr/bin/python should be reused and pointed at
python (>> 2). This should be good enough to get all scripts actively converted
which are not part of the distribution.  If that release is buster, we should
require the use of python2 instead of python now, document that in policy and
provide a lintian check for that.

Matthias

Re: dpkg-buildpackage -B fails

[Andrey Rahmatullin]

On Fri, Aug 04, 2017 at 11:39:50AM +0200, Gudjon I. Gudjonsson wrote:
> Hi list
> 
> I wanted to fix my terrible track record lately, fix bugs and update my 
> packages 
> but I ran into problems with sbuild on my package eric.
> 
> dpkg-buildpackage -B fails with the following error message:
> 
>  dpkg-genbuildinfo --build=any
> dpkg-genbuildinfo: error: binary build with no binary artifacts found; 
> .buildinfo is meaningless
Why are you trying to build arch:any subpackages of a package that doesn't
have arch:any subpackages?

-- 
WBR, wRAR


signature.asc
Description: PGP signature

Re: dpkg-buildpackage -B fails

[Ghislain Vaillant]

On 04/08/17 10:39, Gudjon I. Gudjonsson wrote:

Hi list

I wanted to fix my terrible track record lately, fix bugs and update my packages
but I ran into problems with sbuild on my package eric.

dpkg-buildpackage -B fails with the following error message:

  dpkg-genbuildinfo --build=any
dpkg-genbuildinfo: error: binary build with no binary artifacts found;
.buildinfo is meaningless


Because all your packages listed in d/control are Architecture: all? In 
this case the error message is accurate: dpkg-buildpackage -B yields no 
binary packages.


Ghis

dpkg-buildpackage -B fails

[Gudjon I. Gudjonsson]

Hi list

I wanted to fix my terrible track record lately, fix bugs and update my 
packages 
but I ran into problems with sbuild on my package eric.

dpkg-buildpackage -B fails with the following error message:

 dpkg-genbuildinfo --build=any
dpkg-genbuildinfo: error: binary build with no binary artifacts found; 
.buildinfo is meaningless
dpkg-buildpackage: error: dpkg-genbuildinfo --build=any gave error exit status 
2

I have tried to rearrange the binary-indep and binary-arch fields but without 
success.

Can anyone help?

Regards
Gudjon

Ps. I cannot guarantee I will have time to fix my packages. I will try to but 
if someone more clever with more time wants to adopt any of them or co-
maintain, please go ahead.

Re: MBF for deprecating Python2 usage

[Piotr Ożarowski]

[Scott Kitterman, 2017-08-04]
> Read it.  I remain completely convinced that /usr/bin/python pointing
> at a python3 version is utterly wrong and a disservice to our users.
> Even after we remove python2.7, people will be locally compiling it
> and using it for a decade.
> 
> Such a change would be actively user hostile.
> 
> When python2.7 goes away, /usr/bin/python should go too.
> 
> Before python2.7 goes away, any python3 scripts/apps need to work with
> /usr/bin/python3.  Making them work with /usr/bin/python too doesn't
> actually help anything.  All it does is look pretty for us old timers.
> 
> I've said before and I've yet to see anything to cause me to have any
> doubt on the matter, that as long as I am one of the python-defaults
> maintainers, /usr/bin/python will not point at a python3 version.

same here. Way too many not packaged /usr/bin/python scripts around me
-- 
GPG: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645