Re: Multirelease effort: Moving to Python 3

2013-07-23 Thread Bohuslav Kabrda
- Original Message -
> I'm okay with this portion of the definition.  One note is I would be
> hesitant
> about the timing of python3 being the only python that is installed into the
> minimal buildroot. This should probably happen in rawhide right after
> a branching.
> 

To be more specific, I meant "installed into the minimal buildroot by default", 
which only means building GDB with Python 3, as I've written elsewhere. IMO it 
shouldn't cause any problems, because all the packages that use python 2 during 
build should BR: python2-devel anyway, right? (maybe some packages with build 
scripts written in python 2 may have minor issues, but these should be solvable 
by simply adding BR: python or rewriting to python 3)

> > - livecd can still ship Python 2
> 
> I would consider this to be the goal that we should shoot for, though.  We
> are constantly fighting for space on the install images and we know that
> people who install Fedora would like to have the ability to slim down what
> is installed.  Shooting for no-python2 on the livecds and after that, no
> python2 on the install dvds, and still later, no need for python2 in the
> packages in the repository seem like milestones that have actual real value
> for end users.
> 

Ok, +1

> > - /usr/bin/python points to Python 3
> 
> I am firmly against this.  more depth was in my reply to mmaslano although
> I'll reply to one thing here:
> 
> >   - Please note, that the pep you're referring to also states that "python
> >   should refer to the same target as python2 but may refer to python3 on
> >   some bleeding edge distributions", so this wouldn't really be going
> >   against the pep.
> >
> This is a misinterpretion of the PEP.  (This section is confusing, though:
> "python should refer to the same target as python2" is a recommendation to
> distributions.  "but may refer to python3 on some bleeding edge
> distributions" is a statement of fact for end users to watch out for) See
> the recommendation section:
> 
> "For the time being, it is recommended that python should refer to python2
> (however, some distributions have already chosen otherwise; see the
> Rationale and Migration Notes below)."
> 
> and Future Changes Section:
> http://www.python.org/dev/peps/pep-0394/#future-changes-to-this-recommendation
> 
> "
> It is anticipated that there will eventually come a time where the third
> party ecosystem surrounding Python 3 is sufficiently mature for this
> recommendation to be updated to suggest that the python symlink refer to
> python3 rather than python2.
> 
> This recommendation will be periodically reviewed over the next few years,
> and updated when the core development team judges it appropriate.
> "
> 
> The "may refer to python3" phrase is an acknowledgment that arch has moved
> to /usr/bin/python == python3 and isn't going to revert even though upstream
> thinks it's a... premature time to make that switch.  (To be fair to arch,
> the discussion and PEP happened as a result of arch making that switch so
> they'd already committed to that before the consensus was formed that this
> would be a bad thing to do atthis time.  We don't have that excuse ;-)
> 
> If you'd like to read the discussions for yourself, there are three threads
> linked from the PEP.  An even earlier one is at:
> http://mail.python.org/pipermail/python-dev/2010-November/105252.html
> 

One way or the other, the PEP states that people should be careful about what 
/usr/bin/python points to, since it may be python 2 as well as python 3. This 
gives us the freedom to do the switch when we want and not break the upstream 
expectations (yes, I know that huge number of people expect /usr/bin/python to 
point to python 3, but this pep basically says that it's a bad idea to assume 
that).

> 
> > 
> > > The python package itself should probably also remain python2 due to
> > > dependencies and expectations from other distros and documentation --
> > > I think I'd be -1 to changing this
> > > 
> > > The Fedora live images contain only python3, not python2 -- I'd be
> > > heavily
> > > in favour of this. +1
> > > 
> > > > This is going to be a multirelease effort
> > > > that is going to affect lots of Fedora parts. Since we will need to
> > > > switch
> > > > default package manager from Yum to DNF (which is supposed to work with
> > > > Python 3), we will need to wait for that. I've been told that DNF
> > > > should
> > > > be default in F22, so that's my target, too. That should also give
> > > > everyone else plenty of time to work on other essential packages to
> > > > make
> > > > this happen.
> > > 
> > > Getting there at the same time as we get to DNF sounds like a good
> > > timeline.
> > > (But see my note on anaconda below).  +1
> > > 
> > > > Here is my analysis/proposal:
> > > > Before switching, we need to make sure that everything "important" (*)
> > > > is
> > > > Python 3 compatible. There are three steps I see in this transition:
> > > > 1) Get

Re: Multirelease effort: Moving to Python 3

2013-07-23 Thread Nick Coghlan
On 07/23/2013 05:13 PM, Bohuslav Kabrda wrote:
> One way or the other, the PEP states that people should be careful about what 
> /usr/bin/python points to, since it may be python 2 as well as python 3. This 
> gives us the freedom to do the switch when we want and not break the upstream 
> expectations (yes, I know that huge number of people expect /usr/bin/python 
> to point to python 3, but this pep basically says that it's a bad idea to 
> assume that).

As the migration notes in PEP 394 say: "More conservative distributions
that are less willing to tolerate breakage of third party scripts
continue to alias it to python2. Until the conventions described in this
PEP are more widely adopted, having python invoke python2 will remain
the recommended option."

So I guess it depends if Fedora sees itself as being in the "More
conservative" camp or not :)

>> Also, just to be clear, you do understand why switching the conditionals
>> doesn't work for our existing packages, correct?
>>
> 
> Please be more specific. What do you mean "doesn't work"?
> 
>>> IMO Fedora should lead the way of making Python 3 "the Python" and Python 2
>>> "the old compat version". This also makes sense in the traditional
>>> linux-distro "one version of package" that we should be trying to pursue.
>>>
>>
>> -1.  I'm serious, you've got the wrong conceptual model of the relationship
>> between python2 and python3 stuck in your head and it's coloring what you're
>> trying to achieve in bad ways.  Python3 is not an upgrade to Python2.
>> Python3 is a new language.  It is compatible in many ways.  If you can
>> target recent enough versions (at least python-2.6 but python2.7 is better
>> and python-3.3) you can set out to purposefully code things that work on
>> both languages.  But if you're writing general, working python2 code using
>> idioms and thought processes that you've mastered over the last 10 years,
>> chances are extremely high that not even your simple scripts are going to
>> run without modification.
>>
> 
> And again, I'm saying that your conceptual model is not necessarily the 
> correct one.

For simple scripts, I think Toshio is correct. The basic syntax changes,
like print becoming a function, or the way you bind a caught exception
to a name, are easy to handle for applications and libraries with
something like python-modernize, but that's not going to happen for the
vast majority of sysadmin and general user scripts out there.

Consider the flak Canonical got when they switched /bin/sh from bash to
dash, and exposed all the scripts people had written that used bash
extensions, but still had /bin/sh in the shebang line. The risk in
switching the python symlink to python3 isn't really in breaking
*applications*, it's in breaking *scripts*.

There are *lots* of ways for things to break if Python 2 code that isn't
expecting it is run under Python 3. The "Common Stumbling Blocks"
(http://docs.python.org/dev/whatsnew/3.0.html#common-stumbling-blocks)
in the release notes for 3.0 cover some highlights. Most migration
guides gloss over these, since they're looking at things from the
perspective of development project that are taking steps to get ready
for the change, rather than an end user who has just run "fedup" and
finds that all of their homegrown Python scripts are now throwing syntax
errors and other strange things.

That's why I actually advocate for distros (including Fedora) to design
all their packages to not care, and always using a name qualified with
the major language version. If a sysadmin changes /bin/python to point
to something else, Fedora itself shouldn't be affected at all, only user
scripts.

As the migration notes in PEP 394 say:

  * It is suggested that even distribution-specific packages follow the
python2/python3 convention, even in code that is not intended to operate
on other distributions. This will reduce problems if the distribution
later decides to change the version of the Python interpreter that the
python command invokes, or if a sysadmin installs a custom python
command with a different major version than the distribution default.
Distributions can test whether they are fully following this convention
by changing the python interpreter on a test box and checking to see if
anything breaks.
  * If the above point is adhered to and sysadmins are permitted to
change the python command, then the python command should always be
implemented as a link to the interpreter binary (or a link to a link)
and not vice versa. That way, if a sysadmin does decide to replace the
installed python file, they can do so without inadvertently deleting the
previously installed binary.
  * If the Python 2 interpreter becomes uncommon, scripts should
nevertheless continue to use the python3 convention rather that just
python. This will ease transition in the event that yet another major
version of Python is released.
  * If these conventions are adhered to, it will become the case that
the python command is only