Update:
I'm going to change the development strategy a bit here.
Turns out the first patch for PROTON-490 doesn't work for older versions of
python - <2.5 specifically. I've reverted this change for the time being.
Instead of introducing this piecemeal on trunk, I'd rather develop it on a
branch. By doing this we'll get a complete understanding of the impact of this
work - including the feasibility of supporting older versions of python -
without disrupting trunk.
I've created a kgiusti-python3 branch for this work
FWIW: the six compatibility library claims to only support Python 2.5+. That's
as far back as any of the compatibility tools I've looked at go - none claim to
support 2.4 or previous. That's probably due to the fact that a majority of
the py3 features that replace deprecated 2.x elements where only backported as
far as 2.5. Pre-2.5 simply doesn't have alternatives to these deprecated
language elements.
Once I've got something that works for 2.5+, we can scope the technical debt
required to support pre-2.5.
-K
----- Original Message -----
> From: "Ken Giusti" <kgiu...@redhat.com>
> To: proton@qpid.apache.org
> Sent: Friday, April 17, 2015 9:00:31 AM
> Subject: Re: [RFC] Strategy for porting Proton to Python 3
>
>
>
> ----- Original Message -----
> > From: "Robbie Gemmell" <robbie.gemm...@gmail.com>
> > To: proton@qpid.apache.org
> > Sent: Thursday, April 16, 2015 6:28:18 PM
> > Subject: Re: [RFC] Strategy for porting Proton to Python 3
> >
> > On 16 April 2015 at 23:03, Robbie Gemmell <robbie.gemm...@gmail.com> wrote:
> > > On 16 April 2015 at 20:38, Ken Giusti <kgiu...@redhat.com> wrote:
> > >> And within moments, I hit my first Really Big Problem:
> > >>
> > >> Jython
> > >>
> > >> Yep, turns out Jython can't parse 'futurized' python code. Especially
> > >> dislikes the
> > >>
> > >> except <exception>, <var> ---> except <exception> as <var>
> > >>
> > >> change. Which isn't very helpful, since the 'as' variant is the only
> > >> syntax supported by Python 3 (that comma-syntax has been removed).
> > >>
> > >> This makes porting the python unit tests to python3 a no go. The
> > >> non-test
> > >> python stuff, like the bindings and examples, are not run under jython
> > >> so
> > >> it's not a problem there.
> > >>
> > >> So what to do?
> > >>
> > >> Here's a couple of options that I can think of:
> > >>
> > >> 1) don't port the tests to python 3.
> > >>
> > >> 2) create two copies of the python tests, one 'old' copy for running
> > >> under
> > >> jython, and a new copy for running under python2 and 3. Remove the old
> > >> tests once jython can handle the modern syntax.
> > >>
> > >> 3) Stop running jython tests
> > >>
> > >> 4) A better idea I haven't thought of yet.
> > >>
> > >> I'm pretty much against option 1 - we need to be able to test the
> > >> bindings
> > >> under python 3. I'm open to the others, especially #4.
> > >>
> > >> Opinions?
> > >>
> > >> -K
> > >>
> > >>
> > >
> > > We are using jython 2.5.3, which is the current 'final' release,
>
>
> Ugh - these python2->python3 wrapper modules (like six and future) only work
> with 2.6+ - at least according to their docs.
>
> We'll see how far I can go with those wrappers & jython.
>
> /me remembers answering "Yes I do!" when interviewing for this position and
> being asked "Do you enjoy a challenge?" Damn...
>
>
> > > however on checking the site there have been a couple of RCs for 2.7
> > > in the last month that suggest it might actually be nearing completion
> > > (it has been baking slowly for a couple of years since an initial
> > > beta), so perhaps we could look to upgrade to bring support for 'as'.
> > >
> >
> > I just tried it out for giggles. I cleared out my checkout, flipped
> > the version to 2.7-rc2, ran the build, and went to get a drink while
> > the ~3x sized jar downloaded. I returned to 211 test failures,
> > seemingly from lots of "TypeError: Type not compatible with array
> > type", so there is possibly a little bit of work needed to make such a
> > switch. Some of them passed at least :)
> >
> > I also noticed that Jython 2.7 requires Java 7, so the tests would no
> > longer work on Java 6 if we used it. I'm fine with dropping support
> > for it entirely, but would dislike leaving the compiler target set for
> > Java 6 (as it is now) if we cant actually run the tests on it.
> >
> > >>
> > >>
> > >> ----- Original Message -----
> > >>> From: "Ken Giusti" <kgiu...@redhat.com>
> > >>> To: proton@qpid.apache.org
> > >>> Cc: "Dominic Evans" <dominic.ev...@uk.ibm.com>
> > >>> Sent: Thursday, April 16, 2015 9:54:35 AM
> > >>> Subject: [RFC] Strategy for porting Proton to Python 3
> > >>>
> > >>>
> > >>> Hi all,
> > >>>
> > >>> I'm building on the work done by Dominic and Mickael to get all the
> > >>> proton
> > >>> python bits to work under both python2 and python3. See [1].
> > >>>
> > >>> I think this will entail a lot of little changes to the python sources
> > >>> and
> > >>> the unit tests. Rather than check in a single huge patch, I'm going to
> > >>> break it up over several patches.
> > >>>
> > >>> The first bunch of patches will simply 'modernize' the existing python
> > >>> code.
> > >>> Old style syntax that is not forward compatible with python 3 will be
> > >>> replaced (eg. print "foo" --> print("foo"), etc). I'll use a tool
> > >>> called
> > >>> 'futurize' which is part of the python future toolset [2], [3].
> > >>>
> > >>> Once all python code is updated, then I'll begin introducing python 3
> > >>> specific patches, including the work already done by Dominic and
> > >>> Mickael.
> > >>> Of course I'll verify that none of these changes will break python 2.
> > >>> I've got a local CI system that can build/test in both environments.
> > >>>
> > >>> From a discussion with Dominic, we agreed that it would be A Good Thing
> > >>> to
> > >>> use one of the existing Py2 <--> Py3 abstraction libraries. These
> > >>> libraries
> > >>> provide utilities for writing code that works under both python
> > >>> versions.
> > >>> I've used 'six' in the past [4] and found it quite helpful - it will
> > >>> eliminate a lot of the messy conditional code one has to hack in order
> > >>> to
> > >>> support both languages.
> > >>>
> > >>> However, this library is not part of the standard python library. This
> > >>> means
> > >>> introducing a new dependency.
> > >>>
> > >>> Personally, I don't think this is a big deal - use of 'six' is
> > >>> ubiquitous
> > >>> among python packages. It's available freely via pypi, and though most
> > >>> distros.
> > >>>
> > >>> So that's the Big Question - is everyone comfortable with this
> > >>> additional
> > >>> dependency? Does anyone have a better alternative? Has anyone ported
> > >>> other large python codebases - what was your experience?
> > >>>
> > >>> thanks,
> > >>>
> > >>> -K
> > >>>
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/PROTON-490
> > >>> [2] http://python-future.org/index.html
> > >>> [3] http://python-future.org/futurize_cheatsheet.html
> > >>> [4] https://pythonhosted.org/six/
> > >>>
> > >>> --
> > >>> -K
> > >>>
> > >>
> > >> --
> > >> -K
> >
>
> --
> -K
>
--
-K
