Re: [openstack-dev] [swift] Plan to port Swift to Python 3
Hi, Le 09/10/2015 12:12, vishal yadav a écrit : However I was just checking if you considered using 2to3. I can understand that translation using this tool might not cover every area in the code more specifically custom/3rd party libraries (non-standard python libraries) but IMO it can do fixer translations to much extent. I tried 2to3, modernize and 2to6 tools in the past, but they produce a single giant patch with unwanted changes. These tools are written to add compatibility for all Python versions including Python 2.6 and Python 3.0. In OpenStack, we only care of Python 2.7 and 3.4, so the code can be simpler. For example, we can simply write u"unicode" instead of six.b("unicode"). I wrote the sixer tool for OpenStack. Basically, it's the same than 2to6, except that: - sixer respects OpenStack Coding Style on imports: it groups and sorts imports. It avoids to have to manually fix individual modified import which takes a lot of time - sixer can produce a patch for a single pattern. For example, replace all unicode with six.text_type but nothing else. Since all changes are reviewed carefully in OpenStack, it's important to produce "reviewable" (small) changes. See also my blog article which explains the full rationale: http://haypo.github.io/python3-sixer.html My patch serie of 6 changes to fix most Python 3 issues was almost fully generated by sixer. Sometimes, I had to manually fix a few lines because no tool is perfect ;-) The patch serie: * "py3: Replace unicode with six.text_type" https://review.openstack.org/#/c/232476/ * "py3: Replace urllib imports with six.moves.urllib" https://review.openstack.org/#/c/232536/ * "py3: Use six.reraise() to reraise an exception" https://review.openstack.org/#/c/232537/ * "py3: Replace gen.next() with next(gen)" https://review.openstack.org/#/c/232538/ * "Replace itertools.ifilter with six.moves.filter" https://review.openstack.org/#/c/232539/ * "py3: Replace basestring with six.string_types" https://review.openstack.org/#/c/232540/ Then I will then use sixer on individual files to fix all Python 3 at once. Victor __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Plan to port Swift to Python 3
Victor, I appreciate for your effort. However I was just checking if you considered using 2to3. I can understand that translation using this tool might not cover every area in the code more specifically custom/3rd party libraries (non-standard python libraries) but IMO it can do fixer translations to much extent. If needed custom fixers can also be defined for 2to3. - https://docs.python.org/2/library/2to3.html - https://docs.python.org/3/howto/pyporting.html Thanks, Vishal On Fri, Oct 9, 2015 at 2:34 PM, Thierry Carrez wrote: > Victor Stinner wrote: > > Good news, we made good progress last weeks on porting Swift to Python > > 3, a few changes were merged and all dependencies now work on Python 3. > > We only need two more simple changes to have a working pyhon34 check job: > > > > * "py3: Update pbr and dnspython requirements" > > https://review.openstack.org/#/c/217423/ > > * "py3: Add py34 test environment to tox" > > https://review.openstack.org/#/c/199034/ > > > > With these changes, it will be possible to make the python34 check job > > voting to avoid Python 3 regressions. It's very important to avoid > > regressions, so we cannot go backward again in Python 3 support. > > > > On IRC, it was said that it's better to merge Python 3 changes at the > > beginning of the Mitaka cycle, because Python 3 requires a lot of small > > changes which can likely introduce (subtle) bugs, and it's better to > > catch them early during the development cycle. > > > > John Dickinson prefers incremental and small changes, whereas clayg > > looks to like giant patches to fix all Python 3 issues at once to avoid > > conflicts in other (non-Python3) changes. (Sorry, if I didn't summarized > > correctly the discussion we had yesterday.) > > > > The problem is that it's hard to fix "all" Python 3 issues in a single > > patch, the patch would be super giant and just impossible to review. > > It's also annoying to have to write dozens of small patches: we loose > > time on merge conflicts, rebasing, random gate failures, etc. > > > > I proposed a first patch serie of 6 changes to fix a lot of simple > > Python 3 issues "at once": > > [...] > > > > The overall diff is impressive: "61 files changed, 233 insertions(+), > > 189 deletions(-)" ... but each change is quite simple. It's only one > > pattern replaced with a different pattern. For example, replace > > "unicode" with "six.text_type" (and add "import six" if needed). So > > these changes should be easy to review. > > > > With a working (and voting?) python34 check job and these 6 changes, it > > will be (much) easier to work on porting Swift to Python 3. Following > > patches will be validated by the python34 check job, shorter and > > restricted to a few files. > > > > Victor > > That's great news. Thanks so much for your tireless efforts to get > Python 3 supported everywhere in OpenStack, Victor ! > > -- > Thierry Carrez (ttx) > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [swift] Plan to port Swift to Python 3
Victor Stinner wrote: > Good news, we made good progress last weeks on porting Swift to Python > 3, a few changes were merged and all dependencies now work on Python 3. > We only need two more simple changes to have a working pyhon34 check job: > > * "py3: Update pbr and dnspython requirements" > https://review.openstack.org/#/c/217423/ > * "py3: Add py34 test environment to tox" > https://review.openstack.org/#/c/199034/ > > With these changes, it will be possible to make the python34 check job > voting to avoid Python 3 regressions. It's very important to avoid > regressions, so we cannot go backward again in Python 3 support. > > On IRC, it was said that it's better to merge Python 3 changes at the > beginning of the Mitaka cycle, because Python 3 requires a lot of small > changes which can likely introduce (subtle) bugs, and it's better to > catch them early during the development cycle. > > John Dickinson prefers incremental and small changes, whereas clayg > looks to like giant patches to fix all Python 3 issues at once to avoid > conflicts in other (non-Python3) changes. (Sorry, if I didn't summarized > correctly the discussion we had yesterday.) > > The problem is that it's hard to fix "all" Python 3 issues in a single > patch, the patch would be super giant and just impossible to review. > It's also annoying to have to write dozens of small patches: we loose > time on merge conflicts, rebasing, random gate failures, etc. > > I proposed a first patch serie of 6 changes to fix a lot of simple > Python 3 issues "at once": > [...] > > The overall diff is impressive: "61 files changed, 233 insertions(+), > 189 deletions(-)" ... but each change is quite simple. It's only one > pattern replaced with a different pattern. For example, replace > "unicode" with "six.text_type" (and add "import six" if needed). So > these changes should be easy to review. > > With a working (and voting?) python34 check job and these 6 changes, it > will be (much) easier to work on porting Swift to Python 3. Following > patches will be validated by the python34 check job, shorter and > restricted to a few files. > > Victor That's great news. Thanks so much for your tireless efforts to get Python 3 supported everywhere in OpenStack, Victor ! -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [swift] Plan to port Swift to Python 3
Hi, Good news, we made good progress last weeks on porting Swift to Python 3, a few changes were merged and all dependencies now work on Python 3. We only need two more simple changes to have a working pyhon34 check job: * "py3: Update pbr and dnspython requirements" https://review.openstack.org/#/c/217423/ * "py3: Add py34 test environment to tox" https://review.openstack.org/#/c/199034/ With these changes, it will be possible to make the python34 check job voting to avoid Python 3 regressions. It's very important to avoid regressions, so we cannot go backward again in Python 3 support. On IRC, it was said that it's better to merge Python 3 changes at the beginning of the Mitaka cycle, because Python 3 requires a lot of small changes which can likely introduce (subtle) bugs, and it's better to catch them early during the development cycle. John Dickinson prefers incremental and small changes, whereas clayg looks to like giant patches to fix all Python 3 issues at once to avoid conflicts in other (non-Python3) changes. (Sorry, if I didn't summarized correctly the discussion we had yesterday.) The problem is that it's hard to fix "all" Python 3 issues in a single patch, the patch would be super giant and just impossible to review. It's also annoying to have to write dozens of small patches: we loose time on merge conflicts, rebasing, random gate failures, etc. I proposed a first patch serie of 6 changes to fix a lot of simple Python 3 issues "at once": * "py3: Replace unicode with six.text_type" https://review.openstack.org/#/c/232476/ * "py3: Replace urllib imports with six.moves.urllib" https://review.openstack.org/#/c/232536/ * "py3: Use six.reraise() to reraise an exception" https://review.openstack.org/#/c/232537/ * "py3: Replace gen.next() with next(gen)" https://review.openstack.org/#/c/232538/ * "Replace itertools.ifilter with six.moves.filter" https://review.openstack.org/#/c/232539/ * "py3: Replace basestring with six.string_types" https://review.openstack.org/#/c/232540/ The overall diff is impressive: "61 files changed, 233 insertions(+), 189 deletions(-)" ... but each change is quite simple. It's only one pattern replaced with a different pattern. For example, replace "unicode" with "six.text_type" (and add "import six" if needed). So these changes should be easy to review. With a working (and voting?) python34 check job and these 6 changes, it will be (much) easier to work on porting Swift to Python 3. Following patches will be validated by the python34 check job, shorter and restricted to a few files. Victor __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev