Re: [openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka cycle?

2016-02-19 Thread Thomas Goirand 
On 02/18/2016 08:20 PM, Victor Stinner wrote:
> I discussed with some Trove developers who are interested to start the
> Python 3 port right now. What do you think?

Mitaka b3 is just around the corner (in less than 10 days now), so at
the end, it doesn't change things much, unless all of your patches are
accepted before that. IMO, it's time to get momentum to port things to
Py3. I wish it happens.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka cycle?

2016-02-18 Thread Amrith Kumar 
Great, so in response to your email (below) and Flavio's email [1], I submit to 
you that the way to handle this is as we had discussed at earlier meeting(s) 
and that is to wait for Newton.

Thanks,

-amrith 

[1] http://openstack.markmail.org/thread/4uksb3kmhnagoc5a

> -Original Message-
> From: Victor Stinner [mailto:vstin...@redhat.com]
> Sent: Thursday, February 18, 2016 9:42 AM
> To: OpenStack Development Mailing List (not for usage questions)
> 
> Subject: Re: [openstack-dev] [trove] Start to port Trove to Python 3 in
> Mitaka cycle?
> 
> Le 18/02/2016 14:15, Amrith Kumar a écrit :
> > Let's definitely discuss this again once you have all the changes that
> you feel should be merged for Mitaka ready.
> 
> I don't like working on long patch series. In my experience, after more
> than 4 patches, it's more expensive to maintain the patch serie than to
> write patches. So I prefer to work on few patches, wait until they are
> merged, and then write following patches.
> 
> I'm not going to write dozens of patches. I suggest to do as I done in the
> paste, make progress with baby steps :-)
> 
> For example, my first change only changes the py34 test environment in
> tox.ini, it cannot break anything on Python 2, and it's enough to fix "tox
> -e py34". It is not in conflict with any other pending change.
> https://review.openstack.org/#/c/279098/
> 
>  From this point, we can add a voting gate to be able to validate
> following Python 3 changes.
> 
> 
>  > What I would like to avoid is a dribble of changes where we don't
> know how much more we have coming down the pike.
> 
> You have to be prepared for dozens of small patches. It only depends on
> the size of your project (number of code line numbers) :-)
> 
> To have an idea, you can see the Cinder blueprint which has an
> exhaustive list of all changes made for Python 3:
> https://blueprints.launchpad.net/cinder/+spec/cinder-python3
> 
> I counted 100 patches between June 2015 and February 2016.
> 
> FYI with all my pending patches for Cinder (only 4 changes remain), all
> unit tests will pass on Python 3!
> 
> It also gives you an idea of the time frame: it took me 9 months to port
> Cinder unit tests to Python 3. So more than a single OpenStack cycle (6
> months).
> 
> Since the port is long and painful, I would like to start as soon as
> possible :-)
> 
> 
>  > And while your changes may be "low risk", it does mean that if they
> merge now, the large feature sets that we have committed for this
> release will have to go through the cycle of merge conflicts, rebasing,
> code review, gate ... and so on.
> 
> The principle of technical debt is that the price only is only
> increasing if you wait longer :-) Merging Python 3 today or tomorrow
> doesn't solve the problem of merge conflicts :-)
> 
> It's really up to you to decide to "open the gate" for the flow of
> Python 3 patches, it's also up to you to control how much Python 3
> changes will merged. I can only offer my help to port code. I don't feel
> able to decide when it's the best time to start porting Trove ;-)
> 
> By the way, Gerrit provides a great "Conflicts With" information! It
> also helps to decide if it's ok to merge a Python 3 change, or if it's
> better to focus on the other changes in conflict.
> 
> Victor
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka cycle?

2016-02-18 Thread Victor Stinner 

Le 18/02/2016 14:15, Amrith Kumar a écrit :

Let's definitely discuss this again once you have all the changes that you feel 
should be merged for Mitaka ready.


I don't like working on long patch series. In my experience, after more 
than 4 patches, it's more expensive to maintain the patch serie than to 
write patches. So I prefer to work on few patches, wait until they are 
merged, and then write following patches.


I'm not going to write dozens of patches. I suggest to do as I done in 
the paste, make progress with baby steps :-)


For example, my first change only changes the py34 test environment in 
tox.ini, it cannot break anything on Python 2, and it's enough to fix 
"tox -e py34". It is not in conflict with any other pending change.

https://review.openstack.org/#/c/279098/

From this point, we can add a voting gate to be able to validate 
following Python 3 changes.



> What I would like to avoid is a dribble of changes where we don't 
know how much more we have coming down the pike.


You have to be prepared for dozens of small patches. It only depends on 
the size of your project (number of code line numbers) :-)


To have an idea, you can see the Cinder blueprint which has an 
exhaustive list of all changes made for Python 3:

https://blueprints.launchpad.net/cinder/+spec/cinder-python3

I counted 100 patches between June 2015 and February 2016.

FYI with all my pending patches for Cinder (only 4 changes remain), all 
unit tests will pass on Python 3!


It also gives you an idea of the time frame: it took me 9 months to port 
Cinder unit tests to Python 3. So more than a single OpenStack cycle (6 
months).


Since the port is long and painful, I would like to start as soon as 
possible :-)



> And while your changes may be "low risk", it does mean that if they 
merge now, the large feature sets that we have committed for this 
release will have to go through the cycle of merge conflicts, rebasing, 
code review, gate ... and so on.


The principle of technical debt is that the price only is only 
increasing if you wait longer :-) Merging Python 3 today or tomorrow 
doesn't solve the problem of merge conflicts :-)


It's really up to you to decide to "open the gate" for the flow of 
Python 3 patches, it's also up to you to control how much Python 3 
changes will merged. I can only offer my help to port code. I don't feel 
able to decide when it's the best time to start porting Trove ;-)


By the way, Gerrit provides a great "Conflicts With" information! It 
also helps to decide if it's ok to merge a Python 3 change, or if it's 
better to focus on the other changes in conflict.


Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka cycle?

2016-02-18 Thread Flavio Percoco 

On 18/02/16 13:15 +, Amrith Kumar wrote:

Victor, thanks for the changes and the patch sets.

TL;DR: We've discussed this a couple of times already, once at a Trove 
meeting[1], once at length at the midcycle, and concluded that post-Mitaka is 
the right time to merge changes relative to Python 3. Once you have all the 
changes that you feel should be merged for Mitaka relative to Python 3, let us 
revisit for sure.

-- Longer version --

At this point in the development cycle, the intent is that we work on and 
submit code for accepted and committed projects for the Mitaka cycle, and bug 
fixes. Python 3 was not an accepted and committed project for Trove in the 
Mitaka cycle.

This is not the first time when a "low risk" change set for a project will be 
proposed and someone will want to have it included in the release even at this stage, and 
I don't believe that it will be the last time. For those who would like to work on the 
Python 3 port, I believe that like other multi-commit projects, they can cherry pick your 
code, or make their patches dependent on your changes. I don't believe that a failure to 
merge these into Mitaka would obstruct their ongoing development.

And while your changes may be "low risk", it does mean that if they merge now, 
the large feature sets that we have committed for this release will have to go through 
the cycle of merge conflicts, rebasing, code review, gate ... and so on.

We discussed this matter at some length at a Trove meeting [1], and we 
discussed it again at the mid-cycle. The comment you reference is the result of 
that discussion at the mid-cycle.

If I had my way, I'd rather hold any spare cycles available to get the project 
that we wanted in Mitaka (backup to Ceph [2]), which is currently in jeopardy 
of not making the Mitaka deadlines.

Let's definitely discuss this again once you have all the changes that you feel 
should be merged for Mitaka ready. What I would like to avoid is a dribble of 
changes where we don't know how much more we have coming down the pike. Once 
the committed projects for Mitaka have been merged, it may be reasonable to 
take all of these changes in one set.



My experience from other projects is that py3 patches will come and they'll keep
coming until the gate is made voting. Requesting the folks working on the py3
port to get all the patches ready before doing proper reviews adds a significant
amount of work to the team.

In Glance, Py3 patches have always been small and they have not introduced other
issues (at least that I can remember).

The above is not to ask the Trove team to change the project's priorities but
just to provide feedback from other projects. I do recommend, however, to make
this a priority for newton if it doesn't make it in Mitaka. The Py3 effort has
been huge and it is becoming more and more of an important support to provide.

Flavio



-amrith

--
Amrith Kumar, CTO   | amr...@tesora.com
Tesora, Inc | @amrithkumar
125 CambridgePark Drive, Suite 400  | http://www.tesora.com
Cambridge, MA. 02140| GPG: 0x5e48849a9d21a29b

[1] 
https://wiki.openstack.org/wiki/Trove/MeetingAgendaHistory#Trove_Meeting.2C_Jan_20.2C_2016
[2] https://review.openstack.org/#/c/256057/




-Original Message-
From: Victor Stinner [mailto:vstin...@redhat.com]
Sent: Thursday, February 18, 2016 7:20 AM
To: OpenStack Development Mailing List (not for usage questions)

Subject: [openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka
cycle?

Hi,

When I began to work on porting Trove to Python 3, I was blocked by MySQL-
Python which is not compatible with Python 3. I tried a big change
replacing MySQL-Python with PyMySQL, since other OpenStack services also
moved to PyMySQL. But tests fail and I'm unable to fix them :-/
https://review.openstack.org/#/c/225915/

Recently, I noticed that the dependency is now skipped on Python 3 (thanks
to env markers in requirements.txt), and so "tox -e py34" is able to
create the test environment.

So I abandoned my PyMySQL change (I will reopen it later) and started new
simpler patches following the plan of my Python 3 blueprint for Trove:
https://blueprints.launchpad.net/trove/+spec/trove-python3

In short:

(1) fix the Python 3 gate
(2) make the Python 3 gate voting
(3) port more and more unit tests

My patches:

trove: "Add a minimal py34 test environment"
https://review.openstack.org/#/c/279098/
=> fix "tox -e py34", start with a whitelist of the 3 most basic unit
tests

trove: "Port test_template unit test to Python 3"
https://review.openstack.org/#/c/279119/
=> port another unit test

openstack-infra/project-config: "Add non-voting gate-trove-python34 check"
https://review.openstack.org/#/c/279108/


IMHO these changes are simple and the risk of regression is low, but
amrith wrote me "thanks for your change set but per last trove meeting, I
think this should wait till mitaka is done,

Re: [openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka cycle?

2016-02-18 Thread Amrith Kumar 
Victor, thanks for the changes and the patch sets.

TL;DR: We've discussed this a couple of times already, once at a Trove 
meeting[1], once at length at the midcycle, and concluded that post-Mitaka is 
the right time to merge changes relative to Python 3. Once you have all the 
changes that you feel should be merged for Mitaka relative to Python 3, let us 
revisit for sure.

-- Longer version --

At this point in the development cycle, the intent is that we work on and 
submit code for accepted and committed projects for the Mitaka cycle, and bug 
fixes. Python 3 was not an accepted and committed project for Trove in the 
Mitaka cycle.

This is not the first time when a "low risk" change set for a project will be 
proposed and someone will want to have it included in the release even at this 
stage, and I don't believe that it will be the last time. For those who would 
like to work on the Python 3 port, I believe that like other multi-commit 
projects, they can cherry pick your code, or make their patches dependent on 
your changes. I don't believe that a failure to merge these into Mitaka would 
obstruct their ongoing development.

And while your changes may be "low risk", it does mean that if they merge now, 
the large feature sets that we have committed for this release will have to go 
through the cycle of merge conflicts, rebasing, code review, gate ... and so on.

We discussed this matter at some length at a Trove meeting [1], and we 
discussed it again at the mid-cycle. The comment you reference is the result of 
that discussion at the mid-cycle.

If I had my way, I'd rather hold any spare cycles available to get the project 
that we wanted in Mitaka (backup to Ceph [2]), which is currently in jeopardy 
of not making the Mitaka deadlines.

Let's definitely discuss this again once you have all the changes that you feel 
should be merged for Mitaka ready. What I would like to avoid is a dribble of 
changes where we don't know how much more we have coming down the pike. Once 
the committed projects for Mitaka have been merged, it may be reasonable to 
take all of these changes in one set.

-amrith

--
Amrith Kumar, CTO   | amr...@tesora.com
Tesora, Inc | @amrithkumar
125 CambridgePark Drive, Suite 400  | http://www.tesora.com
Cambridge, MA. 02140| GPG: 0x5e48849a9d21a29b 

[1] 
https://wiki.openstack.org/wiki/Trove/MeetingAgendaHistory#Trove_Meeting.2C_Jan_20.2C_2016
[2] https://review.openstack.org/#/c/256057/



> -Original Message-
> From: Victor Stinner [mailto:vstin...@redhat.com]
> Sent: Thursday, February 18, 2016 7:20 AM
> To: OpenStack Development Mailing List (not for usage questions)
> 
> Subject: [openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka
> cycle?
> 
> Hi,
> 
> When I began to work on porting Trove to Python 3, I was blocked by MySQL-
> Python which is not compatible with Python 3. I tried a big change
> replacing MySQL-Python with PyMySQL, since other OpenStack services also
> moved to PyMySQL. But tests fail and I'm unable to fix them :-/
> https://review.openstack.org/#/c/225915/
> 
> Recently, I noticed that the dependency is now skipped on Python 3 (thanks
> to env markers in requirements.txt), and so "tox -e py34" is able to
> create the test environment.
> 
> So I abandoned my PyMySQL change (I will reopen it later) and started new
> simpler patches following the plan of my Python 3 blueprint for Trove:
> https://blueprints.launchpad.net/trove/+spec/trove-python3
> 
> In short:
> 
> (1) fix the Python 3 gate
> (2) make the Python 3 gate voting
> (3) port more and more unit tests
> 
> My patches:
> 
> trove: "Add a minimal py34 test environment"
> https://review.openstack.org/#/c/279098/
> => fix "tox -e py34", start with a whitelist of the 3 most basic unit
> tests
> 
> trove: "Port test_template unit test to Python 3"
> https://review.openstack.org/#/c/279119/
> => port another unit test
> 
> openstack-infra/project-config: "Add non-voting gate-trove-python34 check"
> https://review.openstack.org/#/c/279108/
> 
> 
> IMHO these changes are simple and the risk of regression is low, but
> amrith wrote me "thanks for your change set but per last trove meeting, I
> think this should wait till mitaka is done, and we can pick it up early in
> newton."
> 
> I discussed with some Trove developers who are interested to start the
> Python 3 port right now. What do you think?
> 
> Maybe we can discuss that in the next Trove meeting?
> https://wiki.openstack.org/wiki/Meetings/TroveMeeting
> (Wednesdays at 18:00 UTC in #openstack-meeting-alt)
> 
> Oops, I just missed the meeting yesterday. I was too slow to write this
> email :-)
> 
> Victor
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>

[openstack-dev] [trove] Start to port Trove to Python 3 in Mitaka cycle?

2016-02-18 Thread Victor Stinner 

Hi,

When I began to work on porting Trove to Python 3, I was blocked by 
MySQL-Python which is not compatible with Python 3. I tried a big change 
replacing MySQL-Python with PyMySQL, since other OpenStack services also 
moved to PyMySQL. But tests fail and I'm unable to fix them :-/

https://review.openstack.org/#/c/225915/

Recently, I noticed that the dependency is now skipped on Python 3 
(thanks to env markers in requirements.txt), and so "tox -e py34" is 
able to create the test environment.


So I abandoned my PyMySQL change (I will reopen it later) and started 
new simpler patches following the plan of my Python 3 blueprint for Trove:

https://blueprints.launchpad.net/trove/+spec/trove-python3

In short:

(1) fix the Python 3 gate
(2) make the Python 3 gate voting
(3) port more and more unit tests

My patches:

trove: "Add a minimal py34 test environment"
https://review.openstack.org/#/c/279098/
=> fix "tox -e py34", start with a whitelist of the 3 most basic unit tests

trove: "Port test_template unit test to Python 3"
https://review.openstack.org/#/c/279119/
=> port another unit test

openstack-infra/project-config: "Add non-voting gate-trove-python34 check"
https://review.openstack.org/#/c/279108/


IMHO these changes are simple and the risk of regression is low, but 
amrith wrote me "thanks for your change set but per last trove meeting, 
I think this should wait till mitaka is done, and we can pick it up 
early in newton."


I discussed with some Trove developers who are interested to start the 
Python 3 port right now. What do you think?


Maybe we can discuss that in the next Trove meeting?
https://wiki.openstack.org/wiki/Meetings/TroveMeeting
(Wednesdays at 18:00 UTC in #openstack-meeting-alt)

Oops, I just missed the meeting yesterday. I was too slow to write this 
email :-)


Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev