Re: [openstack-dev] [nova] Port Nova to Python 3

2015-05-11 Thread Victor Stinner
Hi,

 Oh, this makes me think: how would one fix something like this?

The wiki page already contains some answer:
https://wiki.openstack.org/wiki/Python3#Common_patterns

Don't hesitate to complete the page if needed.

See also my personal list of documents:
https://haypo-notes.readthedocs.org/python.html#port-python-2-code-to-python-3


  def unicode_convert(self, item):
  try:
 return unicode(item, utf-8)
 E   NameError: name 'unicode' is not defined

Use six.text_type.


  def make(self, idp, sp, args):
  md5 = hashlib.md5()
  for arg in args:
  md5.update(arg.encode(utf-8))
 md5.update(sp)
 E   TypeError: Unicode-objects must be encoded before hashing

It depends on the code. Sometimes, you should encode sp in the caller, 
sometimes you should encode just before calling make(). If you don't know, use:

if isinstance(sp, six.text_type):
sp = sp.encode(utf-8)
md5.update(sp)


 and one last one:
 
  def harvest_element_tree(self, tree):
  # Fill in the instance members from the contents of the
  # XML tree.
  for child in tree:
  self._convert_element_tree_to_member(child)
 for attribute, value in tree.attrib.iteritems():
  self._convert_element_attribute_to_member(attribute, value)
 E   AttributeError: 'dict' object has no attribute 'iteritems'

Use six.iteritems().


 BTW, I did this:
 -from Cookie import SimpleCookie
 +try:
 +from Cookie import SimpleCookie
 +except:
 +from http.cookies import SimpleCookie
 
 Is there anything smarter to do with six? What's the rule with
 six.moves? Should I always just use the new location?

I did the same. If it's a very common pattern in your code, you can register 
you own move in six.moves:
https://pythonhosted.org/six/#advanced-customizing-renames

We used that fox mox/mox3 in tests of Oslo projects.


 Also, is this a correct fix for the basestring issue in Py3?
 
 +try:
 +basestring
 +except NameError:
 +basestring = (str,bytes)

I prefer six.string_types.

Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-05-07 Thread Thomas Goirand


On 04/24/2015 10:20 PM, Kevin L. Mitchell wrote:

On Fri, 2015-04-24 at 16:07 -0400, Sean Toner wrote:

What I meant by the worst of both worlds is that you don't get the nice
new features of python3, while simultaneously dealing with the headaches
of making code run under both python versions.  You'll have to do weird
things with imports (for example urllib) and deal with the
inconsistencies between some functions that return strings and some that
return unicode, and some that return bytes.

It's not impossible, but you have to add that extra work while also
depriving yourself of the goodness of python3 only features :)


This is why the 'six' library is such a godsend; this stuff is still not
easy, but the hardest parts, like the imports problem, are already taken
care of by six…and maintaining the bytes/strings/unicode distinction is
actually just as useful in Python 2, it just doesn't have the machinery
to really detect the mixing :)


Oh, this makes me think: how would one fix something like this?

def unicode_convert(self, item):
try:
   return unicode(item, utf-8)
E   NameError: name 'unicode' is not defined

and something like this?

def make(self, idp, sp, args):
md5 = hashlib.md5()
for arg in args:
md5.update(arg.encode(utf-8))
   md5.update(sp)
E   TypeError: Unicode-objects must be encoded before hashing

and one last one:

def harvest_element_tree(self, tree):
# Fill in the instance members from the contents of the
# XML tree.
for child in tree:
self._convert_element_tree_to_member(child)
   for attribute, value in tree.attrib.iteritems():
self._convert_element_attribute_to_member(attribute, value)
E   AttributeError: 'dict' object has no attribute 'iteritems'

I once found a document on the net about how to fix the iteritems 
thingy, but I can't find it again... :(


BTW, I did this:
-from Cookie import SimpleCookie
+try:
+from Cookie import SimpleCookie
+except:
+from http.cookies import SimpleCookie

Is there anything smarter to do with six? What's the rule with 
six.moves? Should I always just use the new location?


Also, is this a correct fix for the basestring issue in Py3?

+try:
+basestring
+except NameError:
+basestring = (str,bytes)

(FYI: I am trying to port pysaml2 to Python 3, and I already have a 
nearly 5k lines patch...)


Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-05-04 Thread Victor Stinner
Hi,

 I wrote a spec to port Nova to Python 3:

   https://review.openstack.org/#/c/176868/

I updated my spec to take in account all comments. Example of changes:

- Explicitly say that Python 2 support is kept (ex: change the title to Adding 
Python 3.4 support to Nova)
- Clarify what are the Nova tests
- Shorter spec

I prefer to exclude Tempest tests, and restrict the scope of the spec to Nova 
unit and functional tests. Most Python 3 issues should be catched by Nova unit 
and functional tests anyway.

Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Victor Stinner
Hi,

Porting OpenStack applications during the Liberty Cycle was discussed last days 
in the thread [oslo] eventlet 0.17.3 is now fully Python 3 compatible.

I wrote a spec to port Nova to Python 3:

   https://review.openstack.org/#/c/176868/

I mentioned the 2 other Python 3 specs for Neutron and Heat.

You can reply to this email, or comment the review, if you want to discuss 
Python 3 in Nova, or if you have any question related to Python 3.

See also the Python 3 wiki page:

   https://wiki.openstack.org/wiki/Python3

Thanks,
Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Rui Chen
Maybe we can add a python3 jenkins job (non-voting) to help us finding out
some potential issue.

2015-04-24 16:34 GMT+08:00 Victor Stinner vstin...@redhat.com:

 Hi,

 Porting OpenStack applications during the Liberty Cycle was discussed last
 days in the thread [oslo] eventlet 0.17.3 is now fully Python 3
 compatible.

 I wrote a spec to port Nova to Python 3:

https://review.openstack.org/#/c/176868/

 I mentioned the 2 other Python 3 specs for Neutron and Heat.

 You can reply to this email, or comment the review, if you want to discuss
 Python 3 in Nova, or if you have any question related to Python 3.

 See also the Python 3 wiki page:

https://wiki.openstack.org/wiki/Python3

 Thanks,
 Victor

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Victor Stinner
Hi,

 If written to use python 3, I hope it will use all the new features of 
 python 3.4 moving forward.

The spec adds Python 3 support, but it keeps Python 2 support. It's too early 
to drop Python 2, Nova is deployed everywhere with Python 2.


 For example, argument annotations, coroutines, asyncio, etc.

Argument annotations are not used in practice :-( There is a PEP under review 
which targets the future Python 3.5 version:

   https://www.python.org/dev/peps/pep-0484/

I'm working actively on asyncio. I wrote a spec to replace eventlet with 
asyncio:

   https://review.openstack.org/#/c/153298/
   superseded by: https://review.openstack.org/#/c/164035/

For OpenStack, I ported asyncio to Python 2: it's the Trollius project:

   https://trollius.readthedocs.org/

I would also prefer to be able to use new shiny Python 3 features, but it's not 
possible yet. We have to move step by step. There is no choice like Python 2 
only or Python 3 only with new Python 3 features. Changes must be done 
incrementally in OpenStack. We all know that.


 At my last workplace, we tried to make our project python2 and 3
 compatible (ie, you could run it under 2.7 or 3.3+) but this was
 the worst of all worlds.

Does it mean that you are against the whole spec?

I don't know any Python project in the wild which was really ported to Python 
3: drop Python 2 support at the same time. Supporting only Python 3 only slowly 
becomes a good choice for *new* projects.

All projects are ported by adding Python 3 support in addition to Python 2 
support. The six module is a great module to limit changes on the source code.

See my early draft patch to port nova to Python 3:

   https://github.com/haypo/nova/commit/bad54bc2b278c7c7cb7fa6cc73d03c70138bd89d

Joe Goron wrote I like how the sha1 starts with 'bad'. Overall that is a 
pretty small patch. ;-)

Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Sean Toner
If written to use python 3, I hope it will use all the new features of 
python 3.4 moving forward.

For example, argument annotations, coroutines, asyncio, etc.  At my last
workplace, we tried to make our project python2 and 3 compatible (ie, 
you could run it under 2.7 or 3.3+) but this was the worst of all 
worlds.

Regards,
Sean

On Friday, April 24, 2015 04:34:48 AM Victor Stinner wrote:
 Hi,
 
 Porting OpenStack applications during the Liberty Cycle was discussed
 last days in the thread [oslo] eventlet 0.17.3 is now fully Python 3
 compatible.
 
 I wrote a spec to port Nova to Python 3:
 
https://review.openstack.org/#/c/176868/
 
 I mentioned the 2 other Python 3 specs for Neutron and Heat.
 
 You can reply to this email, or comment the review, if you want to
 discuss Python 3 in Nova, or if you have any question related to
 Python 3.
 
 See also the Python 3 wiki page:
 
https://wiki.openstack.org/wiki/Python3
 
 Thanks,
 Victor
 
 __
  OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Joshua Harlow

Sean Toner wrote:

If written to use python 3, I hope it will use all the new features of
python 3.4 moving forward.

For example, argument annotations, coroutines, asyncio, etc.  At my last
workplace, we tried to make our project python2 and 3 compatible (ie,
you could run it under 2.7 or 3.3+) but this was the worst of all
worlds.


Likely the reality of things is different from the desires :)

I know we'd all love to just do the above (or some of us would like to), 
but we also need to think about what shiny new features really make the 
quality of nova any better (IMHO some of the above really don't change 
much for the better...)


Out of curiosity why was your experience 'the worst of all worlds'?



Regards,
Sean

On Friday, April 24, 2015 04:34:48 AM Victor Stinner wrote:

Hi,

Porting OpenStack applications during the Liberty Cycle was discussed
last days in the thread [oslo] eventlet 0.17.3 is now fully Python 3
compatible.

I wrote a spec to port Nova to Python 3:

https://review.openstack.org/#/c/176868/

I mentioned the 2 other Python 3 specs for Neutron and Heat.

You can reply to this email, or comment the review, if you want to
discuss Python 3 in Nova, or if you have any question related to
Python 3.

See also the Python 3 wiki page:

https://wiki.openstack.org/wiki/Python3

Thanks,
Victor

__
 OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Clint Byrum
Excerpts from Sean Toner's message of 2015-04-24 05:38:42 -0700:
 If written to use python 3, I hope it will use all the new features of 
 python 3.4 moving forward.
 
 For example, argument annotations, coroutines, asyncio, etc.  At my last
 workplace, we tried to make our project python2 and 3 compatible (ie, 
 you could run it under 2.7 or 3.3+) but this was the worst of all 
 worlds.
 

Sean, this is a very developer-centric way of thinking.

Our operators will likely roll out python3 interpreters at a different
cadence than they roll out releases of OpenStack.

The best of both worlds is working perfectly fine on python 2.7 and
3.4+ until 2.7 is eradicated entirely, so that our operators can manage
change effectively.

The reason for this port is as much because 2.7 has an EOL that is
approximately 5 years away as anything else. If we are not on top of
this, OpenStack will be dependent on dead software rapidly. It has taken
2+ years to get this far with effort happening on the edges. I suspect it
will take another cycle before we start turning on gates, and then another
2 cycles before we're comfortable telling people to run OpenStack on 3.x.

So, we have to be patient with the new shiny, but we have to be very
_impatient_ with broken things.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Mike Bayer



On 4/24/15 4:18 PM, Sean Toner wrote:

On Friday, April 24, 2015 11:20:03 AM Joshua Harlow wrote:

Sean Toner wrote:

If written to use python 3, I hope it will use all the new features
of python 3.4 moving forward.

For example, argument annotations, coroutines, asyncio, etc.  At my
last workplace, we tried to make our project python2 and 3
compatible (ie, you could run it under 2.7 or 3.3+) but this was
the worst of all worlds.

Likely the reality of things is different from the desires :)

I know we'd all love to just do the above (or some of us would like
to), but we also need to think about what shiny new features really
make the quality of nova any better (IMHO some of the above really
don't change much for the better...)

Out of curiosity why was your experience 'the worst of all worlds'?

Like I answered to Victor, it means that you don't get the nice new
features only available in python3 (argument annotations, coroutines,
namespace packaging, etc) while at the same time dealing with the
inconsistencies between python2 and 3 code.


These are all very new features who's acceptance and maturity is yet to 
happen.  Even if these things were available in Python 2 right now, we 
should be very hesitant to sprint down new roads; these aren't assured 
to be winning features.   Also, I work in the realm of Py2k/3k 
cross-compatible code exclusively for several years now and I think 
you're overstating its difficulty, as did the core Python developers 
originally when they made us all believe we'd need a code rewriting tool 
in order to use Python 3, which then led us way down the wrong road 
which we all had to backtrack from.




For example, having to manage some imports, dealing with some functions
now returning bytes instead of str, and all kinds of other fun :)

Here's one example of having to change an import:

try:
 from urllib.request import urlopen
 from urllib.parse import urlparse as urlparse
except ImportError:
 from urllib2 import urlopen
 from urlparse import urlparse
compatibility layers like six handle issues like these pretty simply, 
plus I thought we had some requests-like compatibility layer in place in 
any case.





I also had some code that created a subprocess.Popen object to execute
some command.

i would have used the multiprocessing library instead.



  Some commands output got returned as a regular str, and
some others got returned as bytes.  So I wound up in my class creating a
defensive decorator function that called

if  isinstance(output, bytes):
   return output.decode()

And that's just 2 that popped into my head :)


How would not being Py2k compatible help with dealing with functions 
that randomly return either bytes or strings?





And it was frustrating
having to do extra work, and yet not be able to partake of all the
python3 goodies.

But, if this is a stepping stone to a pure python3 environment even if
that takes until python2 to EOL, I'm ok with that :)


Regards,
Sean

On Friday, April 24, 2015 04:34:48 AM Victor Stinner wrote:

Hi,

Porting OpenStack applications during the Liberty Cycle was
discussed
last days in the thread [oslo] eventlet 0.17.3 is now fully Python
3
compatible.

I wrote a spec to port Nova to Python 3:
 https://review.openstack.org/#/c/176868/

I mentioned the 2 other Python 3 specs for Neutron and Heat.

You can reply to this email, or comment the review, if you want to
discuss Python 3 in Nova, or if you have any question related to
Python 3.

See also the Python 3 wiki page:
 https://wiki.openstack.org/wiki/Python3

Thanks,
Victor

___
___  OpenStack Development Mailing List (not for usage
questions) Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
 OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Sean Toner
On Friday, April 24, 2015 11:20:03 AM Joshua Harlow wrote:
 Sean Toner wrote:
  If written to use python 3, I hope it will use all the new features
  of python 3.4 moving forward.
  
  For example, argument annotations, coroutines, asyncio, etc.  At my
  last workplace, we tried to make our project python2 and 3
  compatible (ie, you could run it under 2.7 or 3.3+) but this was
  the worst of all worlds.
 
 Likely the reality of things is different from the desires :)
 
 I know we'd all love to just do the above (or some of us would like
 to), but we also need to think about what shiny new features really
 make the quality of nova any better (IMHO some of the above really
 don't change much for the better...)
 
 Out of curiosity why was your experience 'the worst of all worlds'?

Like I answered to Victor, it means that you don't get the nice new 
features only available in python3 (argument annotations, coroutines, 
namespace packaging, etc) while at the same time dealing with the 
inconsistencies between python2 and 3 code.

For example, having to manage some imports, dealing with some functions 
now returning bytes instead of str, and all kinds of other fun :)

Here's one example of having to change an import:

try:
from urllib.request import urlopen
from urllib.parse import urlparse as urlparse
except ImportError:
from urllib2 import urlopen
from urlparse import urlparse

I also had some code that created a subprocess.Popen object to execute 
some command.  Some commands output got returned as a regular str, and 
some others got returned as bytes.  So I wound up in my class creating a 
defensive decorator function that called 

if  isinstance(output, bytes):
  return output.decode()

And that's just 2 that popped into my head :)  And it was frustrating 
having to do extra work, and yet not be able to partake of all the 
python3 goodies.

But, if this is a stepping stone to a pure python3 environment even if 
that takes until python2 to EOL, I'm ok with that :)

 
  Regards,
  Sean
  
  On Friday, April 24, 2015 04:34:48 AM Victor Stinner wrote:
  Hi,
  
  Porting OpenStack applications during the Liberty Cycle was
  discussed
  last days in the thread [oslo] eventlet 0.17.3 is now fully Python
  3
  compatible.
  
  I wrote a spec to port Nova to Python 3:
  https://review.openstack.org/#/c/176868/
  
  I mentioned the 2 other Python 3 specs for Neutron and Heat.
  
  You can reply to this email, or comment the review, if you want to
  discuss Python 3 in Nova, or if you have any question related to
  Python 3.
  
  See also the Python 3 wiki page:
  https://wiki.openstack.org/wiki/Python3
  
  Thanks,
  Victor
  
  ___
  ___  OpenStack Development Mailing List (not for usage
  questions) Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  
  __ OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 __
  OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Sean Toner
On Friday, April 24, 2015 09:13:14 AM Victor Stinner wrote:
 Hi,
 
  If written to use python 3, I hope it will use all the new features
  of python 3.4 moving forward.
 
 The spec adds Python 3 support, but it keeps Python 2 support. It's
 too early to drop Python 2, Nova is deployed everywhere with Python
 2.
  For example, argument annotations, coroutines, asyncio, etc.
 
 Argument annotations are not used in practice :-( There is a PEP under
 review which targets the future Python 3.5 version:
 
https://www.python.org/dev/peps/pep-0484/
 

Yeah, that is a shame.  I know many who are vehemently against argument 
annotations or type hinting, but i think it has its usefulness.

 I'm working actively on asyncio. I wrote a spec to replace eventlet
 with asyncio:
 
https://review.openstack.org/#/c/153298/
superseded by: https://review.openstack.org/#/c/164035/
 
 For OpenStack, I ported asyncio to Python 2: it's the Trollius
 project:
 
https://trollius.readthedocs.org/
 
 I would also prefer to be able to use new shiny Python 3 features, but
 it's not possible yet. We have to move step by step. There is no
 choice like Python 2 only or Python 3 only with new Python 3
 features. Changes must be done incrementally in OpenStack. We all
 know that.

I understand.  I hope this is a stepping stone to python 3 only :)

  At my last workplace, we tried to make our project python2 and 3
  compatible (ie, you could run it under 2.7 or 3.3+) but this was
  the worst of all worlds.
 
 Does it mean that you are against the whole spec?
 
 I don't know any Python project in the wild which was really ported to
 Python 3: drop Python 2 support at the same time. Supporting only
 Python 3 only slowly becomes a good choice for *new* projects.
 

This was a proprietary project at another company, so it wasn't open 
source. 

I'm not against the spec as it is.  If nothing else, it paves the way to 
eventually use the nice python 3+ only features :)

What I meant by the worst of both worlds is that you don't get the nice 
new features of python3, while simultaneously dealing with the headaches 
of making code run under both python versions.  You'll have to do weird 
things with imports (for example urllib) and deal with the 
inconsistencies between some functions that return strings and some that 
return unicode, and some that return bytes.

It's not impossible, but you have to add that extra work while also 
depriving yourself of the goodness of python3 only features :)

 All projects are ported by adding Python 3 support in addition to
 Python 2 support. The six module is a great module to limit changes
 on the source code.
 
 See my early draft patch to port nova to Python 3:
 
   
 https://github.com/haypo/nova/commit/bad54bc2b278c7c7cb7fa6cc73d03c70
 138bd89d
 
 Joe Goron wrote I like how the sha1 starts with 'bad'. Overall that
 is a pretty small patch. ;-)
 
 Victor
 
 __
  OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Kevin L. Mitchell
On Fri, 2015-04-24 at 16:07 -0400, Sean Toner wrote:
 What I meant by the worst of both worlds is that you don't get the nice 
 new features of python3, while simultaneously dealing with the headaches 
 of making code run under both python versions.  You'll have to do weird 
 things with imports (for example urllib) and deal with the 
 inconsistencies between some functions that return strings and some that 
 return unicode, and some that return bytes.
 
 It's not impossible, but you have to add that extra work while also 
 depriving yourself of the goodness of python3 only features :)

This is why the 'six' library is such a godsend; this stuff is still not
easy, but the hardest parts, like the imports problem, are already taken
care of by six…and maintaining the bytes/strings/unicode distinction is
actually just as useful in Python 2, it just doesn't have the machinery
to really detect the mixing :)
-- 
Kevin L. Mitchell kevin.mitch...@rackspace.com
Rackspace


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Port Nova to Python 3

2015-04-24 Thread Joshua Harlow

Sean Toner wrote:

On Friday, April 24, 2015 11:20:03 AM Joshua Harlow wrote:

Sean Toner wrote:

If written to use python 3, I hope it will use all the new features
of python 3.4 moving forward.

For example, argument annotations, coroutines, asyncio, etc.  At my
last workplace, we tried to make our project python2 and 3
compatible (ie, you could run it under 2.7 or 3.3+) but this was
the worst of all worlds.

Likely the reality of things is different from the desires :)

I know we'd all love to just do the above (or some of us would like
to), but we also need to think about what shiny new features really
make the quality of nova any better (IMHO some of the above really
don't change much for the better...)

Out of curiosity why was your experience 'the worst of all worlds'?


Like I answered to Victor, it means that you don't get the nice new
features only available in python3 (argument annotations, coroutines,
namespace packaging, etc) while at the same time dealing with the
inconsistencies between python2 and 3 code.


Time to start nova-redux[1] (dubstep remix?), best of luck :)

[1] 
http://www.gossamer-threads.com/lists/openstack/dev/7837?do=post_view_threaded#7837




For example, having to manage some imports, dealing with some functions
now returning bytes instead of str, and all kinds of other fun :)

Here's one example of having to change an import:

try:
 from urllib.request import urlopen
 from urllib.parse import urlparse as urlparse
except ImportError:
 from urllib2 import urlopen
 from urlparse import urlparse

I also had some code that created a subprocess.Popen object to execute
some command.  Some commands output got returned as a regular str, and
some others got returned as bytes.  So I wound up in my class creating a
defensive decorator function that called

if  isinstance(output, bytes):
   return output.decode()

And that's just 2 that popped into my head :)  And it was frustrating
having to do extra work, and yet not be able to partake of all the
python3 goodies.

But, if this is a stepping stone to a pure python3 environment even if
that takes until python2 to EOL, I'm ok with that :)


Regards,
Sean

On Friday, April 24, 2015 04:34:48 AM Victor Stinner wrote:

Hi,

Porting OpenStack applications during the Liberty Cycle was
discussed
last days in the thread [oslo] eventlet 0.17.3 is now fully Python
3
compatible.

I wrote a spec to port Nova to Python 3:
 https://review.openstack.org/#/c/176868/

I mentioned the 2 other Python 3 specs for Neutron and Heat.

You can reply to this email, or comment the review, if you want to
discuss Python 3 in Nova, or if you have any question related to
Python 3.

See also the Python 3 wiki page:
 https://wiki.openstack.org/wiki/Python3

Thanks,
Victor

___
___  OpenStack Development Mailing List (not for usage
questions) Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
 OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev