I have been doing the same thing over the last week. At PyCon Barry
Warsaw, Lennart Regebro, and several others held a porting from 2 to 3
clinic where I got some really great tips. They answered all the issues I
thought would be hard, and I figured I should do the updates while the
fixes were fresh in my mind. So over the last week I have been merging my
Python 2 & 3 code, and I finally finished it yesterday. I cleaned up the
C/generator code today, and tested it by reading & writing to Riak.
So some notes about my approach:
- I went with single code base as well. When I got to the point of
updating the setup to run 2to3 I realized this would be hard (specifically
for tests). The 2 & 3 code was already 99% similar, so I figured single
source is better.
- We run 2.6 and 3.2 at OpenX, and that is what I developed against.
Everything works for 2.6+ and 3.2+.
- The python 2 API might be slightly different now. I still need to do
more testing here to make sure everything works as I expect. String fields
should only accept unicode (u"") now, and byte fields should only accept
bytes/str (b""). Literals ("") are by default str, but if you import
unicode_literals, they become unicode. I'm not sure how strictly protobufs
enforces these type checks, but python2 code that passed in str objects for
string fields might need to be fixed to pass in unicode.
The repo is located here.
https://github.com/openx/python3-protobuf
Also, for those that want to test this I have some simple build
instructions. I don't have much C experience, so I spent a good 20-30
minutes figuring out the build the first time. I had to install gcc-c++,
autoconf and automake. Then in the base directory I ran:
./autogen.sh
./configure
make checks (optional)
make install
Thanks,
Charles
On Wednesday, March 20, 2013 7:53:18 AM UTC-7, Malthe Borch wrote:
>
> I have completed this work: https://github.com/malthe/google-protobuf.
>
> – in as much as that all tests run without fail on both 2.7 and 3.3. I
> have used a single-source approach (which is only really feasible starting
> with those two for syntax compatibility reasons).
>
> Python 2.4, 2.5 and I believe even 2.6 simply aren't going to work. It's
> too much effort.
>
> \malthe
>
> On Saturday, September 22, 2012 3:16:41 AM UTC+2, Charles Law wrote:
>>
>> I've made an attempt to create a Python3 compatible version of protobufs.
>> I have some code that passes pretty much all the unit tests which I've
>> posted here:
>>
>> https://github.com/openx/python3-protobuf
>>
>> I probably won't have a chance to look at this again for a couple weeks
>> if not longer, so I want to get it out there. In my attempt I decided to
>> follow the advice in another post, and treat python3 as a new language. To
>> get python3 working, you'll have to compile the C code. There are also a
>> few issues I ran into along the way:
>>
>> - I decided to use strings where unicode is used in Python 2. I was
>> originally going to try to use bytes/bytearrays, but they do not support
>> >8
>> bit characters, and some of the setup.py tests use "exotic" 16 bit chrs.
>> (Warning: I might have something conceptually wrong here)
>> - There are places where byte data is stored as strings, then
>> converted to unicode. I ended up converting strings (I called them
>> bytestr's) to normal strings. I'm not sure this is done correctly
>> everywhere though.
>> - Data is packed/unpacked using struct.pack/unpack which is done
>> using bytes instead of strings in Python3. I have simple
>> string_to_bytes()
>> and bytes_to_string() functions to do this.
>>
>>
>> What's left is:
>>
>> - There are a couple Exceptions that I don't throw. They are
>> supposed to be where the Python2 code converts from unicode strings to
>> regular strings. I am definitely missing something conceptually here - I
>> haven't figured out how Python 2x supports strings with "exotic"
>> characters, but not strings like u'a\x80a'. If someone can solve this
>> problem & figure out when to throw the exceptions Python3 will be *
>> fully* working.
>>
>>
>> I might have small bits of time here or there but I don't think I can
>> devote the time I need to get this finished for several weeks, so if
>> someone wants to finish this up, feel free to fork this code. If anyone
>> wants to see what I did, the best way to do this is to diff between the
>> latest commit and commit 49ccf5d8b3b688c335dc35bcb9f219eca78c7210.
>> Thanks!
>> Charles
>>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/protobuf?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
