Re: Quick Hacky text_mode Parse in Python
On Tue, Jan 20, 2009 at 3:58 PM, Helder Suzuki heldersuz...@gmail.com wrote:
I sent a partial patch a while ago and I disappeared without completing the
text_format in python, sorry about that.
So far I've only implemented the tokenizer part (w/ test cases), but anyone
is free to use it to implement the parser part (I'd be really glad), and for
some reason I couldn't set up the codereview (I didn't try hard though). For
various reasons I won't be able to touch it in the next few weeks.
This is such an important feature, specially if you use protobufs for
configuration files, it'd be so handy!
Can you sign the contributor agreement:
http://code.google.com/legal/individual-cla-v1.0.html
It's required for us to accept your patch (each contributor has to sign it).
I have reviewed you patch but by some reason I'm unable to send you
the comments.
We have 2 choices:
1. I can apply my own code review comments and submit it in the
internal repository then I will write the rest of the ParseASCII code.
2. You can apply the code review comments below and I'll submit the
patch in external SVN.
In any case you have to sign the agreement above.
Here are my comments on the patch:
google/protobuf/text_format.py:
-__all__ = [ 'MessageToString', 'PrintMessage', 'PrintField',
'PrintFieldValue' ]
+#__all__ = ['MessageToString', 'PrintMessage', 'PrintField', 'PrintFieldValue']
(Draft) 2008/12/08 18:37:10 Why is this line commented?
+class Token(object):
+ TODO(helder): Document me.
(Draft) 2008/12/08 18:37:50 Please add the docstrings.
+class Tokenizer(object):
+ TODO(helder): Document me.
(Draft) 2008/12/08 18:39:09 1-2 sentences will be enough.
+ def ConsumeNumber(self, started_with_zero, started_with_dot):
+ is_float = False
+ if started_with_zero and self.TryConsumeOneOf(('x', 'X')):
+ # A hex number (started with 0x).
+ self.ConsumeOneOrMore(HexDigit, '0x must be followed by hex digits.')
+ elif started_with_zero and self.LookingAt(OctaDigit):
(Draft) 2008/12/08 19:05:42 LookingAt(OctaDigit) - LookingAt(Digit).
It isn't a bug but a hidden feature. This will accept 09 as a decimal
9 (will proceed with the next case - decimal number). I think in
this case we want to error out with a message like the one below -
Numbers starting with leading zero must be in octal.. That's what
Kenton's implementation does.
google/protobuf/internal/text_format_test.py:
+ def testParseInteger(self):
+ self.assertEqual(text_format.ParseInteger('0'), 0)
+ self.assertEqual(text_format.ParseInteger('1'), 1)
+ self.assertEqual(text_format.ParseInteger('012345'), 5349) # base 8
(Draft) 2008/12/08 19:41:49 Can you also add a test which tries to
parse 09. It should result in an error.
Thanks,
Helder
On Tue, Dec 30, 2008 at 12:19 AM, Piotr Findeisen piotr.findei...@gmail.com
wrote:
On Dec 8, 7:27 pm, Petar Petrov pe...@google.com wrote:
On Mon, Dec 8, 2008 at 10:21 AM, Kenton Varda ken...@google.com wrote:
Hey Petar, isn't there a patch someone was trying to submit that
implements
text format parsing? (For real, not by wrapping protoc.) What's the
status
of that?
I'll review it today.
Hopefully the author hasn't forgotten about it.
Hey!
This gonna be a feature I miss really much!
Is there happening anything about this?
regards!
Piotr
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Quick Hacky text_mode Parse in Python
I sent a partial patch a while ago and I disappeared without completing the text_format in python, sorry about that. So far I've only implemented the tokenizer part (w/ test cases), but anyone is free to use it to implement the parser part (I'd be really glad), and for some reason I couldn't set up the codereview (I didn't try hard though). For various reasons I won't be able to touch it in the next few weeks. This is such an important feature, specially if you use protobufs for configuration files, it'd be so handy! Thanks, Helder On Tue, Dec 30, 2008 at 12:19 AM, Piotr Findeisen piotr.findei...@gmail.com wrote: On Dec 8, 7:27 pm, Petar Petrov pe...@google.com wrote: On Mon, Dec 8, 2008 at 10:21 AM, Kenton Varda ken...@google.com wrote: Hey Petar, isn't there a patch someone was trying to submit that implements text format parsing? (For real, not by wrapping protoc.) What's the status of that? I'll review it today. Hopefully the author hasn't forgotten about it. Hey! This gonna be a feature I miss really much! Is there happening anything about this? regards! Piotr --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Quick Hacky text_mode Parse in Python
On Dec 8, 7:27 pm, Petar Petrov pe...@google.com wrote: On Mon, Dec 8, 2008 at 10:21 AM, Kenton Varda ken...@google.com wrote: Hey Petar, isn't there a patch someone was trying to submit that implements text format parsing? (For real, not by wrapping protoc.) What's the status of that? I'll review it today. Hopefully the author hasn't forgotten about it. Hey! This gonna be a feature I miss really much! Is there happening anything about this? regards! Piotr --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Quick Hacky text_mode Parse in Python
Hey Petar, isn't there a patch someone was trying to submit that implements
text format parsing? (For real, not by wrapping protoc.) What's the status
of that?
On Mon, Dec 8, 2008 at 5:03 AM, Nicholas Reid [EMAIL PROTECTED] wrote:
Hi All,
Firstly, just wanted to thank Kenton and the Google team, PB2 is a
beautiful piece of work! Thanks heaps.
I will almost certainly go to some deep circle of Programmer's Hell
for this, but it might be useful for someone until the guys get a
chance to add text_mode message parsing functionality to the Python
API. There are almost certainly more elegant ways of doing this.
Code:
def parse_text_format(message_string, generated_message_type):
Parses the given Protobuf text_format into a new instance of
the given type.
# Should be defined globally somewhere
PROTO_FILENAME = person.proto
# Instance new message
obj = generated_message_type()
# Wrap the protoc command-line utility, expects that 'protoc'
should be on your PATH somewhere
(stdout, stdin) = popen2.popen2(protoc %s --encode=%s %
(PROTO_FILENAME, message_type.DESCRIPTOR.name), bufsize=1024)
# Feed in the message_string in text_format
stdin.write(message_string)
stdin.close()
# Read out the protoc-encoded binary format
binary_string = stdout.read()
stdout.close()
# Parse the resulting binary representation.
obj.ParseFromString(binary_string)
return obj
Example:
Assuming person.proto contains:
message Person {
required string name = 1;
}
Code:
from person_pb2 import *
guido = parse_text_format(name: Guido, person)
Should give you a person object which you can use for nefarious
purposes.
Kind regards,
Nicholas Reid
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Quick Hacky text_mode Parse in Python
On Mon, Dec 8, 2008 at 10:21 AM, Kenton Varda [EMAIL PROTECTED] wrote:
Hey Petar, isn't there a patch someone was trying to submit that implements
text format parsing? (For real, not by wrapping protoc.) What's the status
of that?
I'll review it today.
Hopefully the author hasn't forgotten about it.
On Mon, Dec 8, 2008 at 5:03 AM, Nicholas Reid [EMAIL PROTECTED] wrote:
Hi All,
Firstly, just wanted to thank Kenton and the Google team, PB2 is a
beautiful piece of work! Thanks heaps.
I will almost certainly go to some deep circle of Programmer's Hell
for this, but it might be useful for someone until the guys get a
chance to add text_mode message parsing functionality to the Python
API. There are almost certainly more elegant ways of doing this.
Code:
def parse_text_format(message_string, generated_message_type):
Parses the given Protobuf text_format into a new instance of
the given type.
# Should be defined globally somewhere
PROTO_FILENAME = person.proto
# Instance new message
obj = generated_message_type()
# Wrap the protoc command-line utility, expects that 'protoc'
should be on your PATH somewhere
(stdout, stdin) = popen2.popen2(protoc %s --encode=%s %
(PROTO_FILENAME, message_type.DESCRIPTOR.name), bufsize=1024)
# Feed in the message_string in text_format
stdin.write(message_string)
stdin.close()
# Read out the protoc-encoded binary format
binary_string = stdout.read()
stdout.close()
# Parse the resulting binary representation.
obj.ParseFromString(binary_string)
return obj
Example:
Assuming person.proto contains:
message Person {
required string name = 1;
}
Code:
from person_pb2 import *
guido = parse_text_format(name: Guido, person)
Should give you a person object which you can use for nefarious
purposes.
Kind regards,
Nicholas Reid
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
