Python/uuid bytes field
We are working on a project where we are sending UUIDs between a backedn written in C++ and a Python front end. Our proto file specifies that we want to store these UUIDs in byte fields. This maps to a std::string in C++ which we can assign to manually. We are running into a problem on the Python side though because assigning the raw 16 bytes to the string field in Python gives us the following: exceptions.ValueError: ':\x19\x14\xd1Y\xf7\x11\xde\x9c[\x00\x1eO\xf3! \xd8' isn't in 7-bit ASCII encoding. This does make sense because Python is ensuring that the value is actually a string. My question then is how can we encode a UUID into a 16 byte data structure correctly? Is there a plan to fully support the bytes field type instead of simply mapping it to a string type? Thanks, Aron --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Usage RepeatedPtrField - Advanced memory mgmt
Hi, is there anybody out there who can tell me how the special methods like addcleared(), etc. for repeated field ptr work? I guess I can somehow improve/avoid the allocation overhead, right? Say, I have the following messages, message test { repeated chunk = 1; } message chunk { int value = 1; } The goal is to reduce the memory allocations by using pre-allocated objects. Assume the chunkPtr points to an already allocated object, if I use test.chunk().addAllocated(chunkPtr), does it just attaches the chunkPtr to the reaptedFieldPtr chunk object? Is there any overhead incured by dynamically adding the chunks to the test message? Can I deal with that issue by using the test.chunk().Reserve(xx) method? It seems also that I can maintain a pool of objects by using the addCleared() method, right? This way, future calls to test.chunk().add () will use an element from the pool? I was just wondering about these questions while reading the documentation... maybe you can just tell me if this usage pattern would be ok. Thx for the library and thx for the support Cheers Robert --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Java thread safety
The Java implementation does not contain any mutable singletons, so as long as you are accessing different objects, there is no need to worry about thread-safety. So, yes, multiple threads may call parseFrom() at the same time as long as they are parsing from different streams. On Mon, Jun 15, 2009 at 6:28 PM, Wayne wayne.mene...@gmail.com wrote: In the Java Generated code, there are functions like ParseFrom (CodedInputStream ...) that create protocol buffers messages from a file or other buffer. Can I call these directly from multiple different threads or should I use a wrapper with the synchronized keyword? Thanks, Wayne --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Usage RepeatedPtrField - Advanced memory mgmt
Everything you say sounds right. If you want to know precisely what these methods do, or how much overhead they incur, I encourage you to look at the code in repeated_field.h -- it's not very complicated. However, instead of trying to control all this in detail, it's usually easiest to simply make sure that you reuse your protobuf objects. If you parse a message, clear it, and then parse another message into the same object, it will reuse the memory for sub-objects and such which was allocated the first time. This way it's easy to make sure you aren't doing it wrong. On Tue, Jun 16, 2009 at 9:40 AM, RobertFach robert.f...@inf.tu-dresden.dewrote: Hi, is there anybody out there who can tell me how the special methods like addcleared(), etc. for repeated field ptr work? I guess I can somehow improve/avoid the allocation overhead, right? Say, I have the following messages, message test { repeated chunk = 1; } message chunk { int value = 1; } The goal is to reduce the memory allocations by using pre-allocated objects. Assume the chunkPtr points to an already allocated object, if I use test.chunk().addAllocated(chunkPtr), does it just attaches the chunkPtr to the reaptedFieldPtr chunk object? Is there any overhead incured by dynamically adding the chunks to the test message? Can I deal with that issue by using the test.chunk().Reserve(xx) method? It seems also that I can maintain a pool of objects by using the addCleared() method, right? This way, future calls to test.chunk().add () will use an element from the pool? I was just wondering about these questions while reading the documentation... maybe you can just tell me if this usage pattern would be ok. Thx for the library and thx for the support Cheers Robert --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Usage RepeatedPtrField - Advanced memory mgmt
However, instead of trying to control all this in detail, it's usually easiest to simply make sure that you reuse your protobuf objects. If you parse a message, clear it, and then parse another message into the same object, it will reuse the memory for sub-objects and such which was allocated the first time. This way it's easy to make sure you aren't doing it wrong. So if I understood you right, say my example message test contains 10 chunks, if I clear that message and parse another message containing 9 chunks, no allocation would be performed and 1 cleared chunk object would still be in the object pool? Thx, --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Usage RepeatedPtrField - Advanced memory mgmt
On Tue, Jun 16, 2009 at 10:13 AM, RobertFach robert.f...@inf.tu-dresden.dewrote: So if I understood you right, say my example message test contains 10 chunks, if I clear that message and parse another message containing 9 chunks, no allocation would be performed and 1 cleared chunk object would still be in the object pool? Correct. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Use of repeated
I am experimenting with repeated fields my dot-proto is as follows: message Foo { required string name= 1; optional int32 id = 2; optional int32 num = 3; repeated string sbr = 4; repeated int32 wrk = 5; } I build the msg and use foo.SerializeToString(str) and then sending the retult str.c_str() over a socket; On the receiving end I get the following: libprotobuf FATAL /usr/local/include/google/protobuf/repeated_field.h: 513] CHECK failed: index size(): Aborted If I eliminate the repeated fields from the msg then it all works fine. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---