Python/uuid bytes field

2009-06-16 Thread aronb

We are working on a project where we are sending UUIDs between a
backedn written in C++ and a Python front end. Our proto file
specifies that we want to store these UUIDs in byte fields. This maps
to a std::string in C++ which we can assign to manually. We are
running into a problem on the Python side though because assigning the
raw 16 bytes to the string field in Python gives us the following:

exceptions.ValueError: ':\x19\x14\xd1Y\xf7\x11\xde\x9c[\x00\x1eO\xf3!
\xd8' isn't in 7-bit ASCII encoding.

This does make sense because Python is ensuring that the value is
actually a string. My question then is how can we encode a UUID into a
16 byte data structure correctly? Is there a plan to fully support the
bytes field type instead of simply mapping it to a string type?

Thanks,
Aron
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Usage RepeatedPtrField - Advanced memory mgmt

2009-06-16 Thread RobertFach

Hi,

is there anybody out there who can tell me how the special methods
like addcleared(), etc. for repeated field ptr work?

I guess I can somehow improve/avoid the allocation overhead, right?
Say, I have the following messages,

message test {
  repeated chunk = 1;
}

message chunk {
  int value = 1;
}

The goal is to reduce the memory allocations by using pre-allocated
objects.

Assume the chunkPtr points to an already allocated object, if I use
test.chunk().addAllocated(chunkPtr), does it just attaches the
chunkPtr to the reaptedFieldPtr chunk object? Is there any overhead
incured by dynamically adding the chunks to the test message? Can I
deal with that issue by using the test.chunk().Reserve(xx) method?

It seems also that I can maintain a pool of objects by using the
addCleared() method, right? This way, future calls to test.chunk().add
() will use an element from the pool?

I was just wondering about these questions while reading the
documentation... maybe you can just tell me if this usage pattern
would be ok.

Thx for the library and thx for the support

Cheers Robert
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Re: Java thread safety

2009-06-16 Thread Kenton Varda
The Java implementation does not contain any mutable singletons, so as long
as you are accessing different objects, there is no need to worry about
thread-safety.  So, yes, multiple threads may call parseFrom() at the same
time as long as they are parsing from different streams.

On Mon, Jun 15, 2009 at 6:28 PM, Wayne wayne.mene...@gmail.com wrote:


 In the Java Generated code, there are functions like ParseFrom
 (CodedInputStream ...) that create protocol buffers messages from a
 file or other buffer. Can I call these directly from multiple
 different threads or should I use a wrapper with the synchronized
 keyword?

 Thanks,
 Wayne

 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Re: Usage RepeatedPtrField - Advanced memory mgmt

2009-06-16 Thread Kenton Varda
Everything you say sounds right.  If you want to know precisely what these
methods do, or how much overhead they incur, I encourage you to look at the
code in repeated_field.h -- it's not very complicated.
However, instead of trying to control all this in detail, it's usually
easiest to simply make sure that you reuse your protobuf objects.  If you
parse a message, clear it, and then parse another message into the same
object, it will reuse the memory for sub-objects and such which was
allocated the first time.  This way it's easy to make sure you aren't doing
it wrong.

On Tue, Jun 16, 2009 at 9:40 AM, RobertFach
robert.f...@inf.tu-dresden.dewrote:


 Hi,

 is there anybody out there who can tell me how the special methods
 like addcleared(), etc. for repeated field ptr work?

 I guess I can somehow improve/avoid the allocation overhead, right?
 Say, I have the following messages,

 message test {
  repeated chunk = 1;
 }

 message chunk {
  int value = 1;
 }

 The goal is to reduce the memory allocations by using pre-allocated
 objects.

 Assume the chunkPtr points to an already allocated object, if I use
 test.chunk().addAllocated(chunkPtr), does it just attaches the
 chunkPtr to the reaptedFieldPtr chunk object? Is there any overhead
 incured by dynamically adding the chunks to the test message? Can I
 deal with that issue by using the test.chunk().Reserve(xx) method?

 It seems also that I can maintain a pool of objects by using the
 addCleared() method, right? This way, future calls to test.chunk().add
 () will use an element from the pool?

 I was just wondering about these questions while reading the
 documentation... maybe you can just tell me if this usage pattern
 would be ok.

 Thx for the library and thx for the support

 Cheers Robert
 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Re: Usage RepeatedPtrField - Advanced memory mgmt

2009-06-16 Thread RobertFach

 However, instead of trying to control all this in detail, it's usually
 easiest to simply make sure that you reuse your protobuf objects.  If you
 parse a message, clear it, and then parse another message into the same
 object, it will reuse the memory for sub-objects and such which was
 allocated the first time.  This way it's easy to make sure you aren't doing
 it wrong.

So if I understood you right, say my example message test contains 10
chunks, if I clear that message and parse another message containing 9
chunks, no allocation would be performed and 1 cleared chunk object
would still be in the object pool?

Thx,
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Re: Usage RepeatedPtrField - Advanced memory mgmt

2009-06-16 Thread Kenton Varda
On Tue, Jun 16, 2009 at 10:13 AM, RobertFach
robert.f...@inf.tu-dresden.dewrote:

 So if I understood you right, say my example message test contains 10
 chunks, if I clear that message and parse another message containing 9
 chunks, no allocation would be performed and 1 cleared chunk object
 would still be in the object pool?


Correct.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Use of repeated

2009-06-16 Thread skshe...@gmail.com

I am experimenting with repeated fields my dot-proto is as follows:

message Foo {
required string name= 1;
optional int32  id = 2;
optional int32  num = 3;
repeated string sbr = 4;
repeated int32  wrk = 5;
}

I build the msg and use foo.SerializeToString(str) and then sending
the retult str.c_str() over a socket;

On the receiving end I get the following:

libprotobuf FATAL /usr/local/include/google/protobuf/repeated_field.h:
513] CHECK failed: index  size():
Aborted

If I eliminate the repeated fields from the msg then it all works
fine.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---