[protobuf] Modeling a python dictionary (experimental_map_key etc.)
Hello,
I've been experimenting with protocol buffers on appengine, using
2.3.0. I am trying to basically find the fastest way to serialize a
dictionary, and would like to see if protobuf can improve on
performance I'm seeing with other methods (pickle - ick, simplejson -
better, repr - best so far).
I did a search around on the group and saw this thread from 2008:
http://groups.google.com/group/protobuf/browse_thread/thread/d7fb950ea0ffe076/dae9a45544b9570c?lnk=gstq=python+dictionary#dae9a45544b9570c
The last post included what appeared to be suggested code for
implementing a python dictionary style data structure with protobuf:
message Foo {
optional string name = 1;
optional int32 some_field = 2;
...
}
message Bar {
repeated Foo foo = 1 [experimental_map_key = name];
}
I tried this out in the hopes it might just work as is, but perhaps
predictably enough I run into problems - if I try to use string keys,
it complains (at runtime) that list indices must be integers. I kind
of need my keys to be strings (otherwise I wouldn't really need a
dict...), but out of curiousity I switched to integers and tried
again, this time getting the error list index out of range. It
throws this for any integer index I use.
I guess there might have been a hint in 'experimental' ... :) But is
there anything more I should try before giving up on this? Is there an
alternative way to approach this? My requirements are that a message
'instance' needs to accommodate an unknown number of key-value pairs
with unknown keys...maybe there's another way to do this?
Thanks for any help, it'd be greatly appreciated.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Javascript protocol buffer implementation
Hi all, I am announcing a BSD-licensed javascript protocol buffer implementation, called protojs. The library supports all wire types, as well as packed fields (not autodetection yet), float/double support thanks to jsfromhell.com, and Unicode support. You can find the github site here: http://github.com/sirikata/protojs To install, you need to run ./bootstrap.sh to download and compile antlr, and then run make to build the pbj compiler and compile the sample javascript code. Protojs also supports higher level types, such as enums, nested composites, and precise 64-bit integers (using two 32-bit numbers). It also allows extending the built-in types, and has a library called pbj which includes other helpful types, such as vectors, quaternions and uuids. The library is modelled after python's protocol buffers (same function names). It uses getters and setters on browsers that support them (all but IE), and works without getters and setters at the expense of no runtime type checking. Sadly, javascript currently has no binary datatype. So in place of that the library currently supports serializing/deserializing base64 strings and to arrays of integers. Feel free to let me know if you have any suggestions or bugs, or you can fork the project since it's developed in git. -Patrick -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Issue 167 in protobuf: Python Docs: HasField lacking documentation
Status: New Owner: ken...@google.com Labels: Type-Defect Priority-Medium New issue 167 by danhomerick: Python Docs: HasField lacking documentation http://code.google.com/p/protobuf/issues/detail?id=167 HasField raises ValueError rather than returning False when msg type lacks the requested field. This is the designed behavior, but it isn't documented at: http://code.google.com/apis/protocolbuffers/docs/reference/python/index.html or in the Python Tutorial. -- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] New lightweight Java / J2ME port
Cool! I've added this to the list: http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns On Sat, Feb 20, 2010 at 8:30 AM, Ralf ralf.kist...@gmail.com wrote: I created a new lightweight Java port, specifically intended for J2ME. It is not ready for production yet, but comments are welcome. More info are on the project page on github. Btw the new generator plugin support is great, it made porting much easier! http://github.com/ponderingpanda/protobuf-j2me -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] ANN: Protocol Buffer support in RCF 1.2
I've added this to the RPC system list: http://code.google.com/p/protobuf/w/edit/ThirdPartyAddOns On Sat, Feb 20, 2010 at 2:08 AM, Jarl Lindrud jarl.lind...@gmail.comwrote: Hi all, I would like to announce native Protocol Buffer support in RCF (Remote Call Framework) 1.2. RCF is a portable C++ IPC/RPC framework, supporting a variety of transports and messaging semantics, and currently in use on a number of commercial and open source projects. A more detailed description of RCF is available in the RCF User Guide: http://deltavsoft.com/RcfUserGuide/1.2/ , or if you want you can go straight to the section on Protocol Buffers: http://deltavsoft.com/RcfUserGuide/1.2/rcf_user_guide/Protobufs.html Downloads are available from Google Code: http://code.google.com/p/rcf-cpp/downloads/list Thanks, Jarl Lindrud. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Construct return message using Descriptors.ServiceDescriptor
Right, Descriptor.toProto() returns a DescriptorProto, which is itself a protobuf type. So, calling newBuilderForType() on that is going to return a builder for DescriptorProto, not a builder for the type described. What you want is com.google.protobuf.DynamicMessage. Note that DynamicMessage is slower than the message classes produced by protoc. So if the type in question is actually compiled in to your app, you should use it instead. Typically what you'd do is pass around the type's default instance (MyMessageType.getDefaultInstance()) instead of passing around the descriptor. Note that the Service interface also provides a method getRequestPrototype(MethodDescriptor) which returns the default instance for the type, on which you can then call newBuilderForType(). On Sat, Feb 20, 2010 at 12:54 PM, ph pkirsa...@gmail.com wrote: I'm trying to build service return message using Descriptors.ServiceDescriptor. This does not work: serviceDescriptor.findMethodByName ( methodName ).getOutputType.toProto.newBuilderForType.mergeFrom ( msg ).build msg is byte array It builds DesscriptorProtos.DescriptorProto instead of Message. Is there a way to build message from byte array using method descriptor? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 167 in protobuf: Python Docs: HasField lacking documentation
Updates: Status: Accepted Comment #1 on issue 167 by ken...@google.com: Python Docs: HasField lacking documentation http://code.google.com/p/protobuf/issues/detail?id=167 (No comment was entered for this change.) -- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 166 in protobuf: by_symbol_.insert(iter, make_pair(name, value)); fails with Sun Studio 12
Comment #1 on issue 166 by ken...@google.com: by_symbol_.insert(iter, make_pair(name, value)); fails with Sun Studio 12 http://code.google.com/p/protobuf/issues/detail?id=166 If you change make_pair(name, value) to make_pairstd::string,const FileDescriptorProto*(name, value), does it work? If so, your compiler's definition of the pair template is apparently missing a constructor. -- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 134 in protobuf: Build on Solaris with the native STL
Comment #5 on issue 134 by ken...@google.com: Build on Solaris with the native STL http://code.google.com/p/protobuf/issues/detail?id=134 If someone has actually tested this on the platform in question and confirmed that it worked (tests pass), send me a patch. -- You received this message because you are listed in the owner or CC fields of this issue, or because you starred this issue. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Invitation to connect on LinkedIn
LinkedIn Protocol, I'd like to add you to my professional network on LinkedIn. - xiliu Confirm that you know xiliu tang https://www.linkedin.com/e/isd/1094750935/n1VvIi4p/ -- (c) 2010, LinkedIn Corporation -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Invitation to connect on LinkedIn
Yes sir, as you can see from my LinkedIn profile, I know C++, Java, Python and Ruby. Why, just last weekend I went to Ruby's house for a BBQ and pool party... On Mon, Feb 22, 2010 at 4:28 PM, Henner Zeller henner.zel...@googlemail.com wrote: Wow, that elevates protocol buffers to be a real person ;) -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Invitation to connect on LinkedIn
On Mon, Feb 22, 2010 at 17:00, Dan Homerick danhomer...@gmail.com wrote: Yes sir, as you can see from my LinkedIn profile, I know C++, Java, Python and Ruby. Why, just last weekend I went to Ruby's house for a BBQ and pool party... Fair enough. I like to hang out at Java's place for a good cup of coffee. -h On Mon, Feb 22, 2010 at 4:28 PM, Henner Zeller henner.zel...@googlemail.com wrote: Wow, that elevates protocol buffers to be a real person ;) -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Stopping MSVC from dropping unreferenced objects
Hi, Other than writing a dummy function that explicitly referecnes protobuf messages, do you have any hints for getting Microsoft Visual Studio (2005 in this case), to stop dropping the objects from the binary? I know it's not really a protobuf specific question, but I'm having problems where I read a bunch of protobuf files using the reflection API, but the descriptions are being lost due to linking. My current fix is the dummy function, but it means i have to update the function every time i update the proto files. I was hoping maybe you've seen the problem and have seen a better solution. :) Thanks, Mark On Tue, Feb 23, 2010 at 10:12 AM, Kenton Varda ken...@google.com wrote: My only guess is that your protobuf type wasn't actually referenced anywhere in your program and so was dropped from the binary. Some linkers do this. But it's weird that you have a pointer to an object whose class is not compiled in. Sorry, I don't have any other ideas. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Stopping MSVC from dropping unreferenced objects
We see this problem all the time. Dummy references are the best solution. :( You might also check if your linker has some sort of flag to control the behavior but I have no idea what it may be. On Mon, Feb 22, 2010 at 8:17 PM, Mark Assad mas...@gmail.com wrote: Hi, Other than writing a dummy function that explicitly referecnes protobuf messages, do you have any hints for getting Microsoft Visual Studio (2005 in this case), to stop dropping the objects from the binary? I know it's not really a protobuf specific question, but I'm having problems where I read a bunch of protobuf files using the reflection API, but the descriptions are being lost due to linking. My current fix is the dummy function, but it means i have to update the function every time i update the proto files. I was hoping maybe you've seen the problem and have seen a better solution. :) Thanks, Mark On Tue, Feb 23, 2010 at 10:12 AM, Kenton Varda ken...@google.com wrote: My only guess is that your protobuf type wasn't actually referenced anywhere in your program and so was dropped from the binary. Some linkers do this. But it's weird that you have a pointer to an object whose class is not compiled in. Sorry, I don't have any other ideas. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
