[protobuf] Re: Issue 248 in protobuf: protobuf will not compile without thread library
Updates: Status: Accepted Owner: ken...@google.com Comment #2 on issue 248 by ken...@google.com: protobuf will not compile without thread library http://code.google.com/p/protobuf/issues/detail?id=248 Hmm. I don't think we should automatically fall back to thread-hostile code when no threading library is available -- this could cause really hard-to-debug problems if it happened by accident. But we could certainly provide a way for the user to explicitly ask for this, e.g. a --without-thread-safety configure option. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] 2.4.0 and lazy UTF-8 conversions in Java
On Wed, Dec 1, 2010 at 3:33 AM, Evan Jones ev...@mit.edu wrote: The instanceof approach to switch between the two is a good idea. When I wrote my implementation, I was concerned about the thread-safeness issues, although I don't think I ever considered this particular version. However, I think this can be made thread-safe, even without volatile (although I only understand the JMM enough to be dangerous). Well, Jeremy Manson literally wrote the book on the Java memory model, and he says it works. :) -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Preferred Citation?
There's a (slightly outdated) list of contributors here:
http://code.google.com/p/protobuf/source/browse/trunk/CONTRIBUTORS.txt
But I think it's probably best to list Google as the author. I did not
invent protocol buffers; I just wrote version 2 and open sourced it, and I
had help.
I don't really know how citations work, though.
On Fri, Dec 3, 2010 at 7:25 PM, Dan Homerick danhomer...@gmail.com wrote:
I'm writing up a master's project MS which has used protobufs extensively.
Is there a preferred article for crediting protobufs with? If not, I could
cite it with something like (Bibtex):
@MISC{protobuf,
title={Protocol Buffers},
author={Kenton Varda},
howpublished={\url{http://code.google.com/apis/protocolbuffers/}},
}
but I'm sure that the authors list is woefully incomplete. Is there a more
complete list of authors available?
Alternatively, I could just put the project's URL in a footnote.
- Dan
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 196 in protobuf: Python: Ascii output is not assured to be in utf-8
Updates: Status: Accepted Labels: -FixedIn-2.4.0 Comment #3 on issue 196 by ken...@google.com: Python: Ascii output is not assured to be in utf-8 http://code.google.com/p/protobuf/issues/detail?id=196 Jisi, I'm not convinced that this is fixed. The as_utf parameter simply prevents the printer from escaping character codes = 128. The bug report seems like it may actually be a problem in the parser. Also, round trips should work correctly even if as_utf is not used. We should investigate further, and make sure we have test cases that print and then parse a message containing Unicode characters, both with and without as_utf enabled. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Re: java parse with class known at runtime (and compiled proto)
Evan is correct. The best way to write code which deals with a generic protobuf type is to have it take a default instance as a parameter. From that you can do everything else. This is actually better than passing around Class objects, because it allows users to use DynamicMessages with your code. Using Class objects forces users to use only generated types. Also, Java reflection may be slow or even unavailable on some platforms. On Mon, Dec 6, 2010 at 7:51 AM, Evan Jones ev...@mit.edu wrote: On Dec 6, 2010, at 10:31 , Koert Kuipers wrote: But that doesn't make a parseFrom() in message interface invalid, does it? Indeed some other information outside the raw bytes will be needed to pick to right Message subclass. But that's fine. Oh, sorry, I misunderstood your question, so my answer is somewhat invalid. One could then: 1) pick the right subclass of Message based upon some information outside the raw bytes (in my case something stored in a protobuf wrapper around the raw bytes) 2) call subclass.parseFrom(bytes) now we have to jump through more hoops for step 2 (create instance of Message subclass, newBuilderForType, mergeFrom, isInitialized, build) The MessageLite.Builder interface has a mergeFrom method that does what you want. What you should do is something like: * Get a MessageLite instance for the message type you want to parse (eg. something like MyMessageType.getDefaultInstance(), or MessageLite.getDefaultInstanceForType()) * Hold on to that MessageLite instance in some sort of registry. (HashMapInteger, MessageLite?) * When you get a message, look at the protobuf wrapper to determine the type. * Look up the prototype MessageLite instance in your registry. * Call prototypeInstance.newBuilderForType().mergeFrom(bytes).build() This only creates a single instance of the message each time. The .build() method will automatically check that the message is initialized, so you don't need to call isInitialized (although you may want to catch the exception it could throw?). This Builder pattern is used so that the Message objects are immutable. This means they can be passed between threads without requiring any synchronization. See: http://code.google.com/apis/protocolbuffers/docs/javatutorial.html#builders Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] File already exists in database on Mac OSX
I assume this is happening at startup? Is it possible that you are somehow linking two copies of the same .pb.cc into your binary? There are cases where the linker doesn't catch this, especially when dynamic linking is involved, but sometimes even with static linking. Speaking of which, are you linking dynamically or statically? I am not aware of any outstanding issues that would cause this, but C++ initialization behavior can be crazy complicated and can even vary between platforms. On Mon, Dec 6, 2010 at 8:05 AM, Alex Nixon alex.nixon...@gmail.com wrote: Hello, I'm receiving a File already exists in database error from the protocol buffers library at runtime on Mac OSX, and I'm unsure as to what's causing it or where to look to isolate the problem. I have the exact same codebase compiling on Windows and Linux and they both work fine - so I wondered if this is a protocol buffers issue? That said, I was unable to find any outstanding issues in the bug tracker which match my symptoms. All of my .proto files have different names and are in the same directory. The generated .pb.h and .pb.cc files on Mac OSX are identical to those generated on Linux. Any advice would be much appreciated. Thanks in advance, - Alex Nixon -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 247 in protobuf: Ability to redirect file output to stdout
Comment #2 on issue 247 by ken...@google.com: Ability to redirect file output to stdout http://code.google.com/p/protobuf/issues/detail?id=247 What happens if multiple files are generated? For example, if you have option java_multiple_files = true; in your .proto file. This seems like it could get weird. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] 3 issues changed in protobuf
Updates: Status: Fixed Labels: FixedIn-2.4.0 Comment by liuj...@google.com: Fixed in r358 Affected issues: issue 223: Python Package Doesn't Contain Compiler Plugin Module http://code.google.com/p/protobuf/issues/detail?id=223 issue 224: Protobuf installation error in Cygwin http://code.google.com/p/protobuf/issues/detail?id=224 issue 242: Can no longer install protobuf on pypy http://code.google.com/p/protobuf/issues/detail?id=242 -- You received this message because you are listed in the owner or CC fields of these issues, or because you starred them. You may adjust your issue notification preferences at: http://code.google.com/hosting/settings -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 179 in protobuf: Visual C++ error C1091 when compiling protoc generated code with over 64k descriptor
Updates: Status: New Comment #2 on issue 179 by ken...@google.com: Visual C++ error C1091 when compiling protoc generated code with over 64k descriptor http://code.google.com/p/protobuf/issues/detail?id=179 This proves complicated to fix, because we have lots of code that passes around this blob as a pointer and a size. We'd have to update all that code to pass around a pointer to a more complex data structure which might contain multiple segments. Any fix would have to be used with all compilers, even ones which don't have this limitation. Your example is a 10MB source file. I don't think we care to support 10MB source files. Realistically, if a file is so big that its descriptor is over 64k, the best solution may be to simply split up the file. If you are stuffing an excessive amount of information in custom options, it may be time to consider a different format for that information. I'd like to see more demand before we put effort into fixing this. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Any Python wrappers for the C++ implementation?
On Tue, Dec 7, 2010 at 7:08 PM, Kenton Varda ken...@google.com wrote: Cool. Serialization and parsing themselves should actually be improved even more than that, but having other Python code around it waters down the numbers. :) The times are from a minimal microbenchmark using Python's timeit module: nruns = 1000 nwarmups = 100 es = ... # the protobufs def ser(): return [e.SerializeToString() for e in es] def parse(ses): for se in ses: pb.Email().ParseFromString(se) t = timeit.Timer(lambda:None) t.timeit(nwarmups) print 'noop:', t.timeit(nruns) / nruns t = timeit.Timer(ser) t.timeit(nwarmups) print 'ser:', t.timeit(nruns) / nruns / len(es) ses = ser() t = timeit.Timer(lambda: parse(ses)) t.timeit(nwarmups) print 'parse:', t.timeit(nruns) / nruns / len(es) print 'msg size:', sum(len(se) for se in ses) / len(ses) Also, note that if you explicitly compile C++ versions of your messages and link them into the process, they'll be even faster. (If you don't, the library falls back to DynamicMessage which is not as fast as generated code.) I'm trying to decipher that last hint, but having some trouble - what exactly do you mean / how do I do that? I'm just using protoc --py_out=... and PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp. As for when 2.4.0 might be released, it's hard to say. There's a lot of work to do, and we have a new person doing this release so he has to learn the process. Also, holidays are coming up. So, I'd guess it will be ready sometime in January. Thanks for the estimate; even a ballpark without commitment is useful. -- Yang Zhang http://yz.mit.edu/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 210 in protobuf: Java code should detect incompatible runtime library version
Updates: Cc: liuj...@google.com Comment #13 on issue 210 by liuj...@google.com: Java code should detect incompatible runtime library version http://code.google.com/p/protobuf/issues/detail?id=210 Hmm, not very clear about the maven version compatibility... So suppose you have a pom.xml: project dependencies dependency groupIdcom.google.protobuf/groupId artifactIdprotobuf-java/artiFactId version2.3/version /dependency /dependencies /project will it also accept protobuf-java-2.2.0.jar ? Or did you simply write version2/version? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Any Python wrappers for the C++ implementation?
On Tue, Dec 7, 2010 at 9:19 PM, Yang Zhang yanghates...@gmail.com wrote: Also, note that if you explicitly compile C++ versions of your messages and link them into the process, they'll be even faster. (If you don't, the library falls back to DynamicMessage which is not as fast as generated code.) I'm trying to decipher that last hint, but having some trouble - what exactly do you mean / how do I do that? I'm just using protoc --py_out=... and PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp. I'm not completely sure what I mean, because I don't have much experience with Python C Extensions. Basically I'm saying you should additionally generate C++ code using protoc, the compile that into a C extension (even with no interface), and then load it into your Python process. Simply having the C++ code for your message types present will make them faster. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Issue 210 in protobuf: Java code should detect incompatible runtime library version
Comment #14 on issue 210 by aantono: Java code should detect incompatible runtime library version http://code.google.com/p/protobuf/issues/detail?id=210 It would if you specify, what Maven calls, a range - [1.0,2.0), which would basically read that anything inclusive between 1.0 and 2.0, but excluding the 2.0 itself. If you just leave it as 2.3, that is what Maven calls, a Soft requirement on 2.3 (just a recommendation - helps select the correct version if it matches all ranges), basically means that you would like 2.3 if possible, but maven will do its best to try to select the nearest (by default within the minor range (keeping the same major)) version that would match. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
