Re: [protobuf] Integers are being padded in protobufs on ARM

I'm guessing your cross compiler setup is somehow mucking up endian logic. Do you have a simple reproducible set up. On Apr 28, 2016 2:58 PM, "Brian Savage" wrote: > I am using protobufs on an ARM processor running Ubuntu Linux. Everything > seems to be working, except

Re: [protobuf] NULL character in serialized string

How are you determining the end of the encoded protocol buffer? What language/data type are you using? It sounds to me like you are null termination (perhaps with a C char*?), which isn't going to work to well with a binary structure. Decoding with a null byte, particularly for an integer field,

Re: [protobuf] Re: Why to reinvent the wheel ?

precisely the reason why one would reinvent the wheel: one person's yet another wheel is another person's train tracks. On Mon, Mar 25, 2013 at 4:11 PM, Christopher Smith cbsm...@gmail.comwrote: On Mon, Mar 25, 2013 at 4:25 AM, Vic Devin vfn...@gmail.com wrote: Oliver, yes, Protobuf can do many

Re: [protobuf] Optimizing protoc for Java

Interested. --Chris On Nov 19, 2012, at 4:07 AM, Ryan Fogarty ryan.foga...@gmail.com wrote: I have a repeated primitive field array optimization for the protoc-generated Java source, but before I discuss I would like to gauge interest (and get access to the Protocol Buffer Group).

Re: [protobuf] How to implement packed=true option for a list of enum?

Same way as any other varint wiretype field. --Chris On Aug 30, 2012, at 11:23 AM, Sean Nguyen sontran...@gmail.com wrote: Hi all, Does anybody know how to implement packed=true option for a list of enum? Thanks, Sean Nguyen -- You received this message because you are subscribed

Re: [protobuf] implications of compiling protocol buffers without stl debugging info

On Wed, Aug 8, 2012 at 3:08 PM, Eric J. Holtman e...@holtmans.com wrote: On 8/8/2012 4:51 PM, Chris Morris wrote: I want to keep STL debugging *for the rest of my project*. This leads me to consider compiling the protocol buffers project without STL debugging info. What are the implications

Re: [protobuf] Re: Issue 401 in protobuf: Java generated message field getters should return null or throw an exception for uninitialized fields

Your doBar(Request) should be codegen'd by a plugin. That's the bug. You should only be coding against doBar(Bar). --Chris On Jul 25, 2012, at 3:33 AM, proto...@googlecode.com wrote: Comment #2 on issue 401 by martin.c...@gmail.com: Java generated message field getters should return null

[protobuf] Some difficulties with protobuf on windows

I'm trying, for the first time, to get protocol buffers to work sanely on Windows. It's proving to be harder than I expected. Partly, it's my own fault: I'm building the project with VS11-beta, which I'm sure nobody has bothered to make work (and for good reason, it is right there in the name:

Re: [protobuf] Best way to organize collections?

Three points: There isn't much value in your Attributes message, so DummyAttributes2 would be the way I'd go. List is probably not an ideal name for an field, as it may overlap with other names depending on programming language. Per the style guide, field names should be like_this, not

[protobuf] Re: Some difficulties with protobuf on windows

On Fri, Jul 20, 2012 at 5:41 PM, Christopher Smith cbsm...@gmail.com wrote: 4) Plugins have to be exe's. That kind of sucks if the plugin is in a scripting language. I was looking at adding logic so that if you use the --plugin flag, and if the executable's name ends in .cmd or .bat, it would

Re: [protobuf] Read custom binary file with inner conditions

Protobuf is a library for serializing in its own format, not other predefined binary formats. It is also by design fairly simple with limited integrity checking features intended to help with backwards and forwards compatibility. I don't think it can help you with editing Diablo2 files.

[protobuf] Automated dependency generation?

I have been reworking my Makefile's to do automatic dependency generation proper. I realized that with modern gcc GNU make it gets pretty simple to do this, but protocol buffers were my one stumbling point, as they have dependencies that gcc can't extract. It occurred to me that it ought to be

Re: [protobuf] Re: Odd problem with protobuf implementation in C++

This sounds like something *very* weird is going wrong in your runtime. What does the debugger show as the memory locations of the string fields? the memory locations the allocator provided for the string data? I think if you make a stand alone unit test for this, you won't see this behaviour.

Re: [protobuf] Additional data types

AFAIK the answer is no. A lot of the value of protocol buffers derives from keeping their functionality simple. There are plenty of all singing/all dancing serialization frameworks already. ;-) I think date in particular is fraught with peril. I'd recommend against encodung them as strings.

Re: [protobuf] Additional data types

Nothing prevents you from making a module available for everyone's benefit. If it is broadly useful, it will undoubtedly be universally adopted. --Chris P.S.: What is a decimal type? On Apr 4, 2012, at 2:21 PM, Jawaid Hakim jawaid.ha...@codestreet.com wrote: Date and decimal types are

Re: [protobuf] Additional data types

On Apr 4, 2012, at 2:54 PM, Jawaid Hakim jawaid.ha...@codestreet.com wrote: My group builds applications using use multiple languages, including Java and C#, so a simple int64 for date representation does not work. That there isn't a simple way to do it is a pretty nasty strike against

Re: [protobuf] Message thread safety in Java

Message objects *don't* have mutators and are conceptually a copy of the relevant builder object. --Chris On Feb 20, 2012, at 10:22 AM, Tom Swirly t...@swirly.com wrote: The documentation says it's immutable:

Re: [protobuf] Build tagged versions of protobuf?

You could also just specify the static library name and/or use the static link flag. Not sure why you'd solve this via a rename --Chris On Oct 22, 2011, at 10:43 AM, JonathonS thejunk...@gmail.com wrote: Hi all, is there a way to have protobuf build the libraries with tags? (like

Re: [protobuf] C++0x enum class

You can always modify the compiler with a plugin. --Chris On Tue, Aug 23, 2011 at 12:36 AM, Christopher Head hea...@gmail.com wrote: Hi, Does anyone know if Protobuf's C++ output is ever going to support enum classes? I'd like to be able to define the elements of my enum in just one place

Re: [protobuf] C/C++ to java

I'm going to assume you aren't trying to use protobuf's as a JNI workaround, and rather want it as an rpc solution. I am not aware of a tool that can go from C declarations to protobuf ones. In a lot of ways, the syntaxes are similar, so it isn't much work to convert from C. That said, there

Re: [protobuf] Re: New types?

On Fri, Jul 8, 2011 at 9:46 AM, Eric Hopper omnifari...@gmail.com wrote: I guess. This is an interesting and general problem. Practically every system like protobuf needs to solve it. Python itself, for example, solves it for pickle by allowing you to write custom methods for your classes to

Re: [protobuf] Re: Exceptions when reading protobuf messages in Java which were created in C++

Basic rule: if the memory is used outside the lifetime of the function call, you don't want it on the stack. Async_write very much requires the memory to be around later. --Chris On Jul 6, 2011, at 10:10 AM, platzhirsch konrad.rei...@googlemail.com wrote: I find it difficult to check this by

Re: [protobuf] New types?

You could always extend the compiler, but I bet you could get away with a simple preprocessor that aliases types and represents those larger integers as raw bytes. --Chris On Jun 30, 2011, at 7:30 AM, Eric Hopper omnifari...@gmail.com wrote: I'm building an application around protocol

Re: [protobuf] Re: Serialization of primitive types

I think Gabor wants to avoid the overhead of implementing all that additional bookkeeping as it'd slow down development. Something that would effectively generate a protobuf descriptor so that it'd stay consistent with changes in the Java code. I would suggest looking at the protostuff project:

Re: [protobuf] Best way to deserialize

Best thing is when encoding, group messages by type and preface each group with a type name and FileDescriptorSet which will allow you to decode the rest using a DynamicMessage (see notes on self describing messages on the wiki). As per usual, use coded stream encoding with length prefixing for

Re: [protobuf] Genereating .proto from the existing java models

Are these POJO's or object generated by protoc? If the former, I would say you are SOL. If the latter, you can ask the object for its descriptors and actually dump to protocol buffer version of the .proto, at which point making the .proto is fairly straightforward. --Chris On Mar 18, 2011, at

Re: [protobuf] Re: omitting tag numbers

On Tue, Nov 16, 2010 at 7:28 PM, Kenton Varda ken...@google.com wrote: On Tue, Nov 9, 2010 at 10:42 PM, Christopher Smith cbsm...@gmail.comwrote: This aspect could be mostly mitigated by integrating a metadata header in to files. For systems with this kind of an approach look at Avro Hessian

Re: [protobuf] Re: Why to reinvent the wheel ?

On Tue, Nov 9, 2010 at 6:15 AM, Kalki70 kalki...@gmail.com wrote: On Nov 9, 2:59 am, Kenton Varda ken...@google.com wrote: OK, I looked into this again (something I do once every few years when someone points it out). ASN.1 *by default* has no extensibility, but you can use tags, as I

Re: [protobuf] Re: Why to reinvent the wheel ?

On Tue, Nov 9, 2010 at 6:21 AM, Kalki70 kalki...@gmail.com wrote: On Nov 9, 10:13 am, multijon multi...@gmail.com wrote: As a side note, the company I worked at used ASN.1 for five years to encode all of its product's communication messages (Using PER encoding), with what was supposed to

Re: [protobuf] Re: Why to reinvent the wheel ?

On Tue, Nov 9, 2010 at 6:44 AM, Kalki70 kalki...@gmail.com wrote: Oh, I just found out that you are the developer. It seems I am not the only one who thinks you reinvented the wheel : http://google-opensource.blogspot.com/2008/07/protocol-buffers-googles-data.html Yes, this is not a new

Re: [protobuf] Re: omitting tag numbers

On Mon, Oct 25, 2010 at 4:11 PM, Henner Zeller henner.zel...@googlemail.com wrote: On Mon, Oct 25, 2010 at 16:10, maninder batth batth.manin...@gmail.com wrote: I disagree. You could encode field name in the binary. Then at de- serialization, you can read the field descriptor and

Re: [protobuf] WSDL Vs PB

On Tue, Nov 9, 2010 at 7:56 AM, maninder batth batth.manin...@gmail.comwrote: In typical WS-* webservice, WSDL describes a service interface, abstracts from underlying communication protocol and serialization and deserialization as well as service implementation platform. Where does PB fits

Re: [protobuf] Why to reinvent the wheel ?

On Mon, Nov 8, 2010 at 9:59 PM, Kenton Varda ken...@google.com wrote: The bigger problem with ASN.1, though, is that it is way over-complicated. THIS It has way too many primitive types. It has options that are not needed. The encoding, even though it is binary, is much larger than

Re: [protobuf] ParseFromArray -- in Java

Might as well use a new object. Creating a new object is if anything likely less overhead. --Chris On Oct 25, 2010, at 1:52 PM, ury ury.se...@gmail.com wrote: Hi! Is it possible to reuse the same Protobuf object when unserializing a large number of objects in Java using the equivalent of

Re: [protobuf] Re: can i use protocol buffer to encrypt and send a file

On Sun, Oct 17, 2010 at 10:01 PM, nit nithin.shubhan...@gmail.com wrote: On Oct 14, 4:51 pm, Adam Vartanian flo...@google.com wrote: I have a file which i need to encrypt and send so can i use protocol buffer for this buffer? please do reply me. Protocol buffers don't provide any

[protobuf] Re: What's the point of Protocol Buffers?

There is also the other end of the spectrum, where I am at. With a large Hadoop cluster and terabytes of data, the efficient storage and zippy parsing of protobufs is a huge deal. In many ways, protobufs allow you do do what XML promised, but much more efficiently. The other way to look at them

Re: [protobuf] Re: What's the point of Protocol Buffers?

On Sat, Jul 24, 2010 at 3:22 PM, Oliver Jowett oliver.jow...@gmail.comwrote: Christopher Smith wrote: There is also the other end of the spectrum, where I am at. With a large Hadoop cluster and terabytes of data, the efficient storage and zippy parsing of protobufs is a huge deal

Re: [protobuf] Java UTF-8 encoding/decoding: possible performance improvements

cache? On Mon, May 17, 2010 at 7:59 PM, Christopher Smith cbsm...@gmail.com wrote: This does somewhat suggestive that it might be worthwhile specifically tagging a field as ASCII only. There are enough cases of this that it could be a huge win. On 5/17/10, Evan Jones ev...@mit.edu wrote

Re: [protobuf] EnumValueDescriptor doesn't provide toString()?

classes, so we want to be careful not to mess up that consistency. Instead of calling toString(), you could call TextFormat.printFieldToString() to get a string representation of the field, although it will include the field name. On Mon, May 10, 2010 at 9:17 PM, Christopher Smith cbsm

Re: [protobuf] Java UTF-8 encoding/decoding: possible performance improvements

This does somewhat suggestive that it might be worthwhile specifically tagging a field as ASCII only. There are enough cases of this that it could be a huge win. On 5/17/10, Evan Jones ev...@mit.edu wrote: On May 17, 2010, at 15:38 , Kenton Varda wrote: I see. So in fact your code is quite

Re: [protobuf] Deadlock problems with protobuf static initialization in Java

Never seen it before... and the Java code is pretty extensively used. Surely someone would have hit this before. --Chris On May 15, 2010, at 9:03 PM, Igor Gatis igorga...@gmail.com wrote: Have anyone experienced deadlock problems related to Java protobuf generated messages static

[protobuf] EnumValueDescriptor doesn't provide toString()?

I noticed EnumValueDescriptor uses the default toString() method. Why not override it to call getFullName()? --Chris -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To

Re: [protobuf] EnumValueDescriptor doesn't provide toString()?

it for *all* of them in some consistent way. If we fill them in ad-hoc then they may be inconsistent, and we may not be able to change them to make them consistent without breaking users. On Mon, May 10, 2010 at 9:49 AM, Christopher Smith cbsm...@gmail.com wrote: I noticed EnumValueDescriptor

Re: [protobuf] Re: How to fed PB's generated serialized file to map reduce framework

PB serialization is deterministic, but not necessarily canonical depending on your canonicalization rules. I imagine that can cause issues. In particular, fields with default values look different depending on whether the default value was actually set, and of course the ordering of repeated

Re: [protobuf] Re: Usage of Protocol Buffer with map reduce framework

We do stuff very much like this. Generally we try to encode the PB's as sequence files so that they are splittable. If that doesn't make sense, then we use non-splittable coded streams. --Chris On Apr 19, 2010, at 9:22 PM, stuti awasthi stutic...@gmail.com wrote: Thanks Jason, Yes map

Re: [protobuf] Re: 2.3.0 released

I hate to quibble on this, but strictly speaking: for (int i = 0; i some_vector.size(); i++) is not perfectly valid unless you have verified that some_vector.size() static_castsize_t(std::numeric_limitsint::max()); This would be broken in cases of a very large vector (possible with a

Re: [protobuf] Re: Protocol Buffers using Lzip

One compression algo that I thought would be particularly useful with PB's would be LZO. It lines up nicely with PB's goals of being fast and compact. Have you thought about allowing an integrated LZO stream? --Chris On Wed, Dec 9, 2009 at 12:21 PM, Kenton Varda ken...@google.com wrote: Thanks

[protobuf] FileDescriptorSet mania

Given that there is no Java version of protoc, it has been tempting to see how far one can get with just using FileDescriptorSet's and dynamic messages. I ended up building this short function for iteratively adding all the dependencies of a particular Message in to a FileDescriptorSet. While it

Re: serialize message to UDP socket

Doesn't the UDP packet header effectively provide that length prefix for you? --Chris On Fri, Sep 18, 2009 at 12:26 PM, jayt0...@gmail.com jayt0...@gmail.comwrote: One other thing I wanted to say was that I chose to use CodedOutputStream to send data because ultimately I have to manually

Re: java string literal too long when initializing java.lang.String descriptorData

Heh, I remember complaining about this to James when they first published the byte code spec. It was annoying then and continues to be annoying today. On 7/24/09, Kenton Varda ken...@google.com wrote: How annoying. I'll make sure this or something like it gets into the next release -- which

Re: Java deserialization - any best practices for performances?

The best way to think of it is: Builder : Java Message :: C++ Message : const C++ Message As far as performance goes, it is a common mistake to confuse C/C++ heap memory allocation costs to Java heap allocation. In the common case, allocations in Java are just a few instructions... comperable

Heuristic for getting a particular message type from a FileDescriptorSet

I'm trying to do some work using dynamic message, working from a FileDescriptorSet produced by protoc. The heuristic I've come up with is something like: Parse the binary in to a FileDescriptorSet for each FileDescriptorProto in the set create the FileDescriptor from the FileDescriptorProto

Re: Thread-safe messages

I'd recommend using an atomic swap to do your updates. So you create your new version of the PB localy, and then swap it in to the memory location that is visible to all the other threads. The only real downside is you stress the heap more, and that is probably cheaper/simpler (particularly if

Re: PB's vs ASN.1

No, but in short the advantages over ASN.1 can be summed up as simpler, and for most cases, more efficient. On 6/23/09, Jon M jonme...@yahoo.com wrote: Hello, The system I am currently working on uses ASN.1 at the heart of the client/server communication. I am evaluating PB's for another

Re: 'Streaming' messages (say over a socket)

The normal way to do it is to send each Entity as a separate message. CodedInput/OutputStream is handed for that kind of thing. --Chris On Sun, Jun 14, 2009 at 4:14 PM, Alex Black a...@alexblack.ca wrote: Is there a way to start sending a message before its fully composed? Say we have

Re: Communication methods

http://code.google.com/p/protobuf/wiki/RPCImplementations On Sun, May 24, 2009 at 12:42 PM, SyRenity stas.os...@gmail.com wrote: Hi. Sorry if this was already asked in the past, but does Protocol Buffers provide some IPC communication methods? Or they provide the data encapsulation, and I