Re: [protobuf] Protocol Buffers Encoding Guide erratum

On Thu, May 6, 2021 at 5:06 PM Gregory Bourassa wrote: > > Hi folks, > > Has anyone noticed that the example message which begins the document is > erroneous. At one point (under the Message Structure heading) the authors > claim that: > > 96 01 = 1001 0110 0001 > > which it cannot,

Re: [protobuf] Understanding the binary encoding: Why is there a "01" after certain keys?

What if the field number were 32? (32 << 3) | 2 = 0x102, which won't fit in a byte. The field number is varint-encoded, and since the high bit is set with 16+ (i.e. 0x80), the tag id ends up taking 2 (or more) bytes -- with varint encoding, if the high bit is set, that means the remaining bits are

Re: [protobuf] has() methods in Java

These are gone in proto3. The has* stuff is available with proto2. On Thu, Jul 11, 2019 at 4:54 PM Janac Meena wrote: > > Hello, > > I see in the documentation that there is a way to check for the existence of > a field in a message, i.e. > > message Foo { > string name = 1; > } > > In my Java

Re: [protobuf] Find the message Type from Name of the message

You can get the ->message_type() (assuming the ->type() is TYPE_MESSAGE), which gets you a Descriptor which in turn has a ->name() / ->full_name() function. Cheers, -ilia On Tue, May 21, 2019 at 7:13 PM Kiran Kumar wrote: > > I have protobuf messages defined as below. I need to find the

Re: [protobuf] Protobuf Java Generated Code field name has underscore at the end.

Why not just store the serialized data of the protobuf instead? That's kind of the whole point of protobuf... On Thu, Sep 6, 2018 at 1:27 PM, Chris Zhang wrote: > Hi Adam, > > Thanks for the response. > > We are trying to persist the protobuf generated java object into mongoDB > using Spring

Re: [protobuf] Re: framing header message -- how to know length?

So like I said... you have to write it as To decode it, you can stick the header length as a varint, all via the coded input/output stream. Sample code for encoding: makePacket(Message message) { String name = ... (there has gotta be a way of getting the typename from the message) length =

Re: [protobuf] Re: Java API: creating message by name

Right ... I'm lazy too ... add("FooType", FooType.getDefaultInstance()) ... map.get("FooType").newBuilderForType().mergeFrom(bla).build() Right? On Mon, Jul 9, 2018 at 8:45 PM, John Lilley wrote: > Because I am lazy! I only want to add the wrapper once. > john > > -- > You received this

Re: [protobuf] Re: Java API: creating message by name

Why not just call add() with a Message instance instead? On Mon, Jul 9, 2018 at 7:39 PM, John Lilley wrote: > Darn, there's only one snag. I need the protobuf package name, not the java > package classname, in order to construct the message name that corresponds > to the result of full_name()

Re: [protobuf] Re: Java API: creating message by name

newBuilderForType: "Constructs a new builder for a message of the same type as this message." That's if you have a message, and you want to create a builder for that type of message. Not what you want. The actual types are outside of the library, and need to be looked up via a type registry. I

Re: [protobuf] framing header message -- how to know length?

so looked at ArrayOutputStream, but at first read it appears to require > knowledge of the output size before constructing the stream. True? > Thanks > john > > > On Sat, Jul 7, 2018 at 2:18 PM John Lilley wrote: >> >> Got it, thanks! >> john >> &

Re: [protobuf] framing header message -- how to know length?

;> >>> Thanks! >>> Given that, is there any advantage to a "header message" as opposed to >>> just hand-serializing everything in the header? >>> >>> >>> On Sat, Jul 7, 2018 at 12:45 PM Ilia Mirkin wrote: >>>> >>>

Re: [protobuf] framing header message -- how to know length?

s! >> Given that, is there any advantage to a "header message" as opposed to >> just hand-serializing everything in the header? >> >> >> On Sat, Jul 7, 2018 at 12:45 PM Ilia Mirkin wrote: >>> >>> You need explicit lengths. Usually this is

Re: [protobuf] framing header message -- how to know length?

You need explicit lengths. Usually this is done as . And the header contains the body length in it. In Java, there's a CodedInputStream/OutputStream which makes it easy to consume fixed lengths (push/popLimit) as well as raw varints (as for the initial header length). Other languages have similar

Re: [protobuf] Using ParseFromString to Differentiate Message Types

There's no distinction between message Foo { int32 foo = 1; } and message Bar { int32 bar = 1; } It's all encoded as tags and values, the type and field names aren't on the wire. Unknown tags are ignored (or put into a "unknown tag" list). The framing is entirely up to you. One

Re: [protobuf] Variable-Width Integer Encoding

I doubt you're going to get a nice clean answer. Chances are it's "whatever Sanjay was thinking at the time" which led to the current encoding, maintained throughout the proto versions for backwards compatibility with existing data. While APIs have changed over time, the wire encoding has remained

Re: [protobuf] Is protobuf3 a superset of protobuf2?

The main thing that proto3 lacks and keeps coming up every time I use it is field availability information. All the has_* API is gone for basic types, so you have to supply that information separately, or restructure your logic to not rely on it. -ilia On Fri, Dec 15, 2017 at 6:40 PM,

Re: [protobuf] Re: Any way to "extract all" using protoc or any other tool?

On Sun, Dec 10, 2017 at 11:23 AM, Jim Baldwin wrote: > Perhaps it might help if I understood the output of protoc --decode_raw. > > Here's an example of a .caffemodel file I'm trying to inspect. Is there a > description of what the numbers mean in this file? As I said... it's

Re: [protobuf] Any way to "extract all" using protoc or any other tool?

On Sat, Dec 9, 2017 at 5:52 PM, Jim Baldwin wrote: > I have a protobuf file, and a .proto file that describes the schema. > > The .proto describes dozens of different messages that may be in the > protobuf file. > > I would like to know what messages can be found in the file.

Re: [protobuf] could anyone give me the source code of protobuffer 2.0.3

Not easy to find anymore it seems... looks like all the old releases have been lost in the move to github. That's unfortunate. http://ftp.slackware.com/pub/gsb/gsb64-2.28_slackware64-13.1/source/c/protobuf/protobuf-2.0.3.tar.bz2 http://www.java2s.com/Code/Jar/p/Downloadprotobufjava203jar.htm No

Re: [protobuf] Re: Has Anyone Used protobuf on ucLinux?

By the way, you may be interested in the nanopb and protobuf-c implementations, which likely have fewer dependencies. On Tue, Mar 14, 2017 at 4:50 PM, Doug Lewis wrote: > Ah I see that in the full output of the make process now. > > Yes all I really need is to get the

Re: [protobuf] Re: Has Anyone Used protobuf on ucLinux?

So then you don't need the code that's giving you errors - it's part of protoc. It's been way too many moons since I've looked at the details for how to only build the lib and not protoc, or even if it's still possible, but you might find that that's the path of least resistance. On Tue, Mar 14,

Re: [protobuf] Has Anyone Used protobuf on ucLinux?

This is in the protobuf compiler. Do you plan on compiling protobuf proto definitions into source code on your ucLinux platform itself? On Tue, Mar 14, 2017 at 3:56 PM, Doug Lewis wrote: > I've been tasked with seeing if we can use protobuf on ucLinux (micro c > Linux).

Re: [protobuf] String max length is 255. Correct?

The length is stored as a varint; you should be able to store strings of any length, up to 64-bit. On Wed, Dec 14, 2016 at 8:43 AM, David Mullineux wrote: > The encoding suggests that strings are encoded with a 1 byte length prefix. > This means my strings cannot be larger

Re: [protobuf] Reserved keyword usage in protobuf 2.5.0

On Wed, May 25, 2016 at 5:44 PM, Marcos Juarez wrote: > I ran into a situation today in which I needed to use the "reserved" > keyword in a protobuf definition, so that in the future, nobody would use > those fields. I've attached a screenshot of what the docs say, from >

Re: [protobuf] Re: Has Anyone Successfully Defined "smaller" integers for protocol buffers (C++ or Java)?

If your varint is < 128, it will only take a single byte to get encoded. Even if it's declared as an int64. (Also, consider that what matters isn't data size but number of frames sent.) Anyways, if you wanted to created fixed 8- and 16-bit types, you'd have to add new field types for those, and

Re: [protobuf] Re: Has Anyone Successfully Defined "smaller" integers for protocol buffers (C++ or Java)?

Why not just use a varint? You can use the zigzag variants if you want to store things like -1 efficiently. The only time you'd benefit from having a fixed8/fixed16 field is if the high bits are set, and it'd only be by 1 byte. -ilia On Tue, May 24, 2016 at 11:32 AM, Douglas Lewis

Re: [protobuf] Has Anyone Successfully Defined "smaller" integers for protocol buffers (C++ or Java)?

On Mon, May 23, 2016 at 9:55 AM, Douglas Lewis wrote: > I'm working on a project and we are hoping to use protocol buffers for > messages between embedded and PC based applications. > > One issue I've immediately run into is protocol buffers only has 32 and 64 > bit

Re: [protobuf] Default Values vs Missing Values

e of the tag is not conclusive. > Actually, even packed collection (and strings, and binary data) suffer from > that, as you are "expected" to not include a packed collection with zero > bytes. > > > On Saturday, March 26, 2016 at 1:08:23 PM UTC-7, Ilia Mirkin wrote: >

Re: [protobuf] Default Values vs Missing Values

Encoding is identical... just the API is different. In proto2, you have (in C++) FooMessage->has_field() which will tell you whether a field was present in the encoded version (or has been set prior if you're building a new message). The Java API has something rather similar... hasField() I think?

Re: [protobuf] Default Values vs Missing Values

Use proto2, which has the has_* checks per field. (Using get_* you still get the default value, of course.) It's extremely unfortunate that this functionality was removed in proto3, I see that making proto3 unattractive for all but the simplest uses of protos. I know in almost every protobuf

Re: [protobuf] [Java][Python][TCP] Reading messages that where written with writeDelimited and viceversa

On Tue, Sep 29, 2015 at 9:31 AM, Mike White wrote: > aloha everyone, > > i stumbled across this post as i'm hitting this error as well. since i'm a > newbie to python i had a question on how to implement this. when doing these > steps, > > # Read a message from Java's

Re: [protobuf] Question about data type from binary stream

On Sat, Sep 5, 2015 at 9:36 PM, TheAceInfinity wrote: > If I know that the bytes are a type of varint from a byte stream, is there > then no way to determine whether the 150 should be calculated and > interpreted as an int32 or an sint32? Nope, no information beyond the wire

Re: [protobuf] Question about data type from binary stream

t; 1) ^ ((n) >> 31)) > #define encode_sint64(n) (((n) << 1) ^ ((n) >> 63)) > > Do you have any information on how to go about decoding the bytes if my > original type was sint32 or sint64? > > On Saturday, September 5, 2015 at 7:41:05 PM UTC-6, Ilia Mirkin wrote

Re: [protobuf] Question about data type from binary stream

$ perl -ane 'foreach (@F) { print pack "C", hex($_) }' | protoc --decode_raw 08 96 01 1: 150 Or did you want to see how one might do this by hand? The 96 is misleading since one might be tempted to think that 0x96 == 150 (which it is), but really it's 0x16 + high bit set, which indicates that the

Re: [protobuf] Protocol Buffer for parsing binary message format

No, protobuf has a fixed wire format. You can't use it to describe arbitrary binary formats, only arbitrary data to be passed around. On Wed, Aug 26, 2015 at 3:26 AM, tomcat wrote: > In my application, I need to parse a relatively complex binary message > exchange

Re: [protobuf] Protov3 empty fields

Just to echo this, I think that having empty fields be distinguishables from ones set to the default value is an immensely useful feature, irrespective of whether the field is a primitive type or not. I know I've used it lots of times. On Wed, Jul 1, 2015 at 4:28 PM, John C. j...@excelfore.com

Re: [protobuf] How to get ByteArray from Protobuf in Java

These are the right steps. Make sure that the sender and receiver see the same data. Should be obvious, but protobuf has no framing, so if you want to send a bunch in one stream, you need to handle that. On Jun 5, 2015 2:34 AM, mark77...@gmail.com wrote: I am new to Protobuf. Here is my

Re: [protobuf] protobuf encoded message

0, field number 6 37 = value of field 6 62 = wire type 2, field number 12 65 = length ... not enough bytes to decode. Hence the failure by protoc --decode_raw. On Thu, May 21, 2015 at 2:09 PM, Ilia Mirkin imir...@alum.mit.edu wrote: Hm, that 01 byte seems to be extra somehow. Or there are bytes

Re: [protobuf] protobuf encoded message

Hm, that 01 byte seems to be extra somehow. Or there are bytes after this which are missing. $ perl -ane 'foreach (@F) { print pack C, hex($_) }' | protoc --decode_raw 12 01 40 30 37 62 65 37 36 30 34 32 33 35 32 37 33 30 64 64 63 37 38 35 39 39 38 39 34 66 31 31 37 65 30 37 34 35 36

Re: [protobuf] Using CodedOutputStream to send many messages across network

On Tue, Mar 17, 2015 at 10:15 PM, Sayan Goswami sayan.n...@gmail.com wrote: Hi All, I am using CodedOutputStream and CodedInputStream to transfer messages across a network like so: Client requests: Socket serverSocket = new Socket(hosts, ports); CodedOutputStream outputStream =

Re: [protobuf] Multibye character in protobuf getting corrupted while serializing and serializing.

On Mon, Mar 9, 2015 at 4:19 PM, Nishant Verma nishantma...@gmail.com wrote: There is a multibyte character in one of the string field of the Protobuf messages. When I am serializing or de-serilizig characters are getting corrupted. Please advise solution. Corrupted message: analyst {

Re: [protobuf] Newbie Question regarding multiple messages

Protobuf is a way to encode a message (data) to a byte stream. It does not provide any sort of framing/type/etc information. You have to supply that separately. For example a given endpoint may accept only one type of message. Or you have another standard header type message that describes what

Re: [protobuf] Multibye character in protobuf getting corrupted while serializing and serializing.

-hand anywhere), but what's printed by the text formatter shouldn't matter for communications -- you should be using the binary format. The text format is just there for debug purposes. -ilia On Mon, Mar 9, 2015 at 5:33 PM, Nishant Verma nishantma...@gmail.com wrote: Thanks Ilia Mirkin, What I

Re: [protobuf] same message name in diffent .proto file, how can i fix it?

Use package. https://developers.google.com/protocol-buffers/docs/proto#packages On Wed, Jan 14, 2015 at 2:27 AM, you zhou zhyou...@gmail.com wrote: in my project, file name as module name, like that file(module):battle.proto message SyncMapReq{ } message SyncMapResp{ message Map map =

Re: [protobuf] How To Distinguish signed interger and unsigned interger?

You can't tell from the binary stream. You have to know what data you're decoding to decide exactly how to interpret the decoded varint value. On Tue, Dec 30, 2014 at 3:47 AM, Lai Champion championlai2...@gmail.com wrote: I read the protocol buffer documents,I know compiler use ZigZag to

Re: [protobuf] Determining If a Message is of the Expected Message Format

protobuf messages are typeless. There is no difference between message Foo { int field = 1; } and message Bar { int otherfield = 1; } On Mon, Dec 22, 2014 at 1:32 PM, Ryan O'Meara roj.nemennuh.sar...@gmail.com wrote: Hi, I am using protobuf for a project, and I've run into an interesting

Re: [protobuf] packages and import

I think you want foo_primitives.bar ... going from memory though. On Fri, Nov 7, 2014 at 7:19 PM, Michael Wagner mikepwag...@gmail.com wrote: I would like to do the follwing: foo_primtives.proto: package foo_primitives; messaage bar { // some fields; }; message baz { // some

Re: [protobuf] Parse a .proto file

At no point are you specifying that you want to use the MessagePublish descriptor, so you must still be using the API incorrectly... On Fri, Oct 31, 2014 at 5:10 PM, Pradeep Gollakota pradeep...@gmail.com wrote: Ok… awesome… I do have the .proto’s ahead of time, so I can have them compiled to

Re: [protobuf] Parse a .proto file

On Fri, Oct 31, 2014 at 6:18 PM, Pradeep Gollakota pradeep...@gmail.com wrote: Boolean extension = fieldDescriptor.getOptions().getExtension(Messages.isPii); Shouldn't this use some sort of API that doesn't use the Messages class at all? -ilia -- You received this message because you are

Re: [protobuf] Move nested message.

On Mon, Oct 27, 2014 at 6:33 PM, Shuo Zhou sz...@google.com wrote: Hi, Suppose I got a message as follows: message Foo { message Bar { optional string name = 1; } repeated Bar bar = 1; } Then I find Bar is needed widely as a generic message, and want to move it outside

Re: [protobuf] can protobuf cost less bytes?

I believe you're looking for zigzag encoding wrt negatives. If instead of int32 you call it sint32, that should get a much better encoding for small negative values. On Sun, Oct 19, 2014 at 9:58 AM, Aijing Sun aijings...@gmail.com wrote: 1.if a int value is -1,then through varint 128 serialze is

Re: [protobuf] Message encoding question

On Wed, Oct 15, 2014 at 6:05 AM, Tamás Somhegyi somhegyi.ta...@gmail.com wrote: Hello, I am quite new to Protocol Buffers but I have read the tutorials. There is one part of the encoding which I don't understand. How the message types are encoded? As I see the tutorial mentions the field

Re: [protobuf] How to define Structured Binary file

Not sure what is being asked, but simply having a bytes foo = 1; will cause the presence of the field, the length of the field, and the contents of the field to be encoded into the resulting binary representation (as tagid 1 length varint length bytes). However enforcing maximum lengths/other

Re: [protobuf] What format is DebugString using for bytes?

Octal. It has the same meaning as if you had put that string into a C file. On Wed, Sep 17, 2014 at 3:07 PM, Nick Retallack nickretall...@gmail.com wrote: When I put some arbitrary bytes in a bytes field in a protocol buffer and then print it out with DebugString, it formats the bytes in a

Re: [protobuf] Re: enquiry about bit format of data type bool

On Thu, Jul 24, 2014 at 7:38 PM, ching lu lschin...@gmail.com wrote: Previous example is not good mssage { required bool x1 = 1; required bool x2 = 2; required bool x3 = 3; required bool x4 = 4; required bool x5 = 5; required bool x6 = 6; required bool x7 = 7; required bool x8 = 8;

Re: [protobuf] Find the message type during processing of serialized message at run time!

The serialized message does not contain any type information. You would either have to create a super-message for that, or have some other sort of knowledge of what type to expect. (A problem with the super-message approach is all the copying involved, so if that's a concern, you can just manually

Re: [protobuf] Possible C++ serialization issue with repeated fields

On Wed, Jun 18, 2014 at 12:26 PM, Justin jscad...@gmail.com wrote: My team and I noticed a potential bug in the serialization process, that seems unintended. If I defined a message structure as follows: message X { required Y y = 1; } message Y { repeated Things things = 1;

Re: [protobuf] protoc not generated add_...() method for repeated nested messages

On Wed, Jun 11, 2014 at 6:54 PM, CB cbrund...@gmail.com wrote: I've got a proto file that looks like this: option optimize_for = SPEED; package test; message KeyedStrings { optional string key = 1; repeated string value = 2; } message Map { repeated KeyedStrings map =

Re: [protobuf] protoc not generated add_...() method for repeated nested messages

On Wed, Jun 11, 2014 at 7:04 PM, CB cbrund...@gmail.com wrote: On Wednesday, June 11, 2014 7:01:42 PM UTC-4, Ilia Mirkin wrote: On Wed, Jun 11, 2014 at 6:54 PM, CB cbru...@gmail.com wrote: I've got a proto file that looks like this: option optimize_for = SPEED; package test

Re: [protobuf] Size of string Message not reducing even after serializing with Protocolbuffer

Can you elaborate a bit on the before scenario? (i.e. exactly what are you comparing protobuf to?) Protobuf adds overhead to data (in order to tell which field is which, etc), and does nothing for reducing the sizes of strings. (Integers are usually varint-encoded, so if they're small, they'll

Re: [protobuf] Use different encoding while serializing strings

protobuf does not use \r or \n for any specific purposes. It does, however, assume that it can use any byte it chooses. There is no way to restrict that. You would have to create a protocol to encode arbitrary bytes over your not-binary-safe-channel, which would have to create a way to escape \r

Re: [protobuf] protobuf-java 2.4.X and 2.5.0 are incompatible

While I don't speak for Google, I believe it's fairly well-established that 2.4 and 2.5 are considered to be major releases. Switching between them requires regenerating the java files with protoc, as the internal APIs used by the generated code tend to change. I believe that in general the public

Re: [protobuf] protobuf-java 2.4.X and 2.5.0 are incompatible

On Wed, Apr 16, 2014 at 10:11 PM, Patrick Wendell pwend...@gmail.com wrote: Established based on what conventions? On the convention of how protobuf releases have been done... Would you be happier if it was protobuf2 4.0 and protobuf2 5.0? It means that if you want to write a library that uses

Re: [protobuf] How to parse messages coming via streaming (TCP for example)

PB's don't include any protocol at all. Your RPC mechanism will need to provide for this. It's common to do something like a length pb type of thing; you could even use the CodedOutputStream (or its equivalent, depending on the language you're using) to emit the length as a varint, and then stick

Re: [protobuf] C++ ParseFromCodedStream takes too long

On Thu, Mar 20, 2014 at 3:28 AM, christian.kilpatrick.1...@gmail.com wrote: Hello, I have a message type like this: message SearchResponse { enum Status { OK = 200; BAD_REQUEST = 400; REQUEST_TIMEOUT = 408; INTERNAL_SERVER_ERROR = 500; }

Re: [protobuf] C++ ParseFromCodedStream takes too long

On Thu, Mar 20, 2014 at 8:40 AM, christian.kilpatrick.1...@gmail.com wrote: The message will be changed. Score isn't optional. Every docid has a score. I could try it as you suggested. Without the message. But I'm a little bit confused. I tried it with Java and with C++. Java can parse the

Re: [protobuf] If proto Foo and proto Bar have exactly the same fields, can I use Bar to parse a serialized Foo message?

On Tue, Mar 11, 2014 at 7:28 PM, Zihan Liu updog...@google.com wrote: If proto Foo and proto Bar have exactly the same fields, can I use Bar to parse a serialized Foo message? Yes. The actual message name (or any of the field names for that matter) are not encoded. The only things that matter

Re: [protobuf] Protobuf 2.4.1 headers w/ 2.5.0 link-time library (?)

On Tue, Mar 4, 2014 at 11:52 PM, Jon Wooding jcwood...@gmail.com wrote: Hopefully this question isn't a waste of your time. I'm trying to build a program using OpenLighting Architecture and I'm receiving an error that OLA requires 2.5.0, but installed is 2.4.1. Make sure that your headers are

Re: [protobuf] Serialization problem with repeated nested filed sending through a socket

On Tue, Mar 11, 2014 at 11:09 PM, Ke Wang kewang...@gmail.com wrote: Hi all, I am using google protocol buffer to transmit complex data structures over socket. Here is the .proto file: message MatrixMsg { required string msgType = 1; optional int64 count = 2; optional string extraInfo =

Re: [protobuf] Serialization problem with repeated nested filed sending through a socket

On Wed, Mar 12, 2014 at 9:48 AM, Ke Wang kewang...@gmail.com wrote: Thanks, but when I print out the char* using string.data(), it got truncated too. Right... most print functions will stop when they see a null character... you can't use printf/cout/etc -- you'd have to write a custom print

Re: [protobuf] Serialization problem with repeated nested filed sending through a socket

, at the sever side, I still received the message that got truncated. Does that mean the server hasn't finished received the whole data in mmChar, do I need to do a loop receive until I get all the messages? Thanks! Ke On Wednesday, March 12, 2014 8:56:32 AM UTC-5, Ilia Mirkin wrote: On Wed, Mar 12

Re: [protobuf] How to extract enumeration value with its comments from a protobuf descriptor file ?

On Thu, Feb 27, 2014 at 11:04 AM, r.bertho...@gmail.com wrote: Hello, I have a proto buf file with an enumeration with leading comments. I do this commands to obtains FileDescriptorSet : protoc.exe --cpp_out=proto File.proto --include_imports --include_source_info -oFile.desc I want

Re: [protobuf] Serialize BufferedImage with Proto

On Tue, Feb 25, 2014 at 3:03 PM, jCoder jwch...@gmail.com wrote: I was wondering if anyone had any success implementing this with a HashMap that has a pointer to a BufferedImage. Example: Map thumb = new HashMapString, BufferedImage(); I am currently using Serialization with a custom

Re: [protobuf] Serialize BufferedImage with Proto

object class inside of it, will this slow down serialization? On Wednesday, February 26, 2014 9:10:24 PM UTC-5, Ilia Mirkin wrote: On Tue, Feb 25, 2014 at 3:03 PM, jCoder jwc...@gmail.com wrote: I was wondering if anyone had any success implementing this with a HashMap that has a pointer

Re: [protobuf] Serialize BufferedImage with Proto

to a byte[] would cost just as much if not more time then it currently takes. On Wednesday, February 26, 2014 10:22:20 PM UTC-5, Ilia Mirkin wrote: On Wed, Feb 26, 2014 at 10:13 PM, jCoder jwc...@gmail.com wrote: I was able to change the BufferedImage to an ImageIcon which is Serializable

Re: [protobuf] ruby parse_from_file ?

On Sun, Feb 16, 2014 at 11:09 AM, Jay Wilson jawro...@gmail.com wrote: I created some code that puts 10 GPB messages into a file. When I look at the file with a hex editor all of the messages are in the file. I now want to read the messages back in, processing them 1 at a time. The following

Re: [protobuf] Generate .pb with pb4php library?

A quick search for pb4php turns up https://code.google.com/p/pb4php/ which in turn has a link to http://coderpeek.com/2008-07-17-protocol-buffer-for-php And the example on there is $book = new AddressBook(); $person = $book-add_person(); $person-set_name('Kordulla');

Re: [protobuf] Reading .pb file

You can use protoc --decode_raw to decode it. It will give you a probable decoding of the message -- some specifics can't be known from the encoding alone (e.g. string vs submessage, various types of ints, etc are confusable). You could implement this same functionality using the libraries

Re: [protobuf] Handeling large amount of data

You mentioned you added a reserve function. Are you talking about accelContainer.mutable_data()-Reserve(event.size)? Unfortunately that will still construct the object when calling add_data. You may consider getting rid of the sub-object and just having repeated x/y/z. It's not as nice, but

Re: [protobuf] Handeling large amount of data

, item item item item vs tag item tag item tag item). On Tue, Dec 3, 2013 at 5:23 AM, Mathijs Jonker mathijsjon...@msn.com wrote: Op dinsdag 3 december 2013 10:33:47 UTC+1 schreef Ilia Mirkin: You mentioned you added a reserve function. Are you talking about accelContainer.mutable_data

Re: [protobuf] Re: Parsing sub message not working with certain message

Well, fundamentally, it should all work fine. So you're doing something wrong. Your best bet for getting further help would be supplying a full repro, i.e. the full proto spec + data being parsed. The fact that it says 9 {} and not details {} means that it doesn't seem to know about the details

Re: [protobuf] can client works on 2.4.1 and server works on 2.5.0

Yes, protobuf wire format should remain constant across all versions. If you have any indication that it doesn't, you should report a bug. (However internally, there can be API changes between versions, so you might need to adjust code.) On Fri, Oct 11, 2013 at 4:53 AM, zhouyu...@googlemail.com

Re: [protobuf] Repeated combined with Required

Protobuf is about serializing and deserializing data, not enforcing restrictions on it. (required being the odd case there, and it is often recommended that you avoid it.) Using repeated in the spec, and then enforcing any additional restrictions in your code is the preferred way to go. That way,

Re: [protobuf] enum to string example

You need to get a EnumDescriptor for the field in question, and then retrieve the EnumValueDescriptor using FindValueByNumber. That will have all the name information you need. To get the EnumDescriptor of an enum Foo, you can use Foo_descriptor() which should be in the generated code. I assume

Re: [protobuf] encoding with protobuf java classes

The distinction between text and binary is fairly small and arbitrary. Can you elaborate a bit more as to what you're seeing and what you're expecting to see? In the protobuf encoding, any string values should show up unaltered inside of the stream (not sure why one would expect them to be mangled

Re: [protobuf] How to encode nested Python Protobuf

I believe Chris's suggestion was to base64-encode the full message, not the subproto, or you'll run into other problems down the line, since it sounds like your transport can't handle arbitrary byte sequences. IOW, something like response.response_proto = heartbeatResult.SerializeToString()

Re: [protobuf] Error Setting Repeated Fields

A function is a member of a class. There is no set_field() function if field is repeated. Take a look at https://developers.google.com/protocol-buffers/docs/cpptutorial -- there's add_field() and mutable_field() depending on what you want to do. -ilia On Sat, Sep 7, 2013 at 1:38 PM, Austin Luu

Re: [protobuf] How to encode nested Python Protobuf

On Sun, Aug 25, 2013 at 12:47 AM, chrisg...@gmail.com wrote: heartbeatResult = rpc_pb2.HeartbeatResult() heartbeatResult.service = ALERT_SERVICE heartbeatResult.timestamp = st heartbeatResult.status_cd = rpc_pb2.OK heartbeatResult.status_summary = OK response =

Re: [protobuf] How to encode nested Python Protobuf

On Mon, Aug 26, 2013 at 8:32 PM, chrisg...@gmail.com wrote: Thanks for taking the time to read/reply. GetHeartbeat2 is my original version which I included as well and the way I had thought it should probably work. This code is serializing to string on the inner buffer and then again on the

Re: [protobuf] deserializing unknown protobuf message?

Something like protoc --decode_raw? As for manipulating these in code, there's nothing too great available if you don't have a descriptor at all. You could create a dummy message (no fields) and then get them via unknown fields in the message's reflection object, but they're not easy to

Re: [protobuf] Re: Mapper for protocol buffer entities

Is your basic question is there an easy way to convert between a type I created and a vastly similar type created by protoc? AFAIK the answer is no, definitely not in the core protobuf library. However it shouldn't be too difficult to create something that handles most cases with reflection. (And

Re: [protobuf] Question about size/speed of protobufs with different formats

Well, for regular values, it goes tag value, for a subproto it goes tag length submessage (which in turn has tag value pairs in it). So it depends on the number of fields inside of the message, and how many bytes it is total. But assuming non-edge-case conditions, you're probably better off using

Re: [protobuf] Partial Decoding of message

On Tue, Jul 9, 2013 at 11:36 AM, phreed phr...@gmail.com wrote: On Monday, July 8, 2013 12:59:12 PM UTC-5, Ilia Mirkin wrote: If this is not an option, you can write a custom decoder that just skips over fields you don't need to read. This is a little tricky, but if you're not trying

Re: [protobuf] Partial Decoding of message

On Tue, Jul 9, 2013 at 4:36 PM, phreed phr...@gmail.com wrote: On Tuesday, July 9, 2013 1:51:51 PM UTC-5, Ilia Mirkin wrote: On Tue, Jul 9, 2013 at 11:36 AM, phreed phr...@gmail.com wrote: My situation is a bit different than the original poster. I have a set of content sensitive network

Re: [protobuf] Object that refers to another object of the same type

You can't really have pointers to data with protobuf, only actual data. You might do something like message Node { optional string name; repeated string neighbor; } message Graph { repeated string names; repeated Node nodes; } And use the name (or some other unique identifier) to figure

Re: [protobuf] Running a sample AddPerson.java example - Fails giving Protocol message tag had invalid wire type

I think it wants a file with proto data, not the proto description. Also it wants it proto-encoded, not text-encoded. On Mon, Jul 1, 2013 at 6:00 PM, Dhaval Kotecha dhaval.kote...@gmail.com wrote: Hi All, I am a newbie to Protocol Buffers. I am trying to run the AddPerson example, and it

Re: [protobuf] Byte Offsets into File with Multiple Messages

You also need to keep track of how big your header is. Otherwise when reading the header, it will happily just keep on going, accumulating unknown tags/etc. When you write the file, keep track of the actual offsets you write the protos at. When reading, compare them to what you had when written

Re: [protobuf] Byte Offsets into File with Multiple Messages

More code certainly couldn't hurt. Are you sure that the data is making it into the file OK? Are you opening the file with ios::binary or whatever the flag is? On Tue, Jun 25, 2013 at 6:16 PM, jonathan.w...@gree.co.jp wrote: After checking the offsets written to the header in the file and the

Re: [protobuf] protobuf messages as repeated field?

Try x = packetMessage.checkMessage.balanceUpdates.add() x.tid = 111 x.amount = 222 See https://developers.google.com/protocol-buffers/docs/pythontutorial for more info. On Sun, Jun 23, 2013 at 9:49 AM, Lotte R. Leben lo...@zombietetris.de wrote: Hi, this might be quite the newbie question,

Re: [protobuf] encoding: how to distinguish message and the field

Messages aren't preceded by anything. A message inside of a message is preceeded by a varint that indicates its length (there's also a deprecated begin/end group mechanism that you probably don't have to worry about supporting). Then, for that length after the varint, decoding of the sub-message

  1   2   >