So like I said... you have to write it as
<header length><header which contains message length><message>
To decode it, you can stick the header length as a varint, all via the
coded input/output stream. Sample code for encoding:
makePacket(Message message) {
String name = ... (there has gotta be a way of getting the typename
from the message)
length = message.getSerializedSize()
Header h = Header.newBuilder().setName(name).setLength(length);
CodedOutputStream os = newInstance();
os.writeInt32(h.getSerializedSize());
os.writeMessageNoTag(h);
os.writeMessageNoTag(message);
}
And to decode:
Message readPacket() {
CodedInputStream is;
hlen = is.readInt32();
limit = is.pushLimit(hlen);
Header h = Header.parseFrom(is);
is.popLimit(limit);
is.pushLimit(h.getLength());
messageFactory.getBuilderForType(h.getName()).mergeFrom(is);
}
And you're done. Protobuf is meant for serializing data, not framing
it. Your complaint about lack of framing is well taken, but ... it's
just not part of what protobuf does.
You could just as well have a
message TheOneTrueMessage {
string name;
bytes data;
}
And then always decode that -- this gives you your framing. But this
ends up causing lots of copies of the data.
-ilia
On Tue, Jul 10, 2018 at 11:20 AM, John Lilley <whee...@gmail.com> wrote:
> Just to wrap this up, here's what I found:
> You really have to know how big the header message is, because if you simply
> wrap an input stream around the entire concatenated buffer, the
> deserialization doesn't know where the end of the header is. This strikes
> me as a shortcoming of protobuf, but it is what it is. I used code
> something like(in java):
>
> protected static byte[] makePacket(Message header, Message message) throws
> IOException {
> // We cannot simply concatenate the two messages; protobuf will fail to
> deserialize them.
> // So we add the header length at the front. But we don't know it yet...
> ByteArrayOutputStream os = new ByteArrayOutputStream();
> os.write(new byte[4]); // we will update this later
> CodedOutputStream cos = CodedOutputStream.newInstance(os);
> header.writeTo(cos);
> cos.flush();
> // NOW we know the header length
> int headerLength = os.size() - 4;
> message.writeTo(cos);
> cos.flush();
> byte[] result = os.toByteArray();
> // Set header size
> ByteBuffer bb = ByteBuffer.wrap(result);
> bb.order(ByteOrder.LITTLE_ENDIAN);
> bb.putInt(0, headerLength);
> return result;
> }
>
> Then for deserialization:
>
> protected static Message getResponseMessage(MessageFactory messageFactory,
> byte[] packet) throws Exception {
> ByteBuffer bb = ByteBuffer.wrap(packet);
> bb.order(ByteOrder.LITTLE_ENDIAN);
> int headerLength = bb.getInt(0);
> CommonWrapper.ResponseHeader header;
> {
> InputStream is = new ByteArrayInputStream(packet, 4, headerLength);
> CodedInputStream cis = CodedInputStream.newInstance(is);
> header = CommonWrapper.ResponseHeader.parseFrom(cis);
> }
> InputStream is = new ByteArrayInputStream(packet, 4 + headerLength,
> packet.length - (4 + headerLength));
> return messageFactory.createMessage(header.getResponseMessageType(), is);
> }
>
> The MessageFactory is a hand-rolled class for creating messages from full
> name; but that's for a different thread.
>
> john
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
