Re: Protobuf-net Serialize problem
I've just been looking more - and ParseFromString should do the job. I /suspect/ the problem is the definition a string... you aren't going to be able to treat raw binary in .NET as a String; you are going to have to treat it as a Stream or a byte[], or you can encode it to base-64 (as above) to get something that *can* be treated as a String. Either as binary or a base-64, you would then transfer that to the server. At the server, ProtoBuf again expects raw binary, but in a C++ string (if you have used base-64 as a transfer mechanism, you would have to decode the base-64 back to binary). You should then be able to mount that data into a C++ string (or an istream). My best guess is that at the moment you are losing data during transfer (as above). What mechanism are you using to transfer the data? If it is a network socket (for example) you should be able to just send binary, without ever touching a .NET string; likewise HTTP can be used without creating a .NET string. Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protobuf-net Serialize problem
Marc, The server is using ProtoBuf 2.0.1. I had to upgrade my .net version to the latest protobuf-net in order to have protogen work with optional fields correctly. It seems that the server only processes bytes up to the 1st NULL char when calling ParseFromString. Will this problem be avoided with Base64? Thanks On Apr 2, 5:52 pm, Marc Gravell marc.grav...@gmail.com wrote: (re last post) As with unicode, you can't use ASCII to handle raw binary; either use base-64 (previous post), or send binary. Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protobuf-net Serialize problem
Hi; I'm the author of protobuf-net, and I hope I can help you find what is going on here... The binary format should be compatible between platforms (that is the point, after all). Are you able to share your OrderProto class with me? Either here, or at marc.grav...@gmail.com. The base-64 should be unrelated as long as it is being packed/unpacked correctly. I know of many people using protobuf-net to communicate between both client/ server and windows/linux (mono), so it should work... If you are transporting over a network - then one common issue is that the protocol buffers format is not terminated; messages will bleed into each-other unless you separate messages. You can do this using SerializeWithLengthPrefix and DeserializeWithLengthPrefix, which include the length of the message and only read that much data from the stream. Marc Gravell --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protobuf-net Serialize problem
It seems I was wrong in trying to use ASCII or UTF-8 encoding. This
works.
private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.SerializeOrderProto(stream, proto);
using (StreamReader reader = new StreamReader(stream))
return reader.ReadToEnd();
}
}
Thanks
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Protobuf-net Serialize problem
Actually, I was wrong, this code produces an empty string. The
following code serializes *something* which fails on the server side
on ParseFromString. Any ideas what I have to do to correctly serialize
this?
private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.SerializeOrderProto(stream, proto);
stream.Seek(0, SeekOrigin.Begin);
using (BinaryReader reader = new BinaryReader(stream,
Encoding.ASCII))
{
char[] chars = new char[stream.Length];
for (int i = 0; i stream.Length; i++)
chars[i] = reader.ReadChar();
return new string(chars);
}
}
}
Thanks
On Apr 2, 1:04 pm, test.f...@nomail.please test.f...@gmail.com
wrote:
It seems I was wrong in trying to use ASCII or UTF-8 encoding. This
works.
private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.SerializeOrderProto(stream, proto);
using (StreamReader reader = new StreamReader(stream))
return reader.ReadToEnd();
}
}
Thanks
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Protobuf-net Serialize problem
Quick question - which build are you using? (at client and server)? Based on the field names, the code-generation dates back to November; if the dlls are aged similarly, it is possible that this is an old bug. If possible, can you retry with the current build? I'm just trying the code you posted, to see if I can spot anything... Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protobuf-net Serialize problem
You can't use unicode to transport arbitrary binary data - it won't be
a valid unicode string. If you have to use strings, then base-64 is
the safest option:
private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.SerializeOrderProto(stream, proto);
return Convert.ToBase64String(stream.GetBuffer(), 0,
(int)stream.Length);
}
}
private OrderProto Deserialize(string str)
{
byte[] data = Convert.FromBase64String(str);
using (MemoryStream stream = new MemoryStream(data)) {
return Serializer.DeserializeOrderProto(stream);
}
}
However, if possible, raw binary would be even more efficient.
I've tried the above (with the current build), and it round-trips
fine, giving the same length when repeated.
Does the above fix it at all?
Marc
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Protobuf-net Serialize problem
(re last post) As with unicode, you can't use ASCII to handle raw binary; either use base-64 (previous post), or send binary. Marc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Protobuf-net Serialize problem
Does someone have a Serialize sample for protobuf-net? I'm using VS
2005/C#.
When I try Encoding.ASCII.GetBytes() and Encoding.ASCII.GetString(), I
get an exception invalid wire type - 7. If I
Convert.FromBase64String, serialize/deserialize works but the message
is rejected by the server which is running on Linux and is expecting a
valid protobuf as a string.
private string Serialize(OrderProto proto)
{
using (MemoryStream stream = new MemoryStream())
{
Serializer.SerializeOrderProto(stream, proto);
byte[] data = new byte[stream.Length];
stream.Seek(0, SeekOrigin.Begin);
stream.Read(data, 0, data.Length);
return Convert.ToBase64String(data);
}
}
private OrderProto TestDeserialize(string str)
{
byte[] data = Convert.FromBase64String(str);
MemoryStream stream = new MemoryStream(data);
OrderProto proto = Serializer.DeserializeOrderProto
(stream);
return proto;
}
OrderProto proto = p.GetOrderProto(order.OrderID);
string protoStr = this.Serialize(proto); // but as Base64, this
protobuf is rejected by the server
OrderProto test = TestDeserialize(protoStr);
Another thing I noticed is that if I re-serialize a protobuf I
received from the server, the resulting byte array is roughly 10 bytes
shorter than when came in. But the deserialized class instance looks
correct.
OrderProto order =
Serializer.DeserializeOrderProto(new MemoryStream(contentBytes));
MemoryStream ms = new MemoryStream();
Serializer.SerializeOrderProto(ms, order);
ms.Seek(0, SeekOrigin.Begin);
byte[] buffer = ms.ToArray();
OrderProto test =
Serializer.DeserializeOrderProto(ms);
Thanks
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
