Re: [protobuf] Re: [protobuf-net] Unknow wire-type 7 when serializing List data structure
Hi; it is pretty hard to comment without code. Do you have a minimal repro that shows what you're seeing? The fact that you mention Encoding suggests that you're treating protobuf data as though it were text, which is never a good idea. Also, since this is library specific, you may wish to move this discussion to https://github.com/protobuf-net/protobuf-net instead. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/protobuf/1e2ca6dc-0d90-4247-98f7-66cf17ec67d9o%40googlegroups.com.
Re: [protobuf] Re: [protobuf-net] Unknow wire-type 7 when serializing List data structure
Hello Marc,
Thanks for providing protobuf-net to community.
I am facing same error message with Protobuf-net version 3.0.29. Error
message is "Invalid wire-type (Varint); this usually means you have
over-written a file without truncating or setting the length"
I thought main reason is Encoding type in my case. I passed the Hex string
and having Encoding type = RawEncoding then its work good. But with other
data with Encoding Type = HeatshrinkLzssEncoding, it gives me above
mentioned error. Should it not support this encoding type?
Thanks in advance.
On Thursday, December 24, 2009 at 3:23:32 AM UTC+5:30, Marc Gravell wrote:
>
> OK; now it makes sense. With the description and sample code above, this
> is nothing to do with protobuf-net (specifically), and everything to do
> with how you are doing your file access. If you use File.Open with
> OpenOrCreate on an **existing** file, it doesn't truncate the file. If you
> then write *fewer* bytes than were in the original file, you essentially
> have garbage at the end of the stream.
>
> The "protocol buffers" wire format doesn't (by default) include a length
> prefix for the overall message, since it is designed to be possible to
> merge by appending. You have a few options:
>
> 1: delete the file before re-writing it (although this may impact any ACLs
> etc that you have defined)
> 2: specify FileMode.Truncate rather than FileMode.OpenOrCreate (truncates
> old data *before* writing)
> 3: call stream.SetLength(stream.Position) after writing
> (truncates old data *after* writing)
> 4: use SerializeWithLengthPrefix (but note that this will *retain* old
> garbage)
>
> To reproduce just the stream aspects here, see below.
>
> Marc
>
> using System;
> using System.IO;
>
> static class Program {
> static Random rand = new Random();
> const string Path = @"scratch.bin";
> static void WriteGarbabe(Stream stream, int length) {
> byte[] data = new byte[length];
> rand.NextBytes(data);
> stream.Write(data, 0, length);
> }
> static void ShowLength(string path) {
> Console.WriteLine(new FileInfo(path).Length);
> }
> static void Main() {
> using (Stream dest = File.Open(Path, FileMode.OpenOrCreate)) {
> WriteGarbabe(dest, 50);
> }
> ShowLength(Path); // shows 50 like we expect
> using (Stream dest = File.Open(Path, FileMode.OpenOrCreate)) {
> WriteGarbabe(dest, 40);
> }
> ShowLength(Path); // shows 50, we might have expected 40
> using (Stream dest = File.Open(Path, FileMode.OpenOrCreate)) {
> WriteGarbabe(dest, 30);
> dest.SetLength(dest.Position);
> }
> ShowLength(Path); // shows 30; SetLength works
> using (Stream dest = File.Open(Path, FileMode.Truncate)) {
> WriteGarbabe(dest, 20);
> dest.SetLength(dest.Position);
> }
> ShowLength(Path); // shows 20; Truncate works
> }
> }
>
> 2009/12/22 jeevankodali >
>
>> Finally I was able to reproduce the cause of this error. I am not
>> getting the same error in this sample code but a different one (but I
>> think reason is the same).
>>
>> This is what I did which caused this:
>>
>> 1. First I stored a dictionary, key is int and value is a list into
>> the proto file and loaded this data - this step was fine.
>> 2. Next I changed my code to store just list instead of dictionary in
>> the same file - store was successful
>> 3. I tried to load this new file (which was the same name and location
>> as the original file with dictionary), I was getting the error.
>>
>> But when I deleted the original file and reran the code (which was
>> storing the list), everything was fine.
>>
>> Here is the code:
>>
>>[ProtoContract]
>>public class ProtoTestClass
>>{
>>private string _id;
>>
>>[ProtoMember(1)]
>>public string Id
>>{
>>get { return _id; }
>>set { _id = value; }
>>}
>>}
>>
>> // below code throws error because I am using the same file
>>private static void ProtoTest()
>>{
>>List ownerList = new List
>> ();
>>ProtoTestClass owner = new ProtoTestClass();
>>owner.Id = "799";
>>ownerList.Add(owner);
>>
>>Dictionary> dictionaryList = new
>> Dictionary>();
>>dictionaryList.Add(1, ownerList);
>>
>>using (Stream outfile = File.Open(@"d:\protobuftest.bin",
>> FileMode.OpenOrCreate))
>>{
>>ProtoBuf.Serializer.Serialize(outfile,
>> dictionaryList);
>>}
>>
>>Dictionary> outDictionaryList =
>> new Dictionary>();
>>using (Stream inFile = File.Open(@"d:\protobuftest.bin",
>> FileMode.OpenOrCreate))
>>{
>>outDictionaryList =
>> ProtoBuf.Serializer.Deserialize>>
>> (inFile);
>>}
>>
>>
>>// now write just list
Re: [protobuf] Re: [protobuf-net] Unknow wire-type 7 when serializing List data structure
Really, it is whatever is enough to show the problem. For example, the
following doesn't (for me) show any issues. You might also want to clarify
your setup - could it be some x64/ia64 related bug, for example? (I'm
testing on x86, Win7):
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using ProtoBuf;
[ProtoContract]
public class ProtoBlock {
private string date;
[ProtoMember(1)]
public string Date {
get { return date; }
set { date = value; }
}
}
static class Program {
static void Main() {
ListProtoBlock data = new ListProtoBlock();
RoundTrip(data); // 0
data.Add(new ProtoBlock { Date = abc });
RoundTrip(data); // 1
data.Add(new ProtoBlock { Date = def });
RoundTrip(data); // 2
data.Add(new ProtoBlock { Date = ghi });
RoundTrip(data); // 3
}
static void RoundTrip(ListProtoBlock list) {
string path = foo.bin;
if (File.Exists(path)) File.Delete(path);
using (Stream outfile = File.Open(path, FileMode.OpenOrCreate)) {
ProtoBuf.Serializer.Serialize(outfile, list);
}
ListProtoBlock clone;
using (Stream inFile = File.OpenRead(path)) {
clone =
ProtoBuf.Serializer.DeserializeListProtoBlock(inFile)
?? new ListProtoBlock(); // blanks become null -
limitation
}
Debug.Assert(!ReferenceEquals(clone, list)); // different lists
Debug.Assert(clone.Count == list.Count); // same count
for (int i = 0; i list.Count; i++) {
Debug.Assert(clone[i].Date == list[i].Date); // same data
}
}
}
2009/12/21 jeevankodali jeevankod...@gmail.com
Sorry for the late reply, what exactly do you want in the full
example, I can add the code that I am using, do you need the file as
well (and how can I send the file)?
Thanks
On Dec 19, 5:09 pm, Marc Gravell marc.grav...@gmail.com wrote:
Rechecked with Net20 from the r278 build, and it worked fine. Sorry to be
a
pain, but do you have a complete example that shows the issue?
Marc
2009/12/19 Marc Gravell marc.grav...@gmail.com
OK; I'll re-check with the Net20 variant...
2009/12/19 jeevankodali jeevankod...@gmail.com
Thanks for the reply. I used r278 and Net20 folder in that zip file.
On Dec 18, 2:36 pm, Marc Gravell marc.grav...@gmail.com wrote:
Hmmm... that should work fine; there *was* a related bug in an early
build,
but...
Can I check which version and framework you are using? I've checked
on
r275,
and it works fine as is, for both empty lists and some arbitrary
data
I
made up.
Sorry for the delay, btw - seasonal break, etc...
Marc Gravell (protobuf-net)
--
You received this message because you are subscribed to the Google
Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
protobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
Regards,
Marc
--
Regards,
Marc- Hide quoted text -
- Show quoted text -
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
Regards,
Marc
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Re: [protobuf-net] Unknow wire-type 7 when serializing List data structure
OK; I'll re-check with the Net20 variant... 2009/12/19 jeevankodali jeevankod...@gmail.com Thanks for the reply. I used r278 and Net20 folder in that zip file. On Dec 18, 2:36 pm, Marc Gravell marc.grav...@gmail.com wrote: Hmmm... that should work fine; there *was* a related bug in an early build, but... Can I check which version and framework you are using? I've checked on r275, and it works fine as is, for both empty lists and some arbitrary data I made up. Sorry for the delay, btw - seasonal break, etc... Marc Gravell (protobuf-net) -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.comprotobuf%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- Regards, Marc -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
