Thanks for your help, it makes sense.

On Dec 23, 3:53 pm, Marc Gravell <marc.grav...@gmail.com> wrote:
> OK; now it makes sense. With the description and sample code above, this is
> nothing to do with protobuf-net (specifically), and everything to do with
> how you are doing your file access. If you use File.Open with OpenOrCreate
> on an **existing** file, it doesn't truncate the file. If you then write
> *fewer* bytes than were in the original file, you essentially have garbage
> at the end of the stream.
>
> The "protocol buffers" wire format doesn't (by default) include a length
> prefix for the overall message, since it is designed to be possible to merge
> by appending. You have a few options:
>
> 1: delete the file before re-writing it (although this may impact any ACLs
> etc that you have defined)
> 2: specify FileMode.Truncate rather than FileMode.OpenOrCreate (truncates
> old data *before* writing)
> 3: call stream.SetLength(stream.Position) after writing (truncates old data
> *after* writing)
> 4: use SerializeWithLengthPrefix (but note that this will *retain* old
> garbage)
>
> To reproduce just the stream aspects here, see below.
>
> Marc
>
> using System;
> using System.IO;
>
> static class Program {
>     static Random rand = new Random();
>     const string Path = @"scratch.bin";
>     static void WriteGarbabe(Stream stream, int length) {
>         byte[] data = new byte[length];
>         rand.NextBytes(data);
>         stream.Write(data, 0, length);
>     }
>     static void ShowLength(string path) {
>         Console.WriteLine(new FileInfo(path).Length);
>     }
>     static void Main() {
>         using (Stream dest = File.Open(Path, FileMode.OpenOrCreate)) {
>             WriteGarbabe(dest, 50);
>         }
>         ShowLength(Path); // shows 50 like we expect
>         using (Stream dest = File.Open(Path, FileMode.OpenOrCreate)) {
>             WriteGarbabe(dest, 40);
>         }
>         ShowLength(Path); // shows 50, we might have expected 40
>         using (Stream dest = File.Open(Path, FileMode.OpenOrCreate)) {
>             WriteGarbabe(dest, 30);
>             dest.SetLength(dest.Position);
>         }
>         ShowLength(Path); // shows 30; SetLength works
>         using (Stream dest = File.Open(Path, FileMode.Truncate)) {
>             WriteGarbabe(dest, 20);
>             dest.SetLength(dest.Position);
>         }
>         ShowLength(Path); // shows 20; Truncate works
>     }
>
> }
>
> 2009/12/22 jeevankodali <jeevankod...@gmail.com>
>
>
>
>
>
> > Finally I was able to reproduce the cause of this error. I am not
> > getting the same error in this sample code but a different one (but I
> > think reason is the same).
>
> > This is what I did which caused this:
>
> > 1. First I stored a dictionary, key is int and value is a list into
> > the proto file and loaded this data - this step was fine.
> > 2. Next I changed my code to store just list instead of dictionary in
> > the same file - store was successful
> > 3. I tried to load this new file (which was the same name and location
> > as the original file with dictionary), I was getting the error.
>
> > But when I deleted the original file and reran the code (which was
> > storing the list), everything was fine.
>
> > Here is the code:
>
> >    [ProtoContract]
> >    public class ProtoTestClass
> >    {
> >        private string _id;
>
> >        [ProtoMember(1)]
> >        public string Id
> >        {
> >            get { return _id; }
> >            set { _id = value; }
> >        }
> >    }
>
> > // below code throws error because I am using the same file
> >        private static void ProtoTest()
> >        {
> >            List<ProtoTestClass> ownerList = new List<ProtoTestClass>
> > ();
> >            ProtoTestClass owner = new ProtoTestClass();
> >            owner.Id = "799";
> >            ownerList.Add(owner);
>
> >            Dictionary<int, List<ProtoTestClass>> dictionaryList = new
> > Dictionary<int, List<ProtoTestClass>>();
> >            dictionaryList.Add(1, ownerList);
>
> >            using (Stream outfile = File.Open(@"d:\protobuftest.bin",
> > FileMode.OpenOrCreate))
> >            {
> >                ProtoBuf.Serializer.Serialize(outfile,
> > dictionaryList);
> >            }
>
> >            Dictionary<int, List<ProtoTestClass>> outDictionaryList =
> > new Dictionary<int, List<ProtoTestClass>>();
> >            using (Stream inFile = File.Open(@"d:\protobuftest.bin",
> > FileMode.OpenOrCreate))
> >            {
> >                outDictionaryList =
> > ProtoBuf.Serializer.Deserialize<Dictionary<int, List<ProtoTestClass>>>
> > (inFile);
> >            }
>
> >            // now write just list again in the same file
> >            using (Stream outfile = File.Open(@"d:\protobuftest.bin",
> > FileMode.OpenOrCreate))
> >            {
> >                ProtoBuf.Serializer.Serialize(outfile, ownerList);
> >            }
>
> >            List<ProtoTestClass> outList = new List<ProtoTestClass>();
> >            using (Stream inFile = File.Open(@"d:\protobuftest.bin",
> > FileMode.OpenOrCreate))
> >            {
> >                outList =
> > ProtoBuf.Serializer.Deserialize<List<ProtoTestClass>>(inFile);
> >            }
>
> >            Console.WriteLine("Done with test");
> >         }
>
> > On Dec 22, 5:40 am, Marc Gravell <marc.grav...@gmail.com> wrote:
> > > Really, it is whatever is enough to show the problem. For example, the
> > > following doesn't (for me) show any issues. You might also want to
> > clarify
> > > your setup - could it be some x64/ia64 related bug, for example? (I'm
> > > testing on x86, Win7):
>
> > > using System.Collections.Generic;
> > > using System.Diagnostics;
> > > using System.IO;
> > > using ProtoBuf;
>
> > > [ProtoContract]
> > > public class ProtoBlock {
> > >     private string date;
> > >     [ProtoMember(1)]
> > >     public string Date {
> > >         get { return date; }
> > >         set { date = value; }
> > >     }
>
> > > }
>
> > > static class Program {
> > >     static void Main() {
> > >         List<ProtoBlock> data = new List<ProtoBlock>();
> > >         RoundTrip(data); // 0
> > >         data.Add(new ProtoBlock { Date = "abc" });
> > >         RoundTrip(data); // 1
> > >         data.Add(new ProtoBlock { Date = "def" });
> > >         RoundTrip(data); // 2
> > >         data.Add(new ProtoBlock { Date = "ghi" });
> > >         RoundTrip(data); // 3
> > >     }
> > >     static void RoundTrip(List<ProtoBlock> list) {
> > >         string path = "foo.bin";
> > >         if (File.Exists(path)) File.Delete(path);
> > >         using (Stream outfile = File.Open(path, FileMode.OpenOrCreate)) {
> > >             ProtoBuf.Serializer.Serialize(outfile, list);
> > >         }
> > >         List<ProtoBlock> clone;
> > >         using (Stream inFile = File.OpenRead(path)) {
> > >             clone =
> > > ProtoBuf.Serializer.Deserialize<List<ProtoBlock>>(inFile)
> > >                 ?? new List<ProtoBlock>(); // blanks become null -
> > > limitation
> > >         }
> > >         Debug.Assert(!ReferenceEquals(clone, list)); // different lists
> > >         Debug.Assert(clone.Count == list.Count); // same count
> > >         for (int i = 0; i < list.Count; i++) {
> > >             Debug.Assert(clone[i].Date == list[i].Date); // same data
> > >         }
> > >     }
>
> > > }
>
> > > 2009/12/21 jeevankodali <jeevankod...@gmail.com>
>
> > > > Sorry for the late reply, what exactly do you want in the full
> > > > example, I can add the code that I am using, do you need the file as
> > > > well (and how can I send the file)?
>
> > > > Thanks
>
> > > > On Dec 19, 5:09 pm, Marc Gravell <marc.grav...@gmail.com> wrote:
> > > > > Rechecked with Net20 from the r278 build, and it worked fine. Sorry
> > to be
> > > > a
> > > > > pain, but do you have a complete example that shows the issue?
>
> > > > > Marc
>
> > > > > 2009/12/19 Marc Gravell <marc.grav...@gmail.com>
>
> > > > > > OK; I'll re-check with the Net20 variant...
>
> > > > > > 2009/12/19 jeevankodali <jeevankod...@gmail.com>
>
> > > > > > Thanks for the reply. I used r278 and Net20 folder in that zip
> > file.
>
> > > > > >> On Dec 18, 2:36 pm, Marc Gravell <marc.grav...@gmail.com> wrote:
> > > > > >> > Hmmm... that should work fine; there *was* a related bug in an
> > early
> > > > > >> build,
> > > > > >> > but...
>
> > > > > >> > Can I check which version and framework you are using? I've
> > checked
> > > > on
> > > > > >> r275,
> > > > > >> > and it works fine "as is", for both empty lists and some
> > arbitrary
> > > > data
> > > > > >> I
> > > > > >> > made up.
>
> > > > > >> > Sorry for the delay, btw - seasonal break, etc...
>
> > > > > >> > Marc Gravell (protobuf-net)
>
> > > > > >> --
>
> > > > > >> You received this message because you are subscribed to the Google
> > > > Groups
> > > > > >> "Protocol Buffers" group.
> > > > > >> To post to this group, send email to proto...@googlegroups.com.
> > > > > >> To unsubscribe from this group, send email to
> > > > > >> protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.c­om>
> > <protobuf%2bunsubscr...@googlegroups.c­om>
> > > > <protobuf%2bunsubscr...@googlegroups.c­om>
> > > > > >> .
> > > > > >> For more options, visit this group at
> > > > > >>http://groups.google.com/group/protobuf?hl=en.
>
> > > > > > --
> > > > > > Regards,
>
> > > > > > Marc
>
> > > > > --
> > > > > Regards,
>
> > > > > Marc- Hide quoted text -
>
> > > > > - Show quoted text -
>
> > > > --
>
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Protocol Buffers" group.
> > > > To post to this group, send email to proto...@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.c­om>
> > <protobuf%2bunsubscr...@googlegroups.c­om>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/protobuf?hl=en.
>
> > > --
> > > Regards,
>
> > > Marc- Hide quoted text -
>
> > > - Show quoted text -
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Protocol Buffers" group.
> > To post to this group, send email to proto...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.c­om>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/protobuf?hl=en.
>
> -- ...
>
> read more »- Hide quoted text -
>
> - Show quoted text -

--

You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to proto...@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.


Reply via email to