This is a copy of a conversation I had with Serge a day or so ago, about some problems I'm having with the BinaryReader class (I asked him, because he was the last person to commit code changes here.)
Do you have any thoughts re. the buffering issue? The .net docs for this class are extremely terse, but don't mention any buffering (hence my thoughts about leaving that to the underlying stream.) - Dick
--- Begin Message ---I'm having problems with BinaryReader, and I think I've tracked it down to a change you made a few months ago. Specifically, in BinaryReader.EnsureBuffered(), you added: if (m_buffer_used + needed > m_buffer.Length) { byte[] old_buffer = m_buffer; m_buffer = new byte[m_buffer_used + needed]; Array.Copy(old_buffer, 0, m_buffer, 0, m_buffer_used); + m_buffer_pos = m_buffer_used; } That extra line is moving the buffer read position to the end of the newly grown buffer, skipping unread data whenever EnsureBuffered() is called before the buffer is exhausted. I'm also having trouble with the buffering when it pre-reads too much from the underlying stream. With a trivial test case (attached), I get the following results on Windows and Linux: Windows: $ ./readtest.exe Read string "String 1" Stream pos 9 Read int 1234 Stream pos 13 Read string "String 2" Stream pos 22 Linux (with that m_buffer_pos line removed): :; ./mono/interpreter/mint ../readtest.exe Read string "String 1" Stream pos 22 Read int 1234 Stream pos 22 Read string "String 2" Stream pos 22 The buffer position issue is important for me at the moment because I'm implementing System.Resources.ResourceReader, which reads with BinaryReader but at one point must skip over a number of padding bytes to 8-byte align the next field. - Dickusing System; using System.IO; using System.Text; public class Test { public static int Main () { FileStream instr=new FileStream("binarywriter.data", FileMode.Open); BinaryReader reader=new BinaryReader(instr, Encoding.UTF8); string str; int i; str=reader.ReadString(); Console.WriteLine("Read string \"{0}\"", str); Console.WriteLine("Stream pos {0}", reader.BaseStream.Position); i=reader.ReadInt32(); Console.WriteLine("Read int {0}", i); Console.WriteLine("Stream pos {0}", reader.BaseStream.Position); str=reader.ReadString(); Console.WriteLine("Read string \"{0}\"", str); Console.WriteLine("Stream pos {0}", reader.BaseStream.Position); return 0; } }<<attachment: readtest.exe>>
--- End Message ---
binarywriter.data
Description: Binary data
--- Begin Message ---
Hi Dick,
> I'm having problems with BinaryReader, and I think I've tracked it down
> to a change you made a few months ago.
I see. The changes were made in a hurry, so all sorts of nasty errors possible :-)
I initially planned to rewrite the class, since almost everything was broken (the very
first attempt to read resulted in an exception), but just patched it here and there,
enough to make sure that my own code worked with it.
I will look into this.
Serge
----- Original Message -----
From: "Dick Porter" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, August 13, 2002 7:53 PM
Subject: BinaryReader
> I'm having problems with BinaryReader, and I think I've tracked it down
> to a change you made a few months ago. Specifically, in
> BinaryReader.EnsureBuffered(), you added:
>
>
>
> if (m_buffer_used + needed > m_buffer.Length) {
> byte[] old_buffer = m_buffer;
> m_buffer = new byte[m_buffer_used + needed];
> Array.Copy(old_buffer, 0, m_buffer, 0, m_buffer_used);
> + m_buffer_pos = m_buffer_used;
> }
>
> That extra line is moving the buffer read position to the end of the
> newly grown buffer, skipping unread data whenever EnsureBuffered() is
> called before the buffer is exhausted.
>
> I'm also having trouble with the buffering when it pre-reads too much
> from the underlying stream. With a trivial test case (attached), I get
> the following results on Windows and Linux:
>
> Windows:
> $ ./readtest.exe
> Read string "String 1"
> Stream pos 9
> Read int 1234
> Stream pos 13
> Read string "String 2"
> Stream pos 22
>
>
> Linux (with that m_buffer_pos line removed):
> :; ./mono/interpreter/mint ../readtest.exe
> Read string "String 1"
> Stream pos 22
> Read int 1234
> Stream pos 22
> Read string "String 2"
> Stream pos 22
>
>
> The buffer position issue is important for me at the moment because I'm
> implementing System.Resources.ResourceReader, which reads with
> BinaryReader but at one point must skip over a number of padding bytes
> to 8-byte align the next field.
>
>
> - Dick
>
>
--- End Message ---
--- Begin Message ---> I'm also having trouble with the buffering when it pre-reads too much > from the underlying stream. With a trivial test case (attached), I get > the following results on Windows and Linux: Well, the string/char reading is broken and must be rewritten. In Read(char[] buffer, int index, int count) EnsureBuffered(m_encoding_max_byte * count); This call advances base stream position too far for multibyte encodings. ReadChar(s) has the same side-effect. And so is PeekChar, it also doesn't reset the base stream pos. All in all, major cleanup is needed for this class. What if we ask the original author, maybe he will be interested to look into this? Serge--- End Message ---
--- Begin Message ---On Tue, 2002-08-13 at 21:59, Serge wrote: > > I'm also having trouble with the buffering when it pre-reads too much > > from the underlying stream. With a trivial test case (attached), I get > > the following results on Windows and Linux: > > Well, the string/char reading is broken and must be rewritten. > In Read(char[] buffer, int index, int count) > EnsureBuffered(m_encoding_max_byte * count); > This call advances base stream position too far for multibyte encodings. > ReadChar(s) has the same side-effect. > And so is PeekChar, it also doesn't reset the base stream pos. I'm wondering if we shouldn't just rip all of the buffering out of BinaryReader and let the underlying stream do all that. > All in all, major cleanup is needed for this class. > What if we ask the original author, maybe he will be interested to look into this? Looks like that will be the best option. I'll post to mono-list tomorrow. - Dick--- End Message ---
--- Begin Message ---
On Wed, 2002-08-14 at 13:35, Serge wrote:
> Hi Dick,
>
> > Is it OK if I post some or all of yesterday's conversation to the list?
>
> It's perfectly OK!
Thanks.
By the way, I think I know which of my changes is responsible for fixing
that unicode.c assert you saw:
--- mono/metadata/unicode.c 6 Aug 2002 09:22:07 -0000 1.17
+++ mono/metadata/unicode.c 14 Aug 2002 13:31:33 -0000
@@ -459,7 +459,7 @@
g_assert (chars);
g_assert (bytes);
g_assert (mono_array_length (bytes) > byteIndex);
- g_assert (mono_array_length (chars) >= (byteIndex + byteCount));
+ g_assert (mono_array_length (bytes) >= (byteIndex + byteCount));
g_assert (mono_array_length (chars) > charIndex);
- Dick
--- End Message ---
using System;
using System.IO;
using System.Text;
public class Test {
public static int Main () {
FileStream instr=new FileStream("binarywriter.data", FileMode.Open);
BinaryReader reader=new BinaryReader(instr, Encoding.UTF8);
string str;
int i;
str=reader.ReadString();
Console.WriteLine("Read string \"{0}\"", str);
Console.WriteLine("Stream pos {0}", reader.BaseStream.Position);
i=reader.ReadInt32();
Console.WriteLine("Read int {0}", i);
Console.WriteLine("Stream pos {0}", reader.BaseStream.Position);
str=reader.ReadString();
Console.WriteLine("Read string \"{0}\"", str);
Console.WriteLine("Stream pos {0}", reader.BaseStream.Position);
return 0;
}
}
<<attachment: readtest.exe>>
binarywriter.data
Description: Binary data
