Hi All, Thanks a bunch for all the help. I didn't know about any existing problems with PeekChar(). Thanks for the heads up.
I wrote my own CharReader based upon the Stream object. OMG, talk about the perf increase. I was testing against a 28meg file. Processing time when from 1,000,000+ I/O Reads and 15seconds, down to <4seconds and <500 I/O Reads. Also, you gotta love this one. In case anyone hasn't seen this before (and runs into a problem with ReadChars() dropping data). BinaryReader.ReadChars() Loses Data When It Reaches End-of-File http://support.microsoft.com/default.aspx?scid=kb;en-us;318121 If the BinaryReader.ReadChars method reaches the end of the stream, BinaryReader.ReadChars should return all of the characters that it reads. However, if BinaryReader.ReadChars requests more characters than are available, BinaryReader.ReadChars returns only half of the characters when it reaches the end of the stream. Thanks a bunch for all the help. Cheers! Dave ----- Original Message ----- From: "Sebastian Good" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Thursday, April 14, 2005 11:29 AM Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? I have not observed this behavior, but I have also never used PeekChar -- that's where I'd start looking. Ever since ANSI C made such vague promises about PeekChar I've avoided it like the plague! -----Original Message----- From: Unmoderated discussion of advanced .NET topics. [mailto:[EMAIL PROTECTED] On Behalf Of dave wanta Sent: Thursday, April 14, 2005 11:16 AM To: [email protected] Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? Thanks a bunch for the tip. I hadn't used the BufferedStream before this thread. Ok, so I ran this code against a 29meg file: static void BuffStream(){ int buffersize = 8192; string path = "c:\\temp\\largefile.tmp"; FileStream fs = File.OpenRead( path ); BufferedStream bs = new BufferedStream( fs, buffersize ); BinaryReader br = new BinaryReader( bs ); while( br.PeekChar() != -1 ) { br.ReadChar(); } } against a 30meg file. I pulled up TaskMgr while it's executing, and the number of I/O reads are in the millions (looks like it's going back to the file each time). Plus it is taking minutes to execute. However, when I run this code static void ReadChunk() { int buffersize = 8192; string path = "c:\\temp\\largefile.tmp"; byte[] buffer = new Byte[ buffersize ]; FileStream fs = File.OpenRead( path ); BinaryReader br = new BinaryReader( fs ); int count = br.Read( buffer, 0, buffer.Length ); while( count > 0 ) { count = br.Read( buffer, 0, buffer.Length ); } } TaskMgr only records 3,100 I/O Reads and it executes in < 1sec. Am I using the BufferedStream class/code incorrectly? Granted, in the first method, I'm reading Chars, and the 2nd method byte arrays, but I'm concerned about the I/O reads, and the time to read the file. Am I missing something obvious? Thanks, Dave ----- Original Message ----- From: "Sebastian Good" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Thursday, April 14, 2005 10:59 AM Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? If all you need is buffered input atop your input stream, you might consider using the BufferedStream class. It takes care of the buffering for you. We use this to considerable advantage when we have stream consumers like yours who are interested in one byte (or less!) of the stream at a time, but can't be running to a stream (e.g. from a database BLOB) every time they want data. -----Original Message----- From: Unmoderated discussion of advanced .NET topics. [mailto:[EMAIL PROTECTED] On Behalf Of dave wanta Sent: Thursday, April 14, 2005 10:45 AM To: [email protected] Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? but the lines aren't meaningless. There are other rules of the file format that rely on lines. i.e. Mime/Email. Emails can change encodings for different parts, yet the headers/boundaries are based upon lines. Like I said, I have all this working, in a proof of concept, i'm really just looking to increase perf. So i'm planning on implementing a char[] buffer under the covers. Just looking for some tips/pointers before I go and re-invent the wheel. Thanks a bunch for all the feedback. Cheers! Dave ----- Original Message ----- From: "Bob Provencher" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Thursday, April 14, 2005 10:29 AM Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? > Ok, got it now. You want to read a file you know is some encoding of text, > without changing encodings -- possibly because you don't necessarily > know what encoding you have? > > Makes sense, but unless you always treat them as byte arrays, sooner > or later you will need to change encodings or know the encoding you > want before > going to a string. > > If you are always treating them as byte arrays, I'd suggest that > "lines" are > meaningless. > > -----Original Message----- > From: Unmoderated discussion of advanced .NET topics. > [mailto:[EMAIL PROTECTED] On Behalf Of dave wanta > Sent: Thursday, April 14, 2005 11:14 AM > To: [email protected] > Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? > > parsing text files that can change encodings. > > > > ----- Original Message ----- > From: "Bob Provencher" <[EMAIL PROTECTED]> > To: <[email protected]> > Sent: Thursday, April 14, 2005 10:08 AM > Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? > > > > But, the point is, there may not be any "lines" to read. You may > > get > lucky > > and find a string that looks like an EOL or you may not and blow out > > your buffers trying to read until you find one. > > > > Why not just read fixed length buffers instead of lines? Can you > > tell us what you are trying to do? > > > > -----Original Message----- > > From: Unmoderated discussion of advanced .NET topics. > > [mailto:[EMAIL PROTECTED] On Behalf Of dave wanta > > Sent: Thursday, April 14, 2005 10:46 AM > > To: [email protected] > > Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? > > > > 0x13 0x13, or 0x13, or 0x10. (Same as ReadLine() in StreamReader). > > > > Now, I realize that people say "BinaryReader is used for reading > > binary files". However, There is a ReadChar() and ReadChars() > > method. So, you > can > > build a ReadLine() method. I'm planning on doing just this, but will > > need > to > > maintain my own internal char buffer. I could call ReadChar() and > > just > keep > > checking for 0x13 and 0x10 (which I did as a quick proof of > > concept). However, when doing this against a file, you end up > > accessing the file for each char. > > > > I'm sure I'm not the only one who has needed to do this. Just > > looking for any tips/pointers, before I re-invent the wheel. > > > > Thanks, > > Dave > > > > > > ----- Original Message ----- > > From: "Philip Nelson" <[EMAIL PROTECTED]> > > To: <[email protected]> > > Sent: Thursday, April 14, 2005 9:35 AM > > Subject: Re: [ADVANCED-DOTNET] BinaryReader ReadLine() ?!? > > > > > > > What exactly constitutes a line in a binary file? > > > > > > --- dave wanta <[EMAIL PROTECTED]> wrote: > > > > Hi All, > > > > Has anyone ever implemented a ReadLine() method for a > > > > BinaryReader? I'm sure I'm not the only one implementing this, > > > > yet googling has turned up > > 0. > > > > > > > > > Philip - http://blogs.xcskiwinn.org/panmanphil > > > "There's a difference between righteous anger and just being > > > crabby" > > > - > > Barbara > > > > > > =================================== > > > This list is hosted by DevelopMentorR http://www.develop.com > > > > > > View archives and manage your subscription(s) at > > http://discuss.develop.com > > > > > > > =================================== > > This list is hosted by DevelopMentorR http://www.develop.com > > > > View archives and manage your subscription(s) at > http://discuss.develop.com > > > > =================================== > > This list is hosted by DevelopMentorR http://www.develop.com > > > > View archives and manage your subscription(s) at > http://discuss.develop.com > > > > =================================== > This list is hosted by DevelopMentorR http://www.develop.com > > View archives and manage your subscription(s) at http://discuss.develop.com > > =================================== > This list is hosted by DevelopMentorR http://www.develop.com > > View archives and manage your subscription(s) at http://discuss.develop.com > =================================== This list is hosted by DevelopMentorR http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com =================================== This list is hosted by DevelopMentorR http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com =================================== This list is hosted by DevelopMentorR http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com =================================== This list is hosted by DevelopMentor� http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com =================================== This list is hosted by DevelopMentor� http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
