Yes, Tony is right. You're also haven't really solved the memory issue - you still have two copies of the data.
From: [email protected] [mailto:[email protected]] On Behalf Of Tony McGee Sent: Friday, September 13, 2013 5:43 AM To: ozDotNet Subject: Re: out of memory..urgent...Solution Does that code have a nasty edge case bug? UTF8 is a variable-width encoding so if the buffer is >100kB and a valid multi-byte UTF8 encoded character happens to fall across the boundary of buffers in subsequent loop iterations you might get either an incorrect decoding or an exception thrown. On 13/09/2013 14:46, [email protected]<mailto:[email protected]> wrote: If you are interested..memeory issue was resolved by doing the following... Public Shared Function byteArrayToString(ByVal b() As Byte) As String Dim ss As New System.Text.UTF8Encoding Dim sString As String Dim sb As New StringBuilder Dim cursor As Integer Dim sChunk As String Try ' sString = System.Text.Encoding.UTF8.GetString(b) While cursor < b.Length Dim arr2() As Byte If (cursor + 100000) > (b.Length) Then arr2 = New Byte(b.Length - cursor - 1) {} Array.Copy(b, cursor, arr2, 0, b.Length - cursor) Else arr2 = New Byte(100000 - 1) {} Array.Copy(b, cursor, arr2, 0, 100000) End If sChunk = System.Text.Encoding.UTF8.GetString(arr2) sb.Append(sChunk) cursor += 100000 End While ' sString = ss.GetString(b) Return sb.ToString Catch ex As Exception Throw ex End Try End Function Anthony Melbourne StuffUps...learn from others, share with others! http://www.meetup.com/Melbourne-Ideas-Incubator-Stuffups-Failed-Startups/ ---------------------------------------------------------------------------------- NOTICE : The information contained in this electronic mail message is privileged and confidential, and is intended only for use of the addressee. If you are not the intended recipient, you are hereby notified that any disclosure, reproduction, distribution or other use of this communication is strictly prohibited. If you have received this communication in error, please notify the sender by reply transmission and delete the message without copying or disclosing it. (*13POrtC*) ----------------------------------------------------------------------------------- From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of David Kean Sent: Wednesday, 11 September 2013 2:20 AM To: ozDotNet Subject: RE: out of memory..urgent Memory isn't unlimited. Basically, when you convert from a byte array -> string, you have two copies of the same data (one for the byte array and one for the string) in memory. What exactly are you doing? You are typically better off chunking and reading smaller amounts of data at a time. Use something like a StreamWriter over a stream to automatically handles the byte -> text conversion. From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of [email protected]<mailto:[email protected]> Sent: Monday, September 9, 2013 8:05 PM To: ozDotNet Subject: out of memory..urgent Getting out of memory exception when I try to Dim s as string Dim b() as Byte s=System.Text.Encoding.GetEncoding("utf-8).GetString(b) Definitely something about the length of b..works fine most of the time except if b length is very large Anthony Melbourne StuffUps...learn from others, share with others! http://www.meetup.com/Melbourne-Ideas-Incubator-Stuffups-Failed-Startups/ ---------------------------------------------------------------------------------- NOTICE : The information contained in this electronic mail message is privileged and confidential, and is intended only for use of the addressee. If you are not the intended recipient, you are hereby notified that any disclosure, reproduction, distribution or other use of this communication is strictly prohibited. If you have received this communication in error, please notify the sender by reply transmission and delete the message without copying or disclosing it. (*13POrtC*) -----------------------------------------------------------------------------------
