Does that code have a nasty edge case bug?
UTF8 is a variable-width encoding so if the buffer is >100kB and a valid
multi-byte UTF8 encoded character happens to fall across the boundary of
buffers in subsequent loop iterations you might get either an incorrect
decoding or an exception thrown.
On 13/09/2013 14:46, [email protected] wrote:
If you are interested..memeory issue was resolved by doing the
following...
Public Shared Function byteArrayToString(ByVal b() As Byte) As String
Dim ss As New System.Text.UTF8Encoding
Dim sString As String
Dim sb As New StringBuilder
Dim cursor As Integer
Dim sChunk As String
Try
' sString = System.Text.Encoding.UTF8.GetString(b)
While cursor < b.Length
Dim arr2() As Byte
If (cursor + 100000) > (b.Length) Then
arr2 = New Byte(b.Length - cursor - 1) {}
Array.Copy(b, cursor, arr2, 0, b.Length - cursor)
Else
arr2 = New Byte(100000 - 1) {}
Array.Copy(b, cursor, arr2, 0, 100000)
End If
sChunk = System.Text.Encoding.UTF8.GetString(arr2)
sb.Append(sChunk)
cursor += 100000
End While
' sString = ss.GetString(b)
Return sb.ToString
Catch ex As Exception
Throw ex
End Try
End Function
Anthony
Melbourne StuffUps...learn from others, share with others!
http://www.meetup.com/Melbourne-Ideas-Incubator-Stuffups-Failed-Startups/
----------------------------------------------------------------------------------
NOTICE : The information contained in this electronic mail message is
privileged and confidential, and is intended only for use of the
addressee. If you are not the intended recipient, you are hereby
notified that any disclosure, reproduction, distribution or other use
of this communication is strictly prohibited.
If you have received this communication in error, please notify the
sender by reply transmission and delete the message without copying or
disclosing it. (*13POrtC*)
-----------------------------------------------------------------------------------
*From:*[email protected]
[mailto:[email protected]] *On Behalf Of *David Kean
*Sent:* Wednesday, 11 September 2013 2:20 AM
*To:* ozDotNet
*Subject:* RE: out of memory..urgent
Memory isn't unlimited. Basically, when you convert from a byte array
-> string, you have two copies of the same data (one for the byte
array and one for the string) in memory.
What exactly are you doing? You are typically better off chunking and
reading smaller amounts of data at a time. Use something like a
StreamWriter over a stream to automatically handles the byte -> text
conversion.
*From:*[email protected]
[mailto:[email protected]] *On Behalf Of
*[email protected]
*Sent:* Monday, September 9, 2013 8:05 PM
*To:* ozDotNet
*Subject:* out of memory..urgent
Getting out of memory exception when I try to
Dim s as string
Dim b() as Byte
s=System.Text.Encoding.GetEncoding("utf-8).GetString(b)
Definitely something about the length of b..works fine most of the
time except if b length is very large
Anthony
Melbourne StuffUps...learn from others, share with others!
http://www.meetup.com/Melbourne-Ideas-Incubator-Stuffups-Failed-Startups/
----------------------------------------------------------------------------------
NOTICE : The information contained in this electronic mail message is
privileged and confidential, and is intended only for use of the
addressee. If you are not the intended recipient, you are hereby
notified that any disclosure, reproduction, distribution or other use
of this communication is strictly prohibited.
If you have received this communication in error, please notify the
sender by reply transmission and delete the message without copying or
disclosing it. (*13POrtC*)
-----------------------------------------------------------------------------------
