As a separate data point: - I've seen corruption twice out of several runs (the process hits "self halt") in a 32 bit VM. -- In both cases the first four bytes of the array were 0. - I haven't yet seen a crash or corruption in the 64 bit VM (it's in the process of looping 1000 times).
OS: Ubuntu 16.04. Images: - Pharo-7.0+alpha.build.425.sha.eb0a6fb140ac4a42b1f158ed37717e0650f778b4 (32 Bit) - Pharo-7.0+alpha.build.436.sha.7e0f6d30dca546f3859b32764483b42f4fdcd63c (64 Bit) VM: 5.0-201801110739 Thursday 11 January 09:22:24 CET 2018 gcc 4.8.5 [Production Spur 64-bit VM] CoInterpreter VMMaker.oscog-eem.2302 uuid: 55ec8f63-cdbe-4e79-8f22-48fdea585b88 Jan 11 2018 StackToRegisterMappingCogit VMMaker.oscog-eem.2302 uuid: 55ec8f63-cdbe-4e79-8f22-48fdea585b88 Jan 11 2018 VM: 201801110739 alistair@alistair-xps13:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Date: Wed Jan 10 23:39:30 2018 -0800 $ Plugins: 201801110739 alistair@alistair-xps13:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Linux b07d7880072c 4.13.0-26-generic #29~16.04.2-Ubuntu SMP Tue Jan 9 22:00:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux plugin path: /snap/pharo7/x1/usr/bin/pharo-vm/5.0-201801110739 [default: /snap/pharo7/x1/usr/bin/pharo-vm/5.0-201801110739/] The 32 bit VM is the same version as above. Cheers, Alistair On 18 January 2018 at 16:42, Cyrille Delaunay <[email protected]> wrote: > Hi everyone, > > I just added a new bug entry for an issue we are experimenting since some > times: > > https://pharo.fogbugz.com/f/cases/20982/Random-corrupted-data-when-copying-from-very-large-byte-array > > Here is the description: > > > History: > > This issue has been spotted after experimenting strange behavior with > seaside upload. > After uploading a big file from a web browser, the modeled file generated > within pharo image begins with 4 unexpected bytes. > This issue occurs randomly: sometimes the first 4 bytes are right. Sometimes > the first 4 bytes are wrong. > This issue only occurs with Pharo 6. > This issue occurs for all platforms (Mac, Ubuntu, Windows) > > Steps to reproduce: > > I have been able to set up a small scenario that highlight the issue. > > Download Pharo 6.1 on my Mac (Sierra 10.12.6): > https://pharo.org/web/download > Then, iterate over this process till spotting the issue: > > => start the pharo image > => execute this piece of code in a playground > > 1: > 2: > 3: > 4: > 5: > 6: > > ZnServer startDefaultOn: 1701. > ZnServer default maximumEntitySize: 80* 1024 * 1024. > '/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out | > out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0]. > 18202065 timesRepeat: [ out nextPut: 0 ] > ]. > > => Open a web browser page on: http://localhost:1701/form-test-3 > => Upload the file zip file previously generated ('myzip.zip'). > => If the web page displays: "contents=000000000a00..." (or anything > unexpected), THIS IS THE ISSUE ! > => If the web page displays: "contents=504b03040a00..", the upload worked > fine. You can close the image (without saving) > > > > Debugging: > > > > Bob Arning has been able to reproduce the issue with my scenario. > He dived into the code involved during this process, till reaching some > "basic" methods where he saw the issue occuring. > > Here are the conclusion till there: > => A corruption occurs while reading an input stream with method ZnUtils > class>>readUpToEnd:limit: > The first 4 bytes may be altered randomely. > => The first 4 bytes are initially correctly written to an outputStream. > But, the first 4 bytes of this outputStream gets altered (corrupted), > sometimes when the inner byte array grows OR when performing the final > "outputStream contents" > => Here is a piece of code that reproduce the issue (still randomely. > stopping an restarting the image may change the behavior) > > 1: > 2: > 3: > 4: > 5: > 6: > 7: > 8: > 9: > 10: > 11: > 12: > 13: > 14: > 15: > 16: > 17: > 18: > 19: > 20: > > test4"self test4" | species bufferSize buffer totalRead outputStream > answer inputStream ba byte1 | ba := ByteArray new: 18202085. > ba atAllPut: 99. 1 to: 20 do: [ :i | ba at: i put: (#[80 75 3 4 10 7 > 7 7 7 7 125 83 67 73 7 7 7 7 7 7] at: i) ]. inputStream := ba readStream. > bufferSize := 16384. species := ByteArray. > buffer := species new: bufferSize. > totalRead := 0. > outputStream := nil. > [ inputStream atEnd ] whileFalse: [ | readCount | > readCount := inputStream readInto: buffer startingAt: 1 count: > bufferSize. > totalRead = 0 ifTrue: [ > byte1 := buffer first. > ]. > totalRead := totalRead + readCount. > > outputStream ifNil: [ > inputStream atEnd > ifTrue: [ ^ buffer copyFrom: 1 to: readCount ] > ifFalse: [ outputStream := (species new: bufferSize) > writeStream ] ]. > outputStream next: readCount putAll: buffer startingAt: 1. > byte1 = outputStream contents first ifFalse: [ self halt ]. > ]. > answer := outputStream ifNil: [ species new ] ifNotNil: [ outputStream > contents ]. > byte1 = answer first ifFalse: [ self halt ]. ^answer > > > > suspicions > > This issue appeared with Pharo 6. > > Some people suggested that it could be a vm issue, and to try my little > scenario with the last available vm. > > I am not sure where to find the last available vm. > > I did the test using these elements: > > https://files.pharo.org/image/60/latest.zip > > https://files.pharo.org/get-files/70/pharo-mac-latest.zip/ > > > > The issue is still present > > > > > -- > Cyrille Delaunay
