Re: [fpc-devel] Overflow in TMemoryStream?
> I have committed a patch. Please test and report if it is fixed. > I don't have a 32-bit system available to test on... Tested on win32: the overflow is fixed, 500M gets incremented by 125M. I think the RunError is caused by the way ReallocMem works: growing from 869M to 1086M seems to be done by allocating the new area and copying, so there's a period of almost all of the 2G heap used... ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Overflow in TMemoryStream?
On Mon, 12 Sep 2016, Michael Van Canneyt wrote: So it looks like a 32 vs. 64 bit issue. from the method Realloc : NewCapacity := (5*FCapacity) div 4; // 5*FCapacity can cause overflow Changing this to NewCapacity:=FCapacity + (FCapacity div 4) Will probably fix the issue. I have committed a patch. Please test and report if it is fixed. I don't have a 32-bit system available to test on... Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Overflow in TMemoryStream?
On Sun, 11 Sep 2016, Martok wrote: Hi, yes, I can confirm this as an overflow, but on its own, it should be safe. Above 430MB, the stream doesn't grow by a quarter but just by however much was requested, luckily the branch fails before the wrong capacity could be set. Test: type TMS2 = class(TMemoryStream) end; var ms: TMS2; ds: Int64; begin ds:= 100*1000*1000; ms:= TMS2.Create; ms.SetSize(ds); WriteLn(ds:15,' ', ms.Size:15, ' ', ms.Capacity:15); inc(ds, ds div 10); // grow by less than 25% ms.SetSize(ds); WriteLn(ds:15,' ', ms.Size:15, ' ', ms.Capacity:15); end. with ds=100M, prints: 1 1 13840 11000 11000 125005824<< grew by 1/4*100M with ds=500M, prints: 5 5 52816 55000 55000 550002688<< bug, grew by 1/10*500M Output on linux 64-bit cadwal: >tms 1 1 13840 11000 11000 125005824 cadwal: >tms 5 5 52816 55000 55000 625004544 This is 1/4 cadwal: >tms 86900 86900 869003264 95590 95590 1086255104 So it looks like a 32 vs. 64 bit issue. from the method Realloc : NewCapacity := (5*FCapacity) div 4; // 5*FCapacity can cause overflow Changing this to NewCapacity:=FCapacity + (FCapacity div 4) Will probably fix the issue. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Overflow in TMemoryStream?
Hi, yes, I can confirm this as an overflow, but on its own, it should be safe. Above 430MB, the stream doesn't grow by a quarter but just by however much was requested, luckily the branch fails before the wrong capacity could be set. Test: type TMS2 = class(TMemoryStream) end; var ms: TMS2; ds: Int64; begin ds:= 100*1000*1000; ms:= TMS2.Create; ms.SetSize(ds); WriteLn(ds:15,' ', ms.Size:15, ' ', ms.Capacity:15); inc(ds, ds div 10); // grow by less than 25% ms.SetSize(ds); WriteLn(ds:15,' ', ms.Size:15, ' ', ms.Capacity:15); end. with ds=100M, prints: 1 1 13840 11000 11000 125005824<< grew by 1/4*100M with ds=500M, prints: 5 5 52816 55000 55000 550002688<< bug, grew by 1/10*500M However, with ds=869M, prints: 86900 86900 869003264 95590 18666185569013440 955904000 and mostly crashes with Runtime Error 203 except when I'm step-by-step-debugging it... That looks like a *separate* overflow to me, probably caused by the wild mix of Int64 and Longint that our Streams inherited from Delphi... I don't have RTL built with full symbols right now, maybe someone else can investigate? Martok ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[fpc-devel] Overflow in TMemoryStream?
Hi, While working on the MSEgui fork of classes unit I saw a suspect piece of code in streams.inc: " function TMemoryStream.Realloc(var NewCapacity: PtrInt): Pointer; begin If NewCapacity<0 Then NewCapacity:=0 else begin // if growing, grow at least a quarter if (NewCapacity>FCapacity) and (NewCapacity < (5*FCapacity) div 4) then NewCapacity := (5*FCapacity) div 4; " Isn't there an overflow if the capacity grows above high(ptrint) div 5 (about 430MB on 32 bit)? IIRC there was a discussion on the list about memory problems with big TMemoryStream's. Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
