Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
Thanks for fixing. You're awesome! ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On Tue, 6 Nov 2012, Andrew wrote: On 12-11-06 05:43 PM, Jeppe Græsdal Johansen wrote: It also generates a hint about that: widen.lpr(44,12) Hint: Converting the operands to "Int64" before doing the multiply could prevent overflow errors. Right, keep in mind this example was to simulate the read operation of the TMemoryStream when I traced into the problem. I'm on Ubuntu 12.10 x64. I just re-installed. So this is a real problem. It is a problem with the implementation of TCustomMemoryStream.Read, I fixed it. The other streams are unaffected by this error, and no change to the API is needed. Thanks for persisting and pushing me to dig deeper :-) Permission to go to sleep now ? It's 1 AM... Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On 12-11-06 05:43 PM, Jeppe Græsdal Johansen wrote: It also generates a hint about that: widen.lpr(44,12) Hint: Converting the operands to "Int64" before doing the multiply could prevent overflow errors. Right, keep in mind this example was to simulate the read operation of the TMemoryStream when I traced into the problem. I'm on Ubuntu 12.10 x64. I just re-installed. So this is a real problem. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
Den 07-11-2012 00:12, Andrew skrev: On 12-11-06 05:07 PM, Michael Van Canneyt wrote: Please re-open the issue and I will post an updated program that fails EVERY TIME. Or just use the attached Widen.lpr... :-) Try this FSize:=int64(1024*1024*1024)*Factor; 1024*1024*1024 is handled as a longint on a 32bit machine. Multiplied by a byte will still give a longint, which will overflow if multiplied by 2. It also generates a hint about that: widen.lpr(44,12) Hint: Converting the operands to "Int64" before doing the multiply could prevent overflow errors. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On 12-11-06 05:23 PM, Michael Van Canneyt wrote: Also, it contains a flaw, since the buffer is not initialized, and will contain random data. So the tests at the end will fail and cause an error. But this has nothing to do with range checking. const MyBuffer:Array[0..1] of byte=(0,0); It should be [0,0] in debugger. @ Runtime before I change it. Check it out. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On 12-11-06 05:23 PM, Michael Van Canneyt wrote: Works perfectly here. 64-bit linux, 8Gb mem. Are you trying this on a 32-bit system ? Because then it will of course fail. Also, it contains a flaw, since the buffer is not initialized, and will contain random data. So the tests at the end will fail and cause an error. But this has nothing to do with range checking. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel Fails EVERY time here. And in production too. I exclusively use memory streams. Range Checking / Overflow off line 54 returns -2147483648 when I read only 2 bytes and MyBuffer should be [0,0] since that's what I wrote to the memory stream and shows it was never read into. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On Tue, 6 Nov 2012, Andrew wrote: On 12-11-06 05:07 PM, Michael Van Canneyt wrote: Please re-open the issue and I will post an updated program that fails EVERY TIME. Or just use the attached Widen.lpr... :-) Works perfectly here. 64-bit linux, 8Gb mem. Are you trying this on a 32-bit system ? Because then it will of course fail. Also, it contains a flaw, since the buffer is not initialized, and will contain random data. So the tests at the end will fail and cause an error. But this has nothing to do with range checking. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On 12-11-06 05:07 PM, Michael Van Canneyt wrote:
Please re-open the issue and I will post an updated program that fails
EVERY TIME.
Or just use the attached Widen.lpr... :-)
program Widen;
{$mode objfpc}{$H+}
uses
{$IFDEF UNIX}{$IFDEF UseCThreads}
cthreads,
{$ENDIF}{$ENDIF}
Classes,SysUtils;
function Test_2(Count:LongInt; Factor:Byte):LongInt;
var
FSize:Int64;
FPosition:Int64;
begin
FPosition:=0;
FSize:=(1024*1024*1024)*Factor;
Result:=FSize-FPosition;
end;
function Test_1(Count:LongInt; Factor:Byte):Int64;
var
FSize:Int64;
FPosition:Int64;
begin
FPosition:=0;
FSize:=(1024*1024*1024)*Factor;
Result:=FSize-FPosition;
end;
function Test_3(Count:LongInt; Factor:Byte):LongInt;
const
MyBuffer:Array[0..1] of byte=(0,0);
var
FSize:Int64;
FPosition:Int64;
FS:TMemoryStream;
FWrote:LongInt;
FRead:LongInt;
begin
FS:=TMemoryStream.Create();
Try
FPosition:=0;
FSize:=(1024*1024*1024)*Factor;
While FPosition2) or (MyBuffer[0]<>0) or (MyBuffer[1]<>0) then
raise Exception.Create('Test_3 Failed');
Finally
FS.Free();
end;
end;
begin
Test_1(1024,1); //OK
Test_1(1024,2); //OK
Test_2(1024,1); //OK
//Test_2(1024,2); //Will always fail with range checking on
Test_3(1028,2); //will always fail with range checking on
end.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On Tue, 6 Nov 2012, Andrew wrote:
On 12-11-06 02:50 AM, michael.vancann...@wisa.be wrote:
Since you can only read 2GB (Count is a longint) in a single read
operation, it makes no sense to return a 64-bit result.
Note that the operating system only allows 2Gb reads anyway, even on
64-bit
systems.
Hi Michael,
Thanks but that's not the problem. I can't even write 2bytes to a 2GB
memory/file stream.
See: http://bugs.freepascal.org/view.php?id=23284 for attached test case
shows simple math exactly like read/write in streams
This is another glitch that I must overcome. My sync app can't handle zip
files >=~2GB
The problem is with overflow. FPC raises an exception (as it should).
I realize this is going to affect a couple base classes too.
All you demonstrate is that it is possible to create overflows.
There is nothing wrong with the base classes. They work perfectly.
I've read/written 4Gb files. Probably you do something with the
result which is overflowing your variables.
Michael.
See the below program.
It writes 4Gb to disk and writes the resulting number of bytes,
which is larger than integer, but which I store in an Int64.
When I run the program:
araminta: >./testw
Wrote 4294968320 bytes
As expected. With Range checking.
---
{$mode objfpc}{$h+}
program testw;
uses classes;
var
F : TFileStream;
// 4 Mb buffer.
Buf : Array[0..4*1024*1024] of byte;
I : Integer;
S : Int64;
begin
FillChar(Buf,Sizeof(Buf),' ');
f:=TFileStream.Create('test.dat',fmCreate);
try
For I:=1 to 1024 do
S:=S+F.Write(Buf,SizeOf(Buf));
Finally
F.Free;
end;
Writeln('Wrote ',S,' bytes');
end.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On 12-11-06 02:50 AM, michael.vancann...@wisa.be wrote: Since you can only read 2GB (Count is a longint) in a single read operation, it makes no sense to return a 64-bit result. Note that the operating system only allows 2Gb reads anyway, even on 64-bit systems. Hi Michael, Thanks but that's not the problem. I can't even write 2bytes to a 2GB memory/file stream. See: http://bugs.freepascal.org/view.php?id=23284 for attached test case shows simple math exactly like read/write in streams This is another glitch that I must overcome. My sync app can't handle zip files >=~2GB The problem is with overflow. FPC raises an exception (as it should). I realize this is going to affect a couple base classes too. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: FPC related fairy tale
Am 06.11.2012 09:13, schrieb Michael Schnell: On 11/05/2012 09:28 PM, Sven Barth wrote: the "young mage" => me :) Could you publish a state report on how powerful the actual magic has become ? If you mean how far the m68k port has come now, sure: * currently the only tested "environment" is QEMU's userspace emulation for Coldfire (v4e to be precise, though I currently try to stay with ISA_A without adding ISA_B features - I save that for later ;) ); I've not yet come around to test programs on Aranym (simulates a M68040) or real hardware (I plan to purchase a cfv4e development board though :) ) * the state of the Amiga and Atari targets is not yet known (I'd first need to build binutils for them); I stated that I won't really work on them, but I want them at least compiling and a working "Hello World" program on corresponding emulators (target "palmos" is definitely off for me though :) ) * no work was yet done for m68k-embedded (I plan it for the future once m68k-linux is working good enough) * the test suite does not yet run unattended, because there are some tests that cause a hang in the emulator (not yet investigated) - the compiler itself is another example of such an application (so no native Coldfire compiler yet) * primitve procedural programs do definitely work * StdIO (to/from console/file) is working * the Linux API can be used without problems (so far) * signal handling works (recently I also managed to get the stack trace output working :) ) * command line parameters are handled correctly, environment variables don't work yet * linking to C libraries was not tested (and from what I've seen so far in the startup assembly code it might not work) * building Pascal libraries was not tested and might not work as well * sets are currently problematic as well * 64-Bit operations do work (since the weekend ;) ) * floating point operations don't work currently (includes Currency though it should be an Int64 on m68k...) * the calculation of array element addresses in multi dimensional arrays might currently be wrong (at least I had troubles with the MonthDay array in SysUtils) Nevertheless although it's quite hard work it's also a very rewarding work as you can see that after a bit of work another test compiles or even runs correctly (though most tests currently fail at runtime as you can see below). Here are the testresults from the weekend before I fixed the 64-Bit operations (which caused at least 2 tests to fail, though it could be more as the compiler sometimes uses 64-Bit values for immediates) and the command line parameters. === testresults begin === Total = 5919 (1133:4786) Total number of compilations = 3681 (172:3509) Successfully compiled = 2603 Successfully failed = 906 Compilation failures = 170 Compilation that did not fail while they should = 2 Total number of runs = 2238 (961:1277) Successful runs = 1277 Failed runs = 961 Number units compiled = 121 Number program that should not be run = 241 Number of skipped tests = 389 Number of skipped graph tests = 10 Number of skipped interactive tests = 29 Number of skipped known bug tests = 6 Number of skipped tests for other versions = 4 Number of skipped tests for other cpus = 282 Number of skipped tests for other targets = 58 === testresults end === Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Fwd: [MacPascal] Developing for Android with Pascal and Eclipse
Some people here may also be interested in this. I've already added a link on the wiki at http://wiki.freepascal.org/FPC_JVM_Android_Development From: Phil Hess Date: 5 Nov 2012 03:08:21 GMT+01:00 Date: 5 Nov 2012 03:08:24 GMT+01:00 For a change of pace I looked into integrating FPC's JVM compiler with the Eclipse IDE. Turns out it's not too difficult - I just created a simple console app that you add as a "builder" to the build sequence of your Eclipse Android projects. Here are some notes I've written for getting everything set up: http://web.fastermac.net/~MacPgmr/pba/PbaStatus.html Everything works fine on a Mac, meaning you can target both iOS and Android with Pascal, although sharing source code between iOS and Android apps is minimal at this point. Android challenges include learning the ins and out of Android "activities" and resources, as well as familiarizing yourself with the vast Android runtime. Fortunately, FPC includes the androidr14 unit all ready to go without any additional parsing on our part. And of course you'll be shocked at how slow the Android emulator is - it makes Xcode's Simulator look like something from the future. This is really pretty nifty: developing for a Java RTL using Pascal syntax. Very much analogous to what we do with Pascal vis-a-vis Objective-C. Let me know if I've left anything out or if you spot any obvious mistakes in what I've written up. Thanks. -Phil ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
RE: [fpc-devel] mips-linux and mipsel-linux snapshots available
This is just a compressed installation: if you expand everything into directory /your/base/dir then adding /your/base/dir/bin to PATH environment variable should allow you to use this installation. However, there are indeed at least two known problems: 1) The missing symbolic link between bin/ppcCPU to ../lib/fpc/$FPCVERSION/ppcCPU where CPU is etiher mips or mpisel in our case. I don't know if this is just a limitation of tar or something that could easily be added... 2) A basic fpc.cfg file that gets read by the ppcCPU binary to find the installed ppu and object files. I have a .fpc.cfg file in my HOME directory which contains -Fu/home/pierre/pas/fpc-$fpcversion/lib/fpc/$fpcversion/units/$fpctarget/* This is enough to find all installed ppu and object files as long as you install all your different Free Pascal versions into $HOME/fpc-$fpcversion $fpcversion is simply the output of a 'ppcCPU -iV' call. If you want to know where you could put a configuration file (if you prefer to have a different for each version for example) a good way is to use 'ppcCPU -va' which will output a lot of information, but look for lines containing "Configfile" After those simple operations, you should be able to recompile the whole trunk SVN checkout! Hope this helps, Pierre Muller > -Message d'origine- > De : fpc-devel-boun...@lists.freepascal.org [mailto:fpc-devel- > boun...@lists.freepascal.org] De la part de SkyDiablo > Envoyé : lundi 5 novembre 2012 18:54 > À : fpc-devel@lists.freepascal.org > Objet : Re: [fpc-devel] mips-linux and mipsel-linux snapshots available > > oh, very nice! big thx! > > i would testing this, but please can you add a little (or more detail) > description how to use this snapshot? i see some intrestet files but i > cant to bring together... > > greez & thx, >sky... > > Am 05.11.2012 15:49, schrieb Pierre Muller: > > Due to numerous question about mips/mipsel linux, > > I decided to try to generate snapshot for those systems. > > It finally worked (with OPT=-O- option added) > > > > You can test them at: > > ftp://ftp.freepascal.org/pub/fpc/snapshot/trunk/mipsel-linux/ > > or > > ftp://ftp.freepascal.org/pub/fpc/snapshot/trunk/mips-linux/ > > > > > > Please give feedback, > > > > > > Pierre Muller > > > > ___ > > fpc-devel maillist - fpc-devel@lists.freepascal.org > > http://lists.freepascal.org/mailman/listinfo/fpc-devel > > ___ > fpc-devel maillist - fpc-devel@lists.freepascal.org > http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Memory Streams unable to handle reads past 2GB
On Mon, 5 Nov 2012, Andrew Brunner wrote: Same problem with TMemoryStream.Write :-( On 11/05/2012 10:09 PM, Andrew Brunner wrote: objpas/classes/classesh.inc TCustomMemoryStream function Read(var Buffer; Count: LongInt): LongInt; override; Having Read result declared as LongInt is problematic on 64 bit systems with large streams. Read result MUST be either PtrInt or Int64 but it cannot be LongInt. Since you can only read 2GB (Count is a longint) in a single read operation, it makes no sense to return a 64-bit result. Note that the operating system only allows 2Gb reads anyway, even on 64-bit systems. Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] mips-linux and mipsel-linux snapshots available
oh, very nice! big thx! i would testing this, but please can you add a little (or more detail) description how to use this snapshot? i see some intrestet files but i cant to bring together... greez & thx, sky... Am 05.11.2012 15:49, schrieb Pierre Muller: Due to numerous question about mips/mipsel linux, I decided to try to generate snapshot for those systems. It finally worked (with OPT=-O- option added) You can test them at: ftp://ftp.freepascal.org/pub/fpc/snapshot/trunk/mipsel-linux/ or ftp://ftp.freepascal.org/pub/fpc/snapshot/trunk/mips-linux/ Please give feedback, Pierre Muller ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] fp IDE and gdbint on Debian Wheezy/testing
On 05.11.2012 18:00, Jonas Maebe wrote: On 05 Nov 2012, at 14:40, Vincent Snijders wrote: I found out the cause of *my* problem. The ld provided by Ubuntu (version 2.22) was in /usr/bin. There was also an older version 2.21 in /usr/local/bin which was actually chosen. Fixing that solved my problem. Yes, the default system linker's built-in linker script contains the search path for all system libraries. That is the reason why we don't use the "-T" option the linker complains about, because doing so would tell the linker to ignore the built-in linker script and hence also those system-specific library search paths. We could also very easily get rid of the -T warning by getting rid of all the built-in linker scripts in the compiler, and replacing them with just the parts that are FPC-specific (basically keeping the ".fpc" section), but for reasons I still don't understand some people think this would cause problems. The current setup causes *everything* except sections (symbols, constants, etc) to be linked incorrectly, because the script is being executed twice, first time the built-in script, the second time FPC-supplied one. Fortunately, most linking is governed by built-in script, and most of FPC-supplied script is effectively ignored, that's why we get working executables in the end. I've experimented with removing all but .fpc section from linker script on x86_64-linux and i386-linux targets, and encountered no issues. In the nearest future, I'm going to commit these changes, and also write a test that will fail if linking with current setup. The only argument in favour of keeping the compiler-internal linker scripts for Linux targets that I currently can think of is that it would probably make the built-in ELF linker harder to realise (it would probably have to ask the system linker for the default linker script then). We could also let the internal linker use a compiler-internal linker script and the external linker the system default one, but that's probably not a good idea (ideally the internal and external linker would use the same settings, both from a support and from a predictability perspective). The internal ELF linker has been already implemented for x86 targets, it only misses appropriate RegisterInternalLinker calls in initialization section of t_linux.pas. It uses an internal script. Reusing the system-default script in its entirety would require closely following ld inner structure (e.g. ld composes .rela.dyn section from .rela.text,.rela.data,etc input sections, while internal linker writes it directly), and probably is not worth the trouble. At the same time, I had to implement a basic parser of ld script files anyway, so partially using the system-default script for e.g. search paths can be done fairly easily. Regards, Sergei ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: FPC related fairy tale
On 11/05/2012 09:28 PM, Sven Barth wrote: the "young mage" => me :) Could you publish a state report on how powerful the actual magic has become ? Thanks, -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
