You say something about the "paged pool system memory heap" limiting
available memory to something as low as 491Mb... what is the "paged pool
system memory heap" and are you sure the limit is that low?
I've done a bit of experimenting on my own, using VirtualAlloc and
VirtualLock to see how much physical memory I can get out of my test
systems. Here are the results (the test procedure is available at the
end of this message for anyone interested).
Virtual Machine running Windows XP with 2000 Mb of RAM (yes, 2000, not
2Gb). I was able to get 1441 Mb of RAM.
Next I added as much RAM as VmWare allowd: I increases the RAM for the
XP machine to 2900 Mb, my test app was able to get 1442 Mb of RAM... so
it's not the available RAM that limits the amount of RAM I can allocate
in my Delphi 7 app.
Physical machine running Windows Vista 64 with 4Gb of RAM, everything
else being "default" I was able to get 1483 Mb of RAM.
Next I've done the following experiment. I opened Delphi's Project
Options and I set the absolute minimum for all of min stack size, max
stack size and image base (so I'd allow for the most available address
space in my app). Here are the "upgraded" test results:
Win XP 2000 MB of RAM: I got 1442 Mb max buffer. A 1Mb change!
Win Vista 64 with 4Gb RAM: I got up to 1486 Mb max buffer. A 3Mb
increase, not much for all the trouble I went through.
So there you have it, one may hope to load up to 1.4 Gb of stuff into a
standard Delphi 7 application. This amount might increase by 1Gb if the
Delphi application is marked as being able to handle "negative"
addresses - still 2.4 Gb of stuff is NOT much. And in my case, I tested
with a continuous, huge buffer - that's not typical.
I though I don't need to care about the 64 bit stuff just yet, but for a
personal project (no paying customer) I need to load a 300 Mb Jpeg file
and spit out a modified 300 Mb jpeg file (I need to change the
coordinate system on a scanned map so I can load it onto my PDA's GPS
application). I'm almost certain I will not be able to make it past the
first step - loading the first jpeg and converting it to BMP so I can
work with it :-)
Here's my test procedure:
procedure TForm1.Button1Click(Sender: TObject);
var P:PIntegerArray;
MbNum:Integer;
ByteSize:Integer;
IntCount:Integer;
n:Integer;
begin
MbNum := StrToInt(Edit1.Text);
ByteSize := MbNum * 1024 * 1024;
IntCount := ByteSize div 4;
P := VirtualAlloc(nil, ByteSize, MEM_COMMIT or MEM_RESERVE,
PAGE_READWRITE);
if Assigned(P) then
try
if VirtualLock(P, ByteSize) then
ShowMessageFmt('Lock failed at at %s Mb', [Edit1.Text])
else
begin
// got memory and it's in RAM
for n:=0 to IntCount-1 do
P[n] := n;
for n:=0 to IntCount-1 do
if P[n] <> n then
raise Exception.Create('Bug');
// ok
ShowMessageFmt('OK at %s Mb', [Edit1.Text]);
end;
finally VirtualFree(P, ByteSize, MEM_RELEASE);
end
else
ShowMessageFmt('Failed at %s Mb', [Edit1.Text]);
end;
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On
> Behalf Of Irwin Scollar
> Sent: Monday, June 25, 2007 5:00 PM
> To: [email protected]
> Subject: Delphi Road Map has been published
>
> Ross Levis wrote:
>
> >I'm obviously not aware of all the benefits 64-bit can provide.
> Native
> >64-bit integers, large floating point numbers, access to lots of RAM,
> is all
> >I've thought of. Can you explain why memory mapped files are
> unnecessary in
> >a 64 bit world?
>
> Permit a rather long explanation:
>
> If one has to program for people who work on images whose sizes are
> such that they can no longer be stored as a complete array causing
> the application to run out of paged pool system memory heap (see
> Chapter 7, Memory Management, pp 401 ff. in Microsoft Windows
> Internals, Fourth ed. M. E. Russinovich, D.A. Solomon, MS Press,
> 2005), then an "out of memory" message from the OS results even
> though physical memory may far exceed the 491.875 MB available in Win
> 2000 & XP or 650 MB on WinServer 2003. This is a design limitation
> dating from the origins of Windows NT back in the early 1990's. My
> main machine (SuperMicro, 2 AMD dual core Opterons, with 4 GB
> physical and 3 GB user space using the /3GB switch in boot.ini
> suffers from this.
>
> For example, this limit is easily reached or exceeded when working on
> a full NASA Landsat 7 false color tile. If you declare a canvas for
> this in Delphi so that you can write something to it, the OS will
> crash the program with an "out of memory" error although it really
> means "out of paged pool". If instead, you use a memory mapped file
> which resides on disk and use pointers to return portions to be
> worked on, larger images can be treated although no canvas is
> permitted. Little window pieces can be "cut" out of the main image,
> stored separately, written to and then inserted back again without
> overhead.
>
> Some operations can be agonizingly slow while the system queries the
> disk for additional data that's not currently mapped. If the
> operations are sufficiently local so that a circular set of buffers
> can be used to "roll through" the image, then this speeds things up,
> but if the operations are non-local, then one has to live with the
> speed penalty.
>
> In 64 bit Windows, the maximum pool is 128 GB, so an image can be
> handled as a 2D array and stored and treated entirely in main memory
> independently of locality of an operation. Thus, to answer your
> question, memory mapped files are not necessary as long as the pool
> is still available.
>
> Hopefully, a 64 bit Delphi will have a canvas. A typical non-local
> operation is geometric transformation of a set of multiple images
> for rectification in digital photogrammetry. In the old days of Win
> 9x, one ran into this problem with fairly modest images, but the 32
> bit environment of WinNT solved that for a while. The pool, by the
> way, is shared by all windows, so even less is available than might
> first seem evident.
>
> As the saying goes, "appetite increases with eating". Modern digital
> aerial cameras have largely replaced the older 228 mm side length
> color film scanned at 10-25 microns per pixel with multiple full
> color sensors often with 20K + side lengths and image strips running
> into the sub-terrabyte range. Similar data quantities are
> encountered in satellite images from QuickBird (Digital Globe) and
> others like Google Earth's Keyhole data if paid for. Even a modern
> Hasselblad camera with a 39 Megapixel digital back, if used in high
> dynamic range mode, can also break the OS limits quite easily.
>
> My attempt to use the AWE (Address Windowing Extentions, ref. above,
> p. 399) which theoretically might allow for larger windowed space
> have not succeeded in Delphi 7, although the memory mapped files used
> in HiComponent's TIEBitmap are ok but slow. I have tried both the
> internal Delphi memory manager and FastMM4. Perhaps someone on this
> list can tell me why.
>
> Irwin Scollar
>
> _______________________________________________
> Delphi mailing list -> [email protected]
> http://www.elists.org/mailman/listinfo/delphi
_______________________________________________
Delphi mailing list -> [email protected]
http://www.elists.org/mailman/listinfo/delphi
