Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Konstantin Münning wrote: I would hapily contribute the code for a better MemAvail/MaxAvail function if that's the problem. I would like to be able to use the latest compiler which now I definitely can't as I must have the BP7 compatibility which is very good in 1.0.10. IMO we have two issues here: - compiler compatibility - platform portability The BP7 handling is very platform specific, and it doesn't make much sense to implement the legacy DOS/Go32 API for other (multitasking...) platforms. It would just *prevent* easy porting of such code, when the functions returned some inadequate value on other platforms, at runtime, making the program either fail immediately or crash later. This means that such code *must* be changed (conditionalized...) before it can be compiled for other platforms. Such a change can be enforced when the legacy API only is implemented for DOS compatible target platforms, inavailable on other platforms. But this doesn't mean that the API is completely removed, it should exist and behave as before, on the platforms to which it applies. With regards to compiler compatibility I see no need for special handling of this API, it is standard Pascal with no special syntactical/semantical aberrations? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
This means that such code *must* be changed (conditionalized...) before it can be compiled for other platforms. Such a change can be enforced when the legacy API only is implemented for DOS compatible target platforms, inavailable on other platforms. But this doesn't mean that the API is completely removed, it should exist and behave as before, on the platforms to which it applies. The idea that for dos it is the same is not true, since this is extender dependant. Some extenders might return virtual size allocated by the host OS (DPMI host, or include their own swap. So this is not between dos and the rest, but between real and 32-bit protected mode (and I leave in the middle what 16-bits protected mode does, since that is irrelevant for FPC purposes) And FPC dos programs _are_ protected mode, not real. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Hans-Peter Diettrich wrote: the API is completely removed, it should exist and behave as before, on the platforms to which it applies. It doesn't apply unconditionally to any current fpc target. Even go32v2 might run on windows with changing memory availabilities. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Konstantin Münning wrote:
And in all cases, the program will still crash occasionally because it
runs out of memory in certain cases, because the check and the
allocation do not happen atomically. You can't put something into the
standard Run Time Library which works most of the time, except under
heavy load. Imagine reading the docs and encountering that comment,
you'd immediately start wondering how many more procedures there are
which work most of the time.
Most programs work most of the time except under heavy load. That's not
a good argument. Most small (C) programs don't do any memory
Pascal is not C ;)
availability checks at all. Look at most of the command line tools on
any OS ;-). Of course I would like to have any code behave perfect under
any condition but it has always been a tradeoff between effort
(=expense) and impact. But when there is no memory left and you need
some you can't behave perfect. Please let it be the programmers choice
what to do and how to handle this. A compiler should be providing the
means, not taking them away. Anybody writing a program knows that there
may be flaws. A warning (with an advice how to make it better) should
still be the best for this.
By the way, I've found the following code in compiler/cclasses.pas when
I searched for the whereabouts of maxavail/memavail and there were some
more like this in the sources:
{$ifdef HASGETHEAPSTATUS}
status:=GetFPCHeapStatus;
startmem:=status.CurrHeapUsed;
{$else HASGETHEAPSTATUS}
startmem:=memavail;
{$endif HASGETHEAPSTATUS}
I'm not sure if I understand it right as GetHeapStatus is not documented
well (I've found no reference for the meaning of the fields of the
returned record) but it seems that someone repaced here memavail with
another code which does about the same and has therefore the same flaw.
It has been moved to heap status which contains more info which also reliable on
multitasking systems.
I would hapily contribute the code for a better MemAvail/MaxAvail
function if that's the problem. I would like to be able to use the
latest compiler which now I definitely can't
See below.
as I must have the BP7
compatibility which is very good in 1.0.10.
What about Mark and Release when talking about heap management ;)?
A compromise would be a tpascal unit which is loaded only in -So mode.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
On Mon, 23 May 2005 02:29:48 +0200 Konstantin Münning [EMAIL PROTECTED] wrote: But when there is no memory left and you need some you can't behave perfect. Please let it be the programmers choice what to do and how to handle this. It is. Exceptions, ReturnNil, ... A compiler should be providing the means, not taking them away. Environment changes sometimes you know! Thus, so does the RTL. There are new means now. Anybody writing a program knows that there may be flaws. A warning (with an advice how to make it better) should still be the best for this. Warning: compilation failure; make it better: here on mailing list. I would hapily contribute the code for a better MemAvail/MaxAvail function if that's the problem. I would like to be able to use the latest compiler which now I definitely can't as I must have the BP7 compatibility which is very good in 1.0.10. Well, define these functions in your own compatibility unit then ? Micha ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Florian Klaempfl wrote:
Konstantin Münning wrote:
By the way, I've found the following code in compiler/cclasses.pas when
I searched for the whereabouts of maxavail/memavail and there were some
more like this in the sources:
{$ifdef HASGETHEAPSTATUS}
status:=GetFPCHeapStatus;
startmem:=status.CurrHeapUsed;
{$else HASGETHEAPSTATUS}
startmem:=memavail;
{$endif HASGETHEAPSTATUS}
I'm not sure if I understand it right as GetHeapStatus is not documented
well (I've found no reference for the meaning of the fields of the
returned record) but it seems that someone repaced here memavail with
another code which does about the same and has therefore the same flaw.
It has been moved to heap status which contains more info which also reliable on
multitasking systems.
Yes, I've already found out that there is the GetHeapStatus function but
the use here is also not atomic so there is no difference regarding
the discussed bug. If it is reliable then here's a good information
source for the new MaxAvail :-).
By the way, what's the information provided with GetHeapStatus? I've
found no description in the docs except the record member names. Or
where to look for?
I would hapily contribute the code for a better MemAvail/MaxAvail
function if that's the problem. I would like to be able to use the
latest compiler which now I definitely can't
See below.
as I must have the BP7
compatibility which is very good in 1.0.10.
What about Mark and Release when talking about heap management ;)?
I never had a use for that and as nobody reported them missing...
A compromise would be a tpascal unit which is loaded only in -So mode.
That's I.M.H.O. a good solution. For the affected units I use -So
anyway. Can I do this myself by some compiler option or must it be
included in the compiler source? If you tell me how this unit should be
called I would happily provide it.
Konstantin
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Hi Micha! Thanks for the amusing comments :-). Let me add some of mine. Micha Nelissen wrote: On Mon, 23 May 2005 02:29:48 +0200 Konstantin Münning [EMAIL PROTECTED] wrote: But when there is no memory left and you need some you can't behave perfect. Please let it be the programmers choice what to do and how to handle this. It is. Exceptions, ReturnNil, ... How much can you limit the choice and still pretend that you can choose? A compiler should be providing the means, not taking them away. Environment changes sometimes you know! Thus, so does the RTL. There are new means now. New means are good. I'm happy with this. But the removal of some basics which is not really enforced by the environment is what makes me unhappy. Anybody writing a program knows that there may be flaws. A warning (with an advice how to make it better) should still be the best for this. Warning: compilation failure; make it better: here on mailing list. Hint: a failure is not a warning. I would hapily contribute the code for a better MemAvail/MaxAvail function if that's the problem. I would like to be able to use the latest compiler which now I definitely can't as I must have the BP7 compatibility which is very good in 1.0.10. Well, define these functions in your own compatibility unit then ? The -So autoload of such a unit as discussed with Florian would do the job so you don't have to change the good'ol code ;-). Konstantin P.S.: Please remember - the PC you are using is still around not because it's good but because it's compatible - you can still run the first programs although most of the old games are unplayable on todays hardware ;-). Quite everybody seems to be happy with this bug. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Jonas Maebe wrote:
On 23 mei 2005, at 02:29, Konstantin Münning wrote:
What would you suggest to return under an OS like Windows, Mac OS X or
Linux? The current free memory of the OS? Free memory + buffer cache
- minimal buffer cache size enforced by the OS? The previous +
available swap space - wired (i.e., not swappable to disk) memory?
Just a constant? Something different possibly?
I have written in my previous mail(s) that I think the best return value
would be the biggest value with which a getmem would be successful if
memory conditions haven't changed inbetween.
The only way to know that is to perform getmem's with every increasing
values until one fails.
I was not implying that a getmem(maxavail) must work under any
circumstances on a multitasking OS. But as the OS knows at a given point
what the maximum availible memory for a single allocation call is, so
could a program running on that OS.
Most programs work most of the time except under heavy load. That's not
a good argument.
I beg you pardon? I think the argument that most programs do not work
well under heavy load is an extremely bad argument to encourage even
more buggy programming.
Please don't understand me wrong - I don't encourage the use of these
functions. I'm only speaking about compatibility to old code which was
there and is now missing.
But back to the argument - as on a multitasking system ANY memory
allocation could fail at ANY time, if this was so severe, none of these
OSes should be accepted by the community as they are buggy. But as you
see everybody seems to live happily with this as the chances these
failures to happen are acceptable low. So allowing old programs
initially designed for other environments to live with the same
limits, that's OK I think.
And as to buggy programming - what can you really do when a getmem
fails? Even doing fancy tricks with exceptions and such will not help.
If you haven't added some own memory management (like garbage
collection) to free some unneeded memory in such a case there are only
two things you can do - wait until there is the requested memory
avalable and allowing by this your program to hang indefinitely or fail
in some more or less informative way. Show me one program which can run
without the required memory and does not fail in any way. Are you still
considering it buggy programming?
Please let it be the programmers choice
what to do and how to handle this.
Only using tools that make sense, everything else would be snake oil
(and in reality nothing would be handled).
A compiler should be providing the
means, not taking them away.
It should not provide known defunct means.
It's not the means but the way you are using them. Would you then remowe
the Reset/Rewrite functions as well? In many (old) programs they are
used to check for presence of file(s) to prevent overwriting by first
trying to open them and if this fails then the file can be written...
But as in a multitasking environment the file can appear just between
the Reset and the Rewrite - well, bad bug. There are the means to do
this right by the proper OS functions so maybe this should be enforced
as well!?
(I hope I haven't brought you to another imrpovement idea ;-))
Anybody writing a program knows that there
may be flaws.
No, if you use RTL routines, you expect them to work.
(see above)
A warning (with an advice how to make it better) should
still be the best for this.
People don't read the manual.
Well, then they maybe will not know about the existence of the function
:-). Still a compiler warning for deprecated functions would be nice.
{$ifdef HASGETHEAPSTATUS}
status:=GetFPCHeapStatus;
startmem:=status.CurrHeapUsed;
{$else HASGETHEAPSTATUS}
startmem:=memavail;
{$endif HASGETHEAPSTATUS}
I'm not sure if I understand it right as GetHeapStatus is not documented
well (I've found no reference for the meaning of the fields of the
returned record) but it seems that someone repaced here memavail with
another code which does about the same and has therefore the same flaw.
The above code is not used to check whether there's still enough free
memory left, it's only used to calculate how much memory is used by the
compiler.
I just picked one.
I would hapily contribute the code for a better MemAvail/MaxAvail
function if that's the problem. I would like to be able to use the
latest compiler which now I definitely can't as I must have the BP7
compatibility which is very good in 1.0.10.
You can make a unit which contains a dummy memavail and maxavail which
always return a value of 1GB or so, then your program will also always
work except under heavy load (if that is really what you want). There's
no use in investing lots of time in making very complex routines to
query all sorts of information from the OS and then have it fail in
exactly the same circumstances as when you return a plain constant.
I
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Environment changes sometimes you know! Thus, so does the RTL. There are new means now. New means are good. I'm happy with this. But the removal of some basics which is not really enforced by the environment is what makes me unhappy. If it can't be implemented sanely, it is not a basic. Any of these functions (including *status *avail) basically have only a few uses: 1) test and take decisions on memory use/availability 2) display user visible memory information 3) detect memory leaks. 4) internal use in e.g. unit tests to get a feel for the magnitude (1) is seriously flawed, and should be avoided. Period. Not only the multitasking problem, but also trashing because the program tries to use more than the available physical memory. Either due to other programs or because the total virtual size is reported by the OS. (2) is dangerous since it confuses the user if available memory suddenly increases while the program only requested another block from the system. (3) is a very primitive and somewhat incorrect (what if an automated type happens to allocate a block of the same size somewhere as you deallocate between memavails, it can really confuse. Better use heaptrc) (4) Is the only somewhat valid reason, but requires expert devel interpretation to correct for problems mentioned in (1) and (2). The fact is that the mem* routines are abundant in use for all these 4 types, and we got really a lot of questions and bugreports for these fatally flawed uses. The Delphi eq Getheapstatus is not abused like that (since Delphi is windows oriented and protected mode), so it was left in. I would hapily contribute the code for a better MemAvail/MaxAvail function if that's the problem. This was all considered, and the end conclusion was that a satisfying solution that lifts the problems 1..4 above is impossible. This, and to avoid bugs creeping in while porting is one of the main reasons. If this really bothers you a lot, add a memavail to your compability unit and reroute it to getheapstatus or getfpcheapstatus (which are unfortunately either wrongly, or not documented in 2.0.0) Well, define these functions in your own compatibility unit then ? The -So autoload of such a unit as discussed with Florian would do the job so you don't have to change the good'ol code ;-). IMHO these functions should not be brought back to the main distribution. There were good reasons to remove them, and these still hold. In theory they could be kept for go32v2 only use, but I don't like that either. The few people that have TP compatible code and don't want to fix them despite the warnings, should implement their own versions. P.S.: Please remember - the PC you are using is still around not because it's good but because it's compatible PCS are not that compatible. I can not run my old games, I can not connect my MFM disc, and a lot of old stuff doesn't work on new dos versions. Only the usable subset has survived. FPC actually retains more than the usable subset. (e.g. it retains dos functions that are superseeded by better delphi functions). However the limit is at functions whose use is dangerous, has the possibility of introducing bugs, and you can still run the first programs although most of the old games are unplayable on todays hardware ;-). Quite everybody seems to be happy with this bug. There is a lot that doesn't run. Actually, MSDOS 5.x fails to install on most of todays computers. Compability is a good thing, but the 0.1% of it that is relatively unused and even dangerous should be removed. I can't lowlevel my IDE disc the same way from the BIOS as I did my MFM discs. And that is a good thing, since that would ruin it, or it would fail for no visible reason. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
On 23 mei 2005, at 11:09, Konstantin Münning wrote: The only way to know that is to perform getmem's with every increasing values until one fails. I was not implying that a getmem(maxavail) must work under any circumstances on a multitasking OS. But as the OS knows at a given point what the maximum availible memory for a single allocation call is, so could a program running on that OS. Yes, by trying to allocate that memory. There is no other way that I know of. But back to the argument - as on a multitasking system ANY memory allocation could fail at ANY time, if this was so severe, none of these OSes should be accepted by the community as they are buggy. No, because they do not crash if it fails. They simply return an error which you can check. And this error is consistent. There are no cases where you do not get an error, but that the allocation failed nevertheless (or vice versa). But as you see everybody seems to live happily with this as the chances these failures to happen are acceptable low. So allowing old programs initially designed for other environments to live with the same limits, that's OK I think. These old programs think they are checking for errors, but in practice they are not. And as to buggy programming - what can you really do when a getmem fails? That completely depends on the program. A game will probably terminate with an error, our compiler will terminate with a run time error and a server app will probable reject a new connection or terminate the current session. A compiler should be providing the means, not taking them away. It should not provide known defunct means. It's not the means but the way you are using them. Would you then remowe the Reset/Rewrite functions as well? In many (old) programs they are used to check for presence of file(s) to prevent overwriting by first trying to open them and if this fails then the file can be written... But as in a multitasking environment the file can appear just between the Reset and the Rewrite - well, bad bug. There are the means to do this right by the proper OS functions so maybe this should be enforced as well!? Reset and rewrite also have other functionality. The compiler nor the RTL can enforce proper usage of file locking semantics, just like it can't force you not to have any dataraces. The above code is not used to check whether there's still enough free memory left, it's only used to calculate how much memory is used by the compiler. I just picked one. The compiler never checks whether enough memory is left. It will simply terminate with a run time error if no memory is left anymore. The HeapStatus stuff can also not be used to query how much free OS memory (whatever that may be, see my previous list of possibilities) is left. It can only tell you how much memory has been allocated to the program, how much of that memory is actually in use etc. You can make a unit which contains a dummy memavail and maxavail which always return a value of 1GB or so, then your program will also always work except under heavy load (if that is really what you want). There's no use in investing lots of time in making very complex routines to query all sorts of information from the OS and then have it fail in exactly the same circumstances as when you return a plain constant. I mentioned the plain constant alernative in one of my first mails as it depends on the assumptions you make. Its about compilability of old code and this would also do but a better function would reduce the unavoidable failures to a minimum. There is no better function afaik (other functions will simply require more work to implement, but the end result will be pretty much the same). Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
{$ifdef fpc}const memavail=high(ptrint);maxavail=high(ptrint);{$endif}
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
{$ifdef fpc}const memavail=high(ptrint);maxavail=high(ptrint);{$endif}
That's a nice one :-)
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
On 23 mei 2005, at 11:43, Peter Vreman wrote:
{$ifdef fpc}const memavail=high(ptrint);maxavail=high(ptrint);{$endif}
Then we could maybe also add an optimization that checks whether you
are comparing against the upper or the lower bound of the current type
and completely optimize away the checks :)
Jonas
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Op Mon, 23 May 2005, schreef Jonas Maebe:
On 23 mei 2005, at 11:43, Peter Vreman wrote:
{$ifdef fpc}const memavail=high(ptrint);maxavail=high(ptrint);{$endif}
Then we could maybe also add an optimization that checks whether you
are comparing against the upper or the lower bound of the current type
and completely optimize away the checks :)
Define them as constants and that's exactly what'll happen.
Daniël
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Florian Klaempfl wrote: Konstantin Münning wrote: Hi everybody! Why in fact were MaxAvail,MemAvail,HeapSize removed from the RTL? The explanation in install/doc/whatsnew.txt is puzzling me a bit: - Removed MaxAvail, MemAvail, HeapSize due to their unreliability (bogus/misleading return values) in multitasking environment with swapping Is the incompatibility of the source code preferred over a misleading result? In this case, yes. Better fail totally than under rare circumstance and hard to reproduce because the crashes are caused by a lack of memory causing strange behaviour of memavail in combination with new/getmem. Keep in mind that the code if maxavail=1 then getmem(p,1); is completly useless in any multitasking environment. In general speaking that's not true. What would be the replacement for this? When you skip the check and allocate always then the program will crash anyway when there is not enough memory. What would be the downside of the bogus answer? If there was not enough memory when doing the maxavail call but would have been on the getmem call then there would be one memory low warning (or whatever an else condition action above would be) more than really needed but how often would this flaw happen in comparison to the crash without the check? The only code I would say is really bogus with this is something like getmem(maxavail) which would quite often fail without obvious reason but it is a bad code for a multitasking environment anyway. Such a major change should have a bit better explanation. I just checked the FPC introduction that there is still the claim for excellent compatibility with TP 7.0 which seems I.M.H.O. now to be broken. How should work MaxAvail/MemAvail in a multitasking environment to be TP compatible? It depends a bit on the assumptions you make about the environment. If you assume that you have always any amount of memory (as would it be if you skip the check) then it should return its max. possible value (defining a constant will do). I don't know exactly what value MaxAvail returned before on a Linux system but returning in general the maximal allocatable memory by a single allocation call at the moment of the MaxAvail call would be best. The only problems to be expected would be on a system with a very low memory where most programs would crash/abort/fail anyway or when trying to allocate big chunks of memory. Big chunks for BP7 are 64KB. For todays computers if there are no 64KB more memory left then this IS a very low memory condition so this is not an issue for compatibility. For allocating bigger chunks in a multitasking environment special precautions must be taken anyway. I'm not aware of any standard function which can guarantee this. I still assume that the failure of GetMem causes a Heap-Runtime-Error like in BP7 so checking if the resulting pointer is NIL is worthless ;-). I don't know if the best operation described above is easily possible as it may differ from the previous operation. In this case leaving the original functions and putting a big fat warning in the docs about it would be better than removing them. Printing warnings (like gcc does for deprecated things) on compiling for the affected platforms is also a good option but I don't know how easy this would be. Just my 5 cent. How do you allocate memory in a fashion that is suitable for all environments? I understand that this may lead to strange behaviour of a program but wouldn't it have been better to include a remark in the documentation than to remove the function(s)? So it would be programmer's choice to leave it or to change it to something appropriate for the environment of the program. So, what are now the options? What are the recommended replacements for these functions? What's the best way to make working programs to compile again? By the way, there are still references in the RTL, the examples and the docs to these now missing functions. Some seem to have been missed on cleaning ;-). Have a nice day, ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Konstantin Münning wrote: Florian Klaempfl wrote: Konstantin Münning wrote: Hi everybody! Why in fact were MaxAvail,MemAvail,HeapSize removed from the RTL? The explanation in install/doc/whatsnew.txt is puzzling me a bit: - Removed MaxAvail, MemAvail, HeapSize due to their unreliability (bogus/misleading return values) in multitasking environment with swapping Is the incompatibility of the source code preferred over a misleading result? In this case, yes. Better fail totally than under rare circumstance and hard to reproduce because the crashes are caused by a lack of memory causing strange behaviour of memavail in combination with new/getmem. Keep in mind that the code if maxavail=1 then getmem(p,1); is completly useless in any multitasking environment. In general speaking that's not true. What would be the replacement for this? Try to allocate and check if it worked or let the heap manager throw an exeception if it can't allocate the memory. When you skip the check and allocate always then the program will crash anyway when there is not enough memory. With the check the chances are only less that the program crashes but it can still do so. So your program is still buggy. What would be the downside of the bogus answer? If there was not enough memory when doing the maxavail call but would have been on the getmem call then there would be one memory low warning (or whatever an else condition action above would be) more than really needed but how often would this flaw happen in comparison to the crash without the check? The only code I would say is really bogus with this is something like getmem(maxavail) which would quite often fail without obvious reason but it is a bad code for a multitasking environment anyway. Such a major change should have a bit better explanation. I just checked the FPC introduction that there is still the claim for excellent compatibility with TP 7.0 which seems I.M.H.O. now to be broken. How should work MaxAvail/MemAvail in a multitasking environment to be TP compatible? It depends a bit on the assumptions you make about the environment. If you assume that you have always any amount of memory (as would it be if you skip the check) then it should return its max. possible value (defining a constant will do). I don't know exactly what value MaxAvail returned before on a Linux system but returning in general the maximal allocatable memory by a single allocation call at the moment of the MaxAvail call would be best. The only problems to be expected would be on a system with a very low memory where most programs would crash/abort/fail anyway or when trying to allocate big chunks of memory. Big chunks for BP7 are 64KB. For todays computers if there are no 64KB more memory left then this IS a very low memory condition so this is not an issue for compatibility. For allocating bigger chunks in a multitasking environment special precautions must be taken anyway. I'm not aware of any standard function which can guarantee this. I still assume that the failure of GetMem causes a Heap-Runtime-Error like in BP7 so checking if the resulting pointer is NIL is worthless ;-). I don't know if the best operation described above is easily possible as it may differ from the previous operation. In this case leaving the original functions and putting a big fat warning in the docs about it would be better than removing them. Printing warnings (like gcc does for deprecated things) on compiling for the affected platforms is also a good option but I don't know how easy this would be. Just my 5 cent. How do you allocate memory in a fashion that is suitable for all environments? See above. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
On 22 May 2005, at 20:07, Konstantin Münning wrote: When you skip the check and allocate always then the program will crash anyway when there is not enough memory. No. Either you catch exceptions resulting from a lack of memory and recover, and then you have the same checking as before, except that the check happens atomically (by the OS: you ask for more memory and if there is no more memory, you get an exception). Another possibility to set the global system unit variable ReturnNilIfGrowHeapFails to true, and then you can check after each allocation whether the resulting pointer is nil (if so, not enough memory was available) or not. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
On 23 May 2005, at 00:56, Konstantin Münning wrote: No. Either you catch exceptions resulting from a lack of memory and recover, and then you have the same checking as before, except that the check happens atomically (by the OS: you ask for more memory and if there is no more memory, you get an exception). It seems you are missing a small point. Compatibility. With BP7 there are no exceptions - if you try to allocate more memory than availible you get a runtime error. This runtime error can't be switched off like IO-errors. OK, there are the exit procedures, but there is no recover. There was already no compatibility, because maxavail and memavail simply worked with the currently allocated memory to the program. So if you had bad luck (currently allocated memory was almost used up, even though the OS possibly had 2GB free) you would already get the message that no more memory was available. The structure of a program is completely different when designing it to work with exceptions and recovering from these. At least for the affected parts. It is simply impossible to write a program running under a multitasking system which has virtual memory 100% exactly the same as one running under a single tasking no virtual memory OS. There are other things you have to change as well: assembler code, inline machine code statements, mem[] statement (at least if you're not working under Dos), ... I don't think MaxAvail is so bad that everyone must be enforced to stop using it and is not left to choose by himself. What would you suggest to return under an OS like Windows, Mac OS X or Linux? The current free memory of the OS? Free memory + buffer cache - minimal buffer cache size enforced by the OS? The previous + available swap space - wired (i.e., not swappable to disk) memory? Just a constant? Something different possibly? And in all cases, the program will still crash occasionally because it runs out of memory in certain cases, because the check and the allocation do not happen atomically. You can't put something into the standard Run Time Library which works most of the time, except under heavy load. Imagine reading the docs and encountering that comment, you'd immediately start wondering how many more procedures there are which work most of the time. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] removed MaxAvail,MemAvail,HeapSize
Jonas Maebe wrote:
On 23 May 2005, at 00:56, Konstantin Münning wrote:
No. Either you catch exceptions resulting from a lack of memory and
recover, and then you have the same checking as before, except that
the
check happens atomically (by the OS: you ask for more memory and if
there is no more memory, you get an exception).
It seems you are missing a small point. Compatibility. With BP7 there
are no exceptions - if you try to allocate more memory than availible
you get a runtime error. This runtime error can't be switched off like
IO-errors. OK, there are the exit procedures, but there is no recover.
There was already no compatibility, because maxavail and memavail
simply worked with the currently allocated memory to the program. So if
you had bad luck (currently allocated memory was almost used up, even
though the OS possibly had 2GB free) you would already get the message
that no more memory was available.
You are right. I assumed something like that when I looked at the values
of maxavail but hadn't investigated further and that's how the possible
flaw I mentioned in one of my first mails could look like.
The structure of a program is completely different when designing it to
work with exceptions and recovering from these. At least for the
affected parts.
It is simply impossible to write a program running under a multitasking
system which has virtual memory 100% exactly the same as one running
under a single tasking no virtual memory OS. There are other things
you have to change as well: assembler code, inline machine code
statements, mem[] statement (at least if you're not working under Dos),
...
The only assembler code I had to change was the one I had binary patched
for 32-bit operation. This and mem[] statements must be adjusted anyway
when trying to run on different architectures. So that's not the
trouble. I am still very happy with my units which allow me to write my
programs to run in 16 bit DOS real mode, DOS protected mode and Linux
without changing the main code and this including the GUI (VGA - X11).
Only the device drivers must be rewritten, of course ;-).
I don't think MaxAvail is so bad that everyone must be enforced to stop
using it and is not left to choose by himself.
What would you suggest to return under an OS like Windows, Mac OS X or
Linux? The current free memory of the OS? Free memory + buffer cache
- minimal buffer cache size enforced by the OS? The previous +
available swap space - wired (i.e., not swappable to disk) memory?
Just a constant? Something different possibly?
I have written in my previous mail(s) that I think the best return value
would be the biggest value with which a getmem would be successful if
memory conditions haven't changed inbetween. As this might be difficult
to implement on all supported OSes something between this and the old
one should do.
As I said before - adding an otherwise imperative error handling like
exceptions is still possible in addition to the old code.
By the way, I have never had a noticeable bug in a program (and some of
them are controlling machines running 24/7) because of the discussed
possibility. OK, they are not doing extensive memory
allocation/deallocation but also the possible danger is confined to
non-critical areas - a shortened item list for example would hurt nobody
:-).
And in all cases, the program will still crash occasionally because it
runs out of memory in certain cases, because the check and the
allocation do not happen atomically. You can't put something into the
standard Run Time Library which works most of the time, except under
heavy load. Imagine reading the docs and encountering that comment,
you'd immediately start wondering how many more procedures there are
which work most of the time.
Most programs work most of the time except under heavy load. That's not
a good argument. Most small (C) programs don't do any memory
availability checks at all. Look at most of the command line tools on
any OS ;-). Of course I would like to have any code behave perfect under
any condition but it has always been a tradeoff between effort
(=expense) and impact. But when there is no memory left and you need
some you can't behave perfect. Please let it be the programmers choice
what to do and how to handle this. A compiler should be providing the
means, not taking them away. Anybody writing a program knows that there
may be flaws. A warning (with an advice how to make it better) should
still be the best for this.
By the way, I've found the following code in compiler/cclasses.pas when
I searched for the whereabouts of maxavail/memavail and there were some
more like this in the sources:
{$ifdef HASGETHEAPSTATUS}
status:=GetFPCHeapStatus;
startmem:=status.CurrHeapUsed;
{$else HASGETHEAPSTATUS}
startmem:=memavail;
{$endif HASGETHEAPSTATUS}
I'm not sure if I understand it right as GetHeapStatus is not documented
well
