Re: [Freedos-devel] FreeCOM 0.84-pre6 prerelease
Hi Bart, > sbrk is a little deceiving here since it just administrates a > high-water mark and does not allocate memory from DOS: > old: > https://github.com/tkchia/newlib-ia16/blob/e7c08882834a41d848698a19deae49089c2dab17/libgloss/ia16/dos-sbrk.c > which only returned NULL if heap ptr went beyond 64K (the libc for > ia16-elf-gcc started very bare bones in this respect!) that should be fine then. however IIRC MSC sbrk() would (try to) expand the allocated DOS memory to grow the stack; this would not work well with command XMS-SWAP. > new (just changed by TK Chia) > https://github.com/tkchia/newlib-ia16/blob/newlib-2_4_0-ia16-tkchia/libgloss/ia16/dos-sbrk.c > malloc returns NULL when the end of the heap is reached but this is > not always checked properly in freecom, but at least it does not grow > into the stack any more. > One could easily swap heap and stack in ia16-elf-gcc though I think we > would just see different memory corruption, stack can then go corrupt > other data or even code. possibly a misunderstanding by me: if the stack and heap have clearly defined, separate areas, you are absolutely right. (it made debugging somewhat easier that REN destroyed always the same data, but that's a minor detail). I had understood 'heap grows up toward stack' as some sort of optimization so that the heap can grow at the cost of stack and vice versa. > (that still leaves Turbo C which also grows heap to stack.. and of > course protected mode OSes do it but with virtual memory protection virtual memory stack protection is really cool (at least in the MSVC implementation which happens to be the only one that I am knowledgeable in); all stack checking without any cost. Tom ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] FreeCOM 0.84-pre6 prerelease
Hello Bart, new (just changed by TK Chia) https://github.com/tkchia/newlib-ia16/blob/newlib-2_4_0-ia16-tkchia/libgloss/ia16/dos-sbrk.c malloc returns NULL when the end of the heap is reached but this is not always checked properly in freecom, but at least it does not grow into the stack any more. Yes, I finally got around to implementing the _heaplen variable from Turbo C into gcc-ia16. :) I hope to put out a new release of the whole gcc-ia16 toolchain after I have run the regression tests once through. On the FreeCOM side, I find that the code to locate and patch _heaplen (utils/mkinfres.c, tools/ptchsize.c) will probably also need to be updated to handle the gcc-ia16 output. This turns out to be a bit trickier than I thought. Thank you! -- https://github.com/tkchia/ ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] FreeCOM 0.84-pre6 prerelease
Hi Tom, sbrk is a little deceiving here since it just administrates a high-water mark and does not allocate memory from DOS: old: https://github.com/tkchia/newlib-ia16/blob/e7c08882834a41d848698a19deae49089c2dab17/libgloss/ia16/dos-sbrk.c which only returned NULL if heap ptr went beyond 64K (the libc for ia16-elf-gcc started very bare bones in this respect!) new (just changed by TK Chia) https://github.com/tkchia/newlib-ia16/blob/newlib-2_4_0-ia16-tkchia/libgloss/ia16/dos-sbrk.c malloc returns NULL when the end of the heap is reached but this is not always checked properly in freecom, but at least it does not grow into the stack any more. One could easily swap heap and stack in ia16-elf-gcc though I think we would just see different memory corruption, stack can then go corrupt other data or even code. (that still leaves Turbo C which also grows heap to stack.. and of course protected mode OSes do it but with virtual memory protection they never get close to each other, and this is what can make small mode DOS debugging extra hard). Bart ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] FreeCOM 0.84-pre6 prerelease
Hi Bart, > I tried to limit heap grow a bit using a patch to nanomalloc > https://gist.github.com/f45f2874b16afd8957d019db6182067e > which basically takes any free block at the end into account when it > needs to grow the heap via sbrk, so it does not leave a hole there. using sbrk() inside command.com is forbidden, as the amount to swap in/out is calculated once at init time, unless this code is also fixed. swapping in/out only part of the heap (and thereby destroying its content some of the time) isn't going to make anybody happy. Tom ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] FreeCOM 0.84-pre6 prerelease
Bart,
> you are (in your words) 110% right.
> At the time I had fixed the stack offenders but got lost in debugging
> adapting your stack-checking patch to other compilers (it actually
> checks the heap too, if heap grows to stack).
heap growing to stack *without stack check enabled) is a REAL BAD IDEA. (not
your idea, I know)
let me explain.
ignoring recursion for the moment, maximum stack use
can be statically evaluated
will not change unless you make major changes
you can request the required amount using some linker switch, and
rely that this amount will always be available; no stack checking
in production code necessary.
also: requesting stack (like calling a function) *must not fail*,
otherwise CRASH.
in contrast, malloc() may fail; virtually all malloc ocde runs like
char *p = malloc(some_amount);
if (p == NULL)
{ // fail decently, dont CRASH
printf("unable to allocate");
return FALSE;
}
now consider a heap that grows into the stack:
depending on the programs allocating history there may be
sometimes enough stack, sometimes not.
sometimes you see funny characters in your editor, sometimes the
linked list pointer gets overwritten.
this is just asking for
trouble, and extremely frustrating debugging sessions.
Tom
___
Freedos-devel mailing list
Freedos-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-devel
Re: [Freedos-devel] FreeCOM 0.84-pre6 prerelease
Hello Bart, thanks for offering to help. I now pushed a GCC-portable version of Tom's stack check to freecom's git: https://github.com/FDOS/freecom However if you try to compile it you'll see once you run it that it says 0 stack left quite quickly. The reason is not the stack (there is plenty) but the heap which grows towards the stack. The default heap is very small though (6K), still it grows much further than OW's (with OW the stack is below the heap though but I debugged it a few times). Thanks for the information! I added some further debugging lines --- for temporary use --- to try to show how the heap top address (sbrk) changes over time, and to highlight to the user whenever there is a huge jump in the sbrk: https://gist.github.com/tkchia/c27c47e85550914e37ca85f82952f015 When I run the patched command.com with metados.img, the sbrk gets a huge jump (from 0x18d8 to 0x21e0) around the time of the exec( ) call on the command "xmssize.com 5 >NUL". I find this very strange (since I do not expect exec( ) to need to mess with the heap). There is another huge jump in the sbrk (0x21e0 -> 0x2dd0) around the cmdptr->func( ) of "dir /s/a-d/b a:\fetchbat.zip". It seems this is not merely a case of the heap becoming too fragmented and growing too fast --- though I am not sure yet what it actually is. Thank you! -- https://github.com/tkchia/ ___ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
