Re: [Freedos-kernel] flaws in user HMA memory allocation?
Hi Tom, thanks for your feedback :-) > > 2. It is way slower in file "copy/compare" than other DOSes. > > (maybe we should make FAT and DIR data access smarter?) ... >FreeDOS allocates always the lowest unused cluster, even if >the file should be enlarged by a large ( > 1 cluster) amount. ... which also means more fragmentation ... I think we agree that it would help to remember the location of the first free / last allocated cluster. FAT32 even has a special nfreeclst field for that... One could make read/write_fsinfo work for all FAT types by storing the info in RAM for non-FAT32 drives. This would affect fatfs.c and fattab.c ... Can you explain how find_fat_free decides where to start searching? It might also make a difference to limit how often the FAT32 fsinfo sector is written to disk - depends on how BUFFERS and priorities are processed, you can probably estimate the performance better than I here. >search for a bunch of clusters that are able to fit this request >could accelerate this I think I prefer searching a cluster at a time but in a linear fashion, using a "next free cluster" pointer instead of searching from the start of the FAT each time you need one more cluster. If that is easier, you could start by only implementing it for FAT32, then users can already start doing experiments / compare the FAT32 performance to FAT16 performance :-). > b) COMPARE >if enough (small) files in large enough directories with enough >directory depth are compared, it might matter that fastopen isn't >implemented. Anybody ? Hmmm well I agree that FASTOPEN can be useful in other ways than a cache, but does any MSDOS version have a kernel built-in FASTOPEN? I thought they dropped all FASTOPEN around DOS 5 because they thought that a generic disk cache would be enough? Does FreeDOS for example remember the starting cluster of the current directory and / or a directory which is "being findnext-ed" at the moment? > > - you can say "BUFFERS=-5" if you want "5 and not more than 5" > > (as opposed to "BUFFERS=5" which is "at least 5" in FreeDOS) > > - HMA allocation (int 2f.4a0n) in FreeDOS should allow apps (such > > as Jack's drivers) to use the free HMA space... > yes and no; see www.bttr-software.de/forum/forum_entry.php?id=867 Ah interesting... user allocation is only possible after all DEVICEs are loaded because only then buffers are moved to HMA? Yet BUFFERS and FILES are processed before DEVICE afair, and FreeDOS starts using the HMA for the kernel code as early as possible afair (right after the DEVICE=HIMEM line) so... Are there good reasons to delay HMA buffer / fnode allocations? Anyway. An obvious solution would be to DEVLOAD the UDMA / UDVD drivers to let them enjoy "userspace" allocation of HMA space. Could anybody test that? :-) Eric - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] [Freedos-devel] flaws in user HMA memory allocation?
> 2. It is way slower in file "copy/compare" than other DOSes. > (maybe we should make FAT and DIR data access smarter?) depends on a couple of factors; the huge threat on http://www.drdosprojects.de/forum about FreeDOS is slow was less then scientific a) COPY FreeDOS allocates always the lowest unused cluster, even if the file should be enlarged by a large ( > 1 cluster) amount. this potentially searches a big part of the FAT to find place and writes the file fairly fragmented search for a bunch of clusters that are able to fit this request could accelerate this MSDOS 5 has been reported to allocate buffers on a 'round the disk' alghorithm (continue to search instead of restart from zero); has been reported to slow fragmentation b) COMPARE if enough (small) files in large enough directories with enough directory depth are compared, it might matter that fastopen isn't implemented. Anybody ? > I would like to comment on 1, please correct me if I am wrong... Experts should be Bart, Arkady, Lucho and Tom :-). <<< > - you can say "BUFFERS=-5" if you want "5 and not more than 5" > (as opposed to "BUFFERS=5" which is "at least 5" in FreeDOS) > - HMA allocation (int 2f.4a0n) in FreeDOS should allow apps (such > as Jack's drivers) to use the free HMA space beyond the user- > selected BUFFERS. So even if we fill all space with extra buffers > while the space is unused, we free that space for more useful > things as soon as some app asks for it. > Maybe there is a bug in int 2f.4a0n handling and/or config.c? yes and no; see http://www.bttr-software.de/forum/forum_entry.php?id=867 > - Arkady and Bart in 2004 > - Bart in 2001 > - Bart in 2004 (moved fnodes into HMA) > - Lucho in 2004 > - Tom in 2001 (created inithma.c) the HMA buffer logic came much later ;) Mit freundlichen Grüßen/Kind regards Tom Ehlert +49-241-79886 - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] [Freedos-devel] flaws in user HMA memory allocation?
Hi Japheth,
(changing subj from "Some FreeDOS kernel bugs reported by
Jack R. Ellis" and moving the issue to freedos-kernel...)
> ... just reported a bug/issue found in the FreeDOS kernel:
> http://www.bttr-software.de/forum/...
I guess a few people will have bad mood after reading this,
so it might have been better not to mention that URL ;-)
The language is "angry" there, but the summary is very short.
For some reason, Udo banned Japheth for a moment (??), and
Lucho is great, in particular his "LZ 7.10 MS DOS" (ever
heard of copyright?). FreeDOS, on the other hand, is not ;-)
FreeDOS has 2 big problems compared to LZ MS DOS:
1. It "uses all free HMA space for BUFFERS" even though LZ DOS
shows that, when you also use a separate disk cache, using
more than BUFFERS=3 gives no further improvement anyway. The
HMA should be available for more useful things instead :-).
2. It is way slower in file "copy/compare" than other DOSes.
(maybe we should make FAT and DIR data access smarter?)
I would like to comment on 1, please correct me if I am wrong...
>>> Experts should be Bart, Arkady, Lucho and Tom :-). <<<
- you can say "BUFFERS=-5" if you want "5 and not more than 5"
(as opposed to "BUFFERS=5" which is "at least 5" in FreeDOS)
- HMA allocation (int 2f.4a0n) in FreeDOS should allow apps (such
as Jack's drivers) to use the free HMA space beyond the user-
selected BUFFERS. So even if we fill all space with extra buffers
while the space is unused, we free that space for more useful
things as soon as some app asks for it.
Maybe there is a bug in int 2f.4a0n handling and/or config.c?
It LOOKS okay for me, though: Jack uses int 2f.4a01 -> check
returned BX and later int 2f.4a02.bx -> check returned ES, DI, BX ...
When I use DEBUG to call int 2f.4a01, FreeDOS says bx=2fef es=-1
di=d010" and after int 2f.4a02.1000, it says "bx=1000 es=-1 di=d010"
and when I call int 2f.4a01 again after that, I get bx=1fef es=-1 and
di=e010, so everything looks okay at first glance...? This is on a
system without any BUFFERS=... line. With BUFFERS=10, I get: bx=44b7
di=bb48 / bx=1000 di=bb48 / bx=34b7 di=cb48 :-). With BUFFERS=-10,
I get the same (but fewer actual buffers, as explained above).
Background reading:
kernel/inthndlr.c int2F_12_handler (only 4a.02 and "4a.other"
distinguished in the stable kernel) and AllocateHMASpace
kernel/config.c config_init_buffers (6..99 ok for "min buffers")
... firstAvailableBuf = pbuffer + wantedbuffers; ...
(so buffers after that point can be reused by other HMA users)
Note that pbuffer is a "struct buffer FAR *" while wantedbuffers
is a number and firstAvailableBuf is a "char FAR *" ;-)
kernel/inithma.c HMAalloc function and HMAFree pointer (which is
moved towards the end of the HMA when you allocate space)
kernel/int2f.asm calls int2F_12_handler for ax=4a01 and 4a02 and
for ah=12, which means that Win95 DOS "int 2f.4a03" is NOT yet
implemented. Int 2f.4a01 is "query free HMA space" and
int 2f.4a02 is "allocate HMA space". FreeDOS supports those :-).
Note that we do not round up allocations to paragraphs!
kernel/blockio.c AllocateHMASpace (a bit of a misnomer...) this
function removes buffers which are inside the specified range
of offsets, so the selected space can be used for other things
after calling AllocateHMASpace.
The HMA, if used by DOS, contains code first, then fnodes, then
buffers, unless buffers are too many to fit in the HMA. There
are two "limits": firstAvailableBuf and HMAFree. The latter can
indeed say "everything used" but the former will say "the user
only wants BUFFERS=... so the rest is only used for extra BUFFERS
because nobody else asked for HMA space for other things yet"
People who have worked on user HMA allocation:
- Arkady and Bart in 2004
- Bart in 2001
- Bart in 2004 (moved fnodes into HMA)
- Lucho in 2004
- Tom in 2001 (created inithma.c)
Some possibly relevant revisions and patches:
config.c 164->167 in 2001 (inthndlr.c 164->167 seems unrelated)
854->863 (new "rounding up") in 2004 (824->825 unrelated?)
382->403 in 2002 (?), 274->278 in 2001 (216->232 limit 64->99)
inthndlr.c 844->862 in 2004 and 833->844 in 2004
(274->278 unrelated? 216->232 unrelated?)
inithma.c 880->903 in 2004 ("off by 1") and 274->278 in 2001
(align HMAFree to paragraph in HMAalloc - only used in config.c
but not in inthndlr.c ...)
I hope one of the experts can have a look at the code and find
out whether we have a bug here. Thanks :-)
Eric
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Freedos-kernel mailing list
Freedos-kernel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
