Re: [Freedos-kernel] Re: [Freedos-cvs] kernel/utils exeflat.c,1.9.2.3,1.9.2.4
On Thu, 18 Nov 2004, Arkady V.Belousov wrote: Of course, qsort() if very fast algo (except some specific cases, when it is O(N^2)), but why to do _any_ extra action, when unnecessary? :) Especially, I suggest, (most) linkers do own sorting anyway? I think even bubble sort would be fast enough here. It's just that qsort is convenient because it is provided by the RTL. Hm. I think for simplicity and safety, exeflat should itself move relocation table from executable's header inside executable itself, so that it may be reused by MoveKernel(). This allows to eliminate manual table at kernel.asm:__HMARelocationTableStart. (Yes, I know __HMARelocationTableStart is not plain relocation table, but jump/code table, with jmps and calls to EnableA20. But table from header may be copied (after fixing) into secondary table. This allows to make code, which is more relocatable and may make cross-segments calls and accesses. This also solves issues with standard library routines, which may be used both from non-relocatable low code and from relocatable code.) This is not so easy. Firstly a secondary table would need extra space in low memory. There used to be entries to strcpy et al in the table but I removed them later because that part of the table that was useless after init was still resident. Secondly it would be quite a bit of effort for very little gain. Also think about it: we don't know in advance how big the table will be, it's hard to insert something of arbitrary size in kernel.sys. And then, where's the simplicity? PS: Bart, some time ago you decrease kernel by reordering object files. May you explain what was changed and how you find this? Just reordered by placing more or less similar files close together (asm close to asm, c close to c); that decreased entropy so compression improved. Bart --- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 ___ Freedos-kernel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Re: [Freedos-cvs] kernel/utils exeflat.c,1.9.2.3,1.9.2.4
Hi! 20--2004 18:54 [EMAIL PROTECTED] (Bart Oldeman) wrote to [EMAIL PROTECTED]: Hm. I think for simplicity and safety, exeflat should itself move relocation table from executable's header inside executable itself, so that it may be reused by MoveKernel(). This allows to eliminate manual table at kernel.asm:__HMARelocationTableStart. (Yes, I know __HMARelocationTableStart is not plain relocation table, but jump/code table, with jmps and calls to EnableA20. But table from header may be copied (after fixing) into secondary table. This allows to make code, which is more relocatable and may make cross-segments calls and accesses. This also solves issues with standard library routines, which may be used both from non-relocatable low code and from relocatable code.) BO This is not so easy. Firstly a secondary table would need extra space in BO low memory. ...unless it will be placed into transient/initial portion, which discarded from memory after startup. BO There used to be entries to strcpy et al in the table but I Yes, and then now may be reduced code duplication in asmsupt.asm (which generated both for transient and resident portion). BO removed them later because that part of the table that was useless after BO init was still resident. If you relocation table (RT) will be placed into discardable portion, then it will not still resident, And see the difference: RT is not entry points table, but table with references to places, which need correction, if they refers to relocated code. BO Secondly it would be quite a bit of effort for very little gain. 1. This gives safety (not need to worry about references from initialization part to relocatable resident part; of course, reference from resident portion to transient portion shold be worried). 2. This allows to reduce code (no duplication of same functions, like strcpy). BO Also think about it: we don't know in advance how big BO the table will be, it's hard to insert something of arbitrary size in BO kernel.sys. Of course, I think about this. But, because this table (should be) placed into transient portion, why not _allocate_ oversized table (say, 100-200 items)? Especially, because unused entries will be zero, they will be very well packed by UPX? I see there only one issue: how to reliably find in exeflat, where those place (for RT) resides in executable. BO And then, where's the simplicity? Of course, initial efforts to implement are not very easy, but gain is noticeable. For example, with such relocation you will never encounter troubles, like was issued with RTL-multiplcation functions from OW. The more so, now they may be moved from low/nonrelocatable part (in kernel.asm) into resident part! (Do you remember, how I was try to move Borland's RTL functions into resident part and forget about relocations?) PS: Bart, some time ago you decrease kernel by reordering object files. May you explain what was changed and how you find this? BO Just reordered by placing more or less similar files close together (asm BO close to asm, c close to c); that decreased entropy so compression BO improved. Only compression? I remeber something about reduced executable size, but without words about compression. --- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 ___ Freedos-kernel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Re: [Freedos-cvs] kernel/utils exeflat.c,1.9.2.3,1.9.2.4
On Sat, 20 Nov 2004, Arkady V.Belousov wrote: Yes, and then now may be reduced code duplication in asmsupt.asm (which generated both for transient and resident portion). only for Borland C. For Watcom they are not duplicated (only one CS: there). And anyway it's only a small amount of code. Of course, initial efforts to implement are not very easy, but gain is noticeable. For example, with such relocation you will never encounter troubles, like was issued with RTL-multiplcation functions from OW. That was a different problem. Watcom generated NEAR calls to these functions. Relocation wouldn't help. This point is moot now with one CS, and anyway I like our own versions better :) The more so, now they may be moved from low/nonrelocatable part (in kernel.asm) into resident part! (Do you remember, how I was try to move Borland's RTL functions into resident part and forget about relocations?) Yes you could move these into the resident part if you do runtime patching. But is it worth the effort for a secondary compiler? I will not stop you trying to do this, but I see it as a waste of time myself (other than an exercise in patching). PS: Bart, some time ago you decrease kernel by reordering object files. May you explain what was changed and how you find this? BO Just reordered by placing more or less similar files close together (asm BO close to asm, c close to c); that decreased entropy so compression BO improved. Only compression? I remeber something about reduced executable size, but without words about compression. No it was only compression, how else can reordering save space? The trick to reduce uncompressed executable size was only obtained using a different default calling convention via #pragma. Bart --- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 ___ Freedos-kernel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Re: [Freedos-cvs] kernel/utils exeflat.c,1.9.2.3,1.9.2.4
On Thu, 18 Nov 2004, Arkady V.Belousov wrote:
Small optimization: because BC doesn't knows, that exit() doesn't
returns, better to insert explicit return 0; after exit() or return error
code through return (remaining error handling for main()). For BC, this
allows to join common tails.
I don't think optimizing exeflat.exe's size is very important though ;)
LG + qsort(reloc, header.exRelocItems, sizeof(reloc[0]), compReloc);
Why to spend extra time for sorting relocations?
I think it's easier to read the output that way. Tom's idea.
Does qsort() really take a long time for you in exeflat? I find that hard
to believe, for just ~70 relocations.
LG +for (j = 0; j silentcount; j++)
LG + if (segment == silentSegments[j])
LG + {
LG +silentdone++;
LG +goto dontPrint;
LG + }
Never understand this option. (Though, don't try to study its meaning
deeper). The more so, me always annoying tons of some information about some
relocations, which garbaes output after exeflat.
use a larger backscroll buffer. Anyway, seriously:
The -S0x10 -S0x8B is outdated. 0x10 corresponds to segment 0x70 (device
drivers etc) and 0x8B to the DOS data segment at 0xEB. However over the
years some things were made smaller and the DOS data segment seems to be
at 0xCF or 0xD0 depending on compilation options now.
-S0x10 -S0x6F -S0x70
will catch relocations from these segments. The idea to silence these is
that these relocations are harmless because these segments don't move.
Relocations to other segments may be bugs. With the above options you
should get something like
relocation at 0x:0x0021 -01d6 (far jmp to init code in kernel.asm)
relocation at 0x01d6:0x9f2d -0f2f (seg init_tos in kernel.asm)
relocation at 0x01d6:0xa19c - (LowKernelConfig, main.c)
relocation at 0x01d6:0xc21c -01d6 (MoveKernel, FP_SEG(_HMATextEnd, inithma.c)
I checked them and these relocations are all fine. But if a fifth shows up
unexpectedly we may have a bug.
---
UPXOPT seems indeed unnecessary now.
Bart
---
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
___
Freedos-kernel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Re: [Freedos-cvs] kernel/utils exeflat.c,1.9.2.3,1.9.2.4
Hi! 18--2004 21:21 [EMAIL PROTECTED] (Bart Oldeman) wrote to [EMAIL PROTECTED]: Small optimization: because BC doesn't knows, that exit() doesn't BO I don't think optimizing exeflat.exe's size is very important though ;) Sure. :) But why not make more optimal code from scratch in any case, without thinking will this code often used? LG + qsort(reloc, header.exRelocItems, sizeof(reloc[0]), compReloc); Why to spend extra time for sorting relocations? BO I think it's easier to read the output that way. Tom's idea. BO Does qsort() really take a long time for you in exeflat? I find that hard BO to believe, for just ~70 relocations. Of course, qsort() if very fast algo (except some specific cases, when it is O(N^2)), but why to do _any_ extra action, when unnecessary? :) Especially, I suggest, (most) linkers do own sorting anyway? Ok, ok, above questions are not critical and mostly rhetorical. LG +for (j = 0; j silentcount; j++) Never understand this option. (Though, don't try to study its meaning deeper). The more so, me always annoying tons of some information about some relocations, which garbaes output after exeflat. BO use a larger backscroll buffer. Anyway, seriously: BO The -S0x10 -S0x8B is outdated. 0x10 corresponds to segment 0x70 (device BO drivers etc) and 0x8B to the DOS data segment at 0xEB. However over the BO years some things were made smaller and the DOS data segment seems to be BO at 0xCF or 0xD0 depending on compilation options now. ...and different kernel branches. :) BO -S0x10 -S0x6F -S0x70 BO will catch relocations from these segments. The idea to silence these is BO that these relocations are harmless because these segments don't move. BO Relocations to other segments may be bugs. With the above options you Ok, I see. Thank you for exaplanation. BO should get something like BO relocation at 0x:0x0021 -01d6 (far jmp to init code in kernel.asm) BO relocation at 0x01d6:0x9f2d -0f2f (seg init_tos in kernel.asm) BO relocation at 0x01d6:0xa19c - (LowKernelConfig, main.c) BO relocation at 0x01d6:0xc21c -01d6 (MoveKernel, FP_SEG(_HMATextEnd, inithma.c) BO I checked them and these relocations are all fine. But if a fifth shows up BO unexpectedly we may have a bug. Hm. I think for simplicity and safety, exeflat should itself move relocation table from executable's header inside executable itself, so that it may be reused by MoveKernel(). This allows to eliminate manual table at kernel.asm:__HMARelocationTableStart. (Yes, I know __HMARelocationTableStart is not plain relocation table, but jump/code table, with jmps and calls to EnableA20. But table from header may be copied (after fixing) into secondary table. This allows to make code, which is more relocatable and may make cross-segments calls and accesses. This also solves issues with standard library routines, which may be used both from non-relocatable low code and from relocatable code.) BO --- BO UPXOPT seems indeed unnecessary now. Sure. :) PS: Bart, some time ago you decrease kernel by reordering object files. May you explain what was changed and how you find this? --- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 ___ Freedos-kernel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Re: [Freedos-cvs] kernel/utils exeflat.c,1.9.2.3,1.9.2.4
Hello Arkady, Small optimization: because BC doesn't knows, that exit() doesn't BO I don't think optimizing exeflat.exe's size is very important though ;) Sure. :) But why not make more optimal code from scratch in any case, without thinking will this code often used? And why not leave *existing* code as is - without wasting mental and ethernet bandwidth on it ? LG + qsort(reloc, header.exRelocItems, sizeof(reloc[0]), compReloc); Why to spend extra time for sorting relocations? BO I think it's easier to read the output that way. Tom's idea. BO Does qsort() really take a long time for you in exeflat? I find that hard BO to believe, for just ~70 relocations. Of course, qsort() if very fast algo (except some specific cases, when it is O(N^2)), but why to do _any_ extra action, when unnecessary? :) it's not unnecessary - else I wouldn't have written it. Especially, I suggest, (most) linkers do own sorting anyway? I call people who 'suggest' somethintg that is plain wrong (and they could have seen that themself) idiots. Ok, ok, above questions are not critical and mostly rhetorical. but give some insight. Hm. I think for simplicity and safety, exeflat should itself move relocation table from executable's header inside executable itself, so that it may be reused by MoveKernel(). This allows to eliminate manual table at kernel.asm:__HMARelocationTableStart. (Yes, I know __HMARelocationTableStart is not plain relocation table, but jump/code table, with jmps and calls to EnableA20. But table from header may be copied (after fixing) into secondary table. This allows to make code, which is more relocatable and may make cross-segments calls and accesses. This also solves issues with standard library routines, which may be used both from non-relocatable low code and from relocatable code.) if you can write down a simpler and saver method to do that - fine. else - see my bandwidth comment... tom --- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 ___ Freedos-kernel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Re: [Freedos-cvs] kernel/utils exeflat.c,1.9.2.3,1.9.2.4
Hi! 18--2004 11:24 [EMAIL PROTECTED] (tom ehlert) wrote to Arkady V.Belousov [EMAIL PROTECTED]: te And why not leave *existing* code as is - without wasting mental and te ethernet bandwidth on it ? This is not (yet) existing, this is _new_ code. Especially, I suggest, (most) linkers do own sorting anyway? te I call people who 'suggest' somethintg that is plain wrong (and they could te have seen that themself) idiots. Sometime I study executables, and seen tables was sorted. (Probably, this was caused by applied exepackers, I not remember such details, but they _was_ sorted). --- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 ___ Freedos-kernel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/freedos-kernel
