Hi!
21--2006 02:30 [EMAIL PROTECTED] (Aitor Santamarэa) wrote to
[email protected]:
>> - DISPLAY somehow manages to resist LOADHIGH, Aitor! Maybe it
>> just tries to be too clever...
AS> Not at all. I remind you (discussed long ago) that it happens because
AS> of UPX. The creation of DISPLAY is:
AS> NASM => COM2EXE => UPX
AS> I must use UPX, because (as I asked here long ago) I cannot, as a
AS> device driver (and will not do for EXE, as I will explain for KEYB)
AS> EXPAND my memory block.
AFAIR, device driver not need to "expand" memory block - it already own
all free memory... As for .EXE, then its header already says to executable
loader, how much of memory it required, so not need to expand anything.
AS> Hence I have to book a lot of room with 0's
AS> and then UPX it (code is around 2Kb, the rest to 11KB (using XMS) is
AS> just that bunch of 0's, which is UPX-ed and compressed to less than
AS> 4KB.
But why at all store all zeros in executable? Why not clear memory
after loading (if you need zeros)? Just ensure, that after program loading
there is enough memory for this "bunch of 0's" - for this, ensure, that EXE
header field "min memory" contains required value.
AS> This happens to work perfectly well for MS-DOS (where I can load high
AS> without problem), but I guess that FreeDOS kernel does not like very
AS> much (I suppose) that the program is expanded after being loaded. I
Oops. More details, please.
AS> didn't have time to digg into kernel sources yet to see where the
AS> problem is, but I was also hoping not to release any more .EXEs.
?
AS> However, the bug remains (in my opinion) in the kernel, interesting to
AS> be looked at.
Again, mode details, please!
21--2006 03:41 [EMAIL PROTECTED] (Eric Auer) wrote to Aitor
Santamarэa:
EA> Hi Aitor, your problem is not in FreeDOS, it is in
EA> how your program is organized and in how UPX does
EA> not handle that well...
I suggest this too.
>>> - DISPLAY somehow manages to resist LOADHIGH, Aitor!
>> Not at all. I remind you (discussed long ago) that it happens
>> because of UPX. The creation of DISPLAY is: NASM => COM2EXE => UPX
>> I must use UPX, because (as I asked here long ago) I cannot, as a
>> device driver (and will not do for EXE, as I will explain for KEYB)
>> EXPAND my memory block. Hence I have to book a lot of room with 0's
EA> UPX reports that the EXE would normally be 62 kilobytes...
EA> Checking the header of the de-UPXed version of DISPLAY...
Interesting... Let me check myself. Bellow difference between
distributed original executable and de-UPXed one:
______________O\_/_________________________________\_/O______________
-Pages to load/last page size: 0008/0043h (8 pages/67 bytes)
+Pages to load/last page size: 007B/0047h (123 pages/71 bytes)
-Relocation table offset/size: 001C/0001h (28 bytes/1 dwords)
+Relocation table offset/size: 0020/0000h (32 bytes/0 dwords)
Header size: 0002h (2 para)
- Min/max memory to allocate: 1E62/FFFFh (7778/65535 para)
+ Min/max memory to allocate: 00AD/FFFFh (173/65535 para)
- CS:IP: 0000:0000h
+ CS:IP: FFF0:0100h
- SS:SP: 0F44:FFFEh
+ SS:SP: FFF0:FFFEh
Checksum: 0000h
Overlay #: 0000h (0)
- File size: 00000E43h (3651 bytes)
+ File size: 0000F447h (62535 bytes)
Header size: 00000020h (32 bytes)
- Program image size: 00000E23h (3619 bytes)
+ Program image size: 0000F427h (62503 bytes)
- Required memory to load: 0001F550h (128336 bytes)
+ Required memory to load: 00010000h (65536 bytes)
_____________________________________________________________________
O/~\ /~\O
Wow! UPXed executable requires 128336 bytes to load! This explains, why
DISPLAY resist to load high... Interesting, where is bug/issue? When I
repack DISPLAY by UPX ("upx --8086 --brute"), it again begins require 128k
to load - looks like bad algorithm in UPX. After packing executable by
aPACK ("apack -x"), it now requires only original 64k of memory to load.
Pity, that aPACK doesn't support exe/sys.
EA> is a bit tricky but the main point here is: Your EXE
EA> file tells that it needs to be loaded into a memory
EA> block of at least 128611 (decimal) bytes while, when
EA> you de-UPX it, it tells that it would need only
EA> 65815 (decimal) bytes. Still much for something which
EA> should be loaded high. I think the difference is
Precisely!
EA> caused by your "my stack is 64k and the binary will
EA> decompress to 61.5k" which makes UPX go for "this
EA> file will need 125.5k (4k loaded plus 121k heap) to
EA> be run" for some reason.
I think, this is bug in UPX or this is too greedy algorithm in UPX.
>> This happens to work perfectly well for MS-DOS (where I can load high
>> without problem), but I guess that FreeDOS kernel does not like very
>> much (I suppose) that the program is expanded after being loaded...
EA> Because your program is an EXE, the DOS kernel can figure
EA> out how much it needs to be run. Either your MS DOS has
EA> 128k consecutive UMB space when your run DISPLAY or your
EA> MS DOS is violating its own specs... However, UPX is quite
EA> pessimistic about needed space. If you would NOT UPX your
EA> program, you would only need roughly 64k to load DISPLAY!
Precisely. The more so, if not preserve all zeros in executable, then
it will be reduced even more, than after packing.
EA> There are various possible solutions. If you UPX a DISPLAY.com
EA> file,
No! NEVER-NEVER-NEVER make resident programs as .COM files (I mean,
resident should be compiled as .EXE or converted from .COM to .EXE), because
.COM files don't have header, which notifies system, how much precisely
programs it need! Without this, .COM program may be loaded into too small
UMB block, and, if program doesn't check this, it will overwrite MCB chain
headers!
EA> it will load as soon as only 4k of UMB are free, but
EA> will later abort during deUPXing in RAM (errorlevel 0...).
This is very bad, if program will silently crash or exit without doing
anything useful only because there is not enough memory in block, where it
loaded! AND, if you pack 64k executable (notwithstanding, .COM or .EXE),
then packed header will not say "required 4k of memory to load". Issue
between .COM and .EXE is only when .COM will require _dynamic_ memory (in
given case, if .COM executable will not store all zeros, but will clear
memory after start).
EA> If you make DISPLAY a true EXE
Converted (from .COM to .EXE) executable _is_ "true EXE". Of course,
when you convert, you shouldn't forget to specify, how much memory will be
required after program loading. With my COM2EXE, use option -s.
EA> which knows well about how
EA> much heap it needs and preferrably also has a real stack
EA> (not "initial SS:SP = PSP:fffe"),
For .COM and .COM with .EXE header, this _is_ real stack. But, with my
COM2EXE, if you specify (by -s option), that program requires less than 64k
of memory after load, stack (SP) will be consequently adjusted.
EA> UPX should be able to set
EA> proper values for the minimum heap size. As the decompressor
EA> is f44 paras after the initial CS when you run the UPXed
EA> binary, same as initial SS, I would say you kind of found
EA> an UPX bug.
Value of SS:SP is irrelevant in given case, because after starting
stack (pointer) may be moved in any place.
[...]
EA> You either have to use another compressor, design your
EA> program as a proper EXE with own stack, or ask some UPX
EA> expert to fix the sub-optimal workings of UPX for your
EA> case.
Second advice is ir-relevant, third (I think) is best (if UPX will be
really fixed). And you miss there advise with eliminating zeros from
executables - this allows to not use packer at all (because I doubt that
compressing 2k to, say, 1.4k gives much advantage).
EA> At least you can be sure that this is NOT a FreeDOS
EA> kernel problem.
Precisely.
EA> CCing some involved people...
EA> - Arkady (probably exe header tuning expert)
:)
EA> PS: When DISPLAY is an EXE style SYS, it will have the
EA> same "EXE problem" with UPX,
Interesting, that when I re-pack unpacked executable ("upx --8086
--brute"), UPX says:
UPX 2.01d Markus Oberhumer, Laszlo Molnar & John Reiser Jun 06th 2006
File size Ratio Format Name
-------------------- ------ ----------- -----------
62535 -> 3652 5.84% dos/exe DISPLAY.up
-------------------------------------^^^^^^^
so, I wonder, how Aitor packs his _driver_?
EA> so you still need a better UPX
EA> or better EXE header (where DISPLAY has a normal small stack).
There all fine with stack. Stack of program have nothing with
(de)compression algorithms.
EA> When DISPLAY is a normal classic SYS, it will have the same
EA> problem as COM - possibly aborting at load because the needed
EA> amount of RAM is not known to the kernel - it only is known
EA> for EXE and for EXE style SYS files, as you know.
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Freedos-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/freedos-devel
