Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste On Wed, 12 Apr 2017 04:39:35 -0500, Rugxulo wrote: > It worked fine (redirecting) for me yesterday! I can't imagine why it > wouldn't work for you. > > Oh, before I forget, are you perhaps invoking NDISASM via some .BAT? > Of course a .BAT doesn't really redirect (under FreeCOM) without kludge, > e.g. "%COMSPEC% /c". Ha, yes - you nailed it, Rugxulo. Indeed I was calling ndisasm from within a sneaky ndisasm.bat file (auto-generated by FDNPKG inside my PATH). As said before, I didn't really investigate the problem (for a reason that should be obvious by now) - saw it doesn't work out of the box, went to the "-l" nasm listing, done. But thanks to you the reason is clear now. It's at least the second time this "FDNPKG generates BAT files" thing bites me. I should definitively address this problem eventually. I have to say however that I prefer the "-l" listing anyway, because it preserves the comments (even though it lies a little bit sometimes on the JZ thing). > Maybe you should use Lazy Assembler (LZASM)?! Didn't know that one (too lazy to google it out), but it does sound extremely well suited to me! cheers, Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste On Tue, 11 Apr 2017 02:03:54 -0500, Rugxulo wrote: > AFAIK, the longer one is 386+ only, hence not available with "cpu 8086". The above code assembles with "cpu 8086" (NASM 2.12.02). > Thus, if it still quietly assembles, that is a bug (but I thought that > was long-ago fixed/avoided). Perhaps a bug, didn't investigate. My point is - explicitly mentioning SHORT is always a good idea. Better safe than sorry. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Jerome Shidel Less than 256 bytes of code. Over 2 Megabytes of controversy. Shame on you Mateusz! :-) All joking aside, from what I gather, your bsum is only meant to verify local files were copied correctly from one machine to another. Most forms of checksums should be sufficient for that task. Sure a collision is possible. More bits in the checksum reduces the possibility. But even with a SHA 256 hash, the chance of accidental collision is so remote as to be zero. It is still theoretically possible. How sure do you need to be? Well, you could be absolutely sure by coping them over then back. Then doing a byte level comparison between the original and the copy of a copy. But why? Now as for a man in the middle of your two machines... Even that check could be thwarted by a crafty attacker. So, I assume that the checking done by you bsum is the exact level of certainty you desire. Sure enough. Jerome -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Rugxulo Hi, On Wed, Apr 12, 2017 at 2:03 AM, Mateusz Viste wrote: > On Tue, 11 Apr 2017 22:24:56 -0500, Rugxulo wrote: > >> What disassembler are you using here? I erroneously thought it was NDISASM. > > I don't use ndisasm for a very trivial reason - I am unable to redirect > its output to a file, so I don't really know how other people use it It worked fine (redirecting) for me yesterday! I can't imagine why it wouldn't work for you. > I didn't figure out any quick and easy workaround (again, too stupid). Um, excuse me, but he called you "lazy" and "clueless", not "stupid". I guess we should add "forgetful". ;-)) j/k But I'll point to this anyways, "redir", just for a general tip (in rare case you didn't already know): http://www.delorie.com/djgpp/doc/utils/utils_7.html Oh, before I forget, are you perhaps invoking NDISASM via some .BAT? Of course a .BAT doesn't really redirect (under FreeCOM) without kludge, e.g. "%COMSPEC% /c". > The output I pasted before was copied from the NASM listing (-l). Hmmm, then NASM is being a bit too tricky for its own good. I do (very naively!) wonder whether "warning: 8086 conditional jump extended" would be appropriate. Actually, having "[386]" (etc) in NDISASM output would be nice. (The only workarounds for that are BIEW and QVIEW, IIRC both of which color-code various instructions. Not sure about various debuggers off the top of my head.) > And although I do look at the listing carefully, I do not bother decoding the > opcodes by hand (too lazy!), Maybe you should use Lazy Assembler (LZASM)?!:-P Nah, it needs a separate linker, even for .COM (bah, too slow, we're too lazy!). > I assume that the assembler knows how to > encode mnemonics into opcodes - that's his job after all, not mine. > Ultimately, whether the code is assembled into a "long, 5-byte form of > jump" or "two separate instructions that emulate a jump" is irrelevant to > me - in both cases it's still 5 bytes, that all I need to know. I can't even honestly complain, it's indeed a "feature", not a bug! Not mandatory but certainly nice to have. >> The simple answer is that code size is rarely as important as programmer >> convenience. > > Maybe. But why bother doing assembly then, if not for the control over > what machine code is generated at the end? I was trying to imagine thinking like them, not speaking for myself. I personally like size optimizations in assembly (obviously??). E.g. "add si,2" is three bytes but (times 2) "inc si" is only two! But you won't see a lot of programs that actively try to save such few bytes. Nobody cares. (Well, most other people!) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: "Bret Johnson" > I double-checked, latest NDISASM still decodes as two separate instructions. A disassembler would not report two separate instructions unless the actual compiled code had two separate instructions. Every assembler/compiler I know of does little "tricks" to make the programmer's job a little easier (basically, changes subtle/minor things behind your back). They also all seem to do different "tricks" (even in different versions of the same assembler/compiler), so even in ASM you're not really 100% in control of the resulting code. In ASM you're much more in control than you are in any high-level language, though. > The simple answer is that code size is rarely as important as programmer convenience. There are really several different major things that you must balance: program size, memory footprint, speed, and maintainability. "Programmer convenience" is just a subset of maintainability, which also includes things like program structure, language/compiler/assembler/libraries chosen, documentation, and code commenting. Which of the items has the highest precedence depends on circumstances and goals at the time. TSR's and device drivers, e.g., are very different than foreground programs -- memory footprint really matters (a lot) in TSR's and device drivers, though not necessarily so much in foreground programs (at least not in all foreground programs). It also depends on your intended target audience/hardware. Modern CPU's also do all kinds of "tricks" to increase speed (caching, pipelining, branch prediction, virtual machines/CPU's/memory, memory alignment issues, etc.), so code that is faster on a real 8086/8088 CPU may actually be (relatively) slower in a different environment. Looking up clock-cycles-per-CPU-instruction is a guideline, but not the final answer as to how fast something really is. Smaller size tends to equate to faster speed, though that's not absolute. It is true that a smaller memory footprint is more likely to remain in the CPU cache(s) (at least on CPU's that have caches and have them enabled), so a smaller memory footprint always increases speed (or at least the likelihood of speed) in that sense. -- Bret Johnson There is only one place in the universe where 0+0=1. That is in the mind of an evolutionist. 'Erin' From Happy Days Looks Like Now Is Insane trend-chaser.com http://thirdpartyoffers.juno.com/TGL3141/58ee56e68d39656e6565bst03vuc -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Tom Ehlert >>> Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. >>> 127 (signed) byte range. Hence the billions of workarounds (TASM >>> "jumps", MASM "option ljmp", etc). right. >> I won't argue about what opcode is or is not available on 8086, since I >> did not bother decoding their exact meaning. meaning 'I am a lazy, clueless guy, but write anyway ...' >> I do see however that (NASM >> at least) can assemble JZ and JZ SHORT in two very different forms, JZ >> SHORT being significantly shorter. >> >> 5 B80100 mov ax, 1 >> 6 0003 48 dec ax >> 7 0004 746Ajz short gameover >> >> 5 B80100 mov ax, 1 >> 6 0003 48 dec ax >> 7 0004 7503E9DD01 jz gameover >> >> Of course NASM always uses the short form whenever it's possible, but >> when the jump is too far away it silently uses the longer form, hence the >> need to always specify SHORT if one wants to be sure what's going on. > AFAIK, meaning 'I am completely clueless , but offer my unfounded opinion anyway ...' > the longer one is 386+ only, hence not available with "cpu > 8086". Thus, if it still quietly assembles, that is a bug (but I > thought that was long-ago fixed/avoided). the longer one is 2 instructions instead, automatically generated by NASM because the intended jump goes farther then 127 bte c:\>debug -e 100 1430:0100 00.75 00.03 00.e9 00.dd 00.01 -u 100 1430:0100 7503 JNZ 0105 1430:0102 E9DD01JMP 02E2 Tom -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste Hi Tom, That's nice of you to provide the explanation. I didn't read it completely (too lazy), nor understand it fully (too stupid), but the other clueless guy might find it interesting that he was only half wrong. At the end of the day, I will keep using "JZ SHORT" anyway, 'cause that just works for me. cheers, Mateusz On Tue, 11 Apr 2017 12:33:17 +0200, Tom Ehlert wrote: Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. 127 (signed) byte range. Hence the billions of workarounds (TASM "jumps", MASM "option ljmp", etc). > right. > >>> I won't argue about what opcode is or is not available on 8086, since >>> I did not bother decoding their exact meaning. > > meaning 'I am a lazy, clueless guy, but write anyway ...' > >>> I do see however that (NASM at least) can assemble JZ and JZ SHORT in >>> two very different forms, JZ SHORT being significantly shorter. >>> >>> 5 B80100 mov ax, 1 6 0003 48 dec >>> ax 7 0004 746Ajz short gameover >>> >>> 5 B80100 mov ax, 1 6 0003 48 dec >>> ax 7 0004 7503E9DD01 jz gameover >>> >>> Of course NASM always uses the short form whenever it's possible, but >>> when the jump is too far away it silently uses the longer form, hence >>> the need to always specify SHORT if one wants to be sure what's going >>> on. > >> AFAIK, > meaning 'I am completely clueless , but offer my unfounded opinion > anyway ...' > >> the longer one is 386+ only, hence not available with "cpu 8086". Thus, >> if it still quietly assembles, that is a bug (but I thought that was >> long-ago fixed/avoided). > > the longer one is 2 instructions instead, automatically generated by > NASM because the intended jump goes farther then 127 bte > > > c:\>debug -e 100 1430:0100 00.75 00.03 00.e9 00.dd 00.01 -u 100 > 1430:0100 7503 JNZ 0105 1430:0102 E9DD01JMP > 02E2 > > > Tom -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: dmccunney On Mon, Apr 10, 2017 at 9:36 AM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: > >> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >> with a bigger buffer. > > At the cost of reducing the number of platforms it would be able to run > on. I have to ask. How many folks *have* platforms now it *wouldn't* run on? I suspect the number is *very* small. (Most folks now are trying to get FreeDOS to boot native on a machine rather larger and more powerful than the machines DOS was used on, or running it in a VM. Even folks doing embedded development on IoT devices are probably dealing with fast full 32bit CPUs with more than enough RAM and external storage, and can run a Linux kernel or an RTOS that bears no resemblance to DOS.) > Mateusz __ Dennis https://plus.google.com/u/0/105128793974319004519 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Rugxulo Hi, On Tue, Apr 11, 2017 at 7:06 AM, Mateusz Viste wrote: > > That's nice of you to provide the explanation. I didn't read it > completely (too lazy), nor understand it fully (too stupid), but the > other clueless guy might find it interesting that he was only half wrong. Just to clarify, I'll quote from the A86 text manual: " NOTE that for A86, LONG will have effect only on the operand to an unconditional JMP instruction; not to conditional jumps. That is because conditional jumps farther than 127 bytes are available only on the 386 and later processors. If you run into this problem, then chances are your code is getting out of control--time to rearrange, or to break off some of the intervening code into separate procedures. If you insist upon leaving the code intact, you can replace the conditional jump with an 'IF cond JMP'. " > At the end of the day, I will keep using "JZ SHORT" anyway, 'cause that > just works for me. My problem with always explicitly saying "short" is that it's both unnecessary and verbose. But I (reluctantly) agree that here it's your best option for safety. (Though I would still hide it behind a preprocessor macro or whatever, for brevity.) What disassembler are you using here? I erroneously thought it was NDISASM. I also misremembered NDISASM as saying "jz near" (only) for 386+ jumps. I also misremembered such near/long jumps as being five bytes when they are, in fact, only four. I halfway remembered that the 386+ jz opcode started with 0Fh (sorry, Tom) but wasn't sure offhand without checking. But here your disassembler apparently put two jumps on one line, as if only one, which is weird. I know it's a common idiom, but I didn't think a literally disassembler would use that! (I double-checked, latest NDISASM still decodes as two separate instructions.) 0Fh is actually the opcode to (discontinued, 8086-only) "POP CS". Both FASM and NASM apparently support that (why?), but A86 apparently doesn't. Part of the confusion also lies with NASM. In the old days, you had to explicitly enable -O3 to enable this 8086 conditional jump workaround, e.g. 0.98.39 (by looking at old code of mine). NASM 2.09 started enabling -Ox [sic] by default. However, it appears that even with -O0 nowadays, if you're using "cpu 8086", it will still extend your conditional jumps in this manner. It didn't use to do that automatically, nor for -O0. The simple answer is that code size is rarely as important as programmer convenience. Sure, if you're trying to squeeze every last byte, you'll maybe accidentally be bit by this. But most people, by far, don't care about code size and only care about functionality. I'm only guessing, but that's my impression as to why NASM would always do this by default nowadays. P.S. http://stevemorse.org/8086/ is also an interesting read. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste On Mon, 10 Apr 2017 17:57:59 -0400, dmccunney wrote: > I have to ask. How many folks *have* platforms now it *wouldn't* run > on? I suspect the number is *very* small. Surely, yes. Still, a 700% memory increase for a 10% performance boost just doesn't feel right. I wrote bsum to cover an extreme case - in such context I prefer keeping the memory footprint as small as possible. > Most folks now are trying to get FreeDOS to boot native on a machine > rather larger and more powerful than the machines DOS was used on, or > running it in a VM I'd say that for these machines bsum is irrelevant - they are much better off using md5 or anything else. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste
On Mon, 10 Apr 2017 17:07:30 -0500, Rugxulo wrote:
> Blair's (16-bit, FD) MD5SUM can do all of those hashes as well. Not sure
> if it'd be faster, though.
I believe that's the one I used. If I understand correctly, the original
author is Colin Plumb, and Blair took the maintenance of it at some point.
> Believe me, shrinking size is fairly easy,
If you say so.
> but it's a tradeoff in accidental errors, readability,
> and speed.
Unless it's a goal in itself ("keep the whole thing in 256 bytes"), as is
the case of bsum.
> Irrelevant aesthetics: lines too long (shouldn't be more than 80),
I'll skip all aesthetics remarks, since these are a rather personal thing.
> irrelevant "jz short ..." (when "short" conditional jump is always
> mandatory for "cpu 8086").
I don't think so.
Note that short means "8 bit jump" in this context, and NOT "16 bit jump".
> "section .data align=1" is probably what you intended here. (No need to
> comment it out entirely.
No need to have it either (not in tiny model).
> The program does not end in a CR+LF pair. Thus the output is an
> incomplete line. Not a huge deal but still (sometimes) noticeable.
True. I noticed that command.com adds a CR+LF pair whenever a program
doesn't end with those. This seems to be consistent with both FreeDOS and
MS-DOS, so I thought I'd exploit this to save a few bytes in the program.
> "int 21h // xchg ax, bp // int 21h" is repeated several times. If you
> really want to save space, put "msgquit:" before the first one and "jmp
> short msgquit" for the others (since this is quitting the program
> anyways).
Indeed, that would save 1 byte or 2. Good catch.
> BTW, most asm devs actively hate "loop" in lieu of "dec // jnz". Not
> sure if this would really be worth it, even for your 8086.
Actually my trunk version (svn) does avoid loop in favor of dec/jnz.
The former is shorter by one byte, but 3 times slower than the latter
(5/6 clks vs 2 clks).
> "shl bx, cl" (where CL=4) is also shunned, AFAIK, on 8086 machines, in
> lieu of speedier (times 4) "shl bx,1".
But repeated shl bx,1 is so much bigger. I definitely prefer shl bx,cl,
at least whenever not in performance-critical parts.
> Converting hex nibble to ASCII shouldn't need a jump at all. On the 8086
> all jumps are very slow. Best to avoid them entirely if possible.
> Here you can easily use the old "cmp al, 0Ah // sbb al, 69h // das"
> trick instead. But since you're only printing hex one time (instead of
> thousands), you probably don't care.
Indeed, I care little about jumps there, but still your version might be
shorter, which would make it interesting. Will compare.
Mateusz
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user
--- Internet Rex 2.29
* Origin: capcity2.synchro.net - 502/875-8938 (276:10/901)
--- Synchronet 3.15a-Linux ListGate 1.3
* Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Rugxulo Hi again, On Mon, Apr 10, 2017 at 8:36 AM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: >> >> It would be interesting to see some benchmark numbers for that (for >> various specific tools, 8086, 386, etc). > > Just for the fun of it, I did some quick measures on my 386SX PC, > computing various checksums of a 2 MiB file. Results below. Very interesting > CRC32 (by Colin Plumb) : 26.7s (22%) > MD5 (by Colin Plumb): 52.9s (11%) > SHA1 (by Colin Plumb) : 85.7s (7%) Blair's (16-bit, FD) MD5SUM can do all of those hashes as well. Not sure if it'd be faster, though. > BSUM is the fastest, which is no surprise since the algorithm is > extremely simple (4 CPU instructions). The CRC32 computation by Joe > Forster is surprisingly fast as well. It's 30% slower than bsum and the > binary is 4x times larger (and I suppose the memory usage is also much > higher) but that's still quite impressive for a 32-bit checksum. "30% slower" is machine specific, and I'm quite sure it can be improved. Although his tool does seem to use a fairly big (64512 byte) buffer. *** If extremely bored, check out these "modern" (CRC32C, aka Castagnoli) implementations, which I don't grok: http://stackoverflow.com/questions/17645167/implementing-sse-4-2s-crc32c-in-software http://www.drdobbs.com/parallel/fast-parallelized-crc-computation-using/229401411 *** Of course he also combines (unused) decimal output routine with (used) hex output routine, which unnecessarily (in this case) always uses slow DIV (which you don't need at all for converting to hex). Of course he only needs to call that routine once at the end. It would be much worse result if called more often (e.g. hundreds of times). I've done the same mistake in the past, too. "4x times larger" is only in raw bytes, but in reality it uses a full cluster (as you well know), so even a 256 byte .COM will still use minimum one cluster (e.g. 512 bytes on 1.44 MB floppy). So 1024 isn't really much worse than 512. ;-)Believe me, shrinking size is fairly easy, but it's a tradeoff in accidental errors, readability, and speed. >> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >> with a bigger buffer. > > At the cost of reducing the number of platforms it would be able to run on. > Currently bsum uses an 8K memory buffer to optimize disk reads. Using a > buffer of 64KB increases the overall speed by 10%. Not that much, for a > 700% increase of memory usage. Don't you have an 8086 machine? How much RAM does it have? I had thought most had at least 64 kb of RAM, but I guess that's not accounting for the DOS + shell overhead. Honestly, I wrote several simple hexdump variants in recent months, and the biggest slowdown was my small buffer (only 16 bytes in the .ASM version). The C version is larger but always well-buffered, so it's the fastest. I even got 2x speedup (and noticeable size decrease) by avoiding printf entirely and using my own outhex routine. Okay, so let me break down your source and give some (trivial) comments here. I assume that's okay with you! ;-) Irrelevant aesthetics: lines too long (shouldn't be more than 80), not enough indentation (instructions vs. labels), irrelevant "jz short " (when "short" conditional jump is always mandatory for "cpu 8086"). "section .data align=1" is probably what you intended here. (No need to comment it out entirely. I think default is align=4 or some such, that's probably what you didn't like.) "buff resb 8192" and "mov cx, 8192" should be moved to EQU for clarity (and, even better, as "1024 * 8" constant expression). The program does not end in a CR+LF pair. Thus the output is an incomplete line. Not a huge deal but still (sometimes) noticeable. "int 21h // xchg ax, bp // int 21h" is repeated several times. If you really want to save space, put "msgquit:" before the first one and "jmp short msgquit" for the others (since this is quitting the program anyways). BTW, most asm devs actively hate "loop" in lieu of "dec // jnz". Not sure if this would really be worth it, even for your 8086. "shl bx, cl" (where CL=4) is also shunned, AFAIK, on 8086 machines, in lieu of speedier (times 4) "shl bx,1". But if it's only done extremely rarely then it won't add up to much difference. Only when done thousands of times would you barely even notice. Converting hex nibble to ASCII shouldn't need a jump at all. On the 8086 all jumps are very slow. Best to avoid them entirely if possible. Here you can easily use the old "cmp al, 0Ah // sbb al, 69h // das" trick instead. But since you're only printing hex one time (instead of thousands), you probably don't care. Okay, just wanted to add my $0.02 in case it was (accidentally) helpful. :-) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Rugxulo Hi, On Mon, Apr 10, 2017 at 9:52 PM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 17:07:30 -0500, Rugxulo wrote: > >> irrelevant "jz short ..." (when "short" conditional jump is always >> mandatory for "cpu 8086"). > > I don't think so. > Note that short means "8 bit jump" in this context, and NOT "16 bit jump". Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. 127 (signed) byte range. Hence the billions of workarounds (TASM "jumps", MASM "option ljmp", etc). >> "section .data align=1" is probably what you intended here. (No need to >> comment it out entirely. > > No need to have it either (not in tiny model). But you still have it commented out, so I assume you at least wanted it for descriptive purposes. >> The program does not end in a CR+LF pair. Thus the output is an >> incomplete line. Not a huge deal but still (sometimes) noticeable. > > True. I noticed that command.com adds a CR+LF pair whenever a program > doesn't end with those. This seems to be consistent with both FreeDOS and > MS-DOS, so I thought I'd exploit this to save a few bytes in the program. Most (but not all) FreeCOM versions do this too. But ... that won't work if you redirect the output to file. Then the CR+LF is (still) missing. Of course, if you really need a workaround, afterwards do "echo. >>bsum.out" and don't worry about it. (I still have at least one util with the same problem, but I didn't fix it yet either. Trivial but annoying. Some tools get confused by such incomplete lines.) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste On Mon, 10 Apr 2017 23:30:35 -0500, Rugxulo wrote: > Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. > 127 (signed) byte range. Hence the billions of workarounds (TASM > "jumps", MASM "option ljmp", etc). I won't argue about what opcode is or is not available on 8086, since I did not bother decoding their exact meaning. I do see however that (NASM at least) can assemble JZ and JZ SHORT in two very different forms, JZ SHORT being significantly shorter. 5 B80100 mov ax, 1 6 0003 48 dec ax 7 0004 746Ajz short gameover 5 B80100 mov ax, 1 6 0003 48 dec ax 7 0004 7503E9DD01 jz gameover Of course NASM always uses the short form whenever it's possible, but when the jump is too far away it silently uses the longer form, hence the need to always specify SHORT if one wants to be sure what's going on. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste I have to clarify here that my intention was never to compete in any way with the other algorithms out there. The BSD checksum is a well-known, and pretty weak (16 bits) checksum. The goal behind bsum was only to obtain a checksum tool that would run on my 8086 fast enough for me to not get frustrated, and just good enough to be reasonably sure that the files I just copied from a diskette and then over network-through- parallel-port didn't get corrupted in the process. Mateusz On Mon, 10 Apr 2017 09:48:41 -0700, Ralf Quint wrote: > On 4/10/2017 6:36 AM, Mateusz Viste wrote: >> On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: >>> It would be interesting to see some benchmark numbers for that (for >>> various specific tools, 8086, 386, etc). >> Just for the fun of it, I did some quick measures on my 386SX PC, >> computing various checksums of a 2 MiB file. Results below. >> >> BSUM (by Mateusz Viste) : 6.0s (100%) >> CRC32 (by Joe Forster) : 8.5s (70%) >> CRC32 (by Colin Plumb) : 26.7s (22%) >> MD5 (by Colin Plumb): 52.9s (11%) >> SHA1 (by Colin Plumb) : 85.7s (7%) >> >> BSUM is the fastest, which is no surprise since the algorithm is >> extremely simple (4 CPU instructions). The CRC32 computation by Joe >> Forster is surprisingly fast as well. It's 30% slower than bsum and the >> binary is 4x times larger (and I suppose the memory usage is also much >> higher) but that's still quite impressive for a 32-bit checksum. > Well, most of all, it's kind of comparing apples and oranges. Those > benchmark tests mean nothing if you don't compare them with the number > of possible collisions you get for each of them. > Though that doesn't mean that there aren't use cases where "simple does > it"... >>> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >>> with a bigger buffer. > Nope, won't do a thing. Didn't do much good "back in the days" to use > anything over 16KB and it is even less relevant on modern hard drives > with MBs of cache. Or SSDs... > > Ralf -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste On Tue, 11 Apr 2017 02:52:06 +, Mateusz Viste wrote: > On Mon, 10 Apr 2017 17:07:30 -0500, Rugxulo wrote: >> Converting hex nibble to ASCII shouldn't need a jump at all. On the >> 8086 all jumps are very slow. Best to avoid them entirely if possible. >> Here you can easily use the old "cmp al, 0Ah // sbb al, 69h // das" >> trick instead. But since you're only printing hex one time (instead of >> thousands), you probably don't care. > > Indeed, I care little about jumps there, but still your version might be > shorter, which would make it interesting. Will compare. Checked: your nibble-to-hex version is indeed smaller. Hence it's better than mine both in terms of space (by 3 bytes) and speed (no jump). Nice! Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
[Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste Hello, I needed to verify the integrity of a few files after transferring them to/from my 8086 PC the other day. The obvious method for such task is computing a checksum of the file, like MD5, SHA1, etc... However, on an 8086 this may take ages (even on a fairly fast 386, computing the MD5 sum of a 2 MiB file takes one minute). Since I don't like waiting, I created an alternative tool over the weekend: bsum. bsum is a tiny DOS tool that computes the BSD checksum of a file. It's very tiny: only 256 bytes (half of which is taken by the help screen), so it will easily fit in a single disk sector. A BSD checksum is obviously not as strong as MD5 or SHA1, but it's still more than enough for verifying whether or not a file got corrupted during a transfer. bsum is compatible with 8086 and requires only a few kilobytes of memory. Also, it's very fast. Homepage: http://bsum.sourceforge.net Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Dale E Sterner Would you or anyone else know if there is an 802.11 client for dos? Never heard of one but you guys know alot more than I ever will. cheers DS On Mon, 10 Apr 2017 13:36:24 + (UTC) Mateusz Viste writes: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: > > It would be interesting to see some benchmark numbers for that > (for > > various specific tools, 8086, 386, etc). > > Just for the fun of it, I did some quick measures on my 386SX PC, > computing various checksums of a 2 MiB file. Results below. > > BSUM (by Mateusz Viste) : 6.0s (100%) > CRC32 (by Joe Forster) : 8.5s (70%) > CRC32 (by Colin Plumb) : 26.7s (22%) > MD5 (by Colin Plumb): 52.9s (11%) > SHA1 (by Colin Plumb) : 85.7s (7%) > > BSUM is the fastest, which is no surprise since the algorithm is > extremely simple (4 CPU instructions). The CRC32 computation by Joe > Forster is surprisingly fast as well. It's 30% slower than bsum and > the > binary is 4x times larger (and I suppose the memory usage is also > much > higher) but that's still quite impressive for a 32-bit checksum. > > > Splurge on the memory, give it 32 kb or so. It'll "probably" be > faster > > with a bigger buffer. > > At the cost of reducing the number of platforms it would be able to > run > on. > Currently bsum uses an 8K memory buffer to optimize disk reads. > Using a > buffer of 64KB increases the overall speed by 10%. Not that much, > for a > 700% increase of memory usage. > > Mateusz > > > - - > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Freedos-user mailing list > Freedos-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/freedos-user > ** >From Dale Sterner - MS organic chemistry http://pubs.acs.org/doi/abs/10.1021/jo00975a052 *** Police Urge Americans to Carry This With Them at All Times The Observer http://thirdpartyoffers.juno.com/TGL3141/58eb98a73a29d18a74578st01duc -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Mateusz Viste On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: > It would be interesting to see some benchmark numbers for that (for > various specific tools, 8086, 386, etc). Just for the fun of it, I did some quick measures on my 386SX PC, computing various checksums of a 2 MiB file. Results below. BSUM (by Mateusz Viste) : 6.0s (100%) CRC32 (by Joe Forster) : 8.5s (70%) CRC32 (by Colin Plumb) : 26.7s (22%) MD5 (by Colin Plumb): 52.9s (11%) SHA1 (by Colin Plumb) : 85.7s (7%) BSUM is the fastest, which is no surprise since the algorithm is extremely simple (4 CPU instructions). The CRC32 computation by Joe Forster is surprisingly fast as well. It's 30% slower than bsum and the binary is 4x times larger (and I suppose the memory usage is also much higher) but that's still quite impressive for a 32-bit checksum. > Splurge on the memory, give it 32 kb or so. It'll "probably" be faster > with a bigger buffer. At the cost of reducing the number of platforms it would be able to run on. Currently bsum uses an 8K memory buffer to optimize disk reads. Using a buffer of 64KB increases the overall speed by 10%. Not that much, for a 700% increase of memory usage. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Ralf Quint On 4/10/2017 6:36 AM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: >> It would be interesting to see some benchmark numbers for that (for >> various specific tools, 8086, 386, etc). > Just for the fun of it, I did some quick measures on my 386SX PC, > computing various checksums of a 2 MiB file. Results below. > > BSUM (by Mateusz Viste) : 6.0s (100%) > CRC32 (by Joe Forster) : 8.5s (70%) > CRC32 (by Colin Plumb) : 26.7s (22%) > MD5 (by Colin Plumb): 52.9s (11%) > SHA1 (by Colin Plumb) : 85.7s (7%) > > BSUM is the fastest, which is no surprise since the algorithm is > extremely simple (4 CPU instructions). The CRC32 computation by Joe > Forster is surprisingly fast as well. It's 30% slower than bsum and the > binary is 4x times larger (and I suppose the memory usage is also much > higher) but that's still quite impressive for a 32-bit checksum. Well, most of all, it's kind of comparing apples and oranges. Those benchmark tests mean nothing if you don't compare them with the number of possible collisions you get for each of them. Though that doesn't mean that there aren't use cases where "simple does it"... >> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >> with a bigger buffer. Nope, won't do a thing. Didn't do much good "back in the days" to use anything over 16KB and it is even less relevant on modern hard drives with MBs of cache. Or SSDs... Ralf --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
From: Rugxulo Hi, I haven't tried this (yet), but nice work! On Sun, Apr 9, 2017 at 1:16 PM, Mateusz Viste wrote: > > I needed to verify the integrity of a few files after transferring them > to/from my 8086 PC the other day. The obvious method for such task is > computing a checksum of the file, like MD5, SHA1, etc... These days, MD5 and SHA1 are normally considered broken and obsolete, but they're still good for simple private checks against corruption. Having said that, I normally prefer MD5 myself, but in DOS I often (also) use CRC32, which is fairly universal (and used by archivers like ZIP). > However, on an 8086 this may take ages (even on a fairly fast 386, > computing the MD5 sum of a 2 MiB file takes one minute). It would be interesting to see some benchmark numbers for that (for various specific tools, 8086, 386, etc). I know Blair's (16-bit) MD5SUM is usually half the speed of DOS386's FBMD5 (32-bit). Of course that can vary by cpu family and other factors. Also, like all things, I'm sure there's plenty of room for improvement. AFAIK, the 386 (often with little or no cache?) preferred much smaller code (similar to 8086) vs. 486's pipelined way of preferring simpler instructions. The 486 was also allegedly very sensitive to alignment. I'm not sure many compilers truly took full advantage of those specific cpus. > Since I don't like waiting, I created an alternative tool over the weekend: bsum. > > bsum is a tiny DOS tool that computes the BSD checksum of a file. It's > very tiny: only 256 bytes (half of which is taken by the help screen), so > it will easily fit in a single disk sector. A BSD checksum is obviously > not as strong as MD5 or SHA1, but it's still more than enough for > verifying whether or not a file got corrupted during a transfer. > > bsum is compatible with 8086 and requires only a few kilobytes of memory. > Also, it's very fast. Splurge on the memory, give it 32 kb or so. It'll "probably" be faster with a bigger buffer. > Homepage: http://bsum.sourceforge.net Sounds good. Although I admit to being mostly unfamiliar with BSD cksum. I did recently try to mirror some CRC32 tools to iBiblio, just for completeness. http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/crc32/ Charles Dye's CHKSUM is fairly feature-packed (albeit only CRC32 and DR-DOS XDIR sums), roughly 5 kb. The other CRC32 util is a very simplistic (but good) .COM of roughly 1 kb, which is what I often use in a pinch (mostly due to its small size). And I think the author of that one still frequents FreeDOS mailing lists. Again, feel free to benchmark some of these, and tell us the results. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user --- Internet Rex 2.29 * Origin: capcity2.synchro.net - 502/875-8938 (276:10/901) --- Synchronet 3.15a-Linux ListGate 1.3 * Capitol City Online - Frankfort, KY - telnet://capitolcityonline.net -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Wed, 12 Apr 2017 04:39:35 -0500, Rugxulo wrote: > It worked fine (redirecting) for me yesterday! I can't imagine why it > wouldn't work for you. > > Oh, before I forget, are you perhaps invoking NDISASM via some .BAT? > Of course a .BAT doesn't really redirect (under FreeCOM) without kludge, > e.g. "%COMSPEC% /c". Ha, yes - you nailed it, Rugxulo. Indeed I was calling ndisasm from within a sneaky ndisasm.bat file (auto-generated by FDNPKG inside my PATH). As said before, I didn't really investigate the problem (for a reason that should be obvious by now) - saw it doesn't work out of the box, went to the "-l" nasm listing, done. But thanks to you the reason is clear now. It's at least the second time this "FDNPKG generates BAT files" thing bites me. I should definitively address this problem eventually. I have to say however that I prefer the "-l" listing anyway, because it preserves the comments (even though it lies a little bit sometimes on the JZ thing). > Maybe you should use Lazy Assembler (LZASM)?! Didn't know that one (too lazy to google it out), but it does sound extremely well suited to me! cheers, Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
> I double-checked, latest NDISASM still decodes as two separate instructions. A disassembler would not report two separate instructions unless the actual compiled code had two separate instructions. Every assembler/compiler I know of does little "tricks" to make the programmer's job a little easier (basically, changes subtle/minor things behind your back). They also all seem to do different "tricks" (even in different versions of the same assembler/compiler), so even in ASM you're not really 100% in control of the resulting code. In ASM you're much more in control than you are in any high-level language, though. > The simple answer is that code size is rarely as important as programmer > convenience. There are really several different major things that you must balance: program size, memory footprint, speed, and maintainability. "Programmer convenience" is just a subset of maintainability, which also includes things like program structure, language/compiler/assembler/libraries chosen, documentation, and code commenting. Which of the items has the highest precedence depends on circumstances and goals at the time. TSR's and device drivers, e.g., are very different than foreground programs -- memory footprint really matters (a lot) in TSR's and device drivers, though not necessarily so much in foreground programs (at least not in all foreground programs). It also depends on your intended target audience/hardware. Modern CPU's also do all kinds of "tricks" to increase speed (caching, pipelining, branch prediction, virtual machines/CPU's/memory, memory alignment issues, etc.), so code that is faster on a real 8086/8088 CPU may actually be (relatively) slower in a different environment. Looking up clock-cycles-per-CPU-instruction is a guideline, but not the final answer as to how fast something really is. Smaller size tends to equate to faster speed, though that's not absolute. It is true that a smaller memory footprint is more likely to remain in the CPU cache(s) (at least on CPU's that have caches and have them enabled), so a smaller memory footprint always increases speed (or at least the likelihood of speed) in that sense. -- Bret Johnson There is only one place in the universe where 0+0=1. That is in the mind of an evolutionist. 'Erin' From Happy Days Looks Like Now Is Insane trend-chaser.com http://thirdpartyoffers.juno.com/TGL3141/58ee56e68d39656e6565bst03vuc -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Hi, On Wed, Apr 12, 2017 at 2:03 AM, Mateusz Viste wrote: > On Tue, 11 Apr 2017 22:24:56 -0500, Rugxulo wrote: > >> What disassembler are you using here? I erroneously thought it was NDISASM. > > I don't use ndisasm for a very trivial reason - I am unable to redirect > its output to a file, so I don't really know how other people use it It worked fine (redirecting) for me yesterday! I can't imagine why it wouldn't work for you. > I didn't figure out any quick and easy workaround (again, too stupid). Um, excuse me, but he called you "lazy" and "clueless", not "stupid". I guess we should add "forgetful". ;-)) j/k But I'll point to this anyways, "redir", just for a general tip (in rare case you didn't already know): http://www.delorie.com/djgpp/doc/utils/utils_7.html Oh, before I forget, are you perhaps invoking NDISASM via some .BAT? Of course a .BAT doesn't really redirect (under FreeCOM) without kludge, e.g. "%COMSPEC% /c". > The output I pasted before was copied from the NASM listing (-l). Hmmm, then NASM is being a bit too tricky for its own good. I do (very naively!) wonder whether "warning: 8086 conditional jump extended" would be appropriate. Actually, having "[386]" (etc) in NDISASM output would be nice. (The only workarounds for that are BIEW and QVIEW, IIRC both of which color-code various instructions. Not sure about various debuggers off the top of my head.) > And although I do look at the listing carefully, I do not bother decoding the > opcodes by hand (too lazy!), Maybe you should use Lazy Assembler (LZASM)?!:-P Nah, it needs a separate linker, even for .COM (bah, too slow, we're too lazy!). > I assume that the assembler knows how to > encode mnemonics into opcodes - that's his job after all, not mine. > Ultimately, whether the code is assembled into a "long, 5-byte form of > jump" or "two separate instructions that emulate a jump" is irrelevant to > me - in both cases it's still 5 bytes, that all I need to know. I can't even honestly complain, it's indeed a "feature", not a bug! Not mandatory but certainly nice to have. >> The simple answer is that code size is rarely as important as programmer >> convenience. > > Maybe. But why bother doing assembly then, if not for the control over > what machine code is generated at the end? I was trying to imagine thinking like them, not speaking for myself. I personally like size optimizations in assembly (obviously??). E.g. "add si,2" is three bytes but (times 2) "inc si" is only two! But you won't see a lot of programs that actively try to save such few bytes. Nobody cares. (Well, most other people!) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Tue, 11 Apr 2017 22:24:56 -0500, Rugxulo wrote: > My problem with always explicitly saying "short" is that it's both > unnecessary and verbose. Apparently not so "unnecessary" after all, if one really wants to use a short jump, and not some other contraption. > What disassembler are you using here? I erroneously thought it was > NDISASM. I don't use ndisasm for a very trivial reason - I am unable to redirect its output to a file, so I don't really know how other people use it, and I didn't figure out any quick and easy workaround (again, too stupid). The output I pasted before was copied from the NASM listing (-l). And although I do look at the listing carefully, I do not bother decoding the opcodes by hand (too lazy!), I assume that the assembler knows how to encode mnemonics into opcodes - that's his job after all, not mine. Ultimately, whether the code is assembled into a "long, 5-byte form of jump" or "two separate instructions that emulate a jump" is irrelevant to me - in both cases it's still 5 bytes, that all I need to know. > The simple answer is that code size is rarely as important as programmer > convenience. Maybe. But why bother doing assembly then, if not for the control over what machine code is generated at the end? Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Hi, On Tue, Apr 11, 2017 at 7:06 AM, Mateusz Viste wrote: > > That's nice of you to provide the explanation. I didn't read it > completely (too lazy), nor understand it fully (too stupid), but the > other clueless guy might find it interesting that he was only half wrong. Just to clarify, I'll quote from the A86 text manual: " NOTE that for A86, LONG will have effect only on the operand to an unconditional JMP instruction; not to conditional jumps. That is because conditional jumps farther than 127 bytes are available only on the 386 and later processors. If you run into this problem, then chances are your code is getting out of control--time to rearrange, or to break off some of the intervening code into separate procedures. If you insist upon leaving the code intact, you can replace the conditional jump with an 'IF cond JMP'. " > At the end of the day, I will keep using "JZ SHORT" anyway, 'cause that > just works for me. My problem with always explicitly saying "short" is that it's both unnecessary and verbose. But I (reluctantly) agree that here it's your best option for safety. (Though I would still hide it behind a preprocessor macro or whatever, for brevity.) What disassembler are you using here? I erroneously thought it was NDISASM. I also misremembered NDISASM as saying "jz near" (only) for 386+ jumps. I also misremembered such near/long jumps as being five bytes when they are, in fact, only four. I halfway remembered that the 386+ jz opcode started with 0Fh (sorry, Tom) but wasn't sure offhand without checking. But here your disassembler apparently put two jumps on one line, as if only one, which is weird. I know it's a common idiom, but I didn't think a literally disassembler would use that! (I double-checked, latest NDISASM still decodes as two separate instructions.) 0Fh is actually the opcode to (discontinued, 8086-only) "POP CS". Both FASM and NASM apparently support that (why?), but A86 apparently doesn't. Part of the confusion also lies with NASM. In the old days, you had to explicitly enable -O3 to enable this 8086 conditional jump workaround, e.g. 0.98.39 (by looking at old code of mine). NASM 2.09 started enabling -Ox [sic] by default. However, it appears that even with -O0 nowadays, if you're using "cpu 8086", it will still extend your conditional jumps in this manner. It didn't use to do that automatically, nor for -O0. The simple answer is that code size is rarely as important as programmer convenience. Sure, if you're trying to squeeze every last byte, you'll maybe accidentally be bit by this. But most people, by far, don't care about code size and only care about functionality. I'm only guessing, but that's my impression as to why NASM would always do this by default nowadays. P.S. http://stevemorse.org/8086/ is also an interesting read. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Less than 256 bytes of code. Over 2 Megabytes of controversy. Shame on you Mateusz! :-) All joking aside, from what I gather, your bsum is only meant to verify local files were copied correctly from one machine to another. Most forms of checksums should be sufficient for that task. Sure a collision is possible. More bits in the checksum reduces the possibility. But even with a SHA 256 hash, the chance of accidental collision is so remote as to be zero. It is still theoretically possible. How sure do you need to be? Well, you could be absolutely sure by coping them over then back. Then doing a byte level comparison between the original and the copy of a copy. But why? Now as for a man in the middle of your two machines... Even that check could be thwarted by a crafty attacker. So, I assume that the checking done by you bsum is the exact level of certainty you desire. Sure enough. Jerome -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Hi Tom, That's nice of you to provide the explanation. I didn't read it completely (too lazy), nor understand it fully (too stupid), but the other clueless guy might find it interesting that he was only half wrong. At the end of the day, I will keep using "JZ SHORT" anyway, 'cause that just works for me. cheers, Mateusz On Tue, 11 Apr 2017 12:33:17 +0200, Tom Ehlert wrote: Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. 127 (signed) byte range. Hence the billions of workarounds (TASM "jumps", MASM "option ljmp", etc). > right. > >>> I won't argue about what opcode is or is not available on 8086, since >>> I did not bother decoding their exact meaning. > > meaning 'I am a lazy, clueless guy, but write anyway ...' > >>> I do see however that (NASM at least) can assemble JZ and JZ SHORT in >>> two very different forms, JZ SHORT being significantly shorter. >>> >>> 5 B80100 mov ax, 1 6 0003 48 dec >>> ax 7 0004 746Ajz short gameover >>> >>> 5 B80100 mov ax, 1 6 0003 48 dec >>> ax 7 0004 7503E9DD01 jz gameover >>> >>> Of course NASM always uses the short form whenever it's possible, but >>> when the jump is too far away it silently uses the longer form, hence >>> the need to always specify SHORT if one wants to be sure what's going >>> on. > >> AFAIK, > meaning 'I am completely clueless , but offer my unfounded opinion > anyway ...' > >> the longer one is 386+ only, hence not available with "cpu 8086". Thus, >> if it still quietly assembles, that is a bug (but I thought that was >> long-ago fixed/avoided). > > the longer one is 2 instructions instead, automatically generated by > NASM because the intended jump goes farther then 127 bte > > > c:\>debug -e 100 1430:0100 00.75 00.03 00.e9 00.dd 00.01 -u 100 > 1430:0100 7503 JNZ 0105 1430:0102 E9DD01JMP > 02E2 > > > Tom -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
>>> Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. >>> 127 (signed) byte range. Hence the billions of workarounds (TASM >>> "jumps", MASM "option ljmp", etc). right. >> I won't argue about what opcode is or is not available on 8086, since I >> did not bother decoding their exact meaning. meaning 'I am a lazy, clueless guy, but write anyway ...' >> I do see however that (NASM >> at least) can assemble JZ and JZ SHORT in two very different forms, JZ >> SHORT being significantly shorter. >> >> 5 B80100 mov ax, 1 >> 6 0003 48 dec ax >> 7 0004 746Ajz short gameover >> >> 5 B80100 mov ax, 1 >> 6 0003 48 dec ax >> 7 0004 7503E9DD01 jz gameover >> >> Of course NASM always uses the short form whenever it's possible, but >> when the jump is too far away it silently uses the longer form, hence the >> need to always specify SHORT if one wants to be sure what's going on. > AFAIK, meaning 'I am completely clueless , but offer my unfounded opinion anyway ...' > the longer one is 386+ only, hence not available with "cpu > 8086". Thus, if it still quietly assembles, that is a bug (but I > thought that was long-ago fixed/avoided). the longer one is 2 instructions instead, automatically generated by NASM because the intended jump goes farther then 127 bte c:\>debug -e 100 1430:0100 00.75 00.03 00.e9 00.dd 00.01 -u 100 1430:0100 7503 JNZ 0105 1430:0102 E9DD01JMP 02E2 Tom -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Tue, 11 Apr 2017 02:03:54 -0500, Rugxulo wrote: > AFAIK, the longer one is 386+ only, hence not available with "cpu 8086". The above code assembles with "cpu 8086" (NASM 2.12.02). > Thus, if it still quietly assembles, that is a bug (but I thought that > was long-ago fixed/avoided). Perhaps a bug, didn't investigate. My point is - explicitly mentioning SHORT is always a good idea. Better safe than sorry. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Hi, On Tue, Apr 11, 2017 at 1:26 AM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 23:30:35 -0500, Rugxulo wrote: >> >> Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. >> 127 (signed) byte range. Hence the billions of workarounds (TASM >> "jumps", MASM "option ljmp", etc). > > I won't argue about what opcode is or is not available on 8086, since I > did not bother decoding their exact meaning. I do see however that (NASM > at least) can assemble JZ and JZ SHORT in two very different forms, JZ > SHORT being significantly shorter. > > 5 B80100 mov ax, 1 > 6 0003 48 dec ax > 7 0004 746Ajz short gameover > > 5 B80100 mov ax, 1 > 6 0003 48 dec ax > 7 0004 7503E9DD01 jz gameover > > Of course NASM always uses the short form whenever it's possible, but > when the jump is too far away it silently uses the longer form, hence the > need to always specify SHORT if one wants to be sure what's going on. AFAIK, the longer one is 386+ only, hence not available with "cpu 8086". Thus, if it still quietly assembles, that is a bug (but I thought that was long-ago fixed/avoided). -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Mon, 10 Apr 2017 23:30:35 -0500, Rugxulo wrote: > Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. > 127 (signed) byte range. Hence the billions of workarounds (TASM > "jumps", MASM "option ljmp", etc). I won't argue about what opcode is or is not available on 8086, since I did not bother decoding their exact meaning. I do see however that (NASM at least) can assemble JZ and JZ SHORT in two very different forms, JZ SHORT being significantly shorter. 5 B80100 mov ax, 1 6 0003 48 dec ax 7 0004 746Ajz short gameover 5 B80100 mov ax, 1 6 0003 48 dec ax 7 0004 7503E9DD01 jz gameover Of course NASM always uses the short form whenever it's possible, but when the jump is too far away it silently uses the longer form, hence the need to always specify SHORT if one wants to be sure what's going on. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Hi, On Mon, Apr 10, 2017 at 9:52 PM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 17:07:30 -0500, Rugxulo wrote: > >> irrelevant "jz short ..." (when "short" conditional jump is always >> mandatory for "cpu 8086"). > > I don't think so. > Note that short means "8 bit jump" in this context, and NOT "16 bit jump". Unless I'm mistaken, conditional jumps on 8086 don't go beyond -128 .. 127 (signed) byte range. Hence the billions of workarounds (TASM "jumps", MASM "option ljmp", etc). >> "section .data align=1" is probably what you intended here. (No need to >> comment it out entirely. > > No need to have it either (not in tiny model). But you still have it commented out, so I assume you at least wanted it for descriptive purposes. >> The program does not end in a CR+LF pair. Thus the output is an >> incomplete line. Not a huge deal but still (sometimes) noticeable. > > True. I noticed that command.com adds a CR+LF pair whenever a program > doesn't end with those. This seems to be consistent with both FreeDOS and > MS-DOS, so I thought I'd exploit this to save a few bytes in the program. Most (but not all) FreeCOM versions do this too. But ... that won't work if you redirect the output to file. Then the CR+LF is (still) missing. Of course, if you really need a workaround, afterwards do "echo. >>bsum.out" and don't worry about it. (I still have at least one util with the same problem, but I didn't fix it yet either. Trivial but annoying. Some tools get confused by such incomplete lines.) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
I have to clarify here that my intention was never to compete in any way with the other algorithms out there. The BSD checksum is a well-known, and pretty weak (16 bits) checksum. The goal behind bsum was only to obtain a checksum tool that would run on my 8086 fast enough for me to not get frustrated, and just good enough to be reasonably sure that the files I just copied from a diskette and then over network-through- parallel-port didn't get corrupted in the process. Mateusz On Mon, 10 Apr 2017 09:48:41 -0700, Ralf Quint wrote: > On 4/10/2017 6:36 AM, Mateusz Viste wrote: >> On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: >>> It would be interesting to see some benchmark numbers for that (for >>> various specific tools, 8086, 386, etc). >> Just for the fun of it, I did some quick measures on my 386SX PC, >> computing various checksums of a 2 MiB file. Results below. >> >> BSUM (by Mateusz Viste) : 6.0s (100%) >> CRC32 (by Joe Forster) : 8.5s (70%) >> CRC32 (by Colin Plumb) : 26.7s (22%) >> MD5 (by Colin Plumb): 52.9s (11%) >> SHA1 (by Colin Plumb) : 85.7s (7%) >> >> BSUM is the fastest, which is no surprise since the algorithm is >> extremely simple (4 CPU instructions). The CRC32 computation by Joe >> Forster is surprisingly fast as well. It's 30% slower than bsum and the >> binary is 4x times larger (and I suppose the memory usage is also much >> higher) but that's still quite impressive for a 32-bit checksum. > Well, most of all, it's kind of comparing apples and oranges. Those > benchmark tests mean nothing if you don't compare them with the number > of possible collisions you get for each of them. > Though that doesn't mean that there aren't use cases where "simple does > it"... >>> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >>> with a bigger buffer. > Nope, won't do a thing. Didn't do much good "back in the days" to use > anything over 16KB and it is even less relevant on modern hard drives > with MBs of cache. Or SSDs... > > Ralf -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Tue, 11 Apr 2017 02:52:06 +, Mateusz Viste wrote: > On Mon, 10 Apr 2017 17:07:30 -0500, Rugxulo wrote: >> Converting hex nibble to ASCII shouldn't need a jump at all. On the >> 8086 all jumps are very slow. Best to avoid them entirely if possible. >> Here you can easily use the old "cmp al, 0Ah // sbb al, 69h // das" >> trick instead. But since you're only printing hex one time (instead of >> thousands), you probably don't care. > > Indeed, I care little about jumps there, but still your version might be > shorter, which would make it interesting. Will compare. Checked: your nibble-to-hex version is indeed smaller. Hence it's better than mine both in terms of space (by 3 bytes) and speed (no jump). Nice! Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Mon, 10 Apr 2017 17:07:30 -0500, Rugxulo wrote:
> Blair's (16-bit, FD) MD5SUM can do all of those hashes as well. Not sure
> if it'd be faster, though.
I believe that's the one I used. If I understand correctly, the original
author is Colin Plumb, and Blair took the maintenance of it at some point.
> Believe me, shrinking size is fairly easy,
If you say so.
> but it's a tradeoff in accidental errors, readability,
> and speed.
Unless it's a goal in itself ("keep the whole thing in 256 bytes"), as is
the case of bsum.
> Irrelevant aesthetics: lines too long (shouldn't be more than 80),
I'll skip all aesthetics remarks, since these are a rather personal thing.
> irrelevant "jz short ..." (when "short" conditional jump is always
> mandatory for "cpu 8086").
I don't think so.
Note that short means "8 bit jump" in this context, and NOT "16 bit jump".
> "section .data align=1" is probably what you intended here. (No need to
> comment it out entirely.
No need to have it either (not in tiny model).
> The program does not end in a CR+LF pair. Thus the output is an
> incomplete line. Not a huge deal but still (sometimes) noticeable.
True. I noticed that command.com adds a CR+LF pair whenever a program
doesn't end with those. This seems to be consistent with both FreeDOS and
MS-DOS, so I thought I'd exploit this to save a few bytes in the program.
> "int 21h // xchg ax, bp // int 21h" is repeated several times. If you
> really want to save space, put "msgquit:" before the first one and "jmp
> short msgquit" for the others (since this is quitting the program
> anyways).
Indeed, that would save 1 byte or 2. Good catch.
> BTW, most asm devs actively hate "loop" in lieu of "dec // jnz". Not
> sure if this would really be worth it, even for your 8086.
Actually my trunk version (svn) does avoid loop in favor of dec/jnz.
The former is shorter by one byte, but 3 times slower than the latter
(5/6 clks vs 2 clks).
> "shl bx, cl" (where CL=4) is also shunned, AFAIK, on 8086 machines, in
> lieu of speedier (times 4) "shl bx,1".
But repeated shl bx,1 is so much bigger. I definitely prefer shl bx,cl,
at least whenever not in performance-critical parts.
> Converting hex nibble to ASCII shouldn't need a jump at all. On the 8086
> all jumps are very slow. Best to avoid them entirely if possible.
> Here you can easily use the old "cmp al, 0Ah // sbb al, 69h // das"
> trick instead. But since you're only printing hex one time (instead of
> thousands), you probably don't care.
Indeed, I care little about jumps there, but still your version might be
shorter, which would make it interesting. Will compare.
Mateusz
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Freedos-user mailing list
Freedos-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Mon, 10 Apr 2017 17:57:59 -0400, dmccunney wrote: > I have to ask. How many folks *have* platforms now it *wouldn't* run > on? I suspect the number is *very* small. Surely, yes. Still, a 700% memory increase for a 10% performance boost just doesn't feel right. I wrote bsum to cover an extreme case - in such context I prefer keeping the memory footprint as small as possible. > Most folks now are trying to get FreeDOS to boot native on a machine > rather larger and more powerful than the machines DOS was used on, or > running it in a VM I'd say that for these machines bsum is irrelevant - they are much better off using md5 or anything else. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Hi again, On Mon, Apr 10, 2017 at 8:36 AM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: >> >> It would be interesting to see some benchmark numbers for that (for >> various specific tools, 8086, 386, etc). > > Just for the fun of it, I did some quick measures on my 386SX PC, > computing various checksums of a 2 MiB file. Results below. Very interesting > CRC32 (by Colin Plumb) : 26.7s (22%) > MD5 (by Colin Plumb): 52.9s (11%) > SHA1 (by Colin Plumb) : 85.7s (7%) Blair's (16-bit, FD) MD5SUM can do all of those hashes as well. Not sure if it'd be faster, though. > BSUM is the fastest, which is no surprise since the algorithm is > extremely simple (4 CPU instructions). The CRC32 computation by Joe > Forster is surprisingly fast as well. It's 30% slower than bsum and the > binary is 4x times larger (and I suppose the memory usage is also much > higher) but that's still quite impressive for a 32-bit checksum. "30% slower" is machine specific, and I'm quite sure it can be improved. Although his tool does seem to use a fairly big (64512 byte) buffer. *** If extremely bored, check out these "modern" (CRC32C, aka Castagnoli) implementations, which I don't grok: http://stackoverflow.com/questions/17645167/implementing-sse-4-2s-crc32c-in-software http://www.drdobbs.com/parallel/fast-parallelized-crc-computation-using/229401411 *** Of course he also combines (unused) decimal output routine with (used) hex output routine, which unnecessarily (in this case) always uses slow DIV (which you don't need at all for converting to hex). Of course he only needs to call that routine once at the end. It would be much worse result if called more often (e.g. hundreds of times). I've done the same mistake in the past, too. "4x times larger" is only in raw bytes, but in reality it uses a full cluster (as you well know), so even a 256 byte .COM will still use minimum one cluster (e.g. 512 bytes on 1.44 MB floppy). So 1024 isn't really much worse than 512. ;-)Believe me, shrinking size is fairly easy, but it's a tradeoff in accidental errors, readability, and speed. >> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >> with a bigger buffer. > > At the cost of reducing the number of platforms it would be able to run on. > Currently bsum uses an 8K memory buffer to optimize disk reads. Using a > buffer of 64KB increases the overall speed by 10%. Not that much, for a > 700% increase of memory usage. Don't you have an 8086 machine? How much RAM does it have? I had thought most had at least 64 kb of RAM, but I guess that's not accounting for the DOS + shell overhead. Honestly, I wrote several simple hexdump variants in recent months, and the biggest slowdown was my small buffer (only 16 bytes in the .ASM version). The C version is larger but always well-buffered, so it's the fastest. I even got 2x speedup (and noticeable size decrease) by avoiding printf entirely and using my own outhex routine. Okay, so let me break down your source and give some (trivial) comments here. I assume that's okay with you! ;-) Irrelevant aesthetics: lines too long (shouldn't be more than 80), not enough indentation (instructions vs. labels), irrelevant "jz short ..." (when "short" conditional jump is always mandatory for "cpu 8086"). "section .data align=1" is probably what you intended here. (No need to comment it out entirely. I think default is align=4 or some such, that's probably what you didn't like.) "buff resb 8192" and "mov cx, 8192" should be moved to EQU for clarity (and, even better, as "1024 * 8" constant expression). The program does not end in a CR+LF pair. Thus the output is an incomplete line. Not a huge deal but still (sometimes) noticeable. "int 21h // xchg ax, bp // int 21h" is repeated several times. If you really want to save space, put "msgquit:" before the first one and "jmp short msgquit" for the others (since this is quitting the program anyways). BTW, most asm devs actively hate "loop" in lieu of "dec // jnz". Not sure if this would really be worth it, even for your 8086. "shl bx, cl" (where CL=4) is also shunned, AFAIK, on 8086 machines, in lieu of speedier (times 4) "shl bx,1". But if it's only done extremely rarely then it won't add up to much difference. Only when done thousands of times would you barely even notice. Converting hex nibble to ASCII shouldn't need a jump at all. On the 8086 all jumps are very slow. Best to avoid them entirely if possible. Here you can easily use the old "cmp al, 0Ah // sbb al, 69h // das" trick instead. But since you're only printing hex one time (instead of thousands), you probably don't care. Okay, just wanted to add my $0.02 in case it was (accidentally) helpful. :-) -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Mon, Apr 10, 2017 at 9:36 AM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: > >> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >> with a bigger buffer. > > At the cost of reducing the number of platforms it would be able to run > on. I have to ask. How many folks *have* platforms now it *wouldn't* run on? I suspect the number is *very* small. (Most folks now are trying to get FreeDOS to boot native on a machine rather larger and more powerful than the machines DOS was used on, or running it in a VM. Even folks doing embedded development on IoT devices are probably dealing with fast full 32bit CPUs with more than enough RAM and external storage, and can run a Linux kernel or an RTOS that bears no resemblance to DOS.) > Mateusz __ Dennis https://plus.google.com/u/0/105128793974319004519 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On 4/10/2017 6:36 AM, Mateusz Viste wrote: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: >> It would be interesting to see some benchmark numbers for that (for >> various specific tools, 8086, 386, etc). > Just for the fun of it, I did some quick measures on my 386SX PC, > computing various checksums of a 2 MiB file. Results below. > > BSUM (by Mateusz Viste) : 6.0s (100%) > CRC32 (by Joe Forster) : 8.5s (70%) > CRC32 (by Colin Plumb) : 26.7s (22%) > MD5 (by Colin Plumb): 52.9s (11%) > SHA1 (by Colin Plumb) : 85.7s (7%) > > BSUM is the fastest, which is no surprise since the algorithm is > extremely simple (4 CPU instructions). The CRC32 computation by Joe > Forster is surprisingly fast as well. It's 30% slower than bsum and the > binary is 4x times larger (and I suppose the memory usage is also much > higher) but that's still quite impressive for a 32-bit checksum. Well, most of all, it's kind of comparing apples and oranges. Those benchmark tests mean nothing if you don't compare them with the number of possible collisions you get for each of them. Though that doesn't mean that there aren't use cases where "simple does it"... >> Splurge on the memory, give it 32 kb or so. It'll "probably" be faster >> with a bigger buffer. Nope, won't do a thing. Didn't do much good "back in the days" to use anything over 16KB and it is even less relevant on modern hard drives with MBs of cache. Or SSDs... Ralf --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Would you or anyone else know if there is an 802.11 client for dos? Never heard of one but you guys know alot more than I ever will. cheers DS On Mon, 10 Apr 2017 13:36:24 + (UTC) Mateusz Viste writes: > On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: > > It would be interesting to see some benchmark numbers for that > (for > > various specific tools, 8086, 386, etc). > > Just for the fun of it, I did some quick measures on my 386SX PC, > computing various checksums of a 2 MiB file. Results below. > > BSUM (by Mateusz Viste) : 6.0s (100%) > CRC32 (by Joe Forster) : 8.5s (70%) > CRC32 (by Colin Plumb) : 26.7s (22%) > MD5 (by Colin Plumb): 52.9s (11%) > SHA1 (by Colin Plumb) : 85.7s (7%) > > BSUM is the fastest, which is no surprise since the algorithm is > extremely simple (4 CPU instructions). The CRC32 computation by Joe > Forster is surprisingly fast as well. It's 30% slower than bsum and > the > binary is 4x times larger (and I suppose the memory usage is also > much > higher) but that's still quite impressive for a 32-bit checksum. > > > Splurge on the memory, give it 32 kb or so. It'll "probably" be > faster > > with a bigger buffer. > > At the cost of reducing the number of platforms it would be able to > run > on. > Currently bsum uses an 8K memory buffer to optimize disk reads. > Using a > buffer of 64KB increases the overall speed by 10%. Not that much, > for a > 700% increase of memory usage. > > Mateusz > > > - - > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Freedos-user mailing list > Freedos-user@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/freedos-user > ** >From Dale Sterner - MS organic chemistry http://pubs.acs.org/doi/abs/10.1021/jo00975a052 *** Police Urge Americans to Carry This With Them at All Times The Observer http://thirdpartyoffers.juno.com/TGL3141/58eb98a73a29d18a74578st01duc -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
On Mon, 10 Apr 2017 00:56:17 -0500, Rugxulo wrote: > It would be interesting to see some benchmark numbers for that (for > various specific tools, 8086, 386, etc). Just for the fun of it, I did some quick measures on my 386SX PC, computing various checksums of a 2 MiB file. Results below. BSUM (by Mateusz Viste) : 6.0s (100%) CRC32 (by Joe Forster) : 8.5s (70%) CRC32 (by Colin Plumb) : 26.7s (22%) MD5 (by Colin Plumb): 52.9s (11%) SHA1 (by Colin Plumb) : 85.7s (7%) BSUM is the fastest, which is no surprise since the algorithm is extremely simple (4 CPU instructions). The CRC32 computation by Joe Forster is surprisingly fast as well. It's 30% slower than bsum and the binary is 4x times larger (and I suppose the memory usage is also much higher) but that's still quite impressive for a 32-bit checksum. > Splurge on the memory, give it 32 kb or so. It'll "probably" be faster > with a bigger buffer. At the cost of reducing the number of platforms it would be able to run on. Currently bsum uses an 8K memory buffer to optimize disk reads. Using a buffer of 64KB increases the overall speed by 10%. Not that much, for a 700% increase of memory usage. Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] bsum - compute BSD checksums of your files
Hi, I haven't tried this (yet), but nice work! On Sun, Apr 9, 2017 at 1:16 PM, Mateusz Viste wrote: > > I needed to verify the integrity of a few files after transferring them > to/from my 8086 PC the other day. The obvious method for such task is > computing a checksum of the file, like MD5, SHA1, etc... These days, MD5 and SHA1 are normally considered broken and obsolete, but they're still good for simple private checks against corruption. Having said that, I normally prefer MD5 myself, but in DOS I often (also) use CRC32, which is fairly universal (and used by archivers like ZIP). > However, on an 8086 this may take ages (even on a fairly fast 386, > computing the MD5 sum of a 2 MiB file takes one minute). It would be interesting to see some benchmark numbers for that (for various specific tools, 8086, 386, etc). I know Blair's (16-bit) MD5SUM is usually half the speed of DOS386's FBMD5 (32-bit). Of course that can vary by cpu family and other factors. Also, like all things, I'm sure there's plenty of room for improvement. AFAIK, the 386 (often with little or no cache?) preferred much smaller code (similar to 8086) vs. 486's pipelined way of preferring simpler instructions. The 486 was also allegedly very sensitive to alignment. I'm not sure many compilers truly took full advantage of those specific cpus. > Since I don't like waiting, I created an alternative tool over the weekend: > bsum. > > bsum is a tiny DOS tool that computes the BSD checksum of a file. It's > very tiny: only 256 bytes (half of which is taken by the help screen), so > it will easily fit in a single disk sector. A BSD checksum is obviously > not as strong as MD5 or SHA1, but it's still more than enough for > verifying whether or not a file got corrupted during a transfer. > > bsum is compatible with 8086 and requires only a few kilobytes of memory. > Also, it's very fast. Splurge on the memory, give it 32 kb or so. It'll "probably" be faster with a bigger buffer. > Homepage: http://bsum.sourceforge.net Sounds good. Although I admit to being mostly unfamiliar with BSD cksum. I did recently try to mirror some CRC32 tools to iBiblio, just for completeness. http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/util/file/crc32/ Charles Dye's CHKSUM is fairly feature-packed (albeit only CRC32 and DR-DOS XDIR sums), roughly 5 kb. The other CRC32 util is a very simplistic (but good) .COM of roughly 1 kb, which is what I often use in a pinch (mostly due to its small size). And I think the author of that one still frequents FreeDOS mailing lists. Again, feel free to benchmark some of these, and tell us the results. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
[Freedos-user] bsum - compute BSD checksums of your files
Hello, I needed to verify the integrity of a few files after transferring them to/from my 8086 PC the other day. The obvious method for such task is computing a checksum of the file, like MD5, SHA1, etc... However, on an 8086 this may take ages (even on a fairly fast 386, computing the MD5 sum of a 2 MiB file takes one minute). Since I don't like waiting, I created an alternative tool over the weekend: bsum. bsum is a tiny DOS tool that computes the BSD checksum of a file. It's very tiny: only 256 bytes (half of which is taken by the help screen), so it will easily fit in a single disk sector. A BSD checksum is obviously not as strong as MD5 or SHA1, but it's still more than enough for verifying whether or not a file got corrupted during a transfer. bsum is compatible with 8086 and requires only a few kilobytes of memory. Also, it's very fast. Homepage: http://bsum.sourceforge.net Mateusz -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
