[Freedos-user] Test UIDE Available, For VirtualBox.
Apologies for my late response re: UIDE v.s. VirtualBox. Had to have my (infected!) gall-bladder removed, 25-Jun-2011. NASTY episode, and I am still recovering and moving a bit slow on driver work! Re: recent EDR-DOS forum posts, I say the same as our author Mark Twain once did -- The rumors of my demise are greatly exaggerated! As I have noted, UIDE is designed to run real hardware PC systems, not software emulators. Full-and-complete emulators like VMWare cause no problems. But, from the error reports, VirtualBox seems not to emulate ATA Identify Device commands (gives long timeouts!) and may not handle all EDD BIOS requests (gives EDD BIOS data ERROR messages!). UIDE is written to match EDD BIOS disk I-O addresses with those same addresses from its PCI controller scan. Only the PCI BIOS knows which UltraDMA addresses it assigned to each controller. If such a match fails, any affected hard-disks are IGNORED, as UIDE cannot use UltraDMA for them! With VirtualBox, or other similarly incomplete emulators which prevent UIDE's hard-disk setup logic from succeeding, UIDE can use its call the BIOS logic instead, like it does for diskettes and for any non-SATA/IDE disks that may be in use. I am assuming VirtualBox does NOT affect how the normal BIOS disk I-O routines operate, which would be truly IDIOTIC! So, I have added a /E (emulator) switch to UIDE and UIDE-S. /E makes the drivers call the BIOS on every hard disk I-O request. This means UIDE can go around most of its hard-disk setup logic and simply assume the BIOS knows what it is doing with disk drives. CD/DVD drives that were never part of the PC BIOS still require UIDE's setup logic to work, or they will not get detected and used. /E causes a minor speed loss (5% or less) in UIDE's cache speed, due to calling another driver (the BIOS) and since disks will be unable to use UIDE's XMS cache buffers for direct I-O. But, unlike /N1 which totally ignores hard disks, /E still allows hard-disk data to be cached after BIOS I-O requests, and it ought to let VirtualBox run O.K. with UIDE and UIDE-S. I don't use and don't want VirtualBox, so I must let users tell me if /E works as intended. But, one BIG problem -- Many El Cheapo BIOS programs (including mine!) have no DOS Virtual Data Services logic that tells a driver the 32-bit address of I-O buffers. When a DOS system uses protected-mode (JEMM386 etc.), no VDS logic says the BIOS can only do slow PIO mode transfers! On my system, using JEMM386 and UIDE /E, file copies run 6 times slower, even with UIDE's cache trying to help! Since I don't live in Taiwan or speak Mandarin, there is little I can do re: how mainboard makers design their (wretched) BIOS programs. Thus, users who require VirtualBox and desire to run UIDE /E with it should avoid enabling protected-mode, when possible, to prevent serious El Cheapo BIOS speed losses! I have sent an updated 22-Jul-2011 DRIVERS.ZIP file to Johnson Lam, with the /E switch added in UIDE and UIDE-S, for Johnson to test.He is my partner in these drivers (and a better software tester than me!), so the latest UIDE will not be official until he accepts it. Johnson is unavailable this weekend, so anyone who wishes to try the new UIDE /E on a test basis can E-Mail me, and I will send you the file. Do give me an E-Mail address for you that accepts .ZIP files -- GMail and others do NOT accept them, due to spammers/abuse! My complete E-Mail address is: gykazequios at earthlink dot net Also, Japheth tells me VirtualBox has been updated, and his XDMA32 JLM driver is now O.K. with JEMM386 and the new VirtualBox release.None- theless, I still intend to offer UIDE /E for all users, as there may yet be other emulator problems, and since using call the BIOS logic with hard disks could be valuable in other odd cases as well. Jack R. Ellis -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Test UIDE Available, For VirtualBox.
Op 24-7-2011 16:52, Jack schreef: Apologies for my late response re: UIDE v.s. VirtualBox. Had to have my (infected!) gall-bladder removed, 25-Jun-2011. NASTY episode, and I am still recovering and moving a bit slow on driver work! Re: recent EDR-DOS forum posts, I say the same as our author Mark Twain once did -- The rumors of my demise are greatly exaggerated! Goodluck recovering, Jack. So, I have added a /E (emulator) switch to UIDE and UIDE-S. /E makes the drivers call the BIOS on every hard disk I-O request. This means UIDE can go around most of its hard-disk setup logic and simply assume the BIOS knows what it is doing with disk drives. CD/DVD drives that were never part of the PC BIOS still require UIDE's setup logic to work, or they will not get detected and used. /E causes a minor speed loss (5% or less) in UIDE's cache speed, due to calling another driver (the BIOS) and since disks will be unable to use UIDE's XMS cache buffers for direct I-O. But, unlike /N1 which totally ignores hard disks, /E still allows hard-disk data to be cached after BIOS I-O requests, and it ought to let VirtualBox run O.K. with UIDE and UIDE-S. I don't use and don't want VirtualBox, so I must let users tell me if /E works as intended. Thanks very much for adding this. I can't promise any testing of this in the short run, but hope mr Juan Castro is able to as he started this thread. In the end I might test my complete FreeDOS CD on multiple emulators and physical hardware to catch any remaining issues. For now sticking to VMware. Juan, could you please try these drivers either by emailing Jack or waiting till Johnson Lam has posted them at http://johnson.tmfc.net/dos/driver.html ? DEVICE=C:\FDOS\BIN\UIDE.SYS /D:FDCD0001 /S5 /E (without enabling I/O-APIC and ICH9 chipset as Jeremy Davis posted earlier, thus resorting to normal APIC and PIIX3 chipset from Intel 440FX Pentium-Pro or P2 motherboards) These emulator-specific hacks are slightly annoying though, EMM386/JEMM has (had) them as well due to a VMware specific memory region being in use. Installation routines having to detect PCI-based devices (mainly videocards) might do the trick but involve extra work. slightly offtopic: I'm sticking to the /S5 parameter in general for automated UIDE as only absolute numbers are possible, no percentages (/S50% or /S0.5) with truncating (taking intended minimum amounts in account, e.g n=5/15/25/40/50 or 80=n=4093 minus PCI mapped space) Jack R. Ellis stay healthy!, Bernd -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Test UIDE Available, For VirtualBox.
Hi, On Sun, Jul 24, 2011 at 10:18 AM, Bernd Blaauw bbla...@home.nl wrote: Op 24-7-2011 16:52, Jack schreef: Apologies for my late response re: UIDE v.s. VirtualBox. Had to have my (infected!) gall-bladder removed, 25-Jun-2011. NASTY episode, and I am still recovering and moving a bit slow on driver work! Re: recent EDR-DOS forum posts, I say the same as our author Mark Twain once did -- The rumors of my demise are greatly exaggerated! Goodluck recovering, Jack. Ditto, Godspeed! So, I have added a /E (emulator) switch to UIDE and UIDE-S. Thanks very much for adding this. Ditto. I knew you were quite diligent, but this is beyond the call of duty. I'm surprised, honestly, just expected to have to live with yet another [VBox] bug / quirk that we'd all have to workaround, as usual (Murphy's law). Juan, could you please try these drivers either by emailing Jack or waiting till Johnson Lam has posted them at http://johnson.tmfc.net/dos/driver.html ? It's there already, so I guess he means that Johnson hasn't fully tested yet (until after the weekend). slightly offtopic: I'm sticking to the /S5 parameter in general for automated UIDE as only absolute numbers are possible, no percentages (/S50% or /S0.5) with truncating (taking intended minimum amounts in account, e.g n=5/15/25/40/50 or 80=n=4093 minus PCI mapped space) Why? I mean, sure /S5 is fine, I'm not complaining, but you *could* use Eric's XMSSIZE: http://ericauer.cosmodata.virtuaserver.com.br/soft/specials/xms-size-info-xmssizer.zip EDIT: Oh wait, that wouldn't work in CONFIG.SYS. But wait, you're using DEVLOAD (tricky bastard), so it just might!;-) Jack R. Ellis stay healthy!, Ditto. ;-) -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Test UIDE Available, For VirtualBox.
Op 24-7-2011 20:26, Rugxulo schreef: Why? I mean, sure /S5 is fine, I'm not complaining, but you *could* use Eric's XMSSIZE: http://ericauer.cosmodata.virtuaserver.com.br/soft/specials/xms-size-info-xmssizer.zip EDIT: Oh wait, that wouldn't work in CONFIG.SYS. But wait, you're using DEVLOAD (tricky bastard), so it just might!;-) Jack's driver only accepts certain values. This might be intentional due to specific caching algorythms. Using XMSSIZE I'd still have to use a lot of errorlevel stuff: if not errorlevel 5 , then use /B /N1 /N3 if not errorlevel 10, then use /S5 if not errorlevel .. etc. if errorlevel 80, then use /S%errorlevel% (limited to 255 as that's errorlevel maximum). All above is based on 'how much XMS is currently available' but also ignores the 'how much should I leave available for programs I'd like to run' (8MB system using 5MB cache means trouble for games and Windows). I like automatic calculations and roundings in a program itself, just as is currently done for invalid sizes (unless aborting completely). Purely a cosmetic feature, just like the shown device name on: DEVICE=UIDE.SYS /D:FDCD0001 /N2 (no CD drives handled, UIDE$ char device created, FDCD0001 char device not created, but FDCD0001 char device still listed in sign-on message). Anyway all of this isn't important, just some thoughts. Thanks for mentioning Johnson Lam posted these drivers, I'll update my CD. Makes much easier documentation: if slow in VirtualBox, add /E -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Test UIDE Available, For VirtualBox.
Jack, Except for minimal boot systems, like yours and Lucho's multiboot diskette, I recommend a minimum of 250-MB cache to handle todays' LARGE Windows files. Copying a 100- MB file will take 200-MB of UIDE's cache space (input and output), and some cache space must remain for directories so the NEXT disk operation does not have to re-read them. Having to discard DOS directories, to make room for new data files, is the main loss of speed when using UIDE, so users should allocate as large a UIDE cache as possible. A cache might already know that the directories are more useful because they are accessed more often, so even when a copy of a large file pushes out the directories, they are cached again soon :-) Also, you do not have to cache data on the first write, you can also wait until it is read again before you cache it - lbacache has a command line option to select whether or not to cache on write afair. Of course if data already is cached, you will have to update the cached copy on writes. So the difference is only whether or not to add data to the cache when it is written. Another question is how often people copy 100 MB files on non-Windows DOS systems and whether it bothers them that doing so, with a small cache, will flush out useful data such as FATs and directory data from the cache, temporarily reducing speed. Eric PS: Best wishes for your speedy operation recovery... PPS: know based on I/O patterns, not filesystem awareness. -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Test UIDE Available, For VirtualBox.
Hi Bernd, Jack's driver only accepts certain values. This might be intentional due to specific caching algorythms. Using XMSSIZE I'd still have to use a lot of errorlevel stuff: if not errorlevel 5 , then use /B /N1 /N3 if not errorlevel 10, then use /S5 if not errorlevel .. etc. if errorlevel 80, then use /S%errorlevel% (limited to 255 as that's errorlevel maximum). You can use a factor or simply add a zero, to get 10 to 2550 MB cache. As you say, smaller caches (5, 15, 25, 40, 50 MB) would still need an extra check, but you need not cover ALL steps. A few not enough RAM for 80 MB cache choices will be enough, e.g. 15 MB and 40 MB. All above is based on 'how much XMS is currently available' but also ignores the 'how much should I leave available for programs I'd like to run' (8MB system using 5MB cache means trouble for games and Windows). Again, I think a few steps for low memory systems would be enough and for big system percentages are exact enough imho. Also there are tools like my 503 byte add for further calculations, or use calc387 on newer computers, both in factorial-calc387-add.zip :-) Eric PS: calc387 is free open source software by Craig Hessel, 1024 bytes. -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Test UIDE Available, For VirtualBox.
Eric, Having to discard DOS directories, to make room for new data files, is the main loss of speed when using UIDE ... A cache might already know that the directories are more useful because they are accessed more often ... I did not want to add such logic in UIDE, because I dare not trust that each DOS variant works the same. At run-time, UIDE uses exactly one system resource, the Int 13h vector. This makes it generic and lets it run on all DOS variants. I do not even use interrupts (polling only) since some old Intel chipsets with errata (bugs!) had interrupt problems! ... Also, you do not have to cache data on the first write, you can wait until it is read again before caching it ... This would also add code in UIDE (a table of pending cache disk output areas), that I do not want to include. At 5.5K of run-time code, 4.6K of which can be stashed in the HMA, UIDE is rather efficient as-is, and I want to keep it so. ... Another question is how often people copy 100-MB files on non-Windows DOS systems and whether it bothers them that doing so, with a small cache, will flush out useful data such as FATs and directory data from the cache, temporarily reducing speed. I got into device drivers back in 2003 because I still use V6.22 MS-DOS to backup/restore my Windows/NT system. I bet many folks use DOS as a simple (and cheap!) backup/restore system for Gawd-AWFUL Windows, rather than spending money on some equally Gawd-AWFUL Windows application program! File- copy speed in such operations still matters to me, and since memory is now ludicrously cheap, why not USE it for a cache? PS: Best wishes for your speedy operation recovery ... Thanks! I am up and about, but food with high fat levels still HURTS when I eat it, due to less digestive fluids in my system! The Doc says it can take 18 months to get used to this, so I must simply be patient. Jack R. Ellis -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Test UIDE Available, For VirtualBox.
Jack, A cache might already know that the directories are more useful because they are accessed more often ... Sorry about the misunderstanding - I did not mean that the cache would know what a FAT or directory is, just that those sectors are accessed frequently, giving the cache a chance (e.g. by LRU) to know that it should throw them away later than less useful (less frequently accessed) sectors when new space is needed to cache something. Also, the frequent access means that FAT and directories are likely to be in the cache again soon... ... Also, you do not have to cache data on the first write, you can wait until it is read again before caching it ... This would also add code in UIDE (a table of pending cache disk output areas), that I do not want to include. Not necessarily. the code in lbacache for this is very short. There is a command line argument which sets a flag and then write to disk does the following: - check in which bin cached data for this sector is (you have to do that anyway, to update the cache) - if any (cache hit) update the cached copy. done. (this is needed to keep cache and disk consistent) - else (cache miss) if flag is not set, we are done. (a cache miss for a write does not NEED action...) - else do allocate fresh space in the cache and cache a copy of the written data (anticipating future reads) So if the alloc on write flag is set, the cache will assume that written data will be eventually read again. If the flag is not set, the cache will just write the data to disk and forget about it. It will be cached at the moment when the data is really read from disk again. So if you set the flag, cache memory is consumed faster and if not, reading just written data can be slower :-) UIDE is rather efficient as-is, and I want to keep it so. I understand that. Maybe my little algorithm above is still interesting for UIDE nevertheless, in particular for people who have less RAM and copy big files but do not often use the new copies immediately after copying. many folks use DOS as a simple (and cheap!) backup/restore system for Gawd-AWFUL Windows... In particular those users will usually NOT use their new backup copy right after copying it so they are a good example where alloc on write can be disabled. copy speed in such operations still matters to me, and since memory is now ludicrously cheap, why not USE it for a cache? A cache is good, but in theory, copying many big files is fastest with a cache which only caches metadata and read-ahead content but does not try to keep the written data in memory which will not be read back soon anyway. One COULD (but normally would not) do something like: lbacache cool dir /s nul lbacache temp xcopy ... That would give the directory metadata extra stickiness in the cache so the xcopy does not flush it out so fast, in particular with tuna option (avoid tunw option). However, the above example is a VERY manual way to tune a single XCOPY and as you say, it would add much extra complexity to make the cache REALLY know what directory and fat metadata is... On the other hand, deciding whether or not to set the TUNW alloc-on-write option is quite painless for everybody, as it can be explained with if you have little memory, use tuna, if you have a lot of memory, use tunw when loading. Thanks! I am up and about, but food with high fat levels still HURTS when I eat it, due to less digestive fluids in my system! The Doc says it can take 18 months to get used to this, so I must simply be patient. Yes - and maybe take some substitute enzymes, I guess... Eric PS: The TUNA option makes the cache fully associative so it considers ALL memory slots for caching (slower) while the default is to have only e.g. 16 candidates based on a hash of the sector number in question (16 way assoc.). When new data has to be cached, the least valuable slot of old cached data is replaced, based on access counts. -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
