Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
I have never seen a RAMdisk that was not file-based Maybe we have a misunderstanding here.. The driver itself is of course a file, but the disk is a DOS block device, so for DOS it is just a bunch of sectors. The DOS kernel will then use those sectors as a FAT filesystem which can contain files, sure, but the ramdisk itself has no idea what a (FAT) file is. It only needs to provide an initial state which looks like an empty formatted FAT filesystem. NO misunderstanding, and that is exactly what RDISK and other RAMdisk drivers do. After setting RAM memory to an initial state, a RAMdisk does memory-moves, not file transfers, with a one-to-one correspondence -- every DOS read or write to a RAMdisk causes exactly one XMS move request. Thus, one is better-off viewing a RAMdisk as a FILE handler, rather than a sector handler. Looking from DOS, the RAM disk is a block device. It doesn't handle data on a per-file basis; I don't understand why to call it a file handler then. It handles or provides sectors to DOS and that's it, so one is better off viewing a RAM disk as a sector handler. The Phantom RAM disk presented in the Undocumented DOS books was a file-based RAM disk: it hooked the redirector interface and provided the appropriate functions for each file. These files were stored in Phantom's own file system, in RAM. (Of course it makes more sense to use DOS's built-in file system driver, but the UDOS guys just did it to show how the redirector interface works.) Unless you recently changed RDISK to use the inferior redirector approach (that was a joke), it's still a block device as last time I looked at the source. What do you define as DOS read? Calling the Read from file DOS function? If that's the case then you're wrong, DOS usually calls block devices multiple times to read the FAT and the necessary sectors of the file's data to carry out such a request. If you meant DOS read as reading a single or some sectors from a block device, this is correct. Regards, Christian -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On Thu, 20-Aug-2009 11:27:47, Christian Masloch c...@bttr-software.de wrote: I have never seen a RAMdisk that was not file-based Maybe we have a misunderstanding here ... The driver itself is of course a file, but the disk is a DOS block device ... NO misunderstanding, and that is exactly what RDISK and other RAMdisk drivers do. After setting RAM memory to an initial state, a RAMdisk does memory-moves, not file transfers, with a one-to-one correspondence -- every DOS read or write to a RAMdisk causes exactly one XMS move request. Thus, one is better-off viewing a RAMdisk as a FILE handler, rather than a sector handler. Looking from DOS, the RAM disk is a block device. It doesn't handle data on a per-file basis; I don't understand why to call it a file handler then. It handles or provides sectors to DOS and that's it, so one is better off viewing a RAM disk as a sector handler. The Phantom RAM disk presented in the Undocumented DOS books was a file-based RAM disk: it hooked the redirector interface and provided the appropriate functions for each file. These files were stored in Phantom's own file system, in RAM ... Unless you recently changed RDISK to use the inferior redirector approach (that was a joke), it's still a block device as last time I looked at the source. I have made NO such changes, never WOULD, and never WILL, neither as any sort of insulting joke, nor otherwise! What do you define as DOS read? Calling the Read from file DOS function? If that's the case then you're wrong, DOS usually calls block devices multiple times to read the FAT and the necessary sectors of the file's data to carry out such a request. If you meant DOS read as reading a single or some sectors from a block device, this is correct. Fine, Christian, that is EXACTLY what I meant! Consider just what drives all I-O requests to RDISK or similar drivers. FILES do! Unless some oddball does his OWN requests for his OWN types of data blocks, RDISK will otherwise answers only to FILE requests by the DOS system, using the file DIRECTORY in its RAMdisk memory. To me, that makes RDISK a FILE handler, as distinguished from UIDE which in fact does reads or writes ONLY using LBA addresses and NOT any sort of directory. Thus UIDE is a pure SECTOR handler and Couldn't care LESS about any DOS directory structure. DOS RAMdisk drivers MUST HAVE a directory and a file structure, or DOS is absolutely UNABLE to use them! WHO CARES if it is DOS which knows how to break down files into blocks/sectors/etc. -- DOS and RDISK are in fact handling FILES, NOT sectors addressed ONLY by their LBA addresses as done in UIDE! That dependency on a directory, and on FILES in its directory, are what make every DOS RAMdisk driver a FILE handler FIRST, and a sector handler only as a SECONDARY result, in my opinion! You are free to classify such drivers/handlers however you choose! And, to be honest, I wish you Lots Of Luck dealing with those who MAY NOT be as forgiving of your jokes, then your too-RIGID semantics, as I was!! Jack R. Ellis -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hi! the moment, JEMMEX already is too fancy for many ancient DOS games which use pre-VCPI DOS extenders, so it is good that HIMEMX and JEMM386 are still separately available. Also, isn't JEMM actually FreeDOS's memory manager of choice? I suppose there was such a thing as FreeDOS EMM (that from which JEMM was created) but is it still maintained? No. JEMM386 is the successor of free EMM386 which is the successor of some EMS / UMB / VDS or similar thing of c't. What I mean is that you have to decide whether to use the classic JEMM386 HIMEMX pair (which allows you to decide if you want to load JEMM386 at boot, skipping it if you want to run non VCPI compatible ancient software) or to load the alternative JEMMEX variant which saves a bit more DOS RAM but can only be loaded in one piece... As mentioned on the list recently, any EMM also puts a virtual/vm86 layer between you and your hardware, which can be good for cases where you want to create virtual hardware (SB16, 8042...) but can be bad if you want to do fast lowlevel I/O access. Eric -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hi Jack, Ah okay... So there would be a separate API, and separate memory areas for XMS 2/3 and the new API then? Absolutely! Forgetting XMS V2.0 (only 64-MB), I am suggesting that all XMS V3.0 requests for 4-GB remain as-is, using the first 4-GB of memory. XMS V4.0 commands, if we choose to call them so, will be modified 3.0 commands to deal with memory ABOVE the first 4-GB. Hmmm then on one hand only new apps could use XMS 4, but on the other hand, old apps would not have any side- effects from the existence of XMS 4 :-). Note that the new apps will also have to use XMS 3 to get the 3 GB or similar that will be free in the first 4 GB because no sane XMS 3 app will use much of those 4 GB... ;-) If we do an XMS V4.0 upgrade for 64-GB memory, the scheme used by real mode and protected mode should be consistent for both. True. actually do contain workarounds for Win compat ... Still a BIG problem for drivers like UIDE wanting gigabytes of memory. As said - Windows 3 only has a problem with descriptors for more than 16 MB, so if you have a HIMEM or UIDE implementation of memory-block-copy which creates the descriptors on demand then you can easily copy 1 MB at offset 1 GB of your 2 GB handle and still stay with all offsets relative to the descriptor below 1 MB :-). This is of course not cool for descriptor caching on modern CPU, so a HIMEM could consider only activating this when Windows is active and using full handle sized, fixed offset descriptors for access at all other times :-). Note that I am talking about CPU descriptors, not XMS handles. Basically, apps which only use the XMS API to copy data to/from XMS will not even notice whether they run with a HIMEM with such a Windows limit workaround. At least in Linux and Windows, a pipeline is something provided by the OS kernel ... But we are NOT Linux/Windows, are we?? So all pipeline linked components of the CD/DVD burn toolkit will have to use some other API to communicate directly with each other instead. Yes and no - the metadata of the cache could be kept in the easily accessible area while the data itself can be copied with normal XMS driver calls without extra performance loss, allowing the cache data to stay in non-lockable extra large XMS above the 4 GB boundary. Of course I wonder why you do want a cache of 4 GB instead of just using a RAMDISK...? Agreed -- UIDE's cache tables and binary-search table could remain in normal 4-GB memory, along with its local 128K XMS buffer Okay :-) A RAMdisk handles a more permanent set of logical FILES, using a normal DOS directory. RAMDISKs are also sector-based things, not file-based, but indeed they always contain the same sectors, without dynamic decisions about which sectors will be in RAM and which not. Those who want a SCREAMING-fast DOS system will want BOTH! They could also copy their DOS disk into a RAMDISK at boot, possibly in the background. That would be a combination between a cache and a ramdisk then ;-). I think there are some shareware floppy caches working like this :-) Be kind -- Bernd contributes much to DOS/FreeDOS/etc. same as we do. Sure. Just a comment on the suggestion that he should or could write a pipeline based burn tool himself... No, you just cannot create locks on NEW handles, such which got allocated after Windows took over memory maintenance. Then I hope UIDE/RDISK or other large-memory drivers DO load first! Enjoy working on your miracles :-) Thanks, and you on yours ;-) Thanks :-) Eric -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On Wed 19-Aug-2009 13:06:42, Eric Auer e.a...@jpberlin.de wrote: Ah okay... So there would be a separate API, and separate memory areas for XMS 2/3 and the new API then? Absolutely! Hmmm then on one hand only new apps could use XMS 4, but on the other hand, old apps would not have any side-effects from the existence of XMS 4 :-) Of course not, or people will NOT use the new XMS 4 requests! Note that the new apps will also have to use XMS 3 to get the 3-GB or similar that will be free in the first 4 GB because no sane XMS 3 app will use much of those 4 GB... ;-) I think the user should be allowed to decide this. I would upgrade my drivers to use 64-GB XMS 4 memory only if over 4-GB is desired. For under 4-GB, they can allow memory above or below the 4-GB limit, since below 4-GB memory will surely work faster. As said - Windows 3 only has a problem with descriptors for more than 16 MB ... Note that I am talking about CPU descriptors, not XMS handles. Basically, apps which only use the XMS API to copy data to/from XMS will not even notice whether they run with a HIMEM with such a Windows limit workaround. If you are in fact referring to the descriptors used in an Int 15h AH=87h move-memory request, then I have no-problems in UIDE/RDISK. RDISK data transfers are limited by DOS to 64K bytes or less, so it cannot move more data at any one time. UIDE was written to work with the ACTUAL Int 15h AH=87h logic, present in a BIOS, if there are any protected mode applications which still use only the BIOS and not an EMM driver. I doubt this, but I had to write UIDE with this capability, just-in-case. There were some BIOS routines that do memory moves with CPU interrupts OFF, UIDE will NOT move over 2K bytes at a time. Thus, my drivers ARE doing what you suggest. But we are NOT Linux/Windows, are we?? So all pipeline linked components of the CD/DVD burn toolkit will have to use some other API to communicate directly with each other instead. True ONLY if people want a toolkit and a complex pipeline. If they want to have only ONE stand-alone program that does DVD/BluRay copies for Bernd, that one program can do whatever it wants and can leave the REST of the DOS system alone and at-peace! ... The metadata of the cache could be kept in the easily accessible area ... Agreed -- UIDE's cache tables and binary-search table could stay in normal 4-GB memory, along with its local 128K XMS buffer Okay :-) Another item on which we both agree -- Hallelujah!! A RAMdisk handles a more permanent set of logical FILES, using a normal DOS directory. RAMDISKs are also sector-based things, not file-based, but indeed they always contain the same sectors, without dynamic decisions about which sectors will be in RAM and which not. I have never seen a RAMdisk that was not file-based, same as I have never seen a cache that was not LBA-address (i.e. sector) based. Those who want a SCREAMING-fast DOS system will want BOTH! They could also copy their DOS disk into a RAMDISK at boot, possibly in the background. That would be a combination between a cache and a ramdisk then ;-) I think there are some shareware floppy caches working like this :-) Such a copy is exactly what I suggest in my driver README for users of RDISK. When a system is powered off, all RAMdisk data is lost, and on the next power-up, any permanent RAMdisk files MUST be put back in RAMdisk storage. Compiler temporary and other working files do not need this.However, NOTE that there is a DIFFERENCE between such temp/working files and a cache.A RAMdisk holds ALL of such files. A first-in first-out cache may have to DISCARD some sectors of the files when OTHER data must be cached.So the RAMdisk is quite-a-bit more GUARANTEED to handle a temp/working file faster! Jack R. Ellis -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hi Jack, Note that the new apps will also have to use XMS 3 to get the 3-GB or similar that will be free in the first 4 GB because no sane XMS 3 app will use much of those 4 GB... ;-) I mean if you have some XMS 4 ramdisk then it should also allow combined usage of XMS 3 and 4, otherwise it would leave the first 4 GB unused... Would be somewhat wasteful. As said - Windows 3 only has a problem with descriptors for more than 16 MB ... Note that I am talking about CPU descriptors... then I have no-problems in UIDE/RDISK Depending on whether the used HIMEM is Windows proof but that of course is not the problem of UIDE or RDISK :-) Some old apps used many small XMS handles to push HIMEM towards more Windows friendliness but that of course wasted handles. So all pipeline linked components of the CD/DVD burn toolkit will have to use some other API to communicate directly... If they want to have only ONE stand-alone program that does DVD/BluRay copies for Bernd, that one program can do whatever it wants... Such a program still would have to be written, of course. I have never seen a RAMdisk that was not file-based Maybe we have a misunderstanding here.. The driver itself is of course a file, but the disk is a DOS block device, so for DOS it is just a bunch of sectors. The DOS kernel will then use those sectors as a FAT filesystem which can contain files, sure, but the ramdisk itself has no idea what a (FAT) file is. It only needs to provide an initial state which looks like an empty formatted FAT filesystem. RAMdisk is ... GUARANTEED to handle a temp ... faster! Unless the cache is so big that it never has to discard data and unless you neglect the overhead that it takes the cache to notice this when reading/writing data ;-) Eric -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On Wed 19-Aug-2009 14:21:04, Eric Auer e.a...@jpberlin.de wrote: I mean if you have some XMS 4 ramdisk then it should also allow combined usage of XMS 3 and 4, otherwise it would leave the first 4 GB unused ... Would be somewhat wasteful. I agree, and that is why local data such as UIDE's cache tables would always use normal 4-GB memory. Only LARGE-scale XMS memory such as UIDE's cached DATA would be given the choice of using 4-GB or 64-GB memory. Depending on whether the used HIMEM is Windows proof but that of course is not the problem of UIDE or RDISK :-) Some old apps used many small XMS handles to push HIMEM towards more Windows friendliness but that of course wasted handles. I used V2.06 HIMEM from a server in Hungary as my model for what is now the XMGR driver. Even with V2.06 HIMEM in 1989, Gates and Co. knew-enough NOT to issue ALL of an XMS memory-move as one BIOS request (1989 pre-dated even their own EMM386!). Thus, XMGR also executes real or protected memory moves in small 2K quantities of data, for any old BIOS that may do such moves with interrupts OFF! You should assume HIMEM/EMM drivers are Windows proof re: this problem -- any that are NOT should be DISCARDED immediately!! So all pipeline linked components of the CD/DVD burn toolkit will have to use some other API to communicate directly... If they want to have only ONE stand-alone program that does DVD/ BluRay copies for Bernd, that one program can do whatever it wants... Such a program still would have to be written, of course. ABBBsolutely! Nothing comes free in our world economy, any more, just like the damned used to be free Internet, which is now only a pack of ADVERTISEMENTS everywhere you look! I have never seen a RAMdisk that was not file-based Maybe we have a misunderstanding here.. The driver itself is of course a file, but the disk is a DOS block device, so for DOS it is just a bunch of sectors. The DOS kernel will then use those sectors as a FAT filesystem which can contain files, sure, but the ramdisk itself has no idea what a (FAT) file is. It only needs to provide an initial state which looks like an empty formatted FAT filesystem. NO misunderstanding, and that is exactly what RDISK and other RAMdisk drivers do. After setting RAM memory to an initial state, a RAMdisk does memory-moves, not file transfers, with a one-to-one correspondence -- every DOS read or write to a RAMdisk causes exactly one XMS move request. Thus, one is better-off viewing a RAMdisk as a FILE handler, rather than a sector handler. RAMdisk is ... GUARANTEED to handle a temp ... faster! Unless the cache is so big that it never has to discard data and unless you neglect the overhead that it takes the cache to notice this when reading/writing data ;-) A cache THAT big defeats its pwn purpose, which is to use a SMALL amount of RAM memory to speed-up a much LARGER disk!! UIDE is limited to 2-GB, which some people still believe is a bit high, but in fact hard-disks CAN go up to 2-TERABYTES these days! What you suggest will never happen. Jack R. Ellis -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
JEMMEX could use big pages or PAE or both to access more RAM for various things, but this always leads to the question: How compatible will it be? At the moment, JEMMEX already is too fancy for many ancient DOS games which use pre-VCPI DOS extenders, I think any protected-mode memory manager (EMM386) disables ancient DOS extenders which have to start from real mode (i.e. pre-VCPI). so it is good that HIMEMX and JEMM386 are still separately available. Separately from what? Regards, Christian -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On Tuesday 18 August 2009 15:18 (CEST), Christian Masloch wrote: so it is good that HIMEMX and JEMM386 are still separately available. Separately from what? Separately from each other... Jemmex is a two-in-one driver, but you can use HIMEMX and JEMM386 as two separate drivers, too. Best regards, Mateusz Viste -- You'll find my public OpenPGP key at http://www.viste-family.net/mateusz/pub_key signature.asc Description: This is a digitally signed message part. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On Tue, 18-Aug-2009, Christian Masloch c...@bttr-software.de wrote: so it is good that HIMEMX and JEMM386 are still separately available. Separately from what? Regards, Christian Separately from MS-DOS, FreeDOS, or other DOS variants, since few if-any of their EMM drivers are still under maintenance, and as some have never corrected BUGS! As noted in my drivers' README, I now recommend JEMMEX and JEMM386 only, for systems that need an EMM driver's features (0B000- 0B7FF upper memory, DPMI/VCPI/protected mode, etc.). For such tasks, Japheth IS The ONLY guy in town, as we say!! V4.49/V4.95 EMM386 (V6.22/V7.10 MS-DOS) do not handle many Virtual DMA options properly, and have not since 1994 when I began using such calls. UIDE/UIDEJR do Virtual-DMA locks/unlocks and use the returned lock 32-bit buffer address. I would NOT rely on other Virtual DMA options if using MS-DOS EMM386, since they still FAILED in 2004 when Lucho and I created the first UDMA driver! Jack R. Ellis -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
so it is good that HIMEMX and JEMM386 are still separately available. Separately from what? Separately from MS-DOS, FreeDOS, or other DOS variants, since few if-any of their EMM drivers are still under maintenance, and as some have never corrected BUGS! I support your intention (i.e. JEMM being the best available PM memory manager) but the answer doesn't fit what Eric said: At the moment, JEMMEX already is too fancy for many ancient DOS games which use pre-VCPI DOS extenders, so it is good that HIMEMX and JEMM386 are still separately available. Also, isn't JEMM actually FreeDOS's memory manager of choice? I suppose there was such a thing as FreeDOS EMM (that from which JEMM was created) but is it still maintained? Regards, Christian -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hi Bernd, Jack is right, pipeline based DVD burning saves a lot of space for temp files. Of course you are also right - you cannot easily put temp files on harddisk if the harddisk only has NTFS etc and it would be asking a bit much to add another harddisk. Well, of course DOS USB (storage) drivers are improving a lot now :-). I still like the idea to use memory above 4 GB only for easy things such as RAMDISK... Yes, JEMMEX could use big pages or PAE or both to access more RAM for various things, but this always leads to the question: How compatible will it be? At the moment, JEMMEX already is too fancy for many ancient DOS games which use pre-VCPI DOS extenders, so it is good that HIMEMX and JEMM386 are still separately available. If a new driver provides some kind of XMS 4, but only at the cost of dropping, say, EMS 4 or VCPI support, then that XMS 4 mode should be command line switchable at least :-). By the way, I think that you should not map too much stuff over the now unusable last 0.n GB of the first 4 GB. After all, the remaining area can be used for framebuffers, BIOS and MMIO so whenever it is marked as non-accessible, there will usually be a good reason for that. Same as with UMB. Talking of which, I wonder whether JEMM386 can become even more clairvoiant about avoiding unusable UMB, now that modern hardware and BIOS frequently fail to mark such areas right. 2) Computer has 12GB RAM (hello brand new Core I7 system Hello waste of electricity... 4) DVD can contain more than 4GB, so the file fits on it. Next task would be to find 4 GB of actually nice DOS software. 5) No ISO generation necessary as your above mentioned buffer Indeed, although DOS is not exactly strong in pipelines ;-) 7) No USB sticks around with sufficient capacity and free space Depending on speed, 4 GB USB sticks are 10 - 40 Euro today... For comparison, smallest quad core i7 CPU is already 250 and 6 GB of DDR3 are 130 to 360 Euro at the moment. Same with a much less fancy hardware: 4 GB DDR2 is 40 to 90 Euro and an Athlon64x2 4450E for 40 can also handle 16 GB on nForce630a already... So YES people can have so much RAM (but why? Not even Vista can waste it yet) but NO using RAM is by no means a cost efficient way to have 4 GB of storage in your PC now. With lack of UDF driver for DOS (packet writing) I would already be happy with UDF reading. There were even some attempts to make some UDF enabled SHSUCDX afair...? I know of no other freely usable/distributable DOS optical disk writing software other than CDRECORD and CDRKIT, both stuck on ASPI/SCSI... How about DOSCDROAST? Or is it just a GUI for CDRECORD? Maybe soon enough ReactOS (www.reactos.org) will be a light-enough Windows-replacing environment and work with your streaming recorder program. ... or Linux ... but yes, DVD UDF is tough for DOS. Eric -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hi Jack, No one is suggesting an API only for MEM! Also, my ideas about new XMS commands are only for 64-GB memory, NOT for current XMS commands Ah okay... So there would be a separate API, and separate memory areas for XMS 2/3 and the new API then? It is only necessary if you have tools that want to use more than 4 GB for themselves via protected mode themselves. NO, sir! Protected mode moves are done by JEMM386/JEMMEX, since on systems that use either driver, only THEY can trap and execute an Int 15h AH=80h move-memory request! True - if you have a HIMEMXXL then it will need a way to communicate with some EMM386 if you want to use both in parallel. On the other hand, if the access to XMS style memory beyond 4 GB is implemented in some JEMMEXXL then it would only have to communicate with itself, as it is both HIMEM and EMM386 inside the same driver already. if a driver were to request 64-GB memory BEFORE loading JEMM386 Loading drivers BETWEEN HIMEMX and JEMM386 is a somehow unusual case if my opinion. Combined with the unusually big memory size of more than 4 GB, that adds up to too much unusuality to make UIDE compatibility a priority ;-) In theory, you can allocate a lot of RAM in pieces of 16 MB. This requires revision of many programs, which are NOT expecting True true, yet Windows 3 was so widespread that many drivers actually do contain workarounds for Win compat. And if the XMS move is only done by the XMS driver then the workaround (keep descriptor sizes below 16 MB) can even happen inside HIMEMX itself, needing no changes in other software that uses XMS via the XMS API :-). course it is not trivial to do heavy pipeline things in DOS. Betcha it is a lot MORE trivial than adding 64-GB XMS support! Pipeline and simultaneous I-O schemes concern ONLY application programs that use them, NOT the whole system or its XMS drivers! At least in Linux and Windows, a pipeline is something provided by the OS kernel - but of course you can have two apps which communicate directly to form pipes, too. Well UIDE does not know how to do DMA to 64-bit address space and I do not even know whether it is possible with normal PCI. UIDE could still have its cache in 64-GB memory if it were changed to do all I-O through its local 128-KB XMS buffer. That buffer would have to stay in 32-bit address space, as SATA/UltraDMA chips were designed in the 1994 32-bit days. Interesting. One would assume that SATA/UDMA was more future-proof than that ;-) Doing all I-O thru its buffer costs a small amount of time in UIDE. But if a HUGE cache is needed and a fast-enough CPU for its binary searches is present this COULD be made to work Yes and no - the metadata of the cache could be kept in the easily accessible area while the data itself can be copied with normal XMS driver calls without extra performance loss, allowing the cache data to stay in non-lockable extra large XMS above the 4 GB boundary. Of course I wonder why you do want a cache of 4 GB instead of just using a RAMDISK...? I agree -- I ran Win/NT with only 256KB memory for over 10 years, 256 MB, probably :-) until I had to get more for testing UIDE/RDISK. I never had any problem due to available memory -- Win/NT still crashes now and then, but not due to low memory... I remember that life was quite okay with 16, 32, 64, 192 MB on mixed versions of Windows and Linux for many years indeed. In my opinion, this thread started with the idea that the unused (maybe superfluous) memory of Bernd's 6 GB PC can be used for something SIMPLE such as a RAMDISK so it can serve any purpose at all in DOS ... Agreed -- maybe we should save our time and let Bernd handle his DVD/BluRay copies using a custom DOS program, not a new XMS! Note: Bernd cannot program anything but Warcraft and BAT afair :-) I would rather NOT be unsafe and prefer to lock any XMS used by a SYSTEM driver AGAINST being re-mapped, moved-around... I know I know. My cache also locks memory because it feels bad to have the data of a disk cache swapped out to a disk... :-p. That probably also holds for UIDE, now that I think of it... other Gates Co. fun and games! If XMS locks are unusable when running Win/3, maybe it is best not to support Win/3... No, you just cannot create locks on NEW handles, such which got allocated after Windows took over memory maintenance. Enjoy working on your miracles :-) Eric -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Eric, My Thanks for your support of what you all call pipelining (I call it simultaneous I-O) in our discussions with Bernd, in your earlier post today. No one is suggesting an API only for MEM! Also, my ideas about new XMS commands are only for 64-GB memory, NOT for current XMS commands Ah okay... So there would be a separate API, and separate memory areas for XMS 2/3 and the new API then? Absolutely! Forgetting XMS V2.0 (only 64-MB), I am suggesting that all XMS V3.0 requests for 4-GB remain as-is, using the first 4-GB of memory. XMS V4.0 commands, if we choose to call them so, will be modified 3.0 commands to deal with memory ABOVE the first 4-GB. ... Protected mode moves are done by JEMM386/JEMMEX, since on systems that use either driver, only THEY can trap and execute an Int 15h AH=87h move-memory request! True - if you have a HIMEMXXL then it will need a way to communicate with some EMM386 if you want to use both in parallel. On the other hand, if the access to XMS style memory beyond 4 GB is implemented in some JEMMEXXL then it would only have to communicate with itself, as it is both HIMEM and EMM386 inside the same driver already. If we do an XMS V4.0 upgrade for 64-GB memory, the scheme used by real mode and protected mode should be consistent for both. This is why I suggest any XMS requests using 64-GB memory address- ing (allocates/locks/moves etc.) must include the page number. if a driver were to request 64-GB memory BEFORE loading JEMM386 Loading drivers BETWEEN HIMEMX and JEMM386 is a somehow unusual case if my opinion. IN your opinion, but in fact I agree with you on this point. In theory, you can allocate a lot of RAM in pieces of 16 MB. This requires revision of many programs, which are NOT expecting True true, yet Windows 3 was so widespread that many drivers actually do contain workarounds for Win compat ... Still a BIG problem for drivers like UIDE wanting gigabytes of memory. XMS drivers provide at most 128 handles, and your idea of 16-MB each would take ALL those handles, leaving none for other drivers/programs. At least in Linux and Windows, a pipeline is something provided by the OS kernel ... But we are NOT Linux/Windows, are we?? UIDE could still have its cache in 64-GB memory if it were changed to do all I-O through its local 128-KB XMS buffer. That buffer would have to stay in 32-bit address space, as SATA/UltraDMA chips were designed in the 1994 32-bit days. Interesting. One would assume that SATA/UDMA was more future-proof than that ;-) Maybe newer controllers can access more than 32 bits of I-O bus, but I am not holding my breath, and UIDE must still run OLDER controllers! Doing all I-O thru its buffer costs a small amount of time in UIDE. But if a HUGE cache is needed and a fast-enough CPU for its binary searches is present this COULD be made to work Yes and no - the metadata of the cache could be kept in the easily accessible area while the data itself can be copied with normal XMS driver calls without extra performance loss, allowing the cache data to stay in non-lockable extra large XMS above the 4 GB boundary. Of course I wonder why you do want a cache of 4 GB instead of just using a RAMDISK...? Agreed -- UIDE's cache tables and binary-search table could remain in normal 4-GB memory, along with its local 128K XMS buffer (64K data, and 64K for UltraDMA alignment). Re: caches and RAMdisks, a cache handles dynamically-changing disk data sectors, via LBA addresses and normally a first-in first-out algorithm. A RAMdisk handles a more permanent set of logical FILES, using a normal DOS directory. Users who want SPECIFIC files always available will want a RAMdisk. Those who want to speed-up general sector handling to/from disks will want a cache. Those who want a SCREAMING-fast DOS system will want BOTH! I agree -- I ran Win/NT with only 256KB memory for over 10 years, 256 MB, probably :-) Wait until YOU are age 63, my friend, and typos are a fact of YOUR life, as well! ;) I remember that life was quite okay with 16, 32, 64, 192 MB on mixed versions of Windows and Linux for many years indeed. Now you understand my quite-SERIOUS comments about Do we really NEED all this?? I still say a GOOD 4-GB DOS system can do LOTS of work! Note: Bernd cannot program anything but Warcraft and BAT afair :-) Be kind -- Bernd contributes much to DOS/FreeDOS/etc. same as we do. I would rather NOT be unsafe and prefer to lock any XMS used by a SYSTEM driver AGAINST being re-mapped, moved-around... I know I know. My cache also locks memory because it feels bad to have the data of a disk cache swapped out to a disk... :-p. That probably also holds for UIDE, now that I think of it... A-solutely!! If XMS locks are unusable when running Win/3, maybe it is best not to support Win/3... No, you just cannot create locks on NEW handles, such which got allocated after Windows took over memory
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hello, 2) Do not support a super allocate, just let XMS clients alloc memory as long as it is available - but start in the first 4 GB and/or use memory after that easy area only for big allocs... 3) You need no outside-visible super extended move as long as only XMS access to the extra memory is available. HOWEVER this means that the super XMS driver will have to reject requests to lock XMS memory to a fixed address inside the first 4 GB (which simply is the XMS 3 API limit) as soon as it runs out of space in the first 4 GB. For simplicity, I would not even suggest to move around XMS handles. Instead, I would support locking and fixed addresses only for handles which physically were in the first 4 GB anyway at the moment when they were allocated... Might work, but it's hackish and IMO it's important that a program can tell Himem that it prefers to get physical memory from beyond the 4 GB limit. PS: Regarding I/O virtualization, the MS EMM386 API for this is relatively small, int 2f.4a15 basically. JEMM386 with the JLM driver support infrastructure could probably implement support for int 2f.4a15 as a small JLM driver at some time. Yes. letting software use the extra memory. It is in no way part of the XMS interface to coordinate PAE paging between HIMEM and the software that uses XMS. Of course you are right that PAE paging most likely isn't needed, the simpler approach 32-bit paging with 4 MB pages should do as well. japheth -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Op 14-8-2009 23:37, Jack schreef: Oh, REALLY?? I have been running the DeepBurner program to shoot my CD/RW masters for over two years now. It DOES simultaneous input/output with the disk staying-ahead of the CD/RW and always having the next block of data ready to be written out. It is a Windows program, but it runs on EXACTLY the same computer system that I use for DOS. Given that, and rather than everyone jumping through hoops about super-extended memory, maybe you should contact the DeepBurner people on the Internet, and ask if (A) they have a DOS version or (B) they might send you their sources!! NO reasons you must call UIDE, nor any other DOS drivers -- you can write your OWN simultaneous I-O scheme in DOS to copy your DVDs/BluRays, and it might take 100-MB of buffers maximum, damned well NOT 8.5-GB nor 50-GB!!! Hello Jack, Somehow I think you're missing the point here a bit. Your UIDE and ramdisk together can use all memory below 4GB quite easily already. Point is, I'd prefer UIDE and other programs, and possibly a ramdisk, to use the memory below 4GB; and a ramdisk driver, be it the same driver or not, being able to use/access RAM above 4GB so first 4GB stays available for older/other programs which are limited to this amount. Your recorder program example as claimed above would be able to pipe original source directory tree into ISO into the recorder on the fly. I normally use IMGBURN for recording, though ISO generation + virtual machine is easier option to test things first, before creating a final master on optical disk. Let's go with the following assumptions (as valid in modern computer sales), and see if I can demonstrate my issue. 1) User has a computer with 1 harddisk only, with 1 partition spanning the entire harddisk, formatted as NTFS. 2) Computer has 12GB RAM (hello brand new Core I7 system with triple memory channels) 3) Disk image can create an imagefile of the harddisk, with (compressed) filesize just below 4GB so it fits as single file on FAT32 under DOS. 4) DVD can contain more than 4GB, so the file fits on it. 5) No ISO generation necessary as your above mentioned buffering technique can be used. 6) Not allowed to make alterations to the disk at all. Think forensics if you want :) 7) No USB sticks around with sufficient capacity and free space, or speed, or access from DOS, to be used as storage space. With lack of UDF driver for DOS (packet writing) we'd need to have a FAT32 storage location with at least 4GB capacity, so the file (disk image) that we want to burn to DVD, can be stored. Didn't even mention ISO (see assumption 5), which would double the needed filesize :) I know of no other freely usable/distributable DOS optical disk writing software other than CDRECORD and CDRKIT, both stuck on ASPI/SCSI (as is your DeepBurner program I bet, using SPTI likely). Maybe there does exist a specialised DOS program which does what you do in Windows: source -- pipe -- ISO -- pipe -- disk (Norton Ghost perhaps?). Maybe only Linux based software is suitable (Parted Image etc). Still, I'd like to be able to use DOS, and record/modify optical discs from this environment. Maybe soon enough ReactOS (www.reactos.org) will be a light-enough Windows-replacing environment and work with your streaming recorder program. Thanks for the interesting discussion at the very least :) Bernd -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On Fri, 14 Aug 2009 15:57:55 -0700, Eric Auer e.a...@jpberlin.de wrote: Hi Jack, Re: using memory over 4-GB in DOS, the best way to AVOID many new custom programs is to raise the capacity of XMS drivers... 1) A change to the XMS request which determines available memory, with a change to the FreeDOS MEM program to display all such memory. I believe MEM is currently limited at 4-GB. 2) A new Allocate SUPER-extended memory request, which requests memory above 4-GB if available.This new request would have to return the page number (0 to 15) of the allocated memory, for programs like UIDE that do their own real mode XMS work. It would still return an XMS handle so the Free XMS memory and standard Move XMS memory commands could work as-before. 3) A new Move SUPER-extended memory request, or a change to the current Int 15h AH=87h move-memory request that denotes what page of memory is being moved, for all protected mode move requests (JEMM386/JEMMEX).There are no extra bits in the current Int 15h AH=87h descriptor block, so some sort of new convention would also be needed for large-scale XMS memory. Maybe it can be even more minimal than that... 1) Do not tell MEM about the extra memory, just tell it that you have 3 GB as long as at least 3 GB are still available ... This might materially confuse users, who expect MEM to report actual available XMS memory. Since MEM is only reporting numbers to users I see no reason why MEM cannot remain honest about what is does. 2) Do not support a super allocate, just let XMS clients alloc memory as long as it is available - but start in the first 4 GB and/or use memory after that easy area only for big allocs ... NO good! UIDE and other programs that want to run their OWN real mode XMS moves need to know (A) if XMS above 4-GB is available and (B) which page of 64-GB memory (0 thru 16) was allocated, as they will need to set the 64-GB page registers to do real mode work. The Allocate super-extended memory request is necessary. Of course this means that the 9 GB RAMDISK that Bernd would want for juggling with two big DVD ISOs will need several XMS handles but other RAMDISKS already do the same to support big drives via pure XMS 2 requests which give you at most 64 MB per handle. This also helps Windows 3 compatibility - the XMS handling of Windows seems to have a bug if anybody uses descriptors for 16+ MB areas. Little I can do about Win/3, since UIDE users need a LOT more XMS than only 16-MB. Re: Bernd's 9-GB RAMdisk, this is ABSURD and I have already noted in an earler post that only 100-MB should be adequate for such copies, if using a simultaneous I-O scheme as in the DeepBurner CD/DVD writer program.DeepBurner is for Windows (runs fine on my ancient Windows/NT), but maybe Bernd can find a DOS copy of it, or get its source files and make a similar program to run under DOS. That would save 8.9-GB! 3) You need no outside-visible super extended move as long as only XMS access to the extra memory is available. HOWEVER this means that the super XMS driver will have to reject requests to lock XMS memory to a fixed address inside the first 4 GB (which simply is the XMS 3 API limit) as soon as it runs out of space in the first 4 GB. For simplicity, I would not even suggest to move around XMS handles. Instead, I would support locking and fixed addresses only for handles which physically were in the first 4 GB anyway at the moment when they were allocated... Without knowing the address and page number of XMS memory above 4-GB, UIDE could not use it in real mode, and there may be more programs that would suffer, as well. Actually, I failed to note that XMS locks, and their returned 32-bit memory addresses, would also demand a Lock SUPER-extended memory request, to return the 64-GB page number as well. See the above only for big allocs suggestion for this :-). Note that for example Windows 3 does similar things - it just refuses to let you use XMS call 0c (lock, RBIL table 02762) on handles that were allocated after Windows started ;-). I guess it will return error code ad, lock failed at that time. How often Gates Co. violate their own conventions, simply because they ARE GC! If what you say is true, this poses a BIG problem for drivers using XMS that load after Win/3! My own RDISK could be in that category!I see NO solution for this, except writing-OFF Win/3 as a hopeless piece-of-TRASH!! PS: Regarding I/O virtualization, the MS EMM386 API for this is relatively small, int 2f.4a15 basically. JEMM386 with the JLM driver support infrastructure could probably implement support for int 2f.4a15 as a small JLM driver at some time. I-O virtualization is not my department. PPS: I have the feeling that keeping ONE ISO in RAM should be enough for what Bernd wants, if access is organized well. Of course you still need to find 4 GB of DOS
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hi Jack, 1) Do not tell MEM about the extra memory, just tell it that you have 3 GB as long as at least 3 GB are still available ... This might materially confuse users, who expect MEM to report actual available XMS memory. Since MEM is only reporting numbers to users I see no reason why MEM cannot remain honest about what is does. MEM could - but all XMS 2 and XMS 3 apps will be more confused about a too honest XMS 4 driver... And to add a whole new API only for MEM...? 2) Do not support a super allocate, just let XMS clients alloc memory as long as it is available - but start in the first 4 GB and/or use memory after that easy area only for big allocs ... NO good! UIDE and other programs that want to run their OWN real mode XMS moves need to know (A) if XMS above 4-GB is available and (B) which page of 64-GB memory (0 thru 16) was allocated, as they will need to set the 64-GB page registers to do real mode work. The Allocate super-extended memory request is necessary. It is only necessary if you have tools that want to use more than 4 GB for themselves via protected mode themselves. For something like a RAMDISK, only the XMS driver has to know any details about the true location of the memory... As said, I am trying to suggest using the normal XMS 3 API while still letting software use the extra memory. It is in no way part of the XMS interface to coordinate PAE paging between HIMEM and the software that uses XMS. Of course you are right that protected mode software CAN be interested in protected mode access to XMS. Again, the only thing that I can imagine to make use of more than 4 GB RAM is a RAMDISK, no DOS extender. Little I can do about Win/3, since UIDE users need a LOT more XMS than only 16-MB. In theory, you can allocate a lot of RAM in pieces of 16 MB. In addition, HIMEMX can simply use a per-transfer descriptor strategy instead of a per-handle descriptor strategy. Then you can stay below 16 MB descriptor size for all transfers of less than 16 MB block size even if you allocate 4 GB in a single handle :-) Most transfers (via XMS far call) are between DOS memory and XMS anyway, so size is inherently limited to at most 1 MB most of the time. In short: You can make Windows 3.x happy and still use a lot of XMS... :-). I have already noted in an earler post that only 100-MB should be adequate for such copies, if using a simultaneous I-O scheme as in the DeepBurner CD/DVD writer program.DeepBurner is for Windows (runs fine on my ancient Windows/NT), but maybe Bernd... I know... Linux burner tools such as K3B also use growisofs instead of mkisofs, so they pipeline the filesystem into the burner while it is generated and avoid the need to have one big temp file of the size of the whole CD / DVD disk :-). Of course it is not trivial to do heavy pipeline things in DOS. Without knowing the address and page number of XMS memory above 4-GB, UIDE could not use it in real mode... Well UIDE does not know how to do DMA to 64 bit address space and I do not even know whether it is possible with normal PCI. But again, I think UIDE is not among those tools which would actually gain from using more than 4 GB RAM for one app. You yourself already said that 6 GB RAM are too much for DOS :-) I for example have 1 GB in Linux and my swapfile never does anything useful apart from serving as hibernate image ;-). programs that would suffer, as well. Actually, I failed to note that XMS locks, and their returned 32-bit memory addresses, would also demand a Lock SUPER-extended memory request... In my opinion, this thread started with the idea that the unused (maybe superfluous) memory of Bernd's 6 GB PC can be used for something SIMPLE such as a RAMDISK so it can serve any purpose at all in DOS. Having full support with DMA and 64 bit DOS extenders etc etc would of course be much more work and would need a much wider API as well. Note that for example Windows 3 does similar things - it just refuses to let you use XMS call 0c (lock, RBIL table 02762) on handles that were allocated after Windows started ;-). BIG problem for drivers using XMS that load after Win/3! You can just load them BEFORE Windows - or of course use the XMS via the normal XMS copy function instead of using raw protected mode access. As any RAMDISK will in the end send and receive data via DOS memory, the XMS copy works perfectly for the purpose and the RAMDISK does not need to know about the physical location of the XMS... Things are different for UIDE but again, this will have various issues with the 4 GB boundary and PAE and locking so it does not count as SIMPLE for me ;-). Eric -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Eric, ... Since MEM is only reporting numbers to users I see no reason why MEM cannot remain honest about what is does. MEM could - but all XMS 2 and XMS 3 apps will be more confused about a too honest XMS 4 driver... And to add a whole new API only for MEM...? No one is suggesting an API only for MEM! Also, my ideas about new XMS commands are only for 64-GB memory, NOT for current XMS commands that would continue to run fine for applications expecting and using only the first 4-GB of memory, as we both agree they should! ... UIDE and other programs that want to run their OWN real mode XMS moves need to know (A) if XMS above 4-GB is available and (B) which page of 64-GB memory (0 thru 16) was allocated, as they will need to set the 64-GB page registers to do real mode work. The Allocate super-extended memory request is necessary. It is only necessary if you have tools that want to use more than 4 GB for themselves via protected mode themselves. NO, sir! Protected mode moves are done by JEMM386/JEMMEX, since on systems that use either driver, only THEY can trap and execute an Int 15h AH=80h move-memory request! A real mode system can have drivers like UIDE which do their OWN real mode moves, as the MvData routine does in UIDE, and so it is real mode which needs to be told the 64-GB page number for an XMS memory block. Also, if a driver were to request 64-GB memory BEFORE loading JEMM386, it will have to include the 64-GB page number in move-memory requests, as JEMM386 then will NOT be the driver allocating the XMS and could not know the page number! Better to have one consistent Allocate SUPER-extended memory request, which provides the 64-GB page for either real mode or protected mode systems. Both need it. For something like a RAMDISK, only the XMS driver has to know any details about the true location of the memory ... As I noted above, JEMM386/JEMMEX also need to know the 64-GB page numbers, since it is they (not the XMS driver!) which handle an XMS move in protected mode. As said, I am trying to suggest using the normal XMS 3 API while still letting software use the extra memory. Exactly as I am trying to do. It is in no way part of the XMS interface to coordinate PAE paging between HIMEM and the software that uses XMS. It is, if JEMM386/JEMMEX are present for protected mode! I agree address-extension paging is the responsibility of only XMS drivers that handle over 4-GB of memory. But they need to co-ordinate such page numbers, and offer them both to applications requesting 64-GB memory AND to JEMM386/JEMMEX when using protected mode. Of course you are right that protected mode software CAN be interested in protected mode access to XMS. Again, the only thing that I can imagine to make use of more than 4 GB RAM is a RAMDISK, no DOS extender. UIDE could -- see below. Maybe Alain's database could, as well. Little I can do about Win/3, since UIDE users need a LOT more XMS than only 16-MB. In theory, you can allocate a lot of RAM in pieces of 16 MB. This requires revision of many programs, which are NOT expecting to reserve other than one block of XMS memory with one XMS handle. I have NO intention of adding such code in UIDE/RDISK merely to run Win/3, which I still regard as a hopeless piece-of-junk!So do most others, who now use at-least Win/XT for serious Windows work. I have already noted in an earler post that only 100-MB should be adequate for such copies, if using a simultaneous I-O scheme as in the DeepBurner CD/DVD writer program ... I know... Linux burner tools such as K3B also use growisofs instead of mkisofs, so they pipeline the filesystem into the burner while it is generated and avoid the need to have one big temp file of the size of the whole CD / DVD disk :-). Of course it is not trivial to do heavy pipeline things in DOS. Betcha it is a lot MORE trivial than adding 64-GB XMS support! Pipeline and simultaneous I-O schemes concern ONLY application programs that use them, NOT the whole system or its XMS drivers! Without knowing the address and page number of XMS memory above 4-GB, UIDE could not use it in real mode... Well UIDE does not know how to do DMA to 64-bit address space and I do not even know whether it is possible with normal PCI. UIDE could still have its cache in 64-GB memory if it were changed to do all I-O through its local 128-KB XMS buffer. That buffer would have to stay in 32-bit address space, as SATA/UltraDMA chips were designed in the 1994 32-bit days.Doing all I-O thru its buffer costs a small amount of time in UIDE. But if a HUGE cache is needed and a fast-enough CPU for its binary searches is present this COULD be made to work, certainly for 4-GB caches, maybe more! But again, I think UIDE is not among those tools which would actually gain from using more than 4 GB RAM for one app. You yourself already said that 6 GB RAM are too much for DOS :-) I also noted Gates Co.
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Eric, Sorry, in my last post I meant to say Int 15h AH=87h move- memory requests, not AH=80h. Jack R. Ellis -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Op 14-8-2009 20:17, Jack schreef: Re: using memory over 4-GB in DOS, the best way to AVOID many new I am assuming here that, to keep things simple and still-useful for programs that cannot or will-not be upgraded, the XMS drivers will all behave same-as-before for current XMS commands. User programs would still request up to 4-GB of memory, from the first 4-GB of a large system, and would still use normal XMS requests via handles for their memory as-before. Only the XMS handlers and upgraded large scale XMS programs like UIDE/RDISK need know about and support the new commands above.This AVOIDS a lot of compatibility issues. I don't know if it's easier or harder to do, but maybe best approach is twofold : 1) First 4GB handled traditionally for programs, ramdisks, diskcaches etc. 2) Ramdisk driver that can handle all memory beyond 4GB and leaves anything beyond first 4GB completely alone. Note that the ramdisk size might be beyond 4GB itself as well then on for example a 12GB system But, a much-LARGER issue here is Do we really NEED all this?! There are NOT many systems with over 4-GB, and I expect few would ever need over 4-GB of memory for DOS use, if in fact their FIRST 4-GB is properly managed! Re: my own drivers, as I say in their README file, a proper split of XMS memory between RDISK for all fast files, UIDE for ordinary disk files, and saving some XMS for other programs should yield a SCREAMING-fast DOS system! 6, 12 or 24GB RAM machines are becoming more common now with the Intel Core i7 platform (uses tripple-channel memory, so 3 or 6 banks filled DIMMS, 2 or 4GB per DIMM). My use would mainly be 1) use up all RAM in my system in a usefull way. 2) use RAM as temporary storage for all harddisk-affecting activities, mainly creating drive/disk/partition images, and write them to CD (up to 700MB) / DVD (4.7 or 8.5GB ) /Blu-ray (up to 50GB at the moment). The annoying thing for 2) is I do the following: * Boot optical disk (let's say, 700MB CD-RW) * Image entire contents of it to Ramdisk (taking up 700MB) * extract contents to ramdisk (taking up another 700MB, so total size is 1400MB ramdisk already) * deleting the imagefile (freeing 700MB) * make modifications (for example, creating harddisk backup, or updating certain files) * create updated imagefile (using 700MB) * burn imagefile to same CD-RW I booted from. Currently RAMDISKS are limited to around 3100MB I think, due to 32bit (4GB upper limit) and the PCI address mapping). That would mean a 1550MB DVD at most. With filesystem limitations, maximum for creating an ISO without splitting files would be 2GB anyway (maybe 4, don't know) It has been noted at many business schools that 80% of sales are usually handled by only 20% of a company's inventory -- the same is likely true of today's Gawd-AWFUL sized computer files! If the really NEEDED files load into a 2-GB RAMdisk, and either UIDE or your-choice among other good caching programs is also run, DOS should be PLENTY Fast! for most users. See above. More a 'because it might be possible and would be a nice challenge' usage than anything entirely necessary. If people are up to the challenge, have fun :) If not, then nevermind. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
I pretty much agree. I actually don't really need or even want access to even the full 4GB that a 32-bit CPU allows, but would like what's there to work correctly no matter how much memory there actually is. My newest computer came with 6GB (64-bit Vista), which I multi-boot to DOS. I had to take 4GB out of the computer (leaving it with 2GB) to get DOS to recognize any extended memory at all, thereby leaving Vista in a less-than-optimum configuration. I should also point out that some of my programs require I/O virtualization, which is only available in MS EMM386 or Qualitas 386MAX -- none of the other memory managers are complete enough to work properly. It should also be noted that a 32-bit Windows OS (including XP-32 and Vista-32) are limited to about 3.5 GB. The absolute maximum for 32-bits is 4GB, but you have to leave room for things like video RAM and shadow RAM that don't use real memory. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On Fri, 14 Aug 2009 11:43:17 -0700, Bernd Blaauw bbla...@home.nl wrote: I don't know if it's easier or harder to do, but maybe best approach is twofold : 1) First 4GB handled traditionally for programs, ramdisks, diskcaches etc. 2) Ramdisk driver that can handle all memory beyond 4GB and leaves anything [below?] first 4GB completely alone. This would work. But, if we are to cook up a new scheme to handle memory beyond 4-GB, why limit it only for RAMdisk usage? If the XMS drivers/handlers support such large memory, EVERYBODY could use it! But, a much-LARGER issue here is Do we really NEED all this?! 6, 12 or 24GB RAM machines are becoming more common now with the Intel Core i7 platform (uses triple-channel memory, so 3 or 6 banks filled DIMMS, 2 or 4GB per DIMM). Soon, we will need something like the Enterprise with a guy named Data to hold and run such MONSTERS! I must be in a minority: I still think an efficient computer CAN be achieved with only 4-GB! My use would mainly be 1) use up all RAM in my system in a useful way. 2) use RAM as temporary storage for all harddisk-affecting activities, mainly creating drive/disk/partition images, and write them to CD (up to 700MB) / DVD (4.7 or 8.5GB ) /Blu-ray (up to 50GB at the moment). Can you not read up to 2-GB of new data, while simultaneously writing out the PREVIOUS 2-GB of data??I assume hard-disks are still a LOT faster than writing to CD/DVD drives, so a simultaneous I-O program should work! Or, are simultaneous I-O schemes no-longer taught at computer schools?? [No surprise if so -- No one understands UIDE's binary-search, either!] It has been noted at many business schools that 80% of sales are usually handled by only 20% of a company's inventory -- the same is likely true of today's Gawd-AWFUL sized computer files! See above. More a 'because it might be possible and would be a nice challenge' usage than anything entirely necessary. If people are up to the challenge, have fun :) If not, then nevermind. Most remaining DOS users/programmers, including me, have little time for fun and games in today's CRASHING world economy!I would be up for the challenge of working with Japheth and implementing some sort of over-4GB XMS scheme which benefits ALL users of DOS systems, IF as I noted before such a scheme is REALLY necessary. But a fun and games task, for only 1 highly-specific application like copying a DVD/BluRay disk, is NOT the best use of my time! -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Op 14-8-2009 21:18, Jack schreef: This would work. But, if we are to cook up a new scheme to handle memory beyond 4-GB, why limit it only for RAMdisk usage? If the XMS drivers/handlers support such large memory, EVERYBODY could use it! yes, as long as older programs don't use it by accident. Guess that's what your extended API is for. Most remaining DOS users/programmers, including me, have little time for fun and games in today's CRASHING world economy!I would be up for the challenge of working with Japheth and implementing some sort of over-4GB XMS scheme which benefits ALL users of DOS systems, IF as I noted before such a scheme is REALLY necessary. But a fun and games task, for only 1 highly-specific application like copying a DVD/BluRay disk, is NOT the best use of my time! was it Alain running some kind of DOS database program still? can't think of much else requiring this much memory. And simultaneous operations..we're still in DOS, not modern operating systems. Can't have it all :) Bernd -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
On 14-Aug-2009 21:18, Jack wrote: ... But, if we are to cook up a new scheme to handle memory beyond 4-GB, why limit it only for RAMdisk usage? If the XMS drivers/ handlers support such large memory, EVERYBODY could use it! yes, as long as older programs don't use it by accident. Guess that's what your extended API is for. You betcha, baby! Should not BE any accidents, if older programs have no-idea of the new XMS requests and command-codes! Was it Alain running some kind of DOS database program still? Can't think of much else requiring this much memory. How about your OWN request earlier today for a RAMdisk over 4-GB, which I can equal in RDISK by allowing MULTIPLE 2-GB RAMdisks?? Or if Gates Co.'s Gawd-AWFUL sized data files grow much larger, how about pushing UIDE to a 4-GB maximum cache size, which I already know how to do?? Or, what if Alain (or others) want to have ALL of the above on ONE system?? And simultaneous operations ... we're still in DOS, not modern operating systems. Can't have it all :) Oh, REALLY?? I have been running the DeepBurner program to shoot my CD/RW masters for over two years now. It DOES simultaneous input/output with the disk staying-ahead of the CD/RW and always having the next block of data ready to be written out. It is a Windows program, but it runs on EXACTLY the same computer system that I use for DOS. Given that, and rather than everyone jumping through hoops about super-extended memory, maybe you should contact the DeepBurner people on the Internet, and ask if (A) they have a DOS version or (B) they might send you their sources!! NO reasons you must call UIDE, nor any other DOS drivers -- you can write your OWN simultaneous I-O scheme in DOS to copy your DVDs/BluRays, and it might take 100-MB of buffers maximum, damned well NOT 8.5-GB nor 50-GB!!! -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Over 4-GB for FreeDOS, Etc.
Hi Jack, Re: using memory over 4-GB in DOS, the best way to AVOID many new custom programs is to raise the capacity of XMS drivers... 1) A change to the XMS request which determines available memory, with a change to the FreeDOS MEM program to display all such memory. I believe MEM is currently limited at 4-GB. 2) A new Allocate SUPER-extended memory request, which requests memory above 4-GB if available.This new request would have to return the page number (0 to 15) of the allocated memory, for programs like UIDE that do their own real mode XMS work. It would still return an XMS handle so the Free XMS memory and standard Move XMS memory commands could work as-before. 3) A new Move SUPER-extended memory request, or a change to the current Int 15h AH=87h move-memory request that denotes what page of memory is being moved, for all protected mode move requests (JEMM386/JEMMEX).There are no extra bits in the current Int 15h AH=87h descriptor block, so some sort of new convention would also be needed for large-scale XMS memory. Maybe it can be even more minimal than that... 1) Do not tell MEM about the extra memory, just tell it that you have 3 GB as long as at least 3 GB are still available... 2) Do not support a super allocate, just let XMS clients alloc memory as long as it is available - but start in the first 4 GB and/or use memory after that easy area only for big allocs... Of course this means that the 9 GB RAMDISK that Bernd would want for juggling with two big DVD ISOs will need several XMS handles but other RAMDISKS already do the same to support big drives via pure XMS 2 requests which give you at most 64 MB per handle. This also helps Windows 3 compatibility - the XMS handling of Windows seems to have a bug if anybody uses descriptors for 16+ MB areas. 3) You need no outside-visible super extended move as long as only XMS access to the extra memory is available. HOWEVER this means that the super XMS driver will have to reject requests to lock XMS memory to a fixed address inside the first 4 GB (which simply is the XMS 3 API limit) as soon as it runs out of space in the first 4 GB. For simplicity, I would not even suggest to move around XMS handles. Instead, I would support locking and fixed addresses only for handles which physically were in the first 4 GB anyway at the moment when they were allocated... See the above only for big allocs suggestion for this :-). Note that for example Windows 3 does similar things - it just refuses to let you use XMS call 0c (lock, RBIL table 02762) on handles that were allocated after Windows started ;-). I guess it will return error code ad, lock failed at that time. Eric PS: Regarding I/O virtualization, the MS EMM386 API for this is relatively small, int 2f.4a15 basically. JEMM386 with the JLM driver support infrastructure could probably implement support for int 2f.4a15 as a small JLM driver at some time. PPS: I have the feeling that keeping ONE ISO in RAM should be enough for what Bernd wants, if access is organized well. Of course you still need to find 4 GB of DOS soft first ;-) -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
