Re: [Freedos-kernel] [Freedos-user] Re : Support for 4k byte sectors
On Jan 14, 2012, at 5:26 PM, Eric Auer wrote: are simply too small to be bootable anyway. I wonder if you should be able to boot from nonstandard sector sizes at all - for example a CD- ROM does not even have a FAT filesystem, so why would you boot from a (raw!) CD-ROM then? Booting from cds and raw cd images would be extremely useful. If I had a pure dos system, I'd be more than happy to fiddle with these changes. Problems are, I'm not the worlds best assembly programmer, and I've unfortunately got no way to (currently) test such changes. However, booting from cd should be supported if possible. As you pointed out, the sector size is embeded (or should be) in the boot info, so I see no reason why this couldn't be handled automatically. It can simply assume 512 bytes if no data is available, and then boot normally. I for one would find the ability to boot from different sector sizes useful (as mentioned above), and I'm sure others would too. What we really could use in dos is the ability to designate a drive for booting (like osx has) so you could change your boot drive dynamically during os execution, so if you wanted to boot from second hd or cd, or floppy regardless of what cmos is configured to, this could be done. I realize this would be not a trivial modification, but especially if iso images could be used for this purpose, it would make dos an extremely valuable troubleshooting tool, as well as allowing a person to access all drives regardless of what os / versions might be on them. Plus, the ability to boot from iso images would allow a whole lot of flexibility for testing and other uses as well. I'd of course have no idea how to do such things, but I'm always willing to learn. :) -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] [Freedos-user] Re : Support for 4k byte sectors
Op 14-1-2012 19:48, Bertho Grandpied schreef: It's been a few days and I'm surprised my first mail hasn't been acknowledged in any way, let alone answered; strange, I've been part of various lists before, usually 'newbies' are greeted rather than ignored altogether. So I'll reiterate and articulate the above question just in case it was not clear : have I done something wrong ? should I try posting to the kernel or developers lists instead ? Truth is we're not sure, this 4K sector size thing comes in several versions: 1) emulation with aligned 512byte sector emulation 2) emulation with non-aligned 512byte sector emulation 3) native 4K (especially USB bridges apparently) In other words, this is asking what the plan is for the FreeDOS kernel to be able to mount mass storage devices having 4 kilobytes per sector ? If the disk has traditional BIOS partitioning layout (MBR, primary/logical partitions with FAT16/FAT32 filesystems) then it might be possible for the kernel to work with this as long as it is a data disk. Booting from a partition without 512byte sector-size is probably more challenging, let alone guarantee file(-system) integrity and disk manipulating (defragmentation programs, filesystem checkers like CHKDSK and DOSFSCHK). GPT partitioning scheme isn't supported at all (nor is EFI/UEFI without BIOS emulation). As said in previous message, I for one am ready to test development kernels against my USB disk appliance (1 TiB Iomega Prestige). freedos-kernel would be most appropriate list as the developers on there have most expertise. However they're also the ones who are rarely present due to other interests or obligations, so getting answers can take a while. You could also try freedos-devel for this specific technical question but answers might take as long as getting answers on the users list. People usually don't answer if they don't have the correct answer, thus things stay quiet for a while. -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] [Freedos-user] Re : Support for 4k byte sectors
Replaying a 12 May 2005 mail :-) [Freedos-kernel] Analysis: Support for sector sizes != 512 bytes Hi, I have browsed the CVS kernel a bit, and got the impression that it would not be too hard to support sector sizes below 512 bytes (32, 64, 128 and 256 bytes should be possible). For sector sizes above 512 bytes (e.g. 1024 and 2048), each element in BUFFERS and the DiskTransferBuffer would have to be bigger. Are there DOS versions which support big sectors? How do they solve the buffering problem? I think sector sizes below 64 bytes cannot contain enough of a BPB to be useful. Sector sizes below 32 bytes are definitely not useful, not even a single dirent fits in them. Sector sizes above 512 bytes are at most useful for the 1024 byte (far-east 1.2 MB diskette format / drives which support full 64 MB of a single XMS2 handle as ramdisk without having to support the DOS3.x+ 32bit sector number calls) and 2048 byte (raw sector size on CD-ROM and maybe on magneto-optical drives) cases. Comments about useful sector sizes would be welcome: Smaller (64, 128, 256), normal 512, Bigger (1024, 2048, others). Next part: I believe that the BIOS / kernel drives (diskette, harddisk) only have to handle 512 bytes per sector. Otherwise, you would have to adjust: CalculateFATData, DosDefinePartitions, the kernel driver IOCTLS (e.g. format track), LBA_Transfer, max_transfer (the DMA wrap checker) and maybe floppy_bpb (and -_bpbs). In case you have wondered, directory handling already reads the current sector size from the BPB, so it is flexible. If you want drives with bigger sectors to be detected at boot time, you not only have to get BUFFERs and DiskTransferBuffer bigger and adjust _maxbksize (in the global SDA data structure, which is, by the way, not yet read by FreeDOS itself, as only 512 bytes per sector are supported anyway) but you also have to make InitDiskTransferBuffer bigger as well. For boot purposes you just have to make SYS able to pad unused parts of the boot sector to allow bigger sector sizes - BUT you also have to make SYS able to adjust the boot sector code to that, I remember that not all our boot sectors actually used the BPB bytes-per-sector value (there was not enough space to process it). Smaller sector sizes are simply too small to be bootable anyway. I wonder if you should be able to boot from nonstandard sector sizes at all - for example a CD- ROM does not even have a FAT filesystem, so why would you boot from a (raw!) CD-ROM then? So I suggest to support nonstandard sector sizes ONLY for drives which are connected to separate drivers (e.g. ramdisks, USB drives, ZIP...). To make that possible, BUFFERS and DiskTransferBuffer handling would have to get better (even for smaller sectors - for bigger sectors, the BUFFERS and DiskTransferBuffers themselves, being only _maxbksize big, will be the problem). In addition, getbpb and dskxfer (THE central send/receive a sector to/ from a device driver function) have to be made independent from the hardcoded SEC_SIZE. Instead, they have to use the sector size from the BPB of the drive. Hint: getbpb assumes that the boot sector magic word is at offsets 1fe and 1ff. This IMPLIES a sector size of 512 bytes, too. The GOOD part is that FreeDOS hardly uses anything else than getbpb and dskxfer to communicate disk contents (for which it uses BUFFERS and DiskTransferBuffer as intermediate storage places). rqblockio does call block device drivers directly, too, but not to access sectors, for example, and the other functions all use dskxfer or getbpb directly or indirectly. I hope I have not overlooked things in this review... Summary: - To support bigger sectors, raise _maxbksize and make each element of BUFFERS and DiskTransferBuffer bigger - To support nonstandard sector sizes at boot time, CalculateFATData, InitDiskTransferBuffer and DosDefinePartitions have to be modified - To support nonstandard sector sizes for harddisk / diskette (kernel built-in drivers), you have to adjust the IOCTLs (format track...), LBA_Transfer and max_transfer - To support nonstandard sector sizes at all, you have to modify dskxfer and getbpb and the BUFFERS and DiskTransferBuffer handling has to accept sectors in other sizes than SEC_SIZE, including e.g. sectors which are smaller than _maxbksize... - SYS and boot sectors can only support bigger, not smaller, sector sizes, and SYS might have to do extra patching for those of the boot sectors which do not actually use the boot sector BPB sector size info We should probably start by using _maxbksize in BUFFERS and either only DiskTransferBuffer or both DiskTransferBuffer and InitDiskTransferBuffer to get things a bit clearer... Then we should make SYS and boot sectors able to support bigger sector sizes (not because it is really useful but because this part is more self-contained / easier to maintain than the rest of the kernel). Then the first WORKING support for nonstandard sector sizes can be added by
Re: [Freedos-kernel] [Freedos-user] Re : Support for 4k byte sectors
On Sat, Jan 14, 2012 at 5:26 PM, Eric Auer e.a...@jpberlin.de wrote: Replaying a 12 May 2005 mail :-) [Freedos-kernel] Analysis: Support for sector sizes != 512 bytes Hi, I have browsed the CVS kernel a bit, and got the impression that it would not be too hard to support sector sizes below 512 ... Thank you for reviewing and providing a suggested road map. I have a few other things in my todo queue but once done I will look further into this. I would like to add support for GPT disks first, which I think will be much simpler and I can test. I will need to find a drive that exposes 4KB sectors for testing - ramdisks may work at least initially. As usual my developing time is limited, most of which I use trying to figure out where I left off :-) or persuing other things (lately I have been playing around with Android and how to build it) but I have a little bit of time off in February so maybe I can make significant progress then. Thank you, Jeremy -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] [Freedos-user] Re : Support for 4k byte sectors
Hi Jeremy :-) Replaying a 12 May 2005 mail :-) [Freedos-kernel] Analysis: Support for sector sizes != 512 bytes Thank you for reviewing and providing a suggested road map. Well, that old mail is nice for reference, but I would still say: So I suggest to support nonstandard sector sizes ONLY for drives which are connected to separate drivers (e.g. ramdisks, USB drives, ZIP...). To make that possible... For my current taste, this also implies that we do not NEED any support for sectors ABOVE 512 bytes. It makes memory footprints of buffers etc larger and we would only support 4096 byte sector size for non-boot disks for the moment anyway, where an EXTERNAL (not in the kernel but loaded in config.sys) block device driver could easily map things into a virtual 512 byte sector size. Of course in general, it is nice to be flexible. And some users of ramdisks and romdisks would be happy about support for some SMALLER sector sizes. As for the LARGER than 512 byte sectors, I would really like a CACHE based solution to make access more large block oriented. Maybe including write pooling. SSDs and USB flash sticks would be happy about 4 kB, 32 kB or even more minimum write sizes, of course also with block-aligned writes. I do NOT recommend a full delayed write cache: It is a lot of work to make sure that writes always go to disk before you do something stupid (reboot, crash, change disks) and you already gain a lot of performance with a SMALL pooling buffer IMHO. I have a few other things in my todo queue but once done I will look further into this. I would like to add support for GPT disks first, which I think will be much simpler and I can test... I totally agree and as you see in my mail from a few hours ago, GPT is not hard to implement as long as you only care about FAT. I have a little bit of time off in February so maybe I can make significant progress then. Maybe a strange off-topic, but I have some time off this spring, but instead of programming, I wanted to visit some FreeDOS people in western Europe :-) If you are interested, contact me off-list. Regards, Eric -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
