Re: [Freedos-user] [Freedos-devel] writing a loadable block driver for 4k-sector drive (Questions)
This doesn't belong on Freedos-user then. Maybe Czerno / Bertho is not on freedos-devel? Then they should subscribe to it. Or maybe there is simply too little traffic on freedos-devel, so he preferred to give it a try on freedos-user... I am CCing -devel here, I'm keeping -user as the main recipient until OP moves. If your block device driver shows the DOS kernel a device with 512 byte per sector virtual sectors then the kernel will happily use 512 byte per sector BUFFERS for them... Oh, that's what you meant. This is wrong. Block drivers directly interact with DOS and its buffering scheme. If what you want is to circumvent the restrictions imposed by DOS's buffering scheme (for example, current 512B-/2048B-sector limits of the FDK) you'll have to write your own (FAT) file system driver which interfaces downwards with the block device driver (in your case, USBASPI.SYS) and upwards with DOS's file system redirector interface. This is also wrong. Because the filesystem still is FAT and the kernel does already understand FAT, your driver can present your 4 kB per sector disk as a BLOCK device or in other words: As a bunch of sectors. However, your driver has to transform what the DOS kernel sees of the boot sector and BPB, because the whole idea of having a 4 kB disk support driver for 512 byte per sector DOSes is that it shows the kernel a bunch of 512 BYTE SECTORS (virtual ones) instead of showing the real 4 kB sectors. Apologies, I misunderstood the request. I should have thought of your (Eric) sector size downwards transformation suggestions of earlier. This DOES circumvent the limitation of the DOS kernel: The kernel only sees 512 byte per sector blocks. But as explained above, now your driver will have to spend at least 4 kB of buffer space for the transformations. This is accurate. redirector interface is what file system drivers for non-FAT file systems (eg *CDEX) and networked file systems use to make their files accessible That would mean that you duplicate the whole logics of understanding the FAT filesystem into your driver. Correct. NOTE: Transformation of sector sizes is easiest in FAT32. Other FAT sizes may take more effort. I don't think so. Why would they? Cluster size stays the same, no FAT-related special handling (which would be more complicated for, say, FAT12) is necessary as far as I can tell. Also, you can only transform from 4096 byte units to 512 byte units if the filesystem is at most 2 Terabytes and cluster size is at most 32 or maybe 64 kilobytes. Right, the usual 32 (or for some OSes, 64, 128, etc) KiB cluster size limit still applies here. You CANNOT transform from 512 to 4096 usually, as format tools do not take care to align things in a way that would make this easy enough. True. Of course you CAN transform back a FAT32 partition from an originally 4 kB per sector disk to 4 kB sector disks after you accessed it in 512 byte units and even if you actually stored it on a physically 512 byte per sector disk and used it there. This only needs the boot sector transform to be transformed back :-) Right. Even a file system originally created with 512 B sectors could be transformed into using 4 KiB (or other 512 B sizes) sectors if the file system structures (reserved sector number, FAT size, root size. And the cluster size must be = target's sector size) are properly aligned; as you implied. regards, C. Masloch -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] [Freedos-devel] writing a loadable block driver for 4k-sector drive (Questions)
NOTE: Transformation of sector sizes is easiest in FAT32. Other FAT sizes may take more effort. I don't think so. Why would they? Cluster size stays the same, no FAT-related special handling (which would be more complicated for, say, FAT12) is necessary as far as I can tell. Sorry for replying to my own message, but I just thought of something here again. You're probably referring to the root directory alignment handling, which of course is not needed in FAT32 as its root directory is not in a specifically reserved area. As I noted in our (Eric) previous correspondence, the special case where some rounding is required (which might imply an incompatibility) doesn't usually occur (as formatting software usually leaves no unused directory entries in the last sector). You CANNOT transform from 512 to 4096 usually, as format tools do not take care to align things in a way that would make this easy enough. True. Another additional note, as I went thinking about this a bit. Only cluster values are stored in the FS (think FAT contents, FSINFO, and start cluster fields of directory entries) apart from what is in the *BPB, so a runtime upwards sector size transformation (say, from 512 B to 4 KiB, or from 128 B to 512 B, or whatever) is still possible, just more complicated. I'm not certain, but my intuition is that it'd still be rather trivial to do. As might be thought to be implied, transformation from 512 B away seems rather pointless now, but as I remarked in the previous paragraph, an upwards transformation could be /to/ 512 B if the stored file system (whether it is stored on some physical medium, in a file as an image, on a RAM disk, etc) uses smaller sectors. regards, C. Masloch -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] [Freedos-devel] writing a loadable block driver for 4k-sector drive (Questions)
Another additional note, as I went thinking about this a bit. Only cluster values are stored in the FS (think FAT contents, FSINFO, and start cluster fields of directory entries) apart from what is in the *BPB, so a runtime upwards sector size transformation (say, from 512 B to 4 KiB, or from 128 B to 512 B, or whatever) is still possible, just more complicated. I'm not certain, but my intuition is that it'd still be rather trivial to do. missed mentioning one condition: cluster size must be large enough -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] [Freedos-devel] writing a loadable block driver for 4k-sector drive (Questions)
Hi Chris, NOTE: Transformation of sector sizes is easiest in FAT32. Other FAT sizes may take more effort. You're probably referring to the root directory alignment handling, which of course is not needed in FAT32 as its root directory is not in a specifically reserved area. Correct - in FAT32, the root directory is just a group of data clusters. In FAT12 and FAT16, you specify the size of the root directory instead. However, the size is specified in entries, not sectors, so maybe you are right: FAT12 / FAT16 COULD be as EASY to transform as FAT32. Another aspect is that FAT32 always has multiple special sectors (boot, fsinfo...) before the FAT starts. For FAT12 / FAT16 other numbers than 1 reserved boot sector are UNUSUAL but SHOULD be handled okay by our kernel. Similarily, having more sectors per FAT than strictly needed (because FATs start and end at multiples of 4 kB even if you show virtual 512 byte sectors) is UNUSUAL for ALL FAT sizes (12, 16, 28 / 32) but SHOULD be supported by our kernel: For partition resizing tools, it is easiest to keep FATs and dir data unchanged when shrinking a disk, so your FATs waste some space for info on clusters which are no longer are available. As I noted in our (Eric) previous correspondence, the special case where some rounding is required (which might imply an incompatibility) doesn't usually occur (as formatting software usually leaves no unused directory entries in the last sector). See above: You can always transform a 4096 byte per sector filesystem into one with 8 times more 512 byte sectors, as long as nothing overflows, but the transformed view LOOKS as if you wasted some space in the boot, root or FAT area, e.g. 24 kB per FAT instead of 23 kB when 22.9 are in use. Overflow can happen in size of filesystem and size of cluster, because a cluster must have at most 128 sectors (64 recommended, 256 possible with special DOS versions) and a FAT disk must have at most 2^32 sectors and if you have bigger sectors, you COULD have bigger clusters and filesystems than would be possible with smaller sectors. DOS must be aware that I/O of N sectors will be larger than usual if sectors are larger (64 kB and DMA limits). The INVERSE incompatibility is that you cannot transform a FAT filesystem with really 512 bytes per sector and 23 kB per FAT into something that you could store on a disk that has 4096 bytes per sector easily: You would have to move things around because you will be forced to round up FAT sizes to multiples of the true, larger, sector size... Another additional note, as I went thinking ... Only cluster values are stored in the FS (think FAT contents, FSINFO, and start cluster fields of directory entries) apart from what is in the *BPB Exactly. This is why you can transform sector sizes in BOTH directions, virtual being a multiple or fraction of real iff: - no BPB field overflows (smaller virtual than real sectors) - no calculation overflows (same context, fails if you have a partition larger than 2^32 virtual sectors or a cluster size corresponding to too many small virtual sectors...) and - everything can be expressed in multiples of real sectors (fails if you copy a small-sector filesystem to a big- sector disk unless FATs, root and clusters all happen to be multiples of the big sector size big already anyway) The latter means that if your disk has SMALL sectors, you have to be lucky with alignment and your clusters must be big enough if you want to use it as virtual BIG sectors. You will need additional checks to check for alignment but you are right that transformation works in both directions (bigger/smaller sectors) as far as the general idea goes. However, using a small sector disk to simulate a disk with big sectors feels sort of academic. It might be useful as a way to test big-sector compatibility of a DOS kernel ;-) Another use could be disk-copying a partition from a small- sector disk to a big-sector disk. As said, it only works if you are lucky with alignment of things. I think I prefer USING and COPYING big-sector partitions in small-sector DOS kernels or to small-sector disks, as this is 1. the type of kernel I like to use and 2. easier and almost always possible - unlike the other direction. Eric -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] [Freedos-devel] writing a loadable block driver for 4k-sector drive (Questions)
Correct - in FAT32, the root directory is just a group of data
clusters. In FAT12 and FAT16, you specify the size of the root
directory instead. However, the size is specified in entries,
not sectors, so maybe you are right: FAT12 / FAT16 COULD be as
EASY to transform as FAT32.
This depends on whether rounding (when converting the specification of a
number of directory entries to a number of sectors) is required, and if
so, whether the rounding that is necessary is more than the most rounding
possible for the smaller sector size. Usually, rounding is not necessary
at all because as I noted formatting software wastes none of the space in
the last sector.
Another aspect is that FAT32 always
has multiple special sectors (boot, fsinfo...) before the FAT
starts. For FAT12 / FAT16 other numbers than 1 reserved boot
sector are UNUSUAL but SHOULD be handled okay by our kernel.
Right.
Similarily, having more sectors per FAT than strictly needed
(because FATs start and end at multiples of 4 kB even if you
show virtual 512 byte sectors) is UNUSUAL for ALL FAT sizes
(12, 16, 28 / 32) but SHOULD be supported by our kernel: For
partition resizing tools, it is easiest to keep FATs and dir
data unchanged when shrinking a disk, so your FATs waste some
space for info on clusters which are no longer are available.
Yes.
As I noted in our (Eric) previous correspondence, the special case where
some rounding is required (which might imply an incompatibility) doesn't
usually occur (as formatting software usually leaves no unused directory
entries in the last sector).
See above: You can always transform a 4096 byte per sector
filesystem into one with 8 times more 512 byte sectors, as
long as nothing overflows, but the transformed view LOOKS
as if you wasted some space in the boot, root or FAT area,
e.g. 24 kB per FAT instead of 23 kB when 22.9 are in use.
No, you forgot that if too many root directory entries are unused/wasted
in the last sector, then the transformation driver needs to adjust the
root entries BPB field. Doing that is possible and will render the file
system accessible, but an incompatibility arises if any of those
previously unused/wasted entries are used, as they will remain
inaccessible to drivers natively accessing the FS. Usually, however, a
specific program could trivially modify/fix the FS to use all entries of
the last sector to prepare it for usage with the transformation driver.
Overflow can happen in size of filesystem and size of
cluster, because a cluster must have at most 128 sectors
(64 recommended, 256 possible with special DOS versions)
and a FAT disk must have at most 2^32 sectors and if you
have bigger sectors, you COULD have bigger clusters and
filesystems than would be possible with smaller sectors.
Yes.
DOS must be aware that I/O of N sectors will be larger
than usual if sectors are larger (64 kB and DMA limits).
Interesting. This might raise some issues. However, I think the driver
could be written to handle all of that properly.
The INVERSE incompatibility is that you cannot transform a
FAT filesystem with really 512 bytes per sector and 23 kB
per FAT into something that you could store on a disk that
has 4096 bytes per sector easily: You would have to move
things around because you will be forced to round up FAT
sizes to multiples of the true, larger, sector size...
[...]
- everything can be expressed in multiples of real sectors
(fails if you copy a small-sector filesystem to a big-
sector disk unless FATs, root and clusters all happen to
be multiples of the big sector size big already anyway)
The latter means that if your disk has SMALL sectors, you
have to be lucky with alignment and your clusters must be
big enough if you want to use it as virtual BIG sectors.
You will need additional checks to check for alignment but
you are right that transformation works in both directions
(bigger/smaller sectors) as far as the general idea goes.
No, as I mentioned, even if things are misaligned, as long as the cluster
size is large enough all moving around/translation seems rather trivial.
If I am not mistaken, you'd only have to do some simple calculations on
things like number of reserved sectors, FAT size, and root size, and then
(if it is a runtime transformation driver which is to translate the FS to
DOS on-the-fly) translate sector numbers from the virtual data area,
virtual root area, etc (addressing scheme seen by DOS) to the real data
area, real root area, etc (addressing scheme on the block device) and
vice versa.
Here's some example C-like pseudo-code for the translation from virtual
sector number to a real one, which for now handles only a simplified
case where the FS uses a single FAT:
sec translate_virtual_to_real(sec virtSec) {
sec realSec = virtSec;
if (virtSec = virtLastResSec)
;
else if (virtSec = virtLastFATSec)
realSec +=
