On 4/26/19 3:14 PM, Ivo van Poorten wrote: > On Fri, 26 Apr 2019 14:54:38 -0500 Rob Landley <[email protected]> wrote: >> the gzip second file bug and finish deflate and the new mkfs.vfat and >> ... > > Many years ago I started on mkdosfs, but RL got in the way.
I get in the way a lot. :) > Meant to > continue with it for years, but well, other projects, blabla https://github.com/landley/toybox/commit/055cfcbe5b05 Grass houses and throwing stones... https://www.youtube.com/watch?v=Ka2Eu6LxAKo https://www.youtube.com/watch?v=RzDMCVdPwnE I'm unlikely to complain about somebody _else_'s todo list runnething over. > At the moment, it doesn't do much beyond some calculations and checks, > but it contains a lot of notes at the beginning about corner cases, > pitfalls et cetera. Might be helpful: > > https://github.com/ivop/toybox/blob/master/toys/pending/mkdosfs.c Cool, thanks. I spent ~4 days on this over the holidays and half a .c file and notes, but the problem is I can't go "I don't want to bother with fat12"... you have no choice. It distinguishes fat12 from fat16 by size, and fat32 is this horrible thing layered on top of it, and I have a lot of notes and several bookmarked web pages but mostly I was looking at a hexdump of mkfs.vfat and going "huh". Attached is where I left off, most of the output was a big "fat.txt" file and directory of snippets I have to review and collate. I think I know what to do now, but this is going to require the kind of testing where I'm probably just going to dd files of each 512 byte size from the smallest one that formats (um, 68 blocks I think it was?) through a terabyte or so). It's all historical nonsense and none of it's systematic. I has a sad. (I also, many moons ago, had to fix cross-linked files under DOS with a hex editor on the partition, so "oh right, allocation bitmap with a linked list of used entries..." And I remember old discussions about the long filename patent and Linus pointing out his old code where he'd figured out how to do it as prior art...) I _also_ want to have a "genvfatfs" command produce output like an archiver (qemu does something like this, or did at one point, producing a filesystem image on the fly from a directory for its emulated system; read only and it barfs if you modify files on the host behind its back, of course). The reason I stopped with the mke2fs stuff is I worked out that I want to be able to "gene2fs dir | gzip" which means it can't seek on the output (which the existing gene2fs did and is a certain amount of effort _not_ to do) which means it needs to read all the metadata up front which means I needed to write dirtree first, and that took a while and I haven't cycled back to gene2fs since, kinda wanted to do tar and zip first to see if there was more shareable infrastructure; so far doesn't look like it. But "my version handles "cat tar.gz | tar x" autodetection and debian's doesn't is the same sort of wanting to do the obvious things the other ones didn't bother to. If you're gonna autodetect, stdin should not gratuitously fail to autodetect... Grrr... Oh, and mtools. Toybox should probably have some variant of mtools too, except not "mcopy" and "mtype" dos format but "mls" and "mcat" which is in the pile with "zgrep" and friends as "is there some way to wrap a command in a generic way..." Numerous unresolved design questions there... > Regards, > Ivo Thanks, Rob
/* mkdosfs.c - Make a vfat filesystem. * * Copyright 2018 Rob Landley <[email protected]> * * No obvious standard. Lots of BAD standards. * linux/msdos_fs.h * * mkdosfs -F 12/16/32 -O android path * 12 = sec<32m, 16<2g, else 32 * biggest fat12 is 16404 sectors USE_DUMPVFAT(NEWTOY(dumpvfat, "<1", TOYFLAG_USR|TOYFLAG_BIN)) USE_MKDOSFS(NEWTOY(mkdosfs, "\300N#a#b#c#e#i#k#C#F#I#S#T#B:L:O:[!NC]", TOYFLAG_USR|TOYFLAG_BIN)) USE_MKDOSFS(OLDTOY(newfs_msdos, mkdosfs, TOYFLAG_USR|TOYFLAG_BIN)) USE_MKDOSFS(OLDTOY(mkfs.vfat, mkdosfs, TOYFLAG_USR|TOYFLAG_BIN)) config MKDOSFS bool "mkdosfs" default n help usage: mkdosfs [-@A] [-abceikCFIST NUMBER] [-BLO STRING] FILE -@ Start filesystem at OFFSET bytes -A Align root directory -N No create -B Use boot sector from FILE -C Create file with SIZE -F Type (12, 16, or 32, default based on size) -I Set volume ID (32 bits) -L Set LABEL (11 chars) -O Set OEM string (8 chars) -S Bytes per sector (power of 2, 512-32768) -T Set creation TIMESTAMP (unix time, or path to file) -a Number of sectors in filesystem -b Block size (sectors per block) -c Cluster size (sectors per cluster) -e Number of root directory entries -i Fat32 Info sector location (default 0xffff) -s File system size -f Floppy format -h Drive heads -k Fat32 backup boot sector (default 0xffff) -n Number of File Allocation Tables (1-16, default 2) -m Media descriptor (0xf0 to 0xff) -o Hidden sectors -r Reserved sectors -u Sectors per track config DUMPVFAT bool "dumpvfat" default n help usage: dumpvfat blah blah */ #define FOR_mkdosfs #include "toys.h" // The union lets lib/args.c store arguments for either command. // It's customary to put a space between argument variables and other globals. GLOBALS( char *O, *L, *B; long T, S, I, F, C, k, i, e, c, b, a, at; char *fat; unsigned spc, fds; ) // These fields are out of order. On disk half unaligned, all little endian. struct bootsect { char jmp[3], oem[8], sectors_per_cluster, fat_count, media_type, sig, volume_label[11], system_id[8], end; unsigned short bytes_per_sector, reserved_sectors, root_entries, small_sectors, sectors_per_fat, sectors_per_track, number_of_heads; unsigned hidden_sectors, large_sectors, serial_number; }; struct fat_dirent { char name[11], attr, lcase, ctime_cs; unsigned short ctime, cdate, adate, starthi, time, date, start; unsigned size; }; // Round up to next power of 2. unsigned long long p2roundup(unsigned long long ull) { unsigned long long ull2 = --ull; while (ull) ull2 |= (ull>>=12); return ++ull2; } // cluster to sector static unsigned clus2sec(unsigned cluster) { return (cluster-2)*TT.spc+TT.fds; } // then at 0x200 F8 FF FF // then at 0x600 F8 FF FF // Read/calculate values from a FAT boot sector static void measure_fat(char *boot) { unsigned bytes_per_sector = peek_le(boot+11, 2), sectors_per_cluster = boot[13], reserved_sectors = peek_le(boot+14, 2), fat_count = boot[16], root_entries = peek_le(boot+17, 2), total_sectors = peek_le(boot+19, 2), fat_size = peek_le(boot+22, 2), root_sectors = ((root_entries*32)+511)/512; if (!total_sectors) total_sectors = peek_le(boot+32, 4); if (!fat_size) fat_size = peek_le(boot+36, 4); // first data sector, data sector count unsigned fds = boot[14] + 2*fat_size + root_sectors; unsigned dscount = total_sectors - (boot[14] + root_sectors + 2*fat_size); unsigned clusters = dscount/boot[13]; // starting at 2. // <4085 = fat12, <65525 fat16 // fat_sectors = 256*TT.spc printf("bytes_per_sector=%u\n", bytes_per_sector); printf("sectors_per_cluster=%d\n", sectors_per_cluster); printf("reserved_sectors=%u\n", reserved_sectors); printf("fat_count=%d\n", fat_count); printf("root_entries=%d\n", root_entries); printf("total_sectors=%d\n", total_sectors); printf("fat_size=%d\n", fat_size); printf("first data sector=%d\n", fds); printf("data sector count=%d\n", dscount); printf("clusters=%d\n", clusters); } // Fill out FAT boot sector and allocate TT.fat for a filesystem of this size static void format(char *fat, long long bytes, char *oem) { unsigned sectors = bytes/512, spc, is32 = 28*(bytes >= 1LL<<32), root_ent = 512, root_sectors = ((root_ent*32)+511)/512, fat_sectors; char *s; // <= 16404 sectors = biggest fat12: 8 sectors/cluster memset(fat, 0, 512); // Original PC bios would load the boot sector and execute it. This jumped // past the filesystem data. It's still there because USB flash sticks // perform wear leveling when they detect a FAT filesystem, and how they // detect it varies by vendor. So err on the side of "looking like FAT". fat[0] = 0xeb; fat[1] = is32 ? 0x58 : 0x3c; fat[2] = 0x90; strncpy(fat+3, oem, 8); // OEM fat[11] = 2; // 512 bytes per sector, high byte of little endian # fat[13] = spc = is32 ? 8 : p2roundup((sectors/65536)+1);// sectors per cluster fat[14] = is32 ? 32 : 1; // reserved sectors fat[16] = 2; // how many file allocation tables if (!is32) poke_le(fat+18, root_ent, 2); // fat16 root filesystem entries // small or large sector count if (sectors < 65536) poke_le(fat+19, sectors, 2); // unaligned else *(unsigned *)(fat+32) = sectors; // aligned! fat[21] = 0xf8; // media type "fixed disk", must match first byte of each FAT // sectors per File Allocation Table // fat_size = TODO; if (!is32) *(unsigned short *)(fat+22) = fat_size; else *(unsigned *)(fat+36) = fat_size; fat[24] = 63; // sectors per track, obsolete fat[26] = 64; // number of read/write heads, obsolete if (is32) { fat[44] = 2; // first cluster of root dir fat[48] = 1; // fsinfo starting cluster fat[50] = 6; // backup boot sector } fat[36+is32] = 128; // drive number, obsolete fat[38+is32] = 0x29; poke_le(fat+39+is32, time(0), 4); // Volume ID strcpy(fat+43+is32, "NO NAME FAT%02d ", is32 ? 32 : 16); // type magic fat[510] = 0x55; fat[511] = 0xAA; // first data sector, data sector count TT.fds = fat[14] + 2*fat_size + root_sectors; TT.dscount = sectors - (fat[14] + root_sectors + 2*fat_size); clusters = TT.dscount/fat[13]; // starting at 2. // <4085 = fat12, <65525 fat16 fat_sectors = 256*TT.spc // Initialize everything through root_entries strcpy(fat->magic, "\353\74\220Toybox\0\0\0\2\4\1\0\2\0\2", 19); fat->media_type = 0xf8; fat->physical_disk = 128; fat->signature = 0x29; //*fat->sectors_per_track = 63; //*fat->number_of_heads = 64; // small_sectors, large_sectors // sectors_per_fat, sectors_per_track, number_of_heads // reserved_sectors if (fat32) { fat->magic[1] = 0x58; fat->sectors_per_cluster = 8; fat->reserved_sectors[0] = 0x20; zero root_entries, sectors_per_fat, fat->root_entries[1] = 0; } } void mkdosfs_main(void) { struct fat_bootsect *fat = (void *)toybuf; long long bsz; int bs; // Initialize everything through root_entries strcpy(fat->magic, "\353\74\220Toybox\0\0\0\2\4\1\0\2\0\2", 19); fat->media_type = 0xf8; fat->physical_disk = 128; fat->signature = 0x29; //*fat->sectors_per_track = 63; //*fat->number_of_heads = 64; // small_sectors, large_sectors // sectors_per_fat, sectors_per_track, number_of_heads // reserved_sectors if (fat32) { fat->magic[1] = 0x58; fat->sectors_per_cluster = 8; fat->reserved_sectors[0] = 0x20; zero root_entries, sectors_per_fat, fat->root_entries[1] = 0; } // reserved sectors, hidden sectors // sanity check string(s) OEM, volume label // sanity check media descriptor 0xf0-0xff // info sector or backup sector require fat32 // bytes/sec must be power of 2, at least 512 // block size must be power of 2, at least bytes/sec, at most 128*bytes/sec // sectors/cluster must be power of 2 // create or open, truncate to size, seek to offset xioctl(fd, BLKSSZGET, &bs); xioctl(fd, BLKGETSIZE64, bsz); // sectors per track = 63 heads = 64 // sectors per cluster = 1<8192, 8 <= 1<<17, 16 <= 1<<19 32 <= 1<<21 else 64 // minimum fat16 clusters: 4096-11, mininum fat32: 65536-11 printf("%ld\n", TT.at); } void dumpvfat_main(void) { read(xopen(toys.optargs[1]), toybuf, sizeof(toybuf)); measure_fat(toybuf); }
_______________________________________________ Toybox mailing list [email protected] http://lists.landley.net/listinfo.cgi/toybox-landley.net
