Re: Flash drive internal fragmentation.
On 2010-03-27 21:47, R. Clayton wrote: So, I don't know exactly *how* you'd format a device as "fat32". I used "mkfs.vfat -F 32 /dev/sda1" Ah. I just do mkfs.vfat. According to "man mkfs.vfat": -F FAT-size Specifies the type of file allocation tables used (12, 16 or 32 bit). If nothing is speci‐ fied, mkdosfs will automatically select between 12, 16 and 32 bit, whatever fits better for the file system size. I figure that every HDD from the past 15 years is big enough for mkfs.vfat to choode 32. -- "History does not long entrust the care of freedom to the weak or the timid." Dwight Eisenhower -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4baed085.8010...@cox.net
Re: Flash drive internal fragmentation.
So, I don't know exactly *how* you'd format a device as "fat32". I used "mkfs.vfat -F 32 /dev/sda1" -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k4sx9n7o@ulanbator.myhome.westell.com
Re: Flash drive internal fragmentation.
On 2010-03-27 14:02, Kelly Clowers wrote: On Sat, Mar 27, 2010 at 00:15, Ron Johnson wrote: On 2010-03-26 21:03, R. Clayton wrote: Thanks for your replies to my message. Putting a fat32 fs on the thumb drive did the trick; the block size is 4k, matching what's on the hard drive file systems. fat32 is still not recommended. Use vfat. Aren't those two names for the same thing? Hmmm. "man mkfs" leads me to mkfs.vfat which (along with mkfs.msdos) is an alias for mkdosfs. So, I don't know exactly *how* you'd format a device as "fat32". -- "History does not long entrust the care of freedom to the weak or the timid." Dwight Eisenhower -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bae6d5d.6020...@cox.net
Re: Flash drive internal fragmentation.
On Sat, Mar 27, 2010 at 00:15, Ron Johnson wrote: > On 2010-03-26 21:03, R. Clayton wrote: >> >> Thanks for your replies to my message. Putting a fat32 fs on the thumb >> drive >> did the trick; the block size is 4k, matching what's on the hard drive >> file >> systems. >> > > fat32 is still not recommended. Use vfat. Aren't those two names for the same thing? Cheers, Kelly Clowers -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1840f6971003271202m12f58bd3rd54fafc02ac54...@mail.gmail.com
Re: Flash drive internal fragmentation.
On 2010-03-26 21:03, R. Clayton wrote: Thanks for your replies to my message. Putting a fat32 fs on the thumb drive did the trick; the block size is 4k, matching what's on the hard drive file systems. fat32 is still not recommended. Use vfat. -- "History does not long entrust the care of freedom to the weak or the timid." Dwight Eisenhower -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4badb088.6080...@cox.net
Re: Flash drive internal fragmentation.
Thanks for your replies to my message. Putting a fat32 fs on the thumb drive did the trick; the block size is 4k, matching what's on the hard drive file systems. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/8739zm8qro@ulanbator.myhome.westell.com
Re: Flash drive internal fragmentation.
Ron Johnson wrote: > On 2010-03-24 12:07, Bob McGowan wrote: > [snip] >> >> And, I used VFAT for the filesystem type: > > That's the key difference, since vfat "is" fat32. > Ah, yes. Too long since I've dealt much with FAT of any sort, details get forgotten ... Thanks for the reminder. >> $ mount|grep sde >> /dev/sde on /var/autofs/usb/centon type vfat ... >> > > Try this: > $ mount -lt vat > And for new things to learn ;) -- Bob McGowan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bace130.5080...@symantec.com
Re: Flash drive internal fragmentation.
On 2010-03-24 12:07, Bob McGowan wrote: [snip] And, I used VFAT for the filesystem type: That's the key difference, since vfat "is" fat32. $ mount|grep sde /dev/sde on /var/autofs/usb/centon type vfat ... Try this: $ mount -lt vat -- "History does not long entrust the care of freedom to the weak or the timid." Dwight Eisenhower -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4baa4b9f.1050...@cox.net
Re: Flash drive internal fragmentation.
R. Clayton wrote: > <<--deleted background details-- > > $ ls --block-size=1k -ls > total 896 > 64 -rwxr-xr-x 1 rclayton rclayton 18 2010-03-23 14:42 branniga.htm > 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig1.gif > 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig1.htm > 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig2.gif > 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig2.htm > 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig3.gif > 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig3.htm > 64 -rwxr-xr-x 1 rclayton rclayton 6 2010-03-23 14:42 fig4.gif > 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig4.htm > 64 -rwxr-xr-x 1 rclayton rclayton 5 2010-03-23 14:42 fig5.gif > 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig5.htm > 64 -rwxr-xr-x 1 rclayton rclayton 7 2010-03-23 14:42 fig6.gif > 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig6.htm > 64 -rwxr-xr-x 1 rclayton rclayton 5 2010-03-23 14:42 list1.htm > > $ cd /home/dr-dobbs/articles/CUJ/1990/9001/branniga > > $ ls --block-size=1k -ls > total 88 > 20 -r-x-- 1 rclayton rclayton 18 2010-03-21 08:32 branniga.htm >4 -r-x-- 1 rclayton rclayton 3 2010-03-21 08:32 fig1.gif >4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig1.htm >4 -r-x-- 1 rclayton rclayton 3 2010-03-21 08:32 fig2.gif >4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig2.htm >4 -r-x-- 1 rclayton rclayton 3 2010-03-21 08:32 fig3.gif >4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig3.htm >8 -r-x-- 1 rclayton rclayton 6 2010-03-21 08:32 fig4.gif >4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig4.htm >8 -r-x-- 1 rclayton rclayton 5 2010-03-21 08:32 fig5.gif >4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig5.htm >8 -r-x-- 1 rclayton rclayton 7 2010-03-21 08:32 fig6.gif >4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig6.htm >8 -r-x-- 1 rclayton rclayton 5 2010-03-21 08:32 list1.htm > > $ > > which meant my thumb drive space was getting eaten up by internal > fragmentation. I understand that flash storage has special needs for most > efficient operation, but are block-size overages of an order-of-magnitude or > more really necessary? Can I remake the file system with smaller block sizes? > How? (It is unclear to me if sector size is the same as file block size.) > What bad things happen to flash storage if the file system has 1k block size? > > This happened on a debian testing system updated weekly. > > Curious ... I have a 4G flash drive and have had no issues with it, except for file deletion using a GUI file manager. And you didn't mention if the stick you're using was brand new/never used or not. If you've had files on it and then deleted them using a GUI file browser, it may have simply moved them to a hidden trash folder, which would not free up any space. But you may have already checked that ;) I tried the 'ls' command you used on my flash drive, and got this: $ ls --block-size=1k -ls total 23024 500 -rwxrwxr-x 1 root backup 498 2010-02-22 18:11 C1.jpg 644 -rwxrwxr-x 1 root backup 643 2010-02-22 18:11 C2.jpg 1160 -rwxrwxr-x 1 root backup 1159 2010-02-22 18:11 C3.jpg 852 -rwxrwxr-x 1 root backup 852 2010-02-22 18:11 C4.jpg 4896 -rwxrwxr-x 1 root backup 4895 2010-03-07 10:36 C5.jpg 3564 -rwxrwxr-x 1 root backup 3561 2010-03-07 10:45 C6.jpg 416 -rwxrwxr-x 1 root backup 416 2010-03-14 16:35 C7.jpg 5076 -rwxrwxr-x 1 root backup 5076 2010-03-13 02:43 C8.jpg 5916 -rwxrwxr-x 1 root backup 5913 2010-03-13 03:04 C9.jpg And ... $ ls --block-size=8k -ls total 2878 63 -rwxrwxr-x 1 root backup 63 2010-02-22 18:11 C1.jpg 81 -rwxrwxr-x 1 root backup 81 2010-02-22 18:11 C2.jpg 145 -rwxrwxr-x 1 root backup 145 2010-02-22 18:11 C3.jpg 107 -rwxrwxr-x 1 root backup 107 2010-02-22 18:11 C4.jpg 612 -rwxrwxr-x 1 root backup 612 2010-03-07 10:36 C5.jpg 446 -rwxrwxr-x 1 root backup 446 2010-03-07 10:45 C6.jpg 52 -rwxrwxr-x 1 root backup 52 2010-03-14 16:35 C7.jpg 635 -rwxrwxr-x 1 root backup 635 2010-03-13 02:43 C8.jpg 740 -rwxrwxr-x 1 root backup 740 2010-03-13 03:04 C9.jpg It looks like the numbers are matching up OK (at least on my system - ls --version says "ls (GNU coreutils) 5.97" and still using etch [4.0]). A couple of other differences: My flash drive has no partition table, the filesystem is built on the base device, like a floppy disk. And, I used VFAT for the filesystem type: $ mount|grep sde /dev/sde on /var/autofs/usb/centon type vfat ... I don't know if any of this is really helpful, though it may suggest an alternative or two to play around with. -- Bob McGowan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4baa46dd.90...@symantec.com
Re: Flash drive internal fragmentation.
On 2010-03-24 10:51, R. Clayton wrote: [snip] to an empty 4G thumb drive $ /sbin/fdisk -l /dev/sda Disk /dev/sda: 4009 MB, 4009754624 bytes 124 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 7688 * 512 = 3936256 bytes Disk identifier: 0x Device Boot Start End Blocks Id System /dev/sda1 * 11018 39131616 FAT16 $ and had the copy fail because the thumb drive ran out of space. After poking around a bit, I found the block sizes on the thumb drive were huge: $ cd /mnt/flasha/dr-dobbs/articles/CUJ/1990/9001/branniga $ ls --block-size=1k -ls total 896 64 -rwxr-xr-x 1 rclayton rclayton 18 2010-03-23 14:42 branniga.htm 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig1.gif 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig1.htm [snip] which meant my thumb drive space was getting eaten up by internal fragmentation. I understand that flash storage has special needs for most efficient operation, but are block-size overages of an order-of-magnitude or more really necessary? Can I remake the file system with smaller block sizes? How? (It is unclear to me if sector size is the same as file block size.) What bad things happen to flash storage if the file system has 1k block size? This is a fundamental design flaw of FAT, which was designed for floppy drives. http://en.wikipedia.org/wiki/File_Allocation_Table "To limit the size of the table, disk space is allocated to files in contiguous groups of hardware sectors called clusters." Even with FAT16 and FAT32 increasing the number of clusters on a disk, capacity still outstripped FAT. Thus, cluster size had to greatly increase. Thus the huge block count. I'd try FAT32. The increased FAT should reduce the cluster size. -- "History does not long entrust the care of freedom to the weak or the timid." Dwight Eisenhower -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4baa42e9.5070...@cox.net
Flash drive internal fragmentation.
[ This isn't really a debian question; it's more of a linux question or maybe a file system question, but because this is the most appropriate of the groups I read, I'll pretend it's really important that this happened on a debian system. ] I tried to copy a 1.4G directory $ du -sh dr-dobbs 1.4G dr-dobbs $ to an empty 4G thumb drive $ /sbin/fdisk -l /dev/sda Disk /dev/sda: 4009 MB, 4009754624 bytes 124 heads, 62 sectors/track, 1018 cylinders Units = cylinders of 7688 * 512 = 3936256 bytes Disk identifier: 0x Device Boot Start End Blocks Id System /dev/sda1 * 11018 39131616 FAT16 $ and had the copy fail because the thumb drive ran out of space. After poking around a bit, I found the block sizes on the thumb drive were huge: $ cd /mnt/flasha/dr-dobbs/articles/CUJ/1990/9001/branniga $ ls --block-size=1k -ls total 896 64 -rwxr-xr-x 1 rclayton rclayton 18 2010-03-23 14:42 branniga.htm 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig1.gif 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig1.htm 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig2.gif 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig2.htm 64 -rwxr-xr-x 1 rclayton rclayton 3 2010-03-23 14:42 fig3.gif 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig3.htm 64 -rwxr-xr-x 1 rclayton rclayton 6 2010-03-23 14:42 fig4.gif 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig4.htm 64 -rwxr-xr-x 1 rclayton rclayton 5 2010-03-23 14:42 fig5.gif 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig5.htm 64 -rwxr-xr-x 1 rclayton rclayton 7 2010-03-23 14:42 fig6.gif 64 -rwxr-xr-x 1 rclayton rclayton 1 2010-03-23 14:42 fig6.htm 64 -rwxr-xr-x 1 rclayton rclayton 5 2010-03-23 14:42 list1.htm $ cd /home/dr-dobbs/articles/CUJ/1990/9001/branniga $ ls --block-size=1k -ls total 88 20 -r-x-- 1 rclayton rclayton 18 2010-03-21 08:32 branniga.htm 4 -r-x-- 1 rclayton rclayton 3 2010-03-21 08:32 fig1.gif 4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig1.htm 4 -r-x-- 1 rclayton rclayton 3 2010-03-21 08:32 fig2.gif 4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig2.htm 4 -r-x-- 1 rclayton rclayton 3 2010-03-21 08:32 fig3.gif 4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig3.htm 8 -r-x-- 1 rclayton rclayton 6 2010-03-21 08:32 fig4.gif 4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig4.htm 8 -r-x-- 1 rclayton rclayton 5 2010-03-21 08:32 fig5.gif 4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig5.htm 8 -r-x-- 1 rclayton rclayton 7 2010-03-21 08:32 fig6.gif 4 -r-x-- 1 rclayton rclayton 1 2010-03-21 08:32 fig6.htm 8 -r-x-- 1 rclayton rclayton 5 2010-03-21 08:32 list1.htm $ which meant my thumb drive space was getting eaten up by internal fragmentation. I understand that flash storage has special needs for most efficient operation, but are block-size overages of an order-of-magnitude or more really necessary? Can I remake the file system with smaller block sizes? How? (It is unclear to me if sector size is the same as file block size.) What bad things happen to flash storage if the file system has 1k block size? This happened on a debian testing system updated weekly. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87mxxxen0p@ulanbator.myhome.westell.com
