Re: [freenet-support] Re: [freenet-chat] slightly off topic: fat32 methods of keeping my store folder size
> this was my orginal plan, and i was posting to find out if anyone knew > how to enable a compressed directory in win2k :) but now i think i'll > probably go with a separate 2 gig fat32 partition... that is if i can > figure out how to tweak a fat32 to have smaller blocks. To specify cluster size for a fat32 partition use format x: /FS:FAT32 /A:clustersize where x: is the driveletter. Naturally, the clustersize applies to the entire partition. To compress a directory tree in win2k you *must* either use an NTFS partition or use compression software such as DriveSpace, zipmagic, etc. that is compatible with FAT32. Just noticed, on the help for the format command, there is a /C option which states "Files created on the new volume will be compressed by default" - I guess this only applies to NTFS but it might be worth giving it a go on a FAT32 drive to see what happens! d ___ support mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/support
Re: [freenet-support] Re: [freenet-chat] slightly off topic: fat32 methods of keeping my store folder size
> The negative effects are when you need to store large files, > essentially the file needs to be broken up into smaller chunks, and so takes > longer to store/retrieve. I don't think the size of your FAT table matters > anymore, way back when it was limited and so you were limited by disk size > vs cluster size vs FAT table size. > > Or something like that. Something like that. The three FAT addressing schemes I know of in wide use are FAT12, FAT16 and FAT32 which provide for 2^12, 2^16 and 2^32 clusters (actually I think it's 2^12-64, 2^16-64 or 2^32-256, or something crazy anyway). FAT32 is only 'marginally' slower than FAT16 in practice, and FAT32 is measurably *faster* than NTFS. Generally you wouuld not want to use anything less than FAT32 for a freenet datastore, and if you want a freenet datastore > 2GB you will need to use FAT32, because the largest cluster size allowed by FAT12 and FAT16 is 32768 bytes. The clincher is obviously that not all operating systems support FAT32 (e.g. Windows NT does not "out of the can", but www.sysinternals.com have a FAT32 driver for NT) Not that this should make a big difference - if you're planning to use your datastore in both linux and windows then any old FAT partition should do - you may run into problems with an NTFS partition (I don't know how reliable the NTFS support is in linux). If you were planning to use Windows 2000 or XP exclusively, I might suggest using a compressed directory as your freenet datastore - not because of the disk space this saves (it would save very little, probably - see end of message), but because the compressed clusters are shared within the directory tree so only the last cluster allocated to the directory tree has 'wasted' space at the end. Ideal if you care about space but don't care so much about extra processing time. Although obviously an NTFSv5 feature only (and so not applicable to the original fat32 posting) "OS-level" compressing of directory trees in pre-NTFSv5 operating systems (i.e. before Windows 2000) can be achieved by using commercial software like www.zipmagic.com/zipmagic, which (among other things) makes zip files appear to the operating system as regular 'explorable' folders, or by using the built-in DriveSpace / DoubleSpace utilities to set up a virtual compressed drive. Actually I would make a personal recommendation for zipmagic, it really is rather good, and I'm sure there must be a less expensive alternative available On my NTFS partition, my store currently has about 3% wasted space from cluster allocation. Setting the NTFS Compress flag shrinks it so that my store uses only .5% more space 'on disk' . (Yes - the compressed datastore still uses more than the 'on paper' amount of disk space, mainly due to the encrypted nature of the datastore and its inherent incompressibility coupled with the still necessary cluster allocation) I would therefore expect similar results under FAT32 - that is, a .zip datastore with zipmagic or similar using only about .5% more space than the datastore size on paper. However that is on likely to be true if you can keep the .zip file fragments together... I don't know what zipmagic's fragmentation guarantees are, and I wouldn't be surprised if there were no guarantees whatsoever. It is possible to preallocate files under NTFS using tools such as Contig (www.sysinternals.com) to ensure that they do not fragment - however this only really works in practice for files which do not grow and shrink unpredictably, such as files which are written to rarely but read often. I'm guessing such a tool would be of only limited value for a freenet datastore. I have no idea if comparable utilities are available for use with FAT partitions. dave ___ support mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/support
RE: [freenet-support] Re: [freenet-chat] slightly off topic: fat32 methods of keeping my store folder size
The negative effects are when you need to store large files, essentially the file needs to be broken up into smaller chunks, and so takes longer to store/retrieve. I don't think the size of your FAT table matters anymore, way back when it was limited and so you were limited by disk size vs cluster size vs FAT table size. Or something like that. > -Original Message- > From: [EMAIL PROTECTED] > [mailto:support-admin@;freenetproject.org]On Behalf Of Josh Steiner > Sent: 28 October 2002 14:38 > To: [EMAIL PROTECTED] > Subject: Re: [freenet-support] Re: [freenet-chat] slightly off topic: > fat32 methods of keeping my store folder size > > > good idea, hadnt thought about just making a freenet datastore > partition, what are the negative effects of using smaller cluster sizes? > (there must be some, otherwise they would default to 16 :) perhaps its > just addressing, if you have smaller clusters you have *more* clusters > and therefore have to address more clusters... if this is all, then it > should matter as i'd be having at most a partition of a gig or two for > freenet... > > i'm not sure i'm going to do this though since i dual boot to linux > about 50% of the time i use my computer, so i was planning on setting up > a node on my linux box that shares the same datastore... > > -joschi > > ___ > Josh .. Yoshi .. Joschi .. http://mp3.com/vitriolix > > > > [EMAIL PROTECTED] wrote: > > >>>so (unfortunately) i am forced to use fat32 on my windows > machine where > >>>i run my freenet node... right now here is the size info: > >>> > >>>Size: 391 MB (410,077,960 bytes) > >>>Size on disk: 506 MB (531,529,728 bytes) > >>> > >>> > >>I would recommend putting the freenet datastore on a separate > >>partition if possible. Partition Magic (commercial software, but well > >>worth the money) can do this for you without destroying your existing > >>files. > >> > >>When formatting the new filesystem, try to give it as large a cluster > >>size as possible, and you will cut down on lost space. (Or format it > >>as NTFS, if that's an option). > >> > >> > > > >no, sorry. make the clusters as SMALL as possible, so the last > used cluster by a file is filled up more effectively. > >the clusters that do not fill completely contain the wasted space: > >cluster 256 bytes, data 64 bytes = 192 bytes wasted > >custer 4096 bytes, data 64 bytes = 4032 bytes wasted > > > > > > > >The information transmitted is intended only for the person or > entity to which it is addressed and may contain confidential > and/or privileged material. Any > >review, retransmission, dissemination or other use of, or taking > of any action in reliance upon, this information by persons or > entities other than the intended > >recipient is prohibited. If you received this in error, please > contact the sender and delete the material from any computer. > > > > > > > > > > > > > > > > > > > >___ > >support mailing list > >[EMAIL PROTECTED] > >http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/support > > > > > > > -- > ___ > Josh .. Yoshi .. Joschi .. http://mp3.com/vitriolix > > > > ___ > support mailing list > [EMAIL PROTECTED] > http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/support > ___ support mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/support
[freenet-support] Re: [freenet-chat] slightly off topic: fat32 methods of keeping my store folder size
>>so (unfortunately) i am forced to use fat32 on my windows machine where >>i run my freenet node... right now here is the size info: >> >>Size: 391 MB (410,077,960 bytes) >>Size on disk: 506 MB (531,529,728 bytes) > >I would recommend putting the freenet datastore on a separate >partition if possible. Partition Magic (commercial software, but well >worth the money) can do this for you without destroying your existing >files. > >When formatting the new filesystem, try to give it as large a cluster >size as possible, and you will cut down on lost space. (Or format it >as NTFS, if that's an option). no, sorry. make the clusters as SMALL as possible, so the last used cluster by a file is filled up more effectively. the clusters that do not fill completely contain the wasted space: cluster 256 bytes, data 64 bytes = 192 bytes wasted custer 4096 bytes, data 64 bytes = 4032 bytes wasted The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. ___ support mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/support
