Re: Wrong Shutdown
hmm, on Tue, May 27, 2014 at 07:14:49AM +0200, Otto Moerbeek said that block size is between 4096 and 65536, fragment size between 512 and block size. Both are powers of 2, and block size can be 1, 2, 4, or 8 times fragments size. For media files -b 65536 -i 65536 is fine. If you still have too many inodes, I use -i to reduce the numbers of inodes during newfs, unit is bytes per inode. Newfs reports what it is doing, so you can see how many inodes you are getting. The numbers for -g -and -h matter only at runtime, they do not influence the fs layout during newfs. i smell some great FAQ material here :) [otto@lou:17]$ sudo newfs -N -i 100 -f 65536 -b 65536 /dev/rsd0l would there be an explicit advantage of using ffs2 in this case? is the biggest plus of ffs2 the increased size of all the limits and the fact that inodes are allocated only when needed? -f -- someone whom you reject today, will reject you tomorrow.
Re: Wrong Shutdown
On Tue, May 27, 2014 at 11:06:10AM +0200, frantisek holop wrote: hmm, on Tue, May 27, 2014 at 07:14:49AM +0200, Otto Moerbeek said that block size is between 4096 and 65536, fragment size between 512 and block size. Both are powers of 2, and block size can be 1, 2, 4, or 8 times fragments size. For media files -b 65536 -i 65536 is fine. If you still have too many inodes, I use -i to reduce the numbers of inodes during newfs, unit is bytes per inode. Newfs reports what it is doing, so you can see how many inodes you are getting. The numbers for -g -and -h matter only at runtime, they do not influence the fs layout during newfs. i smell some great FAQ material here :) [otto@lou:17]$ sudo newfs -N -i 100 -f 65536 -b 65536 /dev/rsd0l would there be an explicit advantage of using ffs2 in this case? is the biggest plus of ffs2 the increased size of all the limits and the fact that inodes are allocated only when needed? I'd say that are the only plusses. But they are good enough ;-) The FAQ already contains some material on these issues. I wondert if adding more details would clarify things. Note that disklabel already sets larger blocks sizes for larger partitions. That should do for most uses, though it keeps 8 frags per block. -Otto
Wrong Shutdown
Hello guys, I have a machine with a HardDrive with a slice of 2.7TB, and I have no UPS.. when sometimes I have power failure, and consequently a wrong shutdown, The fsck spends much time to recover the filse system, what can I do? I need to be faster. Thanks in advance. -- Walter Neto Analista Desenvolvedor
Re: Wrong Shutdown
I have a machine with a HardDrive with a slice of 2.7TB, and I have no UPS.. when sometimes I have power failure, and consequently a wrong shutdown, The fsck spends much time to recover the filse system, what can I do? I need to be faster. Get a UPS. fsck is required to ensure the directory hierarchy is coherent.
Re: Wrong Shutdown
Why OpenBSD has no interest in using journal file system? On Mon, May 26, 2014 at 10:48 AM, Theo de Raadt dera...@cvs.openbsd.orgwrote: I have a machine with a HardDrive with a slice of 2.7TB, and I have no UPS.. when sometimes I have power failure, and consequently a wrong shutdown, The fsck spends much time to recover the filse system, what can I do? I need to be faster. Get a UPS. fsck is required to ensure the directory hierarchy is coherent. -- Walter Neto Analista Desenvolvedor
Re: Wrong Shutdown
Why OpenBSD has no interest in using journal file system? Because we simply don't have anyone working on it at the moment. What is so hard to understand about that? We are a group of volunteers! We work on what we want to, and as a group we don't try to overcommit our efforts into specific directions at the impact towards other directions. As far as I know, none of the developers are specifically working on World Peace, either.
Re: Wrong Shutdown
On Mon, May 26, 2014 at 07:58:00AM -0600, Theo de Raadt wrote: [...] As far as I know, none of the developers are specifically working on World Peace, either. That was a work in progress, but it was aborted due to lack of general interest :-/ -- Gilles Chehade https://www.poolp.org @poolpOrg
Re: Wrong Shutdown
On May 26, 2014 9:53 AM, Walter Souza wsouz...@gmail.com wrote: Why OpenBSD has no interest in using journal file system? OpenBSD has great interest in using journal filesystem. Nobody has sent us the diffs that would add one. Ken On Mon, May 26, 2014 at 10:48 AM, Theo de Raadt dera...@cvs.openbsd.org wrote: I have a machine with a HardDrive with a slice of 2.7TB, and I have no UPS.. when sometimes I have power failure, and consequently a wrong shutdown, The fsck spends much time to recover the filse system, what can I do? I need to be faster. Get a UPS. fsck is required to ensure the directory hierarchy is coherent. -- Walter Neto Analista Desenvolvedor
Re: Wrong Shutdown
So do you have interest? I have interest in help.. I love OpenBSD project and I want to use it in everything. And let's work in World Peace too.. :) On Mon, May 26, 2014 at 10:58 AM, Theo de Raadt dera...@cvs.openbsd.orgwrote: Why OpenBSD has no interest in using journal file system? Because we simply don't have anyone working on it at the moment. What is so hard to understand about that? We are a group of volunteers! We work on what we want to, and as a group we don't try to overcommit our efforts into specific directions at the impact towards other directions. As far as I know, none of the developers are specifically working on World Peace, either. -- Walter Neto Analista Desenvolvedor
Re: Wrong Shutdown
So do you have interest? I have interest in help.. I love OpenBSD project and I want to use it in everything. There is a large gap between how do I make fsck faster without buying a UPS and I will help give you guys a working journal filesystem. I don't know you, maybe I am misinterpreting you. And let's work in World Peace too.. :) Your makeup has a smudge, so you don't win.
Re: Wrong Shutdown
2014-05-26 15:52 GMT+02:00 Walter Souza wsouz...@gmail.com: Why OpenBSD has no interest in using journal file system? http://www.openbsd.org/faq/faq8.html#Journaling Please read the FAQ. Best Martin
Re: Wrong Shutdown
On Mon, May 26, 2014 at 11:09 AM, Theo de Raadt dera...@cvs.openbsd.orgwrote: So do you have interest? I have interest in help.. I love OpenBSD project and I want to use it in everything. There is a large gap between how do I make fsck faster without buying a UPS and I will help give you guys a working journal filesystem. I don't know you, maybe I am misinterpreting you. I made the question first because I have learned many different best ways than usual using OBSD, and I think that was better to question first. And let's work in World Peace too.. :) Your makeup has a smudge, so you don't win. -- Walter Neto Analista Desenvolvedor
Re: Wrong Shutdown
On May 26 10:46:30, wsouz...@gmail.com wrote: I have a machine with a HardDrive with a slice of 2.7TB, Why exactly are you using such a huge partition? Do you need to? Can't you use smaller, more manageable partitions? and I have no UPS.. when sometimes I have power failure, and consequently a wrong shutdown, Is your 2.7TB of data at least so valuable that you wold buy a UPS? The fsck spends much time to recover the filse system, what can I do? I need to be faster. How exactly did you create the filesystem? For example, see the mistake I made some time ago: ~$ df -hi /dload Filesystem SizeUsed Avail Capacity iused ifree %iused Mounted on /dev/sd3a 401G377G3.8G99%8501 417481 2% /dload See? It is almost full, but only 2% of the inodes are used. I could have created the filesystem with a fraction of the inodes, and it would be enough, and the fsck would be way faster.
Re: Wrong Shutdown
On May 26, 2014 9:16:17 AM CDT, Martin Schröder mar...@oneiros.de wrote: 2014-05-26 15:52 GMT+02:00 Walter Souza wsouz...@gmail.com: Why OpenBSD has no interest in using journal file system? http://www.openbsd.org/faq/faq8.html#Journaling Please read the FAQ. Best Martin Arguably, Walter might be better served by turning off softdep and ensuring the filesystem is mounted 'sync'. That doesn't solve the fsck speed issue, but it would help ensure no data loss. Note to Walter: a journaling filesystem is not magic, you can (and will) still experience data loss in uncontrolled shutdowns. Journaling just means (roughly) that the metadata and data are written in the correct order so that the filesystem is not in an inconsistent state... not that you won't lose data. (As already pointed out, softdep does much the same thing.) Does running FFS2 improve fsck times? Not something I've ever tested... -Adam -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: Wrong Shutdown
On Mon, May 26, 2014 at 09:32:57AM -0500, Adam Thompson wrote: On May 26, 2014 9:16:17 AM CDT, Martin Schr??der mar...@oneiros.de wrote: 2014-05-26 15:52 GMT+02:00 Walter Souza wsouz...@gmail.com: Why OpenBSD has no interest in using journal file system? http://www.openbsd.org/faq/faq8.html#Journaling Please read the FAQ. Best Martin Arguably, Walter might be better served by turning off softdep and ensuring the filesystem is mounted 'sync'. That doesn't solve the fsck speed issue, but it would help ensure no data loss. Note to Walter: a journaling filesystem is not magic, you can (and will) still experience data loss in uncontrolled shutdowns. Journaling just means (roughly) that the metadata and data are written in the correct order so that the filesystem is not in an inconsistent state... not that you won't lose data. (As already pointed out, softdep does much the same thing.) Does running FFS2 improve fsck times? Not something I've ever tested... Yes it does, in most cases. But the most important is to use large block and/or fragments sizes, if that is acceptable for your use (it wastes space if you have a lot of small files). -Otto
Re: Wrong Shutdown
On Mon, 26 May 2014, Theo de Raadt wrote: From: Theo de Raadt dera...@cvs.openbsd.org To: Walter Souza wsouz...@gmail.com Cc: misc@openbsd.org Date: Mon, 26 May 2014 15:09:03 Subject: Re: Wrong Shutdown ... And let's work in World Peace too.. :) Your makeup has a smudge, so you don't win. That's not makeup! That's the black eye I got in last night's bar brawl :-( Now what's this World Peace thingie? -- Dennis Davis dennisda...@fastmail.fm
Re: Wrong Shutdown
Enable SoftUpdates. /dev/sd0a / ffs rw,softdep 1 1 On 05/26/2014 09:52 AM, Walter Souza wrote: Why OpenBSD has no interest in using journal file system? On Mon, May 26, 2014 at 10:48 AM, Theo de Raadt dera...@cvs.openbsd.orgwrote: I have a machine with a HardDrive with a slice of 2.7TB, and I have no UPS.. when sometimes I have power failure, and consequently a wrong shutdown, The fsck spends much time to recover the filse system, what can I do? I need to be faster. Get a UPS. fsck is required to ensure the directory hierarchy is coherent. -- Salim A. Shaw System Administrator OpenBSD / Free Software Advocate Need security and stability --- Try OpenBSD. BSD license all the way: Sell services, don't lease secrets
Re: Wrong Shutdown
On Mon, May 26, 2014 at 6:56 AM, Salim Shaw salims...@vfemail.net wrote: Enable SoftUpdates. /dev/sd0a / ffs rw,softdep 1 1 Since OpenBSD doesn't have background fsck for softupdates, nor does it have softupdates journaling, how will that solve the original problem? Philip Guenther
Re: Wrong Shutdown
hmm, on Mon, May 26, 2014 at 04:46:04PM +0200, Otto Moerbeek said that Yes it does, in most cases. But the most important is to use large block and/or fragments sizes, if that is acceptable for your use (it wastes space if you have a lot of small files). i meant to ask now for some time, what are (sensible) max values? can't find it in newfs(8), disklabel(8). #size offset fstype [fsize bsize cpg] a:555913152 64 4.2BSD 8192 655361 i dont have an excessively big partition (but big enough for a veeery slow fsck with default newfs values) but it holds only media files, so i dont think i need lots of inodes. so i newfs-ed with -O 2 and big fsize/bsize. but i still have too many inodes. maybe 10x less inodes would suffice? Filesystem SizeUsed Avail Capacity iused ifree %iused Mounted on /dev/sd2a 263G141G122G54% 64861 8730273 1% /home/f/data would these help in any way for media collections? -g avgfilesize The expected average file size for the file system in bytes. -h avgfpdir The expected average number of files per directory on the file system. $ sudo tunefs -N /dev/sd2a tunefs: tuning /dev/sd2a tunefs: current settings of /dev/sd2a maximum contiguous block count 1 maximum blocks per file in a cylinder group 8192 minimum percentage of free space 0% optimization preference: space average file size: 16384 expected number of files per directory: 64 tunefs: no changes made default average file size is rather conservative. and totally untrue for the media collection :) -f -- i am sick and tired of being sick and tired.
Re: Wrong Shutdown
On 2014-05-26, Theo de Raadt dera...@cvs.openbsd.org wrote: I have a machine with a HardDrive with a slice of 2.7TB, and I have no UPS.. when sometimes I have power failure, and consequently a wrong shutdown, The fsck spends much time to recover the filse system, what can I do? I need to be faster. Get a UPS. fsck is required to ensure the directory hierarchy is coherent. For this situation, a UPS only needs to power the machine for a few minutes to shutdown safely, so it doesn't need to be particularly expensive. Quite possibly less than the cost of the drives big enough to hold your 2.7TB filesystem and a backup.
Re: Wrong Shutdown
On Mon, May 26, 2014 at 14:14, Philip Guenther wrote: On Mon, May 26, 2014 at 6:56 AM, Salim Shaw salims...@vfemail.net wrote: Enable SoftUpdates. /dev/sd0a / ffs rw,softdep 1 1 Since OpenBSD doesn't have background fsck for softupdates, nor does it have softupdates journaling, how will that solve the original problem? mount -f for all the speeds.
Re: Wrong Shutdown
On Mon, May 26, 2014 at 11:19:00PM +0200, frantisek holop wrote: hmm, on Mon, May 26, 2014 at 04:46:04PM +0200, Otto Moerbeek said that Yes it does, in most cases. But the most important is to use large block and/or fragments sizes, if that is acceptable for your use (it wastes space if you have a lot of small files). i meant to ask now for some time, what are (sensible) max values? can't find it in newfs(8), disklabel(8). #size offset fstype [fsize bsize cpg] a:555913152 64 4.2BSD 8192 655361 i dont have an excessively big partition (but big enough for a veeery slow fsck with default newfs values) but it holds only media files, so i dont think i need lots of inodes. so i newfs-ed with -O 2 and big fsize/bsize. but i still have too many inodes. maybe 10x less inodes would suffice? Filesystem SizeUsed Avail Capacity iused ifree %iused Mounted on /dev/sd2a 263G141G122G54% 64861 8730273 1% /home/f/data would these help in any way for media collections? -g avgfilesize The expected average file size for the file system in bytes. -h avgfpdir The expected average number of files per directory on the file system. $ sudo tunefs -N /dev/sd2a tunefs: tuning /dev/sd2a tunefs: current settings of /dev/sd2a maximum contiguous block count 1 maximum blocks per file in a cylinder group 8192 minimum percentage of free space 0% optimization preference: space average file size: 16384 expected number of files per directory: 64 tunefs: no changes made default average file size is rather conservative. and totally untrue for the media collection :) -f -- i am sick and tired of being sick and tired. block size is between 4096 and 65536, fragment size between 512 and block size. Both are powers of 2, and block size can be 1, 2, 4, or 8 times fragments size. For media files -b 65536 -i 65536 is fine. If you still have too many inodes, I use -i to reduce the numbers of inodes during newfs, unit is bytes per inode. Newfs reports what it is doing, so you can see how many inodes you are getting. The numbers for -g -and -h matter only at runtime, they do not influence the fs layout during newfs. [otto@lou:16]$ sudo newfs -N -f 65536 -b 65536 /dev/rsd0l newfs: reduced number of fragments per cylinder group from 163839 to 163818 to enlarge last cylinder group /dev/rsd0l: 40959.8MB in 83885696 sectors of 512 bytes 5 cylinder groups of 10238.62MB, 163818 blocks, 40960 inodes each super-block backups (for fsck -b #) at: 128, 20968832, 41937536, 62906240, 83874944, [otto@lou:17]$ sudo newfs -N -i 100 -f 65536 -b 65536 /dev/rsd0l newfs: reduced number of fragments per cylinder group from 163839 to 163833 to enlarge last cylinder group /dev/rsd0l: 40959.8MB in 83885696 sectors of 512 bytes 5 cylinder groups of 10239.56MB, 163833 blocks, 11264 inodes each super-block backups (for fsck -b #) at: 128, 20970752, 41941376, 62912000, 83882624, -Otto
Re: Wrong Shutdown
On Tue, May 27, 2014 at 07:14:49AM +0200, Otto Moerbeek wrote: On Mon, May 26, 2014 at 11:19:00PM +0200, frantisek holop wrote: hmm, on Mon, May 26, 2014 at 04:46:04PM +0200, Otto Moerbeek said that Yes it does, in most cases. But the most important is to use large block and/or fragments sizes, if that is acceptable for your use (it wastes space if you have a lot of small files). i meant to ask now for some time, what are (sensible) max values? can't find it in newfs(8), disklabel(8). #size offset fstype [fsize bsize cpg] a:555913152 64 4.2BSD 8192 655361 i dont have an excessively big partition (but big enough for a veeery slow fsck with default newfs values) but it holds only media files, so i dont think i need lots of inodes. so i newfs-ed with -O 2 and big fsize/bsize. but i still have too many inodes. maybe 10x less inodes would suffice? Filesystem SizeUsed Avail Capacity iused ifree %iused Mounted on /dev/sd2a 263G141G122G54% 64861 8730273 1% /home/f/data would these help in any way for media collections? -g avgfilesize The expected average file size for the file system in bytes. -h avgfpdir The expected average number of files per directory on the file system. $ sudo tunefs -N /dev/sd2a tunefs: tuning /dev/sd2a tunefs: current settings of /dev/sd2a maximum contiguous block count 1 maximum blocks per file in a cylinder group 8192 minimum percentage of free space 0% optimization preference: space average file size: 16384 expected number of files per directory: 64 tunefs: no changes made default average file size is rather conservative. and totally untrue for the media collection :) -f -- i am sick and tired of being sick and tired. block size is between 4096 and 65536, fragment size between 512 and block size. Both are powers of 2, and block size can be 1, 2, 4, or 8 times fragments size. For media files -b 65536 -i 65536 is fine. That -i should be -f If you still have too many inodes, I use -i to reduce the numbers of inodes during newfs, unit is bytes per inode. Newfs reports what it is doing, so you can see how many inodes you are getting. The numbers for -g -and -h matter only at runtime, they do not influence the fs layout during newfs. [otto@lou:16]$ sudo newfs -N -f 65536 -b 65536 /dev/rsd0l newfs: reduced number of fragments per cylinder group from 163839 to 163818 to enlarge last cylinder group /dev/rsd0l: 40959.8MB in 83885696 sectors of 512 bytes 5 cylinder groups of 10238.62MB, 163818 blocks, 40960 inodes each super-block backups (for fsck -b #) at: 128, 20968832, 41937536, 62906240, 83874944, [otto@lou:17]$ sudo newfs -N -i 100 -f 65536 -b 65536 /dev/rsd0l newfs: reduced number of fragments per cylinder group from 163839 to 163833 to enlarge last cylinder group /dev/rsd0l: 40959.8MB in 83885696 sectors of 512 bytes 5 cylinder groups of 10239.56MB, 163833 blocks, 11264 inodes each super-block backups (for fsck -b #) at: 128, 20970752, 41941376, 62912000, 83882624, -Otto
