subject:"Defaults in 10.0 ZFS through bsdinstall"

Re: Defaults in 10.0 ZFS through bsdinstall

2013-11-15 Thread Stefan Esser

Am 15.11.2013 01:00, schrieb Mark Felder:
 On Nov 14, 2013, at 15:04, Teske, Devin devin.te...@fisglobal.com wrote:
 Sounds like a vote for enabling it where-needed by-default (e.g., /var as a 
 whole
 or more selectively, /var/mail)
 
 I'd be OK with FreeBSD taking a stance and moving to noatime by default but 
 we should be consistent across all filesystems that a user can install the OS 
 on from our provided installation media. We should make it obvious to the end 
 users as well.

The cost of atime on UFS and ZFS is quite different. While I
understand that consistency is a plus, I do think that there
is good reason to have different defaults for UFS and ZFS.

It would be good if there was a global parameter which let
the user choose the atime setting for all file-systems where
this setting is not overridden. Such a parameter could be
OFF (noatime) for ZFS and ON for UFS by default, but with
a short explanation about the consequences and a way to
toggle this setting if desired.

Regards, STefan
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Defaults in 10.0 ZFS through bsdinstall

2013-11-15 Thread Stefan Esser

Am 14.11.2013 22:02, schrieb Teske, Devin:
 On Nov 14, 2013, at 12:49 PM, Mark Felder wrote:
 We don't even do installs on UFS with atime disabled by default in fstab
 so why should we so suddenly change course for ZFS?

 
 You've made a good point.

There is major difference between UFS and ZFS: UFS allows in-place
updates of i-node fields (like atime), while ZFS uses COW for all
data, file contents and meta-data like the i-nodes.

With atime ON on UFS you'll see a small number of writes on
file-systems that are only read - we are used to accept that.

On ZFS every update of atime causes a write of the meta-data to
a free location on disk, then updates of all data structures
that reference that meta-data up to the root of the tree (the
uberblock). An update of a few bytes turns out to write tens
of KB for each atime update (within the TXG sync interval, which
defaults to 5 seconds on FreeBSD). If you create snapshots, then
each snapshot will contain a copy of the metadata that was valid
at the time of the snapshot (well, that's not so different from
the situation with UFS snapshots, just that the data structures
are much more complex and larger in the ZFS case). Due to the
ease and speed of snapshot creation with ZFS there probably are
a magnitude or more snapshots on a typical ZFS system than on
one using UFS (I currently have a few hundred and have turned off
periodic snapshot generation on many unimportant file-systems,
already).

I really hope that we get relatime (with minor variations that
were discussed a few months ago) and that we make it the default
in some future release ...

Regards, STefan

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Defaults in 10.0 ZFS through bsdinstall

2013-11-15 Thread Mark Felder



On Fri, Nov 15, 2013, at 3:23, Stefan Esser wrote:
 Am 14.11.2013 22:02, schrieb Teske, Devin:
  On Nov 14, 2013, at 12:49 PM, Mark Felder wrote:
  We don't even do installs on UFS with atime disabled by default in fstab
  so why should we so suddenly change course for ZFS?
 
  
  You've made a good point.
 
 There is major difference between UFS and ZFS: UFS allows in-place
 updates of i-node fields (like atime), while ZFS uses COW for all
 data, file contents and meta-data like the i-nodes.
 
 With atime ON on UFS you'll see a small number of writes on
 file-systems that are only read - we are used to accept that.
 
 On ZFS every update of atime causes a write of the meta-data to
 a free location on disk, then updates of all data structures
 that reference that meta-data up to the root of the tree (the
 uberblock). An update of a few bytes turns out to write tens
 of KB for each atime update (within the TXG sync interval, which
 defaults to 5 seconds on FreeBSD). If you create snapshots, then
 each snapshot will contain a copy of the metadata that was valid
 at the time of the snapshot (well, that's not so different from
 the situation with UFS snapshots, just that the data structures
 are much more complex and larger in the ZFS case). Due to the
 ease and speed of snapshot creation with ZFS there probably are
 a magnitude or more snapshots on a typical ZFS system than on
 one using UFS (I currently have a few hundred and have turned off
 periodic snapshot generation on many unimportant file-systems,
 already).
 
 I really hope that we get relatime (with minor variations that
 were discussed a few months ago) and that we make it the default
 in some future release ...
 

Thanks for this in-depth explanation. I wasn't aware that atime was
quite so expensive on ZFS.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Defaults in 10.0 ZFS through bsdinstall

2013-11-15 Thread Kimmo Paasiala

On Fri, Nov 15, 2013 at 5:36 PM, Mark Felder f...@freebsd.org wrote:


 On Fri, Nov 15, 2013, at 3:23, Stefan Esser wrote:
 Am 14.11.2013 22:02, schrieb Teske, Devin:
  On Nov 14, 2013, at 12:49 PM, Mark Felder wrote:
  We don't even do installs on UFS with atime disabled by default in fstab
  so why should we so suddenly change course for ZFS?
 
 
  You've made a good point.

 There is major difference between UFS and ZFS: UFS allows in-place
 updates of i-node fields (like atime), while ZFS uses COW for all
 data, file contents and meta-data like the i-nodes.

 With atime ON on UFS you'll see a small number of writes on
 file-systems that are only read - we are used to accept that.

 On ZFS every update of atime causes a write of the meta-data to
 a free location on disk, then updates of all data structures
 that reference that meta-data up to the root of the tree (the
 uberblock). An update of a few bytes turns out to write tens
 of KB for each atime update (within the TXG sync interval, which
 defaults to 5 seconds on FreeBSD). If you create snapshots, then
 each snapshot will contain a copy of the metadata that was valid
 at the time of the snapshot (well, that's not so different from
 the situation with UFS snapshots, just that the data structures
 are much more complex and larger in the ZFS case). Due to the
 ease and speed of snapshot creation with ZFS there probably are
 a magnitude or more snapshots on a typical ZFS system than on
 one using UFS (I currently have a few hundred and have turned off
 periodic snapshot generation on many unimportant file-systems,
 already).

 I really hope that we get relatime (with minor variations that
 were discussed a few months ago) and that we make it the default
 in some future release ...


 Thanks for this in-depth explanation. I wasn't aware that atime was
 quite so expensive on ZFS.


What I did on my system when I was still using ZFS was that I set
atime off by default but enabled it explicitly on /var/mail and /home
datasets. The thought was that it's needed for mailboxes in /var/mail
and if I then decide to move the inboxes to user's home directories I
won't get any surprises. Would that be a suitable compromise here?


-Kimmo
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Defaults in 10.0 ZFS through bsdinstall

2013-11-14 Thread Marcus Reid

On Thu, Nov 14, 2013 at 06:35:43PM +, Teske, Devin wrote:
 On Nov 14, 2013, at 10:21 AM, Allan Jude wrote:
  On 2013-11-14 9:34, Reid, Marcus wrote:
  
  Hi,
  
  I noticed a couple of things with the ZFS defaults that result from
  using the new installer in 10.0-BETA3.
  
  One, atime is turned off everywhere by default.  There was a thread
  on this list on June 8 with a subject of 'Changing the default for
  ZFS atime to off?', and from what I can tell the idea of turning off
  atime by default was not a popular one.
  
  Two, and probably less controversial, is that fletcher4 is specified
  exlicitly on the root pool, even though it is default (wouldn't you
  just want to go with the default, in case it changes?)
  
  Marcus

 I have never heard a good argument for having atime on.

Can you address some of the issues that people brought up in the thread
I mentioned earlier?  I'll summarize some:

  - breaks some software (MTAs were mentioned), and the admin should
know when to turn atime on in those cases.

  - any mail program using mbox mail folders uses them to correctly
report which mailboxes have not been read yet

  - Of course it can't be turned off by default.  It is specified by
POSIX.

  - If the overhead were a real problem then file systems would use a
special atime cache

 The performance penalty on ZFS is quite large

This is subjective, and if it were that bad it seems like other
ZFS-based systems would have turned it off as well, or done something
like caching/logging or relatime it to mitigate the performance hit.

 and it also makes your snapshots grow constant.

This does not seem to be the case.  For example, if I snapshot
zroot/usr/src and then tar it up, I generate a bunch of writes and the
snapshot grows to 70.5MB.  However, when I tar it up a second time,
there are writes but the snapshot remains 70.5MB.  It seems to use the
same blocks for subsequent atime updates.

If you made another snapshot, you would see more space used for that
one, of course.

 If you have a use for it, you can turn it on I guess. This would be
 solved by having the dataset editor we're planning for 10.1
 
 Seems easy enough to turn on once installed. And like Allan says, the
 editor for 10.1 would allow changing of the default.

The same case could be made for turning it off, make that an option in
the installer instead.

Some people don't seem to use atime on a regular basis, but I use it a
lot when troubleshooting things, and I think others do to when they get
to the point where they realize how useful it is.

 I'll add that if scripting it, you can (in current state for 10.0)
 change the dataset options at-will.
 
 The fletcher4 thing is a leftover, from older versions of ZFS where
 the default was fletcher2, I suppose we can remove it

While I'm picking nits:

There are some other locally set things that are default (exec set to on
on zroot/var/tmp is one), that might just inherit that from zroot/var.

Also, zroot doesn't appear to be mounted on /zroot, as it is on other
ZFS-based systems.  This has the side-effect of newly created
datasets not having a mountpoint by default, instead of /zroot/dataset.
I don't know of the reason behind this change either.

I'm going to cross-post this to -current, this is a topic that I think
warrants a wider audience.

Thanks,

Marcus

 
 
 Let me put something together.
 - -- 
 Devin
 
 -BEGIN PGP SIGNATURE-
 Comment: GPGTools - https://gpgtools.org
 
 iQEcBAEBCgAGBQJShRf/AAoJEKrMn5R9npq5TYwH/j8RMceM4STpCAoMTAz8zkKn
 gpftMrdf37TnAPHxsXJaiZejlp+D85/a9lglYbB7e4WdgoDX9ZXA0QLtCEmEtN4y
 jz7KNa5oABq1GjDtwm2xQISDIe8yYI+znLSaHA9W18kCUZzi8AOj70C+O5gyY28c
 c3N3j6Y4EZP2wtQnWSgxvCwIX5p86Jvr3QvhrRuuMnKMCTAhzwIx24qcnA4EcTzn
 p9w25CKLtOzF23n3McodaUmibbBCHOYyraLQJ+eJGDXcsPBipls3oiWyiYuqpcQr
 18QpLN1lS35RnULcb5pxnP9Oy9rAwxHKves9Mfu0UoWw3qTjSAf8gsGyu6QaMgc=
 =GjYk
 -END PGP SIGNATURE-
 
 _
 The information contained in this message is proprietary and/or confidential. 
 If you are not the intended recipient, please: (i) delete the message and all 
 copies; (ii) do not disclose, distribute or use the message in any manner; 
 and (iii) notify the sender immediately. In addition, please be aware that 
 any message addressed to our domain is subject to archiving and review by 
 persons other than the intended recipient. Thank you.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Defaults in 10.0 ZFS through bsdinstall

2013-11-14 Thread Teske, Devin

On Nov 14, 2013, at 12:34 PM, Marcus Reid wrote:

On Thu, Nov 14, 2013 at 06:35:43PM +, Teske, Devin wrote:
On Nov 14, 2013, at 10:21 AM, Allan Jude wrote:
On 2013-11-14 9:34, Reid, Marcus wrote:

Hi,

I noticed a couple of things with the ZFS defaults that result from
using the new installer in 10.0-BETA3.

One, atime is turned off everywhere by default. There was a thread
on this list on June 8 with a subject of 'Changing the default for
ZFS atime to off?', and from what I can tell the idea of turning off
atime by default was not a popular one.

Two, and probably less controversial, is that fletcher4 is specified
exlicitly on the root pool, even though it is default (wouldn't you
just want to go with the default, in case it changes?)

Marcus

I have never heard a good argument for having atime on.

Can you address some of the issues that people brought up in the thread
I mentioned earlier? I'll summarize some:

- breaks some software (MTAs were mentioned), and the admin should
know when to turn atime on in those cases.

- any mail program using mbox mail folders uses them to correctly
report which mailboxes have not been read yet

I'm looking at HEAD and I don't see atime=off for the /var dataset.
Knowing that most folks (accepting the defaults) will store their mail
in /var/mail ... does the concern over atime=off still exist?

However, I did notice that before we go creating the /var dataset, we
do set atime=off for the boot pool/dataset.

Is inheritance at-play here? and /var is turning up with atime=off even
though it's not specified?

In that case, I certainly agree we should remove atime=off from the last
place it is used -- the boot pool (nowhere else).

- Of course it can't be turned off by default. It is specified by
POSIX.

- If the overhead were a real problem then file systems would use a
special atime cache

The performance penalty on ZFS is quite large

This is subjective, and if it were that bad it seems like other
ZFS-based systems would have turned it off as well, or done something
like caching/logging or relatime it to mitigate the performance hit.

I do recall discussions about bringing over relatime and making it the
default.

Allan was the one that brought up the performance hit...
Similarly where I $work, we turn it off where need performance (large
storage arrays and such).

I gather it's only a matter of time until relatime is default.

and it also makes your snapshots grow constant.

This does not seem to be the case. For example, if I snapshot
zroot/usr/src and then tar it up, I generate a bunch of writes and the
snapshot grows to 70.5MB. However, when I tar it up a second time,
there are writes but the snapshot remains 70.5MB. It seems to use the
same blocks for subsequent atime updates.

If you made another snapshot, you would see more space used for that
one, of course.

I'll defer to Allan to expand (was his recollection on snapshots)

If you have a use for it, you can turn it on I guess. This would be
solved by having the dataset editor we're planning for 10.1

Seems easy enough to turn on once installed. And like Allan says, the
editor for 10.1 would allow changing of the default.

The same case could be made for turning it off, make that an option in
the installer instead.

Fair enough. Good idea.

Some people don't seem to use atime on a regular basis, but I use it a
lot when troubleshooting things, and I think others do to when they get
to the point where they realize how useful it is.

I'll add that if scripting it, you can (in current state for 10.0)
change the dataset options at-will.

The fletcher4 thing is a leftover, from older versions of ZFS where
the default was fletcher2, I suppose we can remove it

While I'm picking nits:

There are some other locally set things that are default (exec set to on
on zroot/var/tmp is one), that might just inherit that from zroot/var.

Need a +1 from Allan on that.

Also, zroot doesn't appear to be mounted on /zroot, as it is on other
ZFS-based systems. This has the side-effect of newly created
datasets not having a mountpoint by default, instead of /zroot/dataset.
I don't know of the reason behind this change either.

Hmm, wonder if that's been fixed in HEAD already.

I'm going to cross-post this to -current, this is a topic that I think
warrants a wider audience.

Cool. Thanks.
--
Devin

-BEGIN PGP SIGNATURE-
Comment: GPGTools -
https://urldefense.proofpoint.com/v1/url?u=https://gpgtools.org/k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0Ar=LTzUWWrRnz2iN3PtHDubWRSAh9itVJ%2BMUcNBCQ4tyeo%3D%0Am=wXwxswJCpc3K%2Bud90aYI%2B82v%2Fyfs1mrTBzzeCe5XIXw%3D%0As=a9e0697e43ce4744f130997648d0104319221bba6924c28a9493ffa1cf6c3719

iQEcBAEBCgAGBQJShRf/AAoJEKrMn5R9npq5TYwH/j8RMceM4STpCAoMTAz8zkKn
gpftMrdf37TnAPHxsXJaiZejlp+D85/a9lglYbB7e4WdgoDX9ZXA0QLtCEmEtN4y