Re: noatime on ufs2
Am 2024-01-30 01:21, schrieb Warner Losh: On Mon, Jan 29, 2024 at 2:31 PM Olivier Certner wrote: It also seems undesirable to add a sysctl to control a value that the kernel doesn't use. The kernel has to use it to guarantee some uniform behavior irrespective of the mount being performed through mount(8) or by a direct call to nmount(2). I think this consistency is important. Perhaps all auto-mounters and mount helpers always run mount(8) and never deal with nmount(2), I would have to check (I seem to remember that, a long time ago, when nmount(2) was introduced as an enhancement over mount(2), the stance was that applications should use mount(8) and not nmount(2) directly). Even if there were no obvious callers of nmount(2), I would be a bit uncomfortable with this discrepancy in behavior. I disagree. I think Mike's suggestion was better and dealt with POLA and POLA breaking in a sane way. If the default is applied universally in user space, then we need not change the kernel at all. We lose all the chicken and egg problems and the non-linearness of the sysctl idea. I would like to add that a sysctl is some kind of a hidden setting, whereas /etc/fstab + /etc/defaults/fstab is a "right in the face" way of setting filesystem / mount related stuff. [...] It could also be generalized so that the FSTYPE could have different settings for different types of filesystem (maybe unique flags that some file systems don't understand). +1 nosuid for tmpfs comes into my mind here... One could also put it in /etc/defaults/fstab too and not break POLA since that's the pattern we use elsewhere. +1 Anyway, I've said my piece. I agree with Mike that there's consensus for this from the installer, and after that consensus falls away. Mike's idea is one that I can get behind since it elegantly solves the general problem. +1 Bye, Alexander. -- http://www.Leidinger.net alexan...@leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.orgnetch...@freebsd.org : PGP 0x8F31830F9F2772BF signature.asc Description: OpenPGP digital signature
Re: noatime on ufs2
On Mon, Jan 29, 2024 at 2:31 PM Olivier Certner wrote:
> Hi Mike,
>
> I've re-ordered a bit your mail to group some of my comments more
> logically.
>
> > I am not sure a sysctl is a good mechanism for setting the mount default,
> > especially if it is to be set via the kernel environment from
> > /boot/loader.conf. That's an obscure place to find file system defaults.
>
> If the setting has to matter for the root filesystem also (I think it
> should), currently the knob should be set in '/boot/loader.conf'. But if
> "regular" filesystems (those from '/etc/fstab') have an explicit 'atime' or
> 'noatime', '/etc/sysctl.conf' could be enough ('/etc/rc/sysctl' is run very
> early).
>
I strongly oppose this notion to control this from loader.conf. Root is
mounted read-only, so it doesn't matter. That's why I liked Mike's
suggestion: root isn't special.
> > It also seems undesirable to add a sysctl to control a value that the
> > kernel doesn't use.
>
> The kernel has to use it to guarantee some uniform behavior irrespective
> of the mount being performed through mount(8) or by a direct call to
> nmount(2). I think this consistency is important. Perhaps all
> auto-mounters and mount helpers always run mount(8) and never deal with
> nmount(2), I would have to check (I seem to remember that, a long time ago,
> when nmount(2) was introduced as an enhancement over mount(2), the stance
> was that applications should use mount(8) and not nmount(2) directly).
> Even if there were no obvious callers of nmount(2), I would be a bit
> uncomfortable with this discrepancy in behavior.
>
I disagree. I think Mike's suggestion was better and dealt with POLA and
POLA breaking in a sane way. If the default is applied universally in user
space, then we need not change the kernel at all. We lose all the chicken
and egg problems and the non-linearness of the sysctl idea.
> > Note that the root file system is mounted specially in the kernel, but
> the
> > noatime option doesn't need to be set at first while the root is
> read-only.
> > It could be updated by mount when remounting read-write from the startup
> > scripts.
>
> That's true. However, how about other filesystems mounted by rc scripts,
> such as '/tmp'? I agree that this one is not a good example, since the
> 'tmpfs' script ultimately calls 'mdmfs', which ultimately spawns a new
> process to execute mount(8). But I fear that, if we don't have the
> consistency exposed just above, we are going to need to audit other
> programs, including external ones, which is precisely what I wanted to
> avoid with a simple default that applies to everything (hence, implemented
> in the kernel).
>
If it's in fstab as default, then it would be read by whatever updates
things in user space.
> > Instead, I'd like to propose that the default be
> > specified in a new entry in /etc/fstab, where it would be much more
> obvious.
> > For example, a line could be placed at the beginning like:
> >
> > # DeviceMountpoint FStype Options
> > default nonedefault noatime,...
> >
> > It could be retrieved with getfsspec("default") in the fs_mnntops field.
> > I wouldn't include this entry when iterating through the whole file with
> > getfsent() to avoid confusing existing programs. Then mount, and other
> > utilities such as zfs create, could check it explicitly. It should be
> > placed in /etc/fstab when it is created: by bsdinstall when it is used,
> > preferably by having the user select this explicitly, but probably with
> > noatime being the default. It would be in the pre-configured fstab used
> > for VM images and SD card images. Anyone building a root file system by
> > hand would have to deal with this to set a default.
>
> That could be great. And it's not necessarily in contradiction with a
> sysctl. If we have the latter, setting the default could happen through it
> and could be done by some startup script. Then, the only thing not covered
> is the root filesystem, but even this is fixable by parsing the default
> line from the loader itself (it already parses '/etc/fstab' after all) and
> converting that specification to tunables passed to the kernel.
>
I really like Mike's idea. It obviates the need for the sysctl entirely. It
gets around the need to update loader.conf as well. It concentrates the
change in one place and
does so in a way that's not at all atime focused: It could also be
generalized so that the FSTYPE could have different settings for different
types of filesystem (maybe unique flags that some file systems don't
understand).
> > I would then have the mount program look up and apply the default for
> things
> > like mounting a file system manually. Perhaps it could have a -D option
> > to ignore defaults, e.g. for scripts that don't want to be subject to
> local
> > settings.
>
> This is a complication in the case of using sysctl knobs and the kernel
> being in charge of applying them as the
Re: noatime on ufs2
Hi Mike,
I've re-ordered a bit your mail to group some of my comments more logically.
> I am not sure a sysctl is a good mechanism for setting the mount default,
> especially if it is to be set via the kernel environment from
> /boot/loader.conf. That's an obscure place to find file system defaults.
If the setting has to matter for the root filesystem also (I think it should),
currently the knob should be set in '/boot/loader.conf'. But if "regular"
filesystems (those from '/etc/fstab') have an explicit 'atime' or 'noatime',
'/etc/sysctl.conf' could be enough ('/etc/rc/sysctl' is run very early).
> It also seems undesirable to add a sysctl to control a value that the
> kernel doesn't use.
The kernel has to use it to guarantee some uniform behavior irrespective of the
mount being performed through mount(8) or by a direct call to nmount(2). I
think this consistency is important. Perhaps all auto-mounters and mount
helpers always run mount(8) and never deal with nmount(2), I would have to
check (I seem to remember that, a long time ago, when nmount(2) was introduced
as an enhancement over mount(2), the stance was that applications should use
mount(8) and not nmount(2) directly). Even if there were no obvious callers of
nmount(2), I would be a bit uncomfortable with this discrepancy in behavior.
> Note that the root file system is mounted specially in the kernel, but the
> noatime option doesn't need to be set at first while the root is read-only.
> It could be updated by mount when remounting read-write from the startup
> scripts.
That's true. However, how about other filesystems mounted by rc scripts, such
as '/tmp'? I agree that this one is not a good example, since the 'tmpfs'
script ultimately calls 'mdmfs', which ultimately spawns a new process to
execute mount(8). But I fear that, if we don't have the consistency exposed
just above, we are going to need to audit other programs, including external
ones, which is precisely what I wanted to avoid with a simple default that
applies to everything (hence, implemented in the kernel).
> Instead, I'd like to propose that the default be
> specified in a new entry in /etc/fstab, where it would be much more obvious.
> For example, a line could be placed at the beginning like:
>
> # DeviceMountpoint FStype Options
> default nonedefault noatime,...
>
> It could be retrieved with getfsspec("default") in the fs_mnntops field.
> I wouldn't include this entry when iterating through the whole file with
> getfsent() to avoid confusing existing programs. Then mount, and other
> utilities such as zfs create, could check it explicitly. It should be
> placed in /etc/fstab when it is created: by bsdinstall when it is used,
> preferably by having the user select this explicitly, but probably with
> noatime being the default. It would be in the pre-configured fstab used
> for VM images and SD card images. Anyone building a root file system by
> hand would have to deal with this to set a default.
That could be great. And it's not necessarily in contradiction with a sysctl.
If we have the latter, setting the default could happen through it and could be
done by some startup script. Then, the only thing not covered is the root
filesystem, but even this is fixable by parsing the default line from the
loader itself (it already parses '/etc/fstab' after all) and converting that
specification to tunables passed to the kernel.
> I would then have the mount program look up and apply the default for things
> like mounting a file system manually. Perhaps it could have a -D option
> to ignore defaults, e.g. for scripts that don't want to be subject to local
> settings.
This is a complication in the case of using sysctl knobs and the kernel being
in charge of applying them as the defaults. It implies that mount(8) should
know some fixed old defaults, irrespective of the sysctl values. As evoked in
another mail, I think the choice of defaults is really an administrative
matter. If some scripts really need 'atime' to work, I would think that the
administrator should not change the default to 'noatime', else make sure these
scripts explicitly pass 'atime' (or use a line in '/etc/fstab' that specifies
'atime'). Doing the latter seems to be exactly the same effort as having the
same scripts start to use '-D' (whether by configuration or direct
modification).
> It would be plausible to set the default(s) in rc.conf instead, although
> that is more convenient for shell scripts than C programs. It would be
> possible to read output from something like "sysrc filesystem_defaults".
> It would also not be as obvious when setting or checking file system
> configuration.
The non-obvious remark seems to be an argument in favor of having the defaults
in '/etc/fstab'.
> btw, I think there is consensus that noatime is the most useful setting for
> most systems and users. However, I don't think
Re: noatime on ufs2
Hi, > Let me start out by indicating that some bike shed sessions > (snip) > Much of the overall usage is in that "additional attempted span". Ok, so it seems I've misunderstood what you were saying or your intent in this regard. > I will adjust and deal with whatever happens > overall. That is something I know I can do for my > usage context. With a sysctl knob defaulting to 'atime', you won't have to change anything. Thanks for your participation. Regards. -- Olivier Certner signature.asc Description: This is a digitally signed message part.
Re: noatime on ufs2
On Jan 29, 2024, at 02:27, Olivier Certner wrote: > Hi Mark, Hello. Let me start out by indicating that some bike shed sessions are useful overall, even if not contributing to crucial matters. I do not see withdrawing from continued participation with new material as disqualifying of any of the rest of the overall effort in a bike shed session. Given what has already occurred in the session, I've nothing significant to add to the accumulated material and am trying to avoid adding less significant material that wastes effort/time. >> I'm confused: I go to the trouble to produce the same end result >> as your suggested change of defaults would produce, ending up >> with no recording of access times. > > That's nice of you, but unfortunately that's missing the point. First, you > claimed to "seriously care" about access time, so I simply asked about your > use cases, which you have not talked about. Second, your suggestions do not > (in fact, cannot) produce the same end result as what I'm suggesting (change > of default, or have a sysctl to control the default). I've already listed > three use cases in an answer to Warner that can't be covered by modifying > '/etc/fstab', and two of them that can't by just specifying mount options to > mount(8) on the command-line (the auto-mounters). I tried pointing out that my limited usage context is too narrow to make a reasonable generalization from for the overall problem. Auto-mounting is a good example that I'll point out relative to my usage context: What auto-mounting? But auto- mounting should be considered. I had to try to span beyond my usage context to form the options that I expressed. To only consider my usage would have been inappropriate. I can not use my usage context to justify coverage of that additional attempted span. Much of the overall usage is in that "additional attempted span". I will adjust and deal with whatever happens overall. That is something I know I can do for my usage context. >> . . . > . . . > > Is 'noatime' not being the default the biggest problem we currently have in > FreeBSD? I agree it isn't. However, it doesn't mean there is no value in > it. On the contrary, I think it is very important that the project has sane > defaults that match contemporary uses: It reduces the need for tweaks, which > serves both beginners but *also* seasoned administrators. This is in > isolation a very small step in this direction, but there have been others and > more will come. Collectively, they can build up to significant additional > value for the project. === Mark Millard marklmi at yahoo.com
Re: noatime on ufs2
Not responding to a specific message, but following up on the thread:
I am not sure a sysctl is a good mechanism for setting the mount default,
especially if it is to be set via the kernel environment from
/boot/loader.conf. That's an obscure place to find file system defaults.
It also seems undesirable to add a sysctl to control a value that the
kernel doesn't use. Instead, I'd like to propose that the default be
specified in a new entry in /etc/fstab, where it would be much more obvious.
For example, a line could be placed at the beginning like:
# DeviceMountpoint FStype Options
default nonedefault noatime,...
It could be retrieved with getfsspec("default") in the fs_mnntops field.
I wouldn't include this entry when iterating through the whole file with
getfsent() to avoid confusing existing programs. Then mount, and other
utilities such as zfs create, could check it explicitly. It should be
placed in /etc/fstab when it is created: by bsdinstall when it is used,
preferably by having the user select this explicitly, but probably with
noatime being the default. It would be in the pre-configured fstab used
for VM images and SD card images. Anyone building a root file system by
hand would have to deal with this to set a default.
Note that the root file system is mounted specially in the kernel, but the
noatime option doesn't need to be set at first while the root is read-only.
It could be updated by mount when remounting read-write from the startup
scripts.
I would then have the mount program look up and apply the default for things
like mounting a file system manually. Perhaps it could have a -D option
to ignore defaults, e.g. for scripts that don't want to be subject to local
settings.
It would be plausible to set the default(s) in rc.conf instead, although
that is more convenient for shell scripts than C programs. It would be
possible to read output from something like "sysrc filesystem_defaults".
It would also not be as obvious when setting or checking file system
configuration.
btw, I think there is consensus that noatime is the most useful setting for
most systems and users. However, I don't think there is consensus that the
default should be changed for things like mount with no options. I think
that putting a default somewhere fairly obvious could make it more palatable
(less POLA violation). Opinions may vary, though.
Mike
Re: noatime on ufs2
Hi Chris, > Honestly! Gosh... This doesn't start well. > Why do we have to upend decades of usage and understanding? Just > because it's old doesn't mean it's wrong. Who says that exactly? Separately, in case you haven't noticed yet, some things have changed in the past 50 years... > Several weeks of replies confirm my initial > belief -- atime as it is currently implemented, is as it should be. Several weeks? The first mail was sent on January 7th, my first intervention about 'noatime' being the default was on the 9th and your mail is dated the 16th. I would suggest that you revise your implementation of access times. > I haven't seen anything in this thread that wouldn't be better placed in > tuning(7) or tunefs(8). It's not really surprising since, given what you said before that, it seems that you haven't really paid attention to the messages exchanged. > Security and forensics are good reasons to keep atime unchanged. Having access times maintained may occasionally provide a bit more information, especially against script kiddies. But if you think it's a reliable source of information, you're deluded. See my response to Lyndon about the general problem with access times. I also already talked about auditing's needs in my very first mail in this thread. > Any discussion regarding changes to it's current behavior seems folly or > bikeshedding. About bike shedding, see my recent response to Mark, it applies to "folly" as well. > Apologies for the "attitude". Yeah. Most prompts imagined by Poul-Henning Kamp in his bikeshed email would have applied perfectly to your message. Thanks for the thought. -- Olivier Certner signature.asc Description: This is a digitally signed message part.
Re: noatime on ufs2
Hi Mark, > I'm confused: I go to the trouble to produce the same end result > as your suggested change of defaults would produce, ending up > with no recording of access times. That's nice of you, but unfortunately that's missing the point. First, you claimed to "seriously care" about access time, so I simply asked about your use cases, which you have not talked about. Second, your suggestions do not (in fact, cannot) produce the same end result as what I'm suggesting (change of default, or have a sysctl to control the default). I've already listed three use cases in an answer to Warner that can't be covered by modifying '/etc/fstab', and two of them that can't by just specifying mount options to mount(8) on the command-line (the auto-mounters). > My focus was on things like mount command notation and > /etc/fstab notation (that tracks mount defaults) or subroutine > interface equivalents of such things and changing their > behavior without requiring changing the notation already in > place in various files. > Nothing about that of itself > implies that I'd want the defaults for mount notation or /etc/fstab > notation or the like changed --or that I'd want them unchanged. > To narrow of a context for such a judgment about defaults. These two paragraphs seem to contradict themselves. If you've gone to the trouble mentioned above, wasn't it precisely to avoid changing the defaults? Because changing them implies that the exact same mount(8) command-line or line in '/etc/fstab' will have a different effect if 'atime' nor 'noatime' aren't explicitly specified. This is a goal, not an unwanted side effect. > In case the potential confusion is involved I'm wondering if you might be confusing default options per mount (as a line in '/etc/fstab') and system defaults applying to all mounts. By design, the former can't apply to mounts not handled by '/etc/fstab'. The latter applies in any situation, barring explicit specifications by the administrator (or delegated software), which is why it belongs to the kernel (it has to apply to the relevant system calls). > (I've tried to word the above without making new points, > avoiding contributing more to the bike shed material.) Bike shedding has become so popular in these circles that some people see bike shedding where there is none, and/or use it as a tactic to try to disqualify what others are saying. The initial bike shedding email was about a simple, obvious change to sleep(1) that prompted a flame war lasting weeks, with unfriendly fire from the project's people, sometimes from the old timers. We haven't had much of that so far, except perhaps for a few mildly aggressive or emotional emails, and the thread was active for only 10 days. It was then paused for 12 days since I didn't find time to read the latest mails and produce answers until today. What sets it apart also from the sleep(1) example is that I intended to drop an idea and gather reactions before having even produced code, because I wasn't sure on how it would be received and in particular what are the use cases that could be affected. Obtaining this feedback is essential because this project is about people from diverse backgrounds and needs. It also helps in clarifying a particular design, which some answers fulfilled. I'm now at the point where the next step is to put up some code for review for a sysctl knob. Is 'noatime' not being the default the biggest problem we currently have in FreeBSD? I agree it isn't. However, it doesn't mean there is no value in it. On the contrary, I think it is very important that the project has sane defaults that match contemporary uses: It reduces the need for tweaks, which serves both beginners but *also* seasoned administrators. This is in isolation a very small step in this direction, but there have been others and more will come. Collectively, they can build up to significant additional value for the project. Regards. -- Olivier Certner signature.asc Description: This is a digitally signed message part.
Re: noatime on ufs2
Hi Alexander,
> ZFS by default has atime=on. It is our installer which sets atime=off in
> the ZFS properties. I was understanding Warners comment about changing
> ZFS in the sense of changing the ZFS code to have a default of
> atime=off.
>
> I agree with Warner that we should not do that. And in my opinion we
> should keep the other FS which support atime/noatime consistent (those
> which don't support atime/noatime due to technial limitations don't
> count in my opinion)
I've almost addressed this point in an analysis responding to a similar point
by Mark Millard, so let me wrap up and complete it.
The situation of ZFS is not comparable to that of other filesystems because ZFS
keeps a persistent "access time" setting (the 'atime' property) for its
filesystems. ZFS' fundamental way to deal with such kind of settings ("native
control properties") is to automatically apply them at mount, so changing that
part would indeed be a huge POLA violation, and I'm not proposing it (and, to
be clear, never meant to).
The only place where a system default for 'atime' makes sense for ZFS is the
initial value of 'atime' for the root filesystem of each pool on pool creation.
So it's really only a matter of, e.g., changing 'zpool create' to behave as if
'atime=on' or 'atime=off' were explicitly specified (depending on the sysctl's
value; of course, if the user really specified one or the other, his value
prevails). Doing it this way makes ZFS' default for 'atime' completely
invisible (a good thing, IMO). More importantly, after skimming through the
relevant code, it's likely it will work without any kernel code change (whether
in OpenZFS or FreeBSD) at all. There are also other possibilities.
> But I would say this is atypical behavior by people which know
> exactly what they are doing, not what a normal consumer would do. Such
> experts know exactly what they want to do with atime and handle it as
> needed.
The point is rather that nobody, even experts, should bother tuning mounts with
'noatime' when almost all people that care want 'noatime'.
> Those which care about that and know where this makes a difference, have
> it in their finger-memory.
Sure. Does this mean that they must be forced to specify it each time, or even
have scripts for that, each one developing its own? I don't see anything
constructive in requiring that.
> If they are not able to handle that, it is a bug.
> Typical media in desktop use-cases doesn't really need this. If you
> handle media which really _needs_ noatime in such a case, you may want
> to reconsider your way of operating.
I've already mentioned the case of buggy USB keys in another mail, which is a
very annoying use case. Of course, I try to avoid it, but as you know, these
things happen, and this has nothing to do with how I operate but just
adaptation to the world we live in.
> > So limiting action to the installer, while certainly a sensible and
> > pragmatic step, still seems to miss a lot.
>
> Nobody told to only limit this action to the installer.
This seemed to be implicit in some other answers, so I prefer to clarify.
> The pragmatic part here is to ask if it really matters for those use cases.
> For
> mounting by hand I disagree that it matters. For our automounter we
> should do something (at least making sure it is able to handle it, and
> if we don't want to swtich the default at least have a commented out
> entry in the config with a suitable comment). For the desktop helpers it
> is not our responsability, but interested people surely can file a bug
> report upstream.
I disagree for the mounting by hand part (whether to mount USB key or other
external disks). For the rest, yes, it would be nice to change our automounter
and external programs, but all of this becomes moot with a proper default, so
why go through that hassle for the case at hand?
> While I agree that this would be an easy way of globally changing the
> default, what makes noatime special compared to nocover, or nfs4acl, or
> noexec, or nosuid, or whatever other option? Mounting noexec and nosuid
> by default and having those FS be mounted explicitely suid/exec which
> really need it would be a security benefit. And cover/nocover would
> prevent accidental foot-shooting. Where do you want to draw the line
> between "easy" and "explicit"? Only having atime/noatime handled like
> that looks inconsistent to me (which - I hope - not only me thinks is a
> POLA violation).
I'd say this would look incomplete rather than inconsistent, in the sense that
it is always possible to add support for changing the default for other options
later, as this is not a case of already having several similar features that
work differently. Your objection is quite strange in this regard. I don't
think we need to draw the line anywhere now. I like your suggestion about
covering other options. Typically, for my use cases, I would enable by default
