Re: Devices.

[Michael van Elst]

dholland-t...@netbsd.org (David Holland) writes:

>This was all based on the experience of adding discard and adding the
>dispatching for it as a first-class [bc]devsw op rather than an ioctl:
>it was a pain because it ultimately required touching _every_ driver,
>not just the ones that needed to support it, but it was far less of a
>mess than plumbing it as an ioctl would have been.

And it was a questionable decision as you need the discard operations
to queue with other I/O operations.

Re: Devices.

[Mouse]

> We should really get with the times and create a devfs.  I know that
> there are people that disagree with this (likely including you), but
> the archaic device node system causes a lot of headaches and it's
> time that we joined the 21st century.

I am not someone who thinks that we, for any value of 'we", should go
with the new simply because it's new.

I've seen various flavours of devfs.  They have good aspects.  They
also have bad aspects.

I'm curious what headaches you think devfs would fix.  (I know what
headaches *I* think it would fix.  I'm wondering which ones *you* think
are worth bringing it in for.)

The major downside I've seen is that they also render difficult or
impossible a number of clever things that are rarely useful but
borderline essential when they are useful.  Most of those things are
based on devices being "just" filesystem nodes, leading to things such
as the same device appearing more than once in the filesystem
(including places outside /dev), things such as chroot areas with
hand-crafted (and severely cut-down) /dev directories...some of the
devfs variants I've seen don't even support chmod/chown.

By the time you (re)implement all the filesystem operations that give
/dev much of its power, you're pretty much back where you started, only
with a custom filesystem - with new and different bugs and far less
well tested code - on /dev.

devfs is a great thing to experiment with.  NetBSD is - well, was - a
good platform for that.  Perhaps there's a devfs design somewhere
that's solved enough of the issues that it's worth making available.

But I really don't think doing away with conventional device nodes is
sane yet.  Most of the benefits of devfs can be achieved with some kind
of userland interface to the autoconf tree and some userland daemons -
mechanism, not policy.  All the devfses I've seen have, in my opinion,
way too much policy wired into them.

> [...] I think we have enough brain power and enough real world
> examples to be able to not repeat the mistakes of the past.

Not if we do it to "join[] the 21st century".  That's a very wrong
reason to do...well, almost anything technical.

IMO.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: Devices.

[David Holland]

On Sat, May 29, 2021 at 04:17:13PM -0700, John Nemeth wrote:
 >  We should really get with the times and create a devfs.  I
 > know that there are people that disagree with this (likely including
 > you), but the archaic device node system causes a lot of headaches
 > and it's time that we joined the 21st century.  Anything done with
 > devices should be done with idea of a devfs in mind.  Yes, devfs
 > like things have caused a lot of problems on other operating systems,
 > but I think we have enough brain power and enough real world examples
 > to be able to not repeat the mistakes of the past.

The problem with the devfs everyone wants is that it's an automounter,
not a file system, which means that (a) everybody who tries to
implement it as a file system discovers that the abstractions don't
match up and ends up in varying degrees of trouble as a result, and
(b) automounters seem to be inherently awful and devfs implementations
thus are too.

The stuff I'm proposing doesn't affect those issues, and is likely to
make any implementation easier/simpler: it reduces the amount of work
a fs with device special files on it needs to do, and it also provides
the means to look up and "start" any given device or device instance
as its own interface disentangled from the filesystem code.

IMO the only significant problem that devfs solves is storing magic
numbers (device major numbers) on disk, and I think that's readily
(and better) solved by storing driver names instead of major numbers.
Otherwise it's mostly a matter of not cluttering /dev with entries for
hardware you don't have (nice but not a big deal) and not needing to
run MAKEDEV in various circumstances (definitely useful, but not
exactly critical)... all of which would be all very well if a clean
implementation existed, but given that this does not seem to be the
case it doesn't seem to me to be worthwhile.

-- 
David A. Holland
dholl...@netbsd.org

Re: Devices.

[David Holland]

On Sun, May 30, 2021 at 12:00:16AM +0200, Johnny Billquist wrote:
 > On 2021-05-29 22:26, David Holland wrote:
 > > There are a number of infelicities in the way we currently handle the
 > > I/O plumbing for devices in the kernel. These include:
 > 
 > [...]
 > 
 > Just looking/thinking about the ioctl part - you say abolish it inside the
 > kernel. So does that mean that we keep the ioctl() interface for user
 > programs, but we'd create a system function that will translate ioctl()
 > calls from user programs to different function calls based on class and
 > function.

Yes, that's the idea. Getting rid of it at the system call interface
is a much harder problem (between compat, binary emulation, standards,
masses of existing 3rd-party code that calls ioctl explicitly, etc.)
and probably not worth biting off at all, let alone right now.

(Hopefully this also addresses Paul's email.)

This was all based on the experience of adding discard and adding the
dispatching for it as a first-class [bc]devsw op rather than an ioctl:
it was a pain because it ultimately required touching _every_ driver,
not just the ones that needed to support it, but it was far less of a
mess than plumbing it as an ioctl would have been.

 > Which means any time anyone wants to add some new kind of function to a
 > device, you'd have to adjust the ioctl() system call, the class, all
 > devices of that class, and of course your specific driver which you want to
 > do something with the hardware.

That is correct; but normally when you do that (like with discard) you
want it to exist on every device of that class, even if to begin with
most of the implementations are "return ENOSYS".

It is unlikely that ioctl can actually be made to go away entirely
within the kernel or even the vnode side of the kernel (the networking
code is also full of ioctls that this set of plumbing changes would
not affect at all) so there'd probably still be a place to stick
really ad hoc things. But mostly it's better to avoid those,
especially because they gradually accrete into standard interfaces. :-|

Note that even if it does become possible to kill off ioctl at the
driver level it's not going to happen anytime soon, and that there
will also likely need to be some kind of "misc" driver class as well.
I would expect the process of moving in this direction to begin by
carving off obvious device classes (e.g. disks, audio) and then
migrating their ioctls; at some point we'll get to a stage where the
stuff that's left is all atypical and it becomes a waste of time to
try to make up a framework that fits it all.


... something I should have mentioned in the original mail:
centralizing ioctl dispatch also allows having a single copy of any
processing code for the arguments, which isn't a big deal for ordinary
ioctls but should definitely help with robustness as soon as there are
pointers involved or compat32 issues.

-- 
David A. Holland
dholl...@netbsd.org

Re: Devices.

[John Nemeth]

On May 29, 22:52, David Holland wrote:
} On Sat, May 29, 2021 at 05:41:38PM -0400, Mouse wrote:
} 
}  > > For disks, which for historical reasons live in both cdevsw and
}  > > bdevsw, both entries would point at the same disk_dev.
}  > 
}  > I would suggest getting rid of the bdev/cdev distinction.  It is, as
}  > you say, a historical artifact, and IMO it is not serving anyone at
}  > this point.
} 
} It is deeply baked into the system call API and into POSIX, so it's
} not going anywhere. It's been proposed that we should stop having
} block devices, which would have the same net effect; I have no strong
} opinion on that and it doesn't need to be part of this set of changes.

 I was thinking the same thing about getting rid of block
devices.  The only place they should ever be used is an argument
to mount(2) and mount(2) can be adjusted to use a block device
underneath when it is handed a character device.  FreeBSD got rid
of block devices a long time ago.  Doing that as a first step is
likely to simplify things to make other things easier.

}  > > A third question: how does this affect interfaces?
}  > 
}  > As in, network interfaces?  Good question.  I think they should be
}  > device nodes in the filesystem *somehow*.
} 
} That's probably true, but they currently aren't and the plumbing above
} them is unrelated to the VFS device plumbing, so for the time being
} it's a separate issue.
} 
} Disentangling the current situation with device special files on
} filesystems will make it easier to manifest interfaces on disk if we
} ultimately want that.

 We should really get with the times and create a devfs.  I
know that there are people that disagree with this (likely including
you), but the archaic device node system causes a lot of headaches
and it's time that we joined the 21st century.  Anything done with
devices should be done with idea of a devfs in mind.  Yes, devfs
like things have caused a lot of problems on other operating systems,
but I think we have enough brain power and enough real world examples
to be able to not repeat the mistakes of the past.

}-- End of excerpt from David Holland

Re: Devices.

[David Holland]

On Sat, May 29, 2021 at 05:41:38PM -0400, Mouse wrote:
 > There is, however, one thing I think is missing in your rework
 > proposal.  I see nowhere to fit in a one-off driver for idiosyncratic
 > hardware - or as a pseudo-device interface to idiosyncratic kernel
 > code.  I've personally done enough instances of each that I would find
 > it close to crippling if support for them were to be lost.

It's unlikely to be possible to avoid having a device class called
"miscellaneous" that such things could be stuffed into, because, as
you note, they exist.

 > > For disks, which for historical reasons live in both cdevsw and
 > > bdevsw, both entries would point at the same disk_dev.
 > 
 > I would suggest getting rid of the bdev/cdev distinction.  It is, as
 > you say, a historical artifact, and IMO it is not serving anyone at
 > this point.

It is deeply baked into the system call API and into POSIX, so it's
not going anywhere. It's been proposed that we should stop having
block devices, which would have the same net effect; I have no strong
opinion on that and it doesn't need to be part of this set of changes.

 > > (2) Abolish ioctl inside the kernel, or at least within the device
 > > tree.
 > 
 > This would, I think be possibly the hardest part of dealing with
 > idiosyncratic drivers such as I sketched above.  Some of the drivers
 > I've done use ioctls, some use I/O on minor sub-numbers (eg, the low
 > three bits of the minor number select a functional interface whereas
 > the rest of the bits are the "true" minor number), but most of them
 > need something of the sort.

Getting rid of it altogether is likely too optimistic for reality. See
next email (upcoming).

 > > A third question: how does this affect interfaces?
 > 
 > As in, network interfaces?  Good question.  I think they should be
 > device nodes in the filesystem *somehow*.

That's probably true, but they currently aren't and the plumbing above
them is unrelated to the VFS device plumbing, so for the time being
it's a separate issue.

Disentangling the current situation with device special files on
filesystems will make it easier to manifest interfaces on disk if we
ultimately want that.

-- 
David A. Holland
dholl...@netbsd.org

Re: Devices.

[Johnny Billquist]

On 2021-05-29 22:26, David Holland wrote:

There are a number of infelicities in the way we currently handle the
I/O plumbing for devices in the kernel. These include:


[...]

Just looking/thinking about the ioctl part - you say abolish it inside 
the kernel. So does that mean that we keep the ioctl() interface for 
user programs, but we'd create a system function that will translate 
ioctl() calls from user programs to different function calls based on 
class and function.


Which means any time anyone wants to add some new kind of function to a 
device, you'd have to adjust the ioctl() system call, the class, all 
devices of that class, and of course your specific driver which you want 
to do something with the hardware.


Or did I misunderstand something here?

I'm not trying to shoot anything down at this time. Just trying to 
understand the implications of the suggestion.


I sortof like the idea of getting some generalization, which classes 
would bring. But the problem with ioctl causing duplication and all kind 
of efforts is at the same time also a strength in that you can add 
something new to a device without having to care at all about all other 
devices and code. Since nothing else really tries to understand ioctls. 
Flexible, but more effort if you want coherent behavior.


  Johnny

--
Johnny Billquist  || "I'm on a bus
  ||  on a psychedelic trip
email: b...@softjar.se ||  Reading murder books
pdp is alive! ||  tryin' to stay hip" - B. Idol

Re: Devices.

[Paul Goyette]

On Sat, 29 May 2021, David Holland wrote:




Anyhow, I think this architecture addresses all the problems cited.
The critical question is: what have I overlooked? There are probably
some issues I've thought about but failed to remember to discuss
above; there are also probably some issues I've not thought about or
am completely unaware of.


One thing that would concern me is compat...  :)  It's great if, for
example, we can rip out ioctl() but if we have to maintain it to
provide compatability that might increase the problem by an order of
magnitude.

Some sort of transition planning would need to exist in the overall
plan, and well-communicated to users.


++--+---+
| Paul Goyette   | PGP Key fingerprint: | E-mail addresses: |
| (Retired)  | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org   |
++--+---+

Re: Devices.

[Mouse]

> There are a number of infelicities in the way we currently handle the
> I/O plumbing for devices in the kernel.  These include:  [...]

Thank you.  I don't agree with all of that (which I daresay astonishes
nobody), but much of it sounds like a codification of some vague unease
that's been floating around half-formed in the back of my mind
for...well, gradually forming for almost as long as I've been using
Unix variants, meaning something like 40 years at this point.

There is, however, one thing I think is missing in your rework
proposal.  I see nowhere to fit in a one-off driver for idiosyncratic
hardware - or as a pseudo-device interface to idiosyncratic kernel
code.  I've personally done enough instances of each that I would find
it close to crippling if support for them were to be lost.

> For disks, which for historical reasons live in both cdevsw and
> bdevsw, both entries would point at the same disk_dev.

I would suggest getting rid of the bdev/cdev distinction.  It is, as
you say, a historical artifact, and IMO it is not serving anyone at
this point.

> (2) Abolish ioctl inside the kernel, or at least within the device
> tree.

This would, I think be possibly the hardest part of dealing with
idiosyncratic drivers such as I sketched above.  Some of the drivers
I've done use ioctls, some use I/O on minor sub-numbers (eg, the low
three bits of the minor number select a functional interface whereas
the rest of the bits are the "true" minor number), but most of them
need something of the sort.

> A third question: how does this affect interfaces?

As in, network interfaces?  Good question.  I think they should be
device nodes in the filesystem *somehow*.  For example, at present
there is no way to grant someone tcpdump access to le0 without granting
tcpdump access to all interfaces, short of a custom setuid program, or
something even more contrived such as bridge and a special-purpose tap
instance.  Seems to me this really should be doable with as little as
chown (or chgrp) and chmod, much as it currently is for (say) ttys.

I also believe Bluetooth should be just another AF rather than being
its own weird mess tacked onto the side of the networking subsystem.  I
don't know if that's worth taking on, but, given the magnitude of this
overhaul, it seems like a good opportunity to fix things like that, or
the aforementioned legacy cdev/bdev split.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Devices.

[David Holland]

There are a number of infelicities in the way we currently handle the
I/O plumbing for devices in the kernel. These include:
   - cloning devices exist but as currently implemented violate
 layering abstractions;
   - every file system needs to have cutpaste vnode ops tables for
 device vnodes;
   - the split between block and character devices was never
 particularly well anchored in reality (e.g. tapes) but has aged
 poorly since as new classes of devices have appeared;
   - because we don't distinguish between different classes of devices
 nearly all device ops travel through the system as ioctls;
   - because of the ensuing complexity, dispatching of ioctls is a
 mess; there are many cases where ioctl handlers match some ioctls
 specifically and pass anything else on, which introduces
 opportunities for various kinds of bugs;
   - adding a new device-level operation to cdevsw or bdevsw requires
 touching every driver, including those it is completely
 irrelevant to;
   - if you have multiple sets of device nodes on a system (e.g. in
 chroots) operations that affect the device nodes themselves can
 behave differently or strangely depending on which copy you touch
 (we have had multiple generations of hacks to mitigate this, and
 none have been completely satisfactory);
   - because we don't distinguish device classes in any modern sense
 of the term they cannot be addressed or reasoned about in system
 config or things like kauth policies;
   - and probably other things I haven't thought of.

I've been mumbling on and off for a long time about various parts of
this problem, and I think it's time to propose a unified architecture
for a solution. Note that the changes required are nontrivial and this
is not going to happen all at once (or anytime soon); the goal of
blathering about it is to try to reach agreement on a place that we
want to get to eventually... and also, to smoke out any places where
the proposed architecture won't actually work or is inconsistent with
what happens on the ground.

There are four major interconnected sets of changes I have in mind to
address these problems.

(1) Create explicit device classes. This would be adding a layer of
indirection between struct cdevsw/bdevsw and drivers; so e.g. a mouse
driver would, instead of declaring a struct cdevsw, declare a struct
mouse_dev containing operations on mice, and the cdevsw entry would
point at this. For disks, which for historical reasons live in both
cdevsw and bdevsw, both entries would point at the same disk_dev.

(2) Abolish ioctl inside the kernel, or at least within the device
tree. Given separate device classes, each operation needed can be made
its own operation on that device class (that is, a function pointer in
struct foo_dev) with the ensuing large increase in clarity about what
the operations are and where they need to be implemented. Plus this
way we get type safety for the arguments.

(3) Rearrange the way operations dispatch to devices. The traditional
model is that opening a device gives you a device vnode, and device
ops are dispatched to the device driver by looking up the major number
and indirecting through either the cdevsw or bdevsw table, and passing
the minor number as an argument. (This throws away all information
about how or when the device was opened, which is why cloners needed
to do something different.) The proposed method is that device vnodes
resolve the identity of the device when first loaded, and at runtime
point to the e.g. struct disk_dev rather than remembering the major
number; and for cloners they point to a device instance structure
created when the device is opened, which holds the per-instance data
the cloner needs. (It's not clear to me right now if only cloners
should get instance structures, or if for uniformity it makes sense to
allocate them for every driver, or if it should be a property of
certain device classes -- the overhead of a allocating a handful of
extra small structures isn't important, so it's mostly a code
complexity question.) Operations on devices go to the device vnode and
are then sent on to the driver directly; the cdevsw and bdevsw tables
are used only when devices are first looked up. All ioctls are turned
into explicit device-level operations in the device vnode's ioctl op.

(3a) Further down the line it might make sense to make devices _not_
vnodes but instead make them different instances of struct file
(either one for all devices or even one for each device class) -- this
would move ioctl dispatching up a layer, which would be an
improvement. But I don't think this needs to be part of the initial
plan. Plus 

(4) Make device vnodes fs-independent and rearrange how looking them
up works. Get rid of the extra ops table for devices that every fs has
to have (and also the one for fifos); instead, make fs-level special
file vnodes mostly-inert objects that don't support anything much
besides getattr 

Re: Is there a command to change btime (creation time of files)?

[Christos Zoulas]

> On May 29, 2021, at 12:22 PM, Martin Husemann  wrote:
> 
> On Sat, May 29, 2021 at 02:03:42PM -, Christos Zoulas wrote:
>> The baroque procedure is described in the manual page of utimes(2) where
>> one would expect it :-)
> 
> We should also add a command for it to fsdb(8).
> 
> Martin

Done :-)

christos


signature.asc
Description: Message signed with OpenPGP

Re: Is there a command to change btime (creation time of files)?

[Martin Husemann]

On Sat, May 29, 2021 at 02:03:42PM -, Christos Zoulas wrote:
> The baroque procedure is described in the manual page of utimes(2) where
> one would expect it :-)

We should also add a command for it to fsdb(8).

Martin

Re: Is there a command to change btime (creation time of files)?

[Christos Zoulas]

In article <2828.1622233...@jinx.noi.kre.to>,
Robert Elz   wrote:
>Date:Fri, 28 May 2021 16:49:26 +
>From:Kenny 
>Message-ID: 
>
>
>  | I am using NetBSD 9.2 (amd64) with ZFS as file system and I have
>  | not found a command to change btime for my files.
>
>Don't bother, the birth time is a total waste of space.  It is used
>by nothing that matters, and is useful for nothing at all.   It is
>mostly unsupported (in that the stat command is just about the only
>way to view it, and while it is possible to set it, but not arbitrarily,
>the procedure is baroque and not worth explaining).

The baroque procedure is described in the manual page of utimes(2) where
one would expect it :-)

christos