Hi again, On Mon, Mar 15, 2021 at 07:58:33PM +1100, William ML Leslie wrote: > On Mon, 15 Mar 2021 at 05:19, Olaf Buddenhagen <[email protected]> > wrote: > > On Thu, Feb 25, 2021 at 06:48:11PM +1100, William ML Leslie wrote:
> > > I am still hopeful that someone will figure out how to do async > > > right at some point. > It's somewhat easy to stay as close to io_uring as possible, having > ring buffers for messages ready to be sent and messages to be > received. It's quite unusual to treat generic async IPC as the same as async I/O... Though to be frank, it actually is the same in the IPC approach I'm envisioning :-) The issue with io_uring however is that it's designed for the monolithic system use case, where all I/O is handled by the kernel. In a microkernel environment, rather than having a shared memory protocol between userspace processes and the kernel, the obvious approach is to have these implemented directly between the clients and servers instead... Of course that doesn't help if we want a unified I/O queue for requests to multiple servers: however, it's not immediately obvious to me that implementing a mechanism for that in the kernel (instead of client-side handling) is indeed a good idea... > We're a bit spoiled in the capability space, though. Many of our > protocols include the ability to send a message before its target or > arguments become available, as well as the ability to only have a > message sent on receipt or failure of a previous message. [...] > The third solution is to add logic to the kernel to perform further > sends when a message is received, and complicating the kernel at all > is frankly a bit scary (and how should we time-bound that?). If we take that to the logical extreme, we get grafting, i.e. uploading arbitrary logic expressed in a safe language -- a concept that has been proposed in academia decades ago, but AFAIK hasn't made it into the mainsteam until recently, in the form of eBPF in Linux... Intuitively, it doesn't feel like accounting would be much different than for individually issued kernel requests?... Of course the question is whether it's really worthwhile. It would arguably provide more motivation for having I/O queue handling in the kernel: since in many cases, the grafted logic in the kernel could handle things without a roundtrip to the client, thus saving context switches. However, it's not immediately clear to me that the downsides from the indirection through the kernel wouldn't actually outweigh the savings... Another question is whether grafting using eBPF (or some other language) isn't actually powerful enough to implement custom queue handling, rather than needing a fixed mechanims for that in the kernel?... For my design, for a while I was thinking about implementing handling of more or less complex generic logic in IPC requests -- though in my case that would be handled by the server handling the initial request (and passing on to other servers as needed) rather than the kernel. However, I realised that the way I'm approaching IPC, it is actually quite natural to express the major use cases in the IPC protocol -- thus mostly eliminating the motivation for doing generic logic... > I plan to do a little more on a per-process basis. A few turns before > a process is scheduled, we make sure to page-in anything the process > is about to touch. A capability provides the means for a process to > say which addresses it will soon access, in addition to the > instruction pointer. That's an interesting idea... Not sure though whether there are actual use cases for that: in theory, it could reduce the latency for processes that are resumed after having been idle long enough to get paged out -- but only if they know in advance that they need to resume soon... I can't think of a situation where a process needs to react quickly to an event scheduled long in advance? > An aside: I absolutely want to have optional orthogonal persistence > per-application. Imagine having emacs up and ready to go the moment > you log into your machine. Yes please. How would that work per-application? Don't we have to restore the application's environment (including capabilities) -- meaning it has to be more or less system-wide?... Either way: yes, I totally want the ability to seamlessly resume any activities (that do NOT include Emacs in my case :-P ) after logouts, power cycles etc. Indeed I consider it among the two or three most important must-have features of my design. (Maybe *the* most important one? It's hard to rank them...) However, I don't intend to implement this with orthogonal persistence based on preserving the entire memory image of processes. This type of persistence mechanism is very tempting, because it feels like it provides very desirable functionality with very little effort. The problem is at the edges, where it doesn't help: things like upgrading software; migrating part of the environment to a different system instance; recovering from crashes involving memory corruption... Of course transparent orthogonal persistence doesn't *preclude* handling these situations: we just need to serialise all precious state to carry it forward explicitly... The thing is, once we have such a serialisation mechanism, why do we need the other persistence mechanism at all? Better make serialisation the sole persistence mechanism; ensuring it works really well, rather than just being a poorly maintained backup mechanism for special cases... (Shap will probably tell me that I got it all wrong or something: but the truth is that my conclusions on this matter haven't budged over the past 15 years -- and I can't imagine them budging over the next 15 :-) ) > > I don't see a fundamental difference... Whether it's ext2, SLS, or > > some sort of RAM disk: in each case you need a driver that can be > > loaded by the bootloader?... > > It's just a matter of complexity. The various pieces that implement > the SLS are less than 5000 lines, wheras libstore is over 7000 on its > own; libdiskfs 12000, and then libext2 on top of that. But yes, it's > somewhat like an initrd. Why would that matter, though? You aren't limited in the size of the image loaded by bootloader, are you?... -antrik-
