Re: kqueue: SIGIO?

[David Holland]

On Wed, Sep 30, 2015 at 11:28:35PM +0700, Robert Elz wrote:
 > It also fits with the only safe thing that's really possible to do in a
 > single handler being to set a variable and return (or exit the process)
 > (ie: the main loop has to check a variable anyway, whether signal delivery
 > is traditional, or via Mouse's suggested mechanism).
 > 
 > The issue with it is how one would ever safely clear the variable again,
 > while avoiding race conditions - 

An atomic read-modify-write cycle; other than vintage cpus have these
in hardware.

Anyhow, infiniband had to come up with some scheme for this; borrowing
that rather than reinventing it (or not borrowing it if it sucks,
instead of reinventing it by accident) seems like a good starting
point.

-- 
David A. Holland
dholl...@netbsd.org

Re: kqueue: SIGIO?

[Mouse]

> On the other hand, if kernel changes would be needed (for example to
> make SIGIO work with kqueue() on NetBSD) then we really should
> evaluate whether or not there is a better change that could be made
> to handle the situation, rather than just blindly making NetBSD the
> same as linux.   What that might be though I have no idea.

The first thing that comes to mind is a syscall that tells the kernel
to deliver signals, or at least certain signals, by changing a memory
location rather than arranging to execute code.  (I have trouble
imagining an architecture on which checking a volatile int variable is
more expensive than a syscall into the kernel.)

It is true, though, that that's more-than-zero cost in the loop.  But
it might be close enough to zero to be acceptable.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: kqueue: SIGIO?

[Thor Lancelot Simon]

On Wed, Sep 30, 2015 at 12:30:36AM +0200, Joerg Sonnenberger wrote:
> On Tue, Sep 29, 2015 at 10:09:51PM +0200, Rhialto wrote:
> > On Tue 29 Sep 2015 at 13:22:08 +0200, Tobias Nygren wrote:
> > > Here is the relevant bit of the talk if you are curious:
> > > 
> > > https://www.youtube.com/watch?v=t400SmZlnO8=youtu.be=1888
> > 
> > So he wants a signal when a message is available in a kqueue, in other
> > words, can be read with kevent(2).
> 
> Why oh why. I thought the X server finally got rid of the
> overcomplicated signal handlers. If there is any kind of load going on,
> the signal sending is more costly than occassional querying the kqueue
> for (other active) entries. If there is no load, it doesn't make a
> difference.

What he said.  I've owned some fairly performance-critical single threaded
event driven code in my time, and it is my opinion that trying to use
signals to achieve client fairness is almost always a huge mistake.

The trick is almost always to structure the event loop so that checking
for work from another client is nearly zero-cost, and scales much less
than linearly with the number of clients.  Given the very high cost of
handling a signal, it is pretty darned hard to do worse.

The way to get yourself in trouble is to chase false "optimizations"
involving processing-to-completion of too much work from a single client
at once, or shortcuts involving handing off data directly from one
client to another.  The latter, at least, are really classic priority
inversion bugs in disguise.

In practice, the X server has a shared memory transport to most clients
and a shared memory interface to the display hardware; it should seldom
have syscalls to do.  Arranging for nearly zero-cost "look aside" at
some other *properly designed and structured* shared memory source of
client requests should be pretty easy.  Does the problem actually have
to do with the mouse and keyboard?  Mouse's idea of having the kernel
write a flag word instead of interrupting the process seems like a 
very nice fit if so.

-- 
  Thor Lancelot Simont...@panix.com

  "We cannot usually in social life pursue a single value or a single moral
   aim, untroubled by the need to compromise with others."  - H.L.A. Hart

Re: kqueue: SIGIO?

[Robert Elz]

Date:Wed, 30 Sep 2015 09:45:32 -0400
From:Thor Lancelot Simon 
Message-ID:  <20150930134532.ga25...@panix.com>

  | Does the problem actually have to do with the mouse and keyboard?

The server also needs to deal with (potential) network connections from
clients - most people these days might only run clients on the same system
as the server, and so can use shared mem, but not everyone is so limited
(I know I run across-net connections, even if it is just from a xen DomU
client to the X server running on the Dom0 - but I also do real over
ethernet/wireless X connections too on occasion).   Those connections
will never be the high performance kind, but nor should they be starved
by some other local high performance shared-mem using local client.

  | Mouse's idea of having the kernel
  | write a flag word instead of interrupting the process seems like a 
  | very nice fit if so.

It also fits with the only safe thing that's really possible to do in a
single handler being to set a variable and return (or exit the process)
(ie: the main loop has to check a variable anyway, whether signal delivery
is traditional, or via Mouse's suggested mechanism).

The issue with it is how one would ever safely clear the variable again,
while avoiding race conditions - when a signal handler sets the variable,
it is all user code, and can use locking to be safe, one cannot lock out the
kernel though.   But maybe, given this is supposed to be rare, a sys call
to clear the var, after detecting it set, would be acceptable - or just
switch to a different var for subsequent notifications using the original
sys call, after which the first one is just a variable again, and can be
cleared normally (though that would require an indirect reference to
check it, and so greater cost for that.)

kre

Re: kqueue: SIGIO?

[Joerg Sonnenberger]

On Wed, Sep 30, 2015 at 07:37:10AM -0400, Mouse wrote:
> > On the other hand, if kernel changes would be needed (for example to
> > make SIGIO work with kqueue() on NetBSD) then we really should
> > evaluate whether or not there is a better change that could be made
> > to handle the situation, rather than just blindly making NetBSD the
> > same as linux.   What that might be though I have no idea.
> 
> The first thing that comes to mind is a syscall that tells the kernel
> to deliver signals, or at least certain signals, by changing a memory
> location rather than arranging to execute code.  (I have trouble
> imagining an architecture on which checking a volatile int variable is
> more expensive than a syscall into the kernel.)

Well, you can easily get that by just running a second thread that does
nothing but monitor the kqueue and deliver notification to the main
thread. That's a pretty standard design and all the OpenGL likely has
put at least one other thread into the X servere anyway.

Joerg

Re: kqueue: SIGIO?

[Mouse]

>> Mouse's idea of having the kernel write a flag word instead of
>> interrupting the process seems like a very nice fit if so.
> The issue with it is how one would ever safely clear the variable
> again, [...]

This is not difficult: you do it by not clearing the variable.

For the sake of argument and brevity, let us suppose a suitable type
for the variable in question is unsigned int.  Then the kernel, instead
of _setting_ the variable, can _increment_ the variable, and userland
can do something like

volatile unsigned int sigflag;
unsigned int chksigflag;
unsigned int lastsigflag;

sigflag = 0;
lastsigflag = 0;
handle_via_flag_variable(SIGIO,); // flag to sigaction()?
while (1) { // main loop
...
chksigflag = sigflag;
if (chksigflag != lastsigflag) {
lastsigflag = chksigflag;
...handle the signal...
}
...
}

Obviously, there is still a potential issue; if the kernel delivers
exactly k*2^32 (for integer k, assuming 32-bit unsigned ints) signals,
between userland checks, userland will miss them.  I don't consider
this a big enough risk to worry about; if it really bothers you, make
it long long int instead - there is some risk of value tearing in the
read on many architectures, but, since the kernel's increment is atomic
with respect to userland, the worst it will do is delay noticing the
signal by one trip around the loop.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

kqueue: SIGIO?

[Thomas Klausner]

Hi!

During the XDC last week, Keith Packard talked about a select(2) issue
in xserver he would like to fix with epoll and its support for SIGIO.
Is there a similar feature in kqueue in NetBSD?

 Thomas

Re: kqueue: SIGIO?

[Tobias Nygren]

On Tue, 29 Sep 2015 12:56:36 +0200
Joerg Sonnenberger  wrote:

> On Tue, Sep 29, 2015 at 11:31:20AM +0200, Thomas Klausner wrote:

> > During the XDC last week, Keith Packard talked about a select(2) issue
> > in xserver he would like to fix with epoll and its support for SIGIO.
> > Is there a similar feature in kqueue in NetBSD?
> 
> It would be easier to answer that question if we know what issue he was
> actually talking about.

>From my (limited) understanding it is about preventing starvation of
other X clients when one client is hammering the server with requests.

When they are processing events for one client in a tight loop they
don't want to periodically check if other clients are ready for I/O,
but would instead like to get notified via a signal.

Here is the relevant bit of the talk if you are curious:

https://www.youtube.com/watch?v=t400SmZlnO8=youtu.be=1888

Re: kqueue: SIGIO?

[Joerg Sonnenberger]

On Tue, Sep 29, 2015 at 11:31:20AM +0200, Thomas Klausner wrote:
> Hi!
> 
> During the XDC last week, Keith Packard talked about a select(2) issue
> in xserver he would like to fix with epoll and its support for SIGIO.
> Is there a similar feature in kqueue in NetBSD?

It would be easier to answer that question if we know what issue he was
actually talking about.

Joerg

Re: kqueue: SIGIO?

[Joerg Sonnenberger]

On Tue, Sep 29, 2015 at 10:09:51PM +0200, Rhialto wrote:
> On Tue 29 Sep 2015 at 13:22:08 +0200, Tobias Nygren wrote:
> > Here is the relevant bit of the talk if you are curious:
> > 
> > https://www.youtube.com/watch?v=t400SmZlnO8=youtu.be=1888
> 
> So he wants a signal when a message is available in a kqueue, in other
> words, can be read with kevent(2).

Why oh why. I thought the X server finally got rid of the
overcomplicated signal handlers. If there is any kind of load going on,
the signal sending is more costly than occassional querying the kqueue
for (other active) entries. If there is no load, it doesn't make a
difference.

Joerg

Re: kqueue: SIGIO?

[Rhialto]

On Tue 29 Sep 2015 at 13:22:08 +0200, Tobias Nygren wrote:
> Here is the relevant bit of the talk if you are curious:
> 
> https://www.youtube.com/watch?v=t400SmZlnO8=youtu.be=1888

So he wants a signal when a message is available in a kqueue, in other
words, can be read with kevent(2).

I do notice there are message queues (mq(3)) which are file descriptors
and have such functionality. Does that mean that mq_notify(3) might work
on any file descriptor, or is that too optimistic in hoping that these
things are implemented orthogonally?

-Olaf.
-- 
___ Olaf 'Rhialto' Seibert  -- The Doctor: No, 'eureka' is Greek for
\X/ rhialto/at/xs4all.nl-- 'this bath is too hot.'


pgpsL3oyNyHuS.pgp
Description: PGP signature