Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Robert Elz
Date:Fri, 10 Aug 2018 19:17:55 -0400 (EDT)
From:Mouse 
Message-ID:  <201808102317.taa24...@stone.rodents-montreal.org>

  | However, whether it means "nyah nyah, I'm not telling you this" or
  | "don't worry, this won't be on the exam"

Not either I would think - the former is not an attitude I'd expect,
(and all the code was there, so was available to everyone who
could see the comment anyway) and I cannot imagine what exam
anyone could possibly have ever been even contemplating, so 
that version makes little sense.

  | Actually, as I recall, the (rest of the) comment _did_ explain it, but
  | only briefly, and in a way that would be incomprehensible to those who
  | only partially understood the underlying mechanisms, so perhaps it
  | doesn't entirely count.

It did, though the rest of it was more of a "this happens" comment as I
recall, kind of like "I wave my wand and the rabbit appears" - you know
it is all true as you can see it happening, what you don't understand
(or are not expected to understand) is how that it is made to happen.
In both cases, with sufficient study and access to what is really
going on, it is possible to find out, but most people don't bother.

kre



Re: Finding an available fss device

2018-08-10 Thread Robert Elz
I doubt that your new proposed ioctl() is a very good
interface - to do what you really need you would require
the equivalent of a "test & set" otherwise all you are
doing is creating a race condition - even though it is,
because of the low number of users, one that is
unlikely to matter very often.

If the interface were useful, I see no point in taking and
releasing a mutex in order to read one bit - the bit is either
set or not, and is either stable set or not (if it is stable,
the mutex achieves nothing, if it is about to, or has just
changed, then that's just the race condition, the
ioctl would see the value either before, or after,
that change, the mutex doesn't help you know
which, so it doesn't really achieve anything).

If the mutex were useful, you'e used the wrong one,
FSS_ACTIVE is set/cleared under the control of
sc_lock not sc_slock (which is kind of strange as
other flags in the same word are controlled by sc_slock
which does not look like it would be reliable to me).

What I think I'd do is change the code for FSSIOCSET
to be something like

case FSSIOCSET:
error = 0;
if ((flag & FWRITE) == 0)
error = EPERM;
else if ((sc->sc_flags & FSS_ACTIVE) != 0)
error = EBUSY;
if (error == 0) {
mutex_enter(>sc_lock);
error = fss_create_snapshot(sc, fss, l);
if (error == 0)
sc->sc_uflags = fss->fss_flags;
mutex_exit(>sc_lock); 
}
break;

(apologies for any indentation screwups, it looks OK
as I am typing it, but that might not be what it looks like
to anyone else...)

With that you have more the test & set operation, which
should not block if the device is active already.

You could also re-order the two initial tests, so you could
do a read-only open, attempt this operation, which would
always fail then, but with errno==EBUSY if the device is
active, and errno==EPERM otherwise - which would provice
the (racy) just see if it is available operation.

But you need advice from someone who unserstands the locking
issues, and can see if they are used properly with this code
now, and what is really needed to get what you want - don't
just believe this because it looks right to me.

kre



Re: Finding an available fss device

2018-08-10 Thread Emmanuel Dreyfus
Emmanuel Dreyfus  wrote:

> Perhaps the right way is to add a FSSIOBUSY ioctl that would
> use mutex_tryenter and return EBUSY if the device is in use?

I propose the change below, so that we can find an available /dev/fss*
device without hanging:

--- sys/dev/fss.c.orig
+++ sys/dev/fss.c
@@ -427,8 +427,17 @@
mutex_exit(>sc_slock);
error = 0;
break;
 
+   case FSSIOBUSY:
+   if (mutex_tryenter(>sc_slock) == 0) {
+   error = EBUSY;
+   break;
+   }
+   error = (sc->sc_flags & FSS_ACTIVE) ? EBUSY : 0;
+   mutex_exit(>sc_slock);
+   break;
+
default:
error = EINVAL;
break;
}
--- sys/dev/fssvar.h.orig
+++ sys/dev/fssvar.h
@@ -56,8 +56,10 @@
 #define FSSIOCGET  _IOR('F', 1, struct fss_get)/* Status */
 #define FSSIOCCLR  _IO('F', 2) /* Unconfigure */
 #define FSSIOFSET  _IOW('F', 3, int)   /* Set flags */
 #define FSSIOFGET  _IOR('F', 4, int)   /* Get flags */
+#define FSSIOBUSY  _IO('F', 6) /* Is busy? */
+
 #ifdef _KERNEL
 #include 
 
 struct fss_set50 {


-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Mouse
>>   | Ancient BSD tradition is not to explain these things :-(
>> Older than that.   Don't you remember
>>  you are not expected to understand this
>> (or wording very similar) in ancient 4th/5th edition unix.
> The explanation for that comment I've read somewhere was that it
> really meant "it will not be in the exam" (which is a wonderful story
> even if it's not true :).

I've seen a similar explanation.  I am inclined to believe it.

However, whether it means "nyah nyah, I'm not telling you this" or
"don't worry, this won't be on the exam" (or whatever else), it still
is part of a tradition of not explaining things.

Actually, as I recall, the (rest of the) comment _did_ explain it, but
only briefly, and in a way that would be incomprehensible to those who
only partially understood the underlying mechanisms, so perhaps it
doesn't entirely count.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Edgar Fuß
> Are you sure it _only_ happens in the do/while and _never_ in the
> preceding if?
Well, the three times I looked at the backtrace carefully, yes (I have photos).
The next two or three times, I'm not completely sure.

> In any case, it's just a diagnostic, not a protocol for a robust
> software system to rely on.  If it doesn't work, can try another one.
OK, I'll give it a try. Will I need to declare the global variable volatile?


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Valery Ushakov
On Sat, Aug 11, 2018 at 00:46:26 +0700, Robert Elz wrote:

> Date:Fri, 10 Aug 2018 08:03:55 -0400
> From:Greg Troxel 
> Message-ID:  
> 
>   | Ancient BSD tradition is not to explain these things :-(
> 
> Older than that.   Don't you remember
>   you are not expected to understand this
> (or wording very similar) in ancient 4th/5th edition unix.

The explanation for that comment I've read somewhere was that it
really meant "it will not be in the exam" (which is a wonderful story
even if it's not true :).

-uwe


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Taylor R Campbell
> Date: Fri, 10 Aug 2018 19:48:40 +0200
> From: Edgar Fuß 
> 
> > Yes -- isn't that the symptom you're seeing, or did I miss something?
> It's the mutex_oncpu in the while condition that crashes, not the on in the
> if condition above the do.

Are you sure it _only_ happens in the do/while and _never_ in the
preceding if?

I don't see any reason to distinguish ordering on other CPUs between
the two calls to mutex_oncpu.  It is possible that the spinlock
backoff in the do/while opens a window for a race condition wide
enough that you only see it in the do/while condition.

> > It doesn't really matter since (a) only one thread ever sets the
> > variable, (b) there are no invariants around it, and (c) you never
> > dereference it.  So, as soon as unp_gc decides it will use a
> > particular socket, it should just store the pointer to that socket in
> > some global unp_gc_current_socket, and when it's done (before closef),
> > it should set unp_gc_current_socket to null; then in soput/sofree,
> > just KASSERT(so != unp_gc_current_socket).
> But couldn't the thread that KASSERTs read a stale copy that unp_gc() 
> nulled out but the null value didn't make it to the right CPU/cache/whatever?

Conceivably, yes.  Then you would have a false positive for your test.
I would guess that (a) that won't happen a lot, and (b) it'll be clear
on scrutiny that it's a false positive.

But I also don't think it's very likely to have false positives,
because in correct code, soclose won't be called with a socket
associated with a file that has a positive reference count, which is
all managed under fp->f_lock.

In any case, it's just a diagnostic, not a protocol for a robust
software system to rely on.  If it doesn't work, can try another one.


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Edgar Fuß
> Yes -- isn't that the symptom you're seeing, or did I miss something?
It's the mutex_oncpu in the while condition that crashes, not the on in the
if condition above the do.

> It doesn't really matter since (a) only one thread ever sets the
> variable, (b) there are no invariants around it, and (c) you never
> dereference it.  So, as soon as unp_gc decides it will use a
> particular socket, it should just store the pointer to that socket in
> some global unp_gc_current_socket, and when it's done (before closef),
> it should set unp_gc_current_socket to null; then in soput/sofree,
> just KASSERT(so != unp_gc_current_socket).
But couldn't the thread that KASSERTs read a stale copy that unp_gc() 
nulled out but the null value didn't make it to the right CPU/cache/whatever?


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Robert Elz
Date:Fri, 10 Aug 2018 08:03:55 -0400
From:Greg Troxel 
Message-ID:  

  | Ancient BSD tradition is not to explain these things :-(

Older than that.   Don't you remember
you are not expected to understand this
(or wording very similar) in ancient 4th/5th edition unix.

kre



Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Taylor R Campbell
> Date: Fri, 10 Aug 2018 19:05:28 +0200
> From: Edgar Fuß 
> 
> > More likely is that the socket is prematurely freed before unp_gc 
> > grabs it at all.
> But then the mutex_oncpu() in the if above the do...while would panic, no?

Yes -- isn't that the symptom you're seeing, or did I miss something?

> > You could do something like create a global variable that stores the
> > socket pointer that unp_gc is currently working on, shortly before it
> > tries solock, and kassert that soclose isn't given that.
> I thought about something like that but thought that was too simplistic.
> How do I synchronize the variable value amongst threads?

It doesn't really matter since (a) only one thread ever sets the
variable, (b) there are no invariants around it, and (c) you never
dereference it.  So, as soon as unp_gc decides it will use a
particular socket, it should just store the pointer to that socket in
some global unp_gc_current_socket, and when it's done (before closef),
it should set unp_gc_current_socket to null; then in soput/sofree,
just KASSERT(so != unp_gc_current_socket).


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Edgar Fuß
> More likely is that the socket is prematurely freed before unp_gc 
> grabs it at all.
But then the mutex_oncpu() in the if above the do...while would panic, no?

> You could do something like create a global variable that stores the
> socket pointer that unp_gc is currently working on, shortly before it
> tries solock, and kassert that soclose isn't given that.
I thought about something like that but thought that was too simplistic.
How do I synchronize the variable value amongst threads?


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Taylor R Campbell
> Date: Fri, 10 Aug 2018 17:51:11 +0200
> From: Edgar Fuß 
> 
> I'm currently running an 8.0_STABLE kernel on the machine (with 6.1_STABLE 
> userland) and no panics so far. This smay be
> -- luck
> -- different timing that doesn't trigger the race
> -- a bug fixed since 6.1
> 
> If someone remembers a bug in this area fixed since 6.1, we can stop here.

I don't remember a specific bug but that is entirely plausible.  There
have been some possibly relevant changes, e.g. uipc_usrreq.c 1.175.

> What I guess is that that -16L is MUTEX_THREAD and was put there by 
> MUTEX_DESTROY(), called by mutex_destroy() called by soput() by another 
> thread that ran during the preemtion-enabled phase.
> 
> Any other ideas on how that -16L could go there?

This sounds plausible.

> Could I install some hack, that, in soput(), would panic if the socket to be
> freed is the one unp_gc() is currently working on? If that would trigger, 
> we'd 
> get a useful traceback, no? And if that panic doesn't trigger, but the other 
> one does, we'd know that some of my assumptions were wrong.

They are unlikely to coincide like that.  More likely is that the
socket is prematurely freed before unp_gc grabs it at all.

You could do something like create a global variable that stores the
socket pointer that unp_gc is currently working on, shortly before it
tries solock, and kassert that soclose isn't given that.  But I'm not
optimistic that there's enough of a window there for you to catch
anyone in the act.


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Edgar Fuß
Thank you both for the detailed explanations and audits!

I'm currently running an 8.0_STABLE kernel on the machine (with 6.1_STABLE 
userland) and no panics so far. This smay be
-- luck
-- different timing that doesn't trigger the race
-- a bug fixed since 6.1

If someone remembers a bug in this area fixed since 6.1, we can stop here.


> if indeed the problem is a use-after-free arising from 
> soclose/sofree/soput that happened too early.
What I'm sure about is that the first solock() in unp_gc() calls mutex_enter() 
and the preemtion-enabling loop in mutex_vector_enter evaluates mutex_oncpu() 
(the one in the while condition) on an owner value of -16L.

What I guess is that that -16L is MUTEX_THREAD and was put there by 
MUTEX_DESTROY(), called by mutex_destroy() called by soput() by another 
thread that ran during the preemtion-enabled phase.

Any other ideas on how that -16L could go there?

Could I install some hack, that, in soput(), would panic if the socket to be
freed is the one unp_gc() is currently working on? If that would trigger, we'd 
get a useful traceback, no? And if that panic doesn't trigger, but the other 
one does, we'd know that some of my assumptions were wrong.


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Taylor R Campbell
> Date: Fri, 10 Aug 2018 08:03:55 -0400
> From: Greg Troxel 
> 
> Edgar Fuß  writes:
> 
> > Is there a general scheme/rule/proposal how to prevent a mutex that someone 
> > is trying to mutex_enter() from being teared down by someone else calling 
> > mutex_destroy() on it during that?
> 
> Not really.  Basically it's a bug to destroy a mutex that could possibly
> be in use.  So there has to be something else protecting getting at the
> mutex that is being destroyed.

Correct.

> > Specifically, I'm trying to understand what should prevent a thread from 
> > destroying a socket's lock (by sofree()/soput()) while unp_gc() is trying 
> > to acquire that lock.

The unp_gc thread assumes that holding a reference to the associated
struct file, by incrementing f_count, prevents the socket from being
freed.  Certainly that prevents closef from calling fo_close, which
for a socket is soo_close, which calls soclose, which calls sofree,
which calls soput.

There are other calls to soclose:

- kern/subr_tftproot.c, tftproot_getfile
  => only for in-kernel use, not assocated with files
- kern/uipc_syscalls.c, makesocket
  => but only before the socket has been associated with a file
- kern/uipc_syscalls.c, sys_socketpair
  => looks wrong to me but unlikely to trigger
  => maybe fd_abort should wait for unp_gc
  => or maybe this should call closef, not soclose
- kern/uipc_syscalls.c, pipe1 (if PIPE_SOCKETPAIR is enabled)
  => same as sys_socketpair

The remaining calls to soclose are internal to various file systems or
rump and I don't think the sockets they work on have been assocated
with anyone's struct file:

- miscfs/fifofs/fifo_vnops.c
- netsmb/smb_trantcp.c
- nfs/krpc_subr.c
- nfs/nfs_boot.c
- nfs/nfs_bootdhcp.c
- nfs/nfs_socket.c
- rump/net/lib/libnetinet/netinet_component.c
- rump/net/lib/libsockin/sockin.c

There are also other calls to sofree:

- kern/uipc_socket.c, socreate error branch
  => not relevant, hasn't been associated with a file yet
- kern/uipc_socket.c, soabort
  => only for sockets that haven't been associated with files yet

The remaining calls to sofree are in network protocols, mostly in the
detach routines.  When called via ->pr_detach, the socket in question
is not associated with a file.  But there are other callers and I've
run out of steam to audit all of the other callers of these functions
to confirm that they apply only to sockets not associated with files,
or to find where they don't.

- kern/uipc_usrreq.c, unp_drop
- net/raw_cb.c, raw_detach
  => called by raw->pr_detach, route->pr_detach, key->pr_detach
  => called by raw_disconnect only if SS_NOFDREF is set, meaning
 there is no associated file
- netatalk/ddp_usrreq.c, ddp_detach
  => called by atalk->pr_detach and atalk->pr_abort, no associated file
- netcan/can_pcb.c, can_pcbdetach
  => called by can->pr_detach
  => called by can_pcbdisconnect only if SS_NOFDREF is set
- netinet/in_pcb.c, in_pcbdetach
  => called by dccp_close
 ---> oops, we have a callout_stop/halt bug here
 ---> out of steam to study further
  => called by in_pcbdisconnect only if SS_NOFDREF is set
  => called by rip->pr_detach
  => called by tcp_close
 ---> out of steam to study further
  => called by tcp->pr_attach
  => called by udp->pr_detach
- netinet/sctp_pcb.c, sctp_incpb_free
  => out of steam to study further
- netinet6/in6_pcb.c, in6_pcbdetach
  => out of steam to study further, looks the same as in_pcbdetach
- netnatm/natm.c, natm_detach
  => called by natm->pr_detach

There's also one other call to soput:

- sonewconn
  => only for sockets that haven't been assocated with files yet

More eyeballs on these code paths might find a problem, if indeed the
problem is a use-after-free arising from soclose/sofree/soput that
happened too early.


Re: To test presence of CVE-2018-6922 ( TCP vulnerability) in NetBSD5.1

2018-08-10 Thread Ripunjay Tripathi
Thanks for the link.

On Fri, Aug 10, 2018 at 3:19 PM Maxime Villard  wrote:

> Le 10/08/2018 à 11:18, Ripunjay Tripathi a écrit :
> > I am trying to test presence of CVE-2018-6922 [...]
>
> NetBSD 5 is not supported anymore, and NetBSD 6 is about to reach EOL. So
> there is no way this is ever going to be fixed in NetBSD 5.
>
> I know that. I am interested in understanding if someone has already
tested the presence OR could help me in my attempts to reproduce this.
I also need to fix this therefore wanted to be sure if my understanding of
code tcp_input() is correct.

> There was a small conversation about the issue yesterday, in case you're
> interested:
>
> http://mail-index.netbsd.org/tech-net/2018/08/09/msg007004.html
>
> Maxime
>


Finding an available fss device

2018-08-10 Thread Emmanuel Dreyfus
Hello

How are user processes supposed to find an unused fss device?
In dump(8) code,  there is an iteration on /dev/rfss* trying to
performan an ioctl FSSIOCSET. The code tests for EBUSY on failure,
but in my experience that struggles to happen: if the device is 
already in use, the ioctl will sleep in the kernel for ages before 
getting a reply.

This is something I can even experience with fssconfig -l, which 
hangs for a while if dump is running.

Is there another way? I thought about searching vnode in kernel to 
check if the device is already used by someone else, but that looks 
overkill. 

Perhaps the right way is to add a FSSIOBUSY ioctl that would
use mutex_tryenter and return EBUSY if the device is in use?

-- 
Emmanuel Dreyfus
m...@netbsd.org


Re: How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Greg Troxel

Edgar Fuß  writes:

> I know very little about locking so that's probably a stupid question, but:

Don't worry - locking is probably the hardest thing to get right.

> Is there a general scheme/rule/proposal how to prevent a mutex that someone 
> is trying to mutex_enter() from being teared down by someone else calling 
> mutex_destroy() on it during that?

Not really.  Basically it's a bug to destroy a mutex that could possibly
be in use.  So there has to be something else protecting getting at the
mutex that is being destroyed.

> Specifically, I'm trying to understand what should prevent a thread from 
> destroying a socket's lock (by sofree()/soput()) while unp_gc() is trying 
> to acquire that lock.

I would expect (without reading the code) that there would be some lock
on the socket structure (using list here; the type is not the point),
and there would be a

  acquire socket_list lock
  find socket
  lock socket
  unlock sockt_list

or alternatively

  acquire socket_list lock
  find socket
  unlink socket from the list
  unlock sockt_list

  do whatever to the socket

So there has to be a rule about when various things are valid based on
being in various higher-level data structures.  In an ideal world this
rule would be clearly explained in the source code.  Ancient BSD
tradition is not to explain these things :-(


signature.asc
Description: PGP signature


Re: Too many PMC implementations

2018-08-10 Thread Maxime Villard

I saw the thread [Re: Sample based profiling] on tech-userlevel@, I'm not
subscribed to this list but I'm answering here because it's related to
tprof among other things.

I agree that it would be better to retire gprof in base, because there are
more powerful tools now, and also advanced hardware support (PMC, PEBS,
ProcessorTrace).

But in particular, it would be nice to retire the "kernel gprof". That is,
the MD/MI pieces that are surrounded by #ifdef GPROF. This kind of
profiling is weak, and misses many aspects of execution (branch prediction,
cache misses, heavy instructions, etc) that are offered by tprof.

I already dropped NENTRY() from x86, so KGPROF is officially not supported
there anymore. I think it has never worked on amd64.


Re: To test presence of CVE-2018-6922 ( TCP vulnerability) in NetBSD5.1

2018-08-10 Thread Mouse
>> I am trying to test presence of CVE-2018-6922 [...]
> NetBSD 5 is not supported anymore, and NetBSD 6 is about to reach
> EOL.  So there is no way this is ever going to be fixed in NetBSD 5.

That's a bit of an overstatement.  Not fixed _by NetBSD_, perhaps, but
there are at least a few people still using and, to some extent,
maintaining EOLed NetBSD.  I, for example, still run and evolve 5.2,
among others.

> There was a small conversation about the issue yesterday, in case
> you're interested: [...]

But NetBSD is vulnerable if the threat model includes malicious
attacks, even if it is resistant against pathological behaviour
provoked by random fragment loss.  (For that matter, it's not clear
from the reply whether the statement applies to all NetBSD or only
recent NetBSD - though code inspection makes it appear it's true of
1.4T and 5.2 and presumably everything in between.)

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B


Re: To test presence of CVE-2018-6922 ( TCP vulnerability) in NetBSD5.1

2018-08-10 Thread Maxime Villard

Le 10/08/2018 à 11:18, Ripunjay Tripathi a écrit :

I am trying to test presence of CVE-2018-6922 [...]


NetBSD 5 is not supported anymore, and NetBSD 6 is about to reach EOL. So
there is no way this is ever going to be fixed in NetBSD 5.

There was a small conversation about the issue yesterday, in case you're
interested:

http://mail-index.netbsd.org/tech-net/2018/08/09/msg007004.html

Maxime


Re: ddb input via IPMI virtual console

2018-08-10 Thread Edgar Fuß
But what do I gain if the real keyboard and the internal-USB keyboard emulated 
by IPMI's console redirection are on different USB ports?


Re: ddb input via IPMI virtual console

2018-08-10 Thread Edgar Fuß


> ddb should be able to talk to the console keyboard (via polling)
What exactly does that mean in case the console keyboard is wscins?


Re: ddb input via IPMI virtual console

2018-08-10 Thread Edgar Fuß


>  Sorry, my description wasn't clear. 
I did understand it.
The drawback is you don't see the kernel messages (including the fact the 
machne paniced) on the VGA monitor. You would need to use IPMI.


How to prevent a mutex being _enter()ed from being _destroy()ed?

2018-08-10 Thread Edgar Fuß
I know very little about locking so that's probably a stupid question, but:

Is there a general scheme/rule/proposal how to prevent a mutex that someone 
is trying to mutex_enter() from being teared down by someone else calling 
mutex_destroy() on it during that?

Specifically, I'm trying to understand what should prevent a thread from 
destroying a socket's lock (by sofree()/soput()) while unp_gc() is trying 
to acquire that lock.


re: 8.0 performance issue when running build.sh?

2018-08-10 Thread matthew green
> > >> 100.002054 14.18 kernel_lock
> > >>  47.43 846  6.72 kernel_lockfileassoc_file_delete+20
> > >>  23.73 188  3.36 kernel_lockintr_biglock_wrapper+16
> > >>  16.01 203  2.27 kernel_lockscsipi_adapter_request+63

also note that this is 14.18ms of locked time across 26 minutes.
this isn't a contention point.


.mrg.


Re: panic: biodone2 already

2018-08-10 Thread Michael van Elst
m...@netbsd.org (Emmanuel Dreyfus) writes:

>I can tell that in vfs_bio.c, bread() -> bio_doread() will call
>VOP_STRATEGY once for the offendinf buf_t, but biodone() is called twice
>in interrupt context for the buf_t, leading to the biodone2 already
>panic later.

>Since you know the xbd code you could save me some time: where do we go
>below VOP_SRATEGY? 

The buffer is passed to xbdstrategy that tries to run the I/O queue.
-> on error it finishes with biodone
regulary it calls dk_strategy that iterates over xbd_diskstart.
-> on error from xbd_diskstart it again finishes with biodone

xbd_diskstart tries to push a single buffer, it either
- queues the buffer to the "hardware" (a hypervisor event) or
- returns EAGAIN to retry the operation or
- returns another error to fail the operation

a queued operation eventually returns with a call to xbd_handler.
- for every buffer returned, dk_done is called which finally ends
  in invoking biodone.

hypervisor events link to a 'xbd_req' structure (and per request,
which can be I/O but also a cache flush operation). For I/O requests
the 'xbd_req' structure points to the buffer.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."


Re: 8.0 performance issue when running build.sh?

2018-08-10 Thread Jason Thorpe


> On Aug 9, 2018, at 10:40 AM, Thor Lancelot Simon  wrote:
> 
> Actually, I wonder if we could kill off the time spent by fileassoc.  Is
> it still used only by veriexec?  We can easily option that out of the build
> box kernels.

Indeed.  And there are better ways to do what veriexec does, in any case.

-- thorpej



Re: 8.0 performance issue when running build.sh?

2018-08-10 Thread Brett Lymn
 
 I would be interested to finish that off, I need to make some time to
get to doing it though.
I have been sitting on some changes to veriexec for ~ years that
change it from locking everything to using reference counts and
condition variables which removes some nasty hacks I did.  I have not
committed the changes because the kernel would sometimes deadlock and
I was having trouble tracking down why.  Perhaps I was looking in the
wrong spot for the error and it was fileassoc all along that was
causing the deadlock.

- Original Message -
From: "Mindaugas Rasiukevicius" 
To:"Jason Thorpe" 
Cc:
Sent:Fri, 10 Aug 2018 00:12:23 +0100
Subject:Re: 8.0 performance issue when running build.sh?

 Jason Thorpe  wrote:
 > 
 > 
 > > On Aug 9, 2018, at 10:40 AM, Thor Lancelot Simon  wrote:
 > > 
 > > Actually, I wonder if we could kill off the time spent by
fileassoc. Is
 > > it still used only by veriexec? We can easily option that out of
the
 > > build box kernels.
 > 
 > Indeed. And there are better ways to do what veriexec does, in any
case.
 > 

 Many years ago I wrote a diff to make fileassoc MP-safe:

 http://www.netbsd.org/~rmind/fileassoc.diff

 If somebody wants to finish -- I am glad to help.

 -- 
 Mindaugas




Re: verbose vflushbuf()

2018-08-10 Thread Michael van Elst
dholland-t...@netbsd.org (David Holland) writes:

>Probably, but I don't think it's supposed to happen and possibly it
>should be a panic:

It can regularly happen under load and the retry is supposed to handle that
condition. Still, it shouldn't occur frequently, so in this case there
is a problem.


-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."