Re: CVS commit: src/lib/libpthread

2020-06-09 Thread Andrew Doran
On Thu, Jun 04, 2020 at 06:26:07PM +, Martin Husemann wrote:

> On Wed, Jun 03, 2020 at 10:10:24PM +0000, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Wed Jun  3 22:10:24 UTC 2020
> > 
> > Modified Files:
> > src/lib/libpthread: pthread.c pthread_cond.c pthread_mutex.c
> > 
> > Log Message:
> > Deal with a couple of problems with threads being awoken early due to
> > timeouts or cancellation where:
> 
> Not sure if it is caused by this commit or joergs TSD/malloc, but today
> most of the libpthread tests time out on aarch64, while everything but
> a few minor nits was fine on May 30.

I tried with everything updated and it seems to be hanging on exit in
jemalloc.  Haven't looked further yet.

http://www.netbsd.org/~ad/2020/barrier.txt

Andrew


 
> Martin
> 
> (this is on a hummingboard pulse 4-core board)
> lib/libpthread/t_barrier (398/848): 1 test cases
> barrier: [300.034098s] Failed: Test case timed out after 300 seconds
> [300.034568s]
> 
> lib/libpthread/t_cond (399/848): 9 test cases
> bogus_timedwaits: [300.016677s] Failed: Test case timed out after 300 
> seconds
> broadcast: [300.023040s] Failed: Test case timed out after 300 seconds
> cond_timedwait_race: [300.034567s] Failed: Test case timed out after 300 
> seconds
> condattr: [0.013756s] Passed.
> destroy_after_cancel: [300.025649s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock: [300.023630s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock_static_init: [300.022909s] Failed: Test case timed 
> out after 300 seconds
> signal_delay_wait: [300.031310s] Failed: Test case timed out after 300 
> seconds
> signal_wait_race: [300.025552s] Failed: Test case timed out after 300 
> seconds
> [2400.221956s]
> 
> lib/libpthread/t_condwait (400/848): 2 test cases
> cond_wait_mono: [2.023083s] Passed.
> cond_wait_real: [2.017972s] Passed.
> [4.042078s]
> 
> lib/libpthread/t_detach (401/848): 1 test cases
> pthread_detach: [300.021242s] Failed: Test case timed out after 300 
> seconds
> [300.021708s]
> 
> lib/libpthread/t_equal (402/848): 1 test cases
> pthread_equal: [300.027646s] Failed: Test case timed out after 300 seconds
> [300.028295s]
> 
> lib/libpthread/t_fork (403/848): 1 test cases
> fork: [300.018077s] Failed: Test case timed out after 300 seconds
> [300.018975s]
> 
> lib/libpthread/t_fpu (404/848): 1 test cases
> fpu: [0.020578s] Passed.
> [0.021314s]
> 
> lib/libpthread/t_join (405/848): 1 test cases
> pthread_join: [0.021632s] Passed.
> [0.022411s]
> 
> lib/libpthread/t_kill (406/848): 1 test cases
> simple: [300.033759s] Failed: Test case timed out after 300 seconds
> [300.034528s]
> 
> lib/libpthread/t_mutex (407/848): 7 test cases
> mutex1: [300.017542s] Failed: Test case timed out after 300 seconds
> mutex2: [300.023064s] Failed: Test case timed out after 300 seconds
> mutex3: [300.023259s] Failed: Test case timed out after 300 seconds
> mutex4: [300.023015s] Failed: Test case timed out after 300 seconds
> mutex5: [300.023495s] Failed: Test case timed out after 300 seconds
> mutexattr1: [0.013769s] Passed.
> mutexattr2: [0.025483s] Passed.
> [1500.154160s]
> 
> lib/libpthread/t_name (408/848): 1 test cases
> name: [0.014262s] Passed.
> [0.014951s]
> 
> ...


Re: CVS commit: src/tests/lib/libc/sys

2020-06-06 Thread Andrew Doran
On Sat, Jun 06, 2020 at 06:11:21PM +, Jason R Thorpe wrote:

> Module Name:  src
> Committed By: thorpej
> Date: Sat Jun  6 18:11:21 UTC 2020
> 
> Modified Files:
>   src/tests/lib/libc/sys: t_lwp_create.c
> 
> Log Message:
> Add a test case to ensure that _lwp_create() fails with the
> expected error code when a bad new-lwp-id pointer is passed.

Was thinking of just this yesterday - thanks!  It's a fragile bit of code
and has been fixed 3x already this year.

Andrew


Re: CVS commit: src/lib/libpthread

2020-06-06 Thread Andrew Doran
On Thu, Jun 04, 2020 at 09:28:09PM +, Andrew Doran wrote:

> On Thu, Jun 04, 2020 at 06:26:07PM +, Martin Husemann wrote:
> 
> > On Wed, Jun 03, 2020 at 10:10:24PM +0000, Andrew Doran wrote:
> > > Module Name:  src
> > > Committed By: ad
> > > Date: Wed Jun  3 22:10:24 UTC 2020
> > > 
> > > Modified Files:
> > >   src/lib/libpthread: pthread.c pthread_cond.c pthread_mutex.c
> > > 
> > > Log Message:
> > > Deal with a couple of problems with threads being awoken early due to
> > > timeouts or cancellation where:
> > 
> > Not sure if it is caused by this commit or joergs TSD/malloc, but today
> > most of the libpthread tests time out on aarch64, while everything but
> > a few minor nits was fine on May 30.
> 
> I'll try to reproduce it over the weekend.

This works OK for me on aarch64 with libpthread updated in isolation.  I'll
try updating more stuff tomorrow.

Andrew

> 
> Andrew
>  
> > Martin
> > 
> > (this is on a hummingboard pulse 4-core board)
> > lib/libpthread/t_barrier (398/848): 1 test cases
> > barrier: [300.034098s] Failed: Test case timed out after 300 seconds
> > [300.034568s]
> > 
> > lib/libpthread/t_cond (399/848): 9 test cases
> > bogus_timedwaits: [300.016677s] Failed: Test case timed out after 300 
> > seconds
> > broadcast: [300.023040s] Failed: Test case timed out after 300 seconds
> > cond_timedwait_race: [300.034567s] Failed: Test case timed out after 
> > 300 seconds
> > condattr: [0.013756s] Passed.
> > destroy_after_cancel: [300.025649s] Failed: Test case timed out after 
> > 300 seconds
> > signal_before_unlock: [300.023630s] Failed: Test case timed out after 
> > 300 seconds
> > signal_before_unlock_static_init: [300.022909s] Failed: Test case timed 
> > out after 300 seconds
> > signal_delay_wait: [300.031310s] Failed: Test case timed out after 300 
> > seconds
> > signal_wait_race: [300.025552s] Failed: Test case timed out after 300 
> > seconds
> > [2400.221956s]
> > 
> > lib/libpthread/t_condwait (400/848): 2 test cases
> > cond_wait_mono: [2.023083s] Passed.
> > cond_wait_real: [2.017972s] Passed.
> > [4.042078s]
> > 
> > lib/libpthread/t_detach (401/848): 1 test cases
> > pthread_detach: [300.021242s] Failed: Test case timed out after 300 
> > seconds
> > [300.021708s]
> > 
> > lib/libpthread/t_equal (402/848): 1 test cases
> > pthread_equal: [300.027646s] Failed: Test case timed out after 300 
> > seconds
> > [300.028295s]
> > 
> > lib/libpthread/t_fork (403/848): 1 test cases
> > fork: [300.018077s] Failed: Test case timed out after 300 seconds
> > [300.018975s]
> > 
> > lib/libpthread/t_fpu (404/848): 1 test cases
> > fpu: [0.020578s] Passed.
> > [0.021314s]
> > 
> > lib/libpthread/t_join (405/848): 1 test cases
> > pthread_join: [0.021632s] Passed.
> > [0.022411s]
> > 
> > lib/libpthread/t_kill (406/848): 1 test cases
> > simple: [300.033759s] Failed: Test case timed out after 300 seconds
> > [300.034528s]
> > 
> > lib/libpthread/t_mutex (407/848): 7 test cases
> > mutex1: [300.017542s] Failed: Test case timed out after 300 seconds
> > mutex2: [300.023064s] Failed: Test case timed out after 300 seconds
> > mutex3: [300.023259s] Failed: Test case timed out after 300 seconds
> > mutex4: [300.023015s] Failed: Test case timed out after 300 seconds
> > mutex5: [300.023495s] Failed: Test case timed out after 300 seconds
> > mutexattr1: [0.013769s] Passed.
> > mutexattr2: [0.025483s] Passed.
> > [1500.154160s]
> > 
> > lib/libpthread/t_name (408/848): 1 test cases
> > name: [0.014262s] Passed.
> > [0.014951s]
> > 
> > ...


Re: [stos, again] Re: CVS commit: src/sys/arch/amd64

2020-06-04 Thread Andrew Doran
On Thu, Jun 04, 2020 at 02:35:17AM +0200, Kamil Rytarowski wrote:

> On 04.06.2020 00:42, Andrew Doran wrote:
> > On Wed, Jun 03, 2020 at 02:03:22AM +0200, Kamil Rytarowski wrote:
> > 
> >> On 03.06.2020 01:49, Andrew Doran wrote:
> >>> On the assembly thing recall that recently you expressed a desire to 
> >>> remove
> >>> all of the amd64 assembly string functions from libc because of 
> >>> sanitizers -
> >>> I invested my time to do up a little demo to try and show you why that's 
> >>> not
> >>> a good idea:
> >>>
> >>>   http://mail-index.netbsd.org/port-amd64/2020/04/19/msg003226.html
> >>
> >> Please note that interceptors for string functions are not just some
> >> extra burden, but also very useful approach to feedback a fuzzer through
> >> additional coverage.
> >>
> >> At least libFuzzer and honggfuzz wrap many kinds of string functions and
> >> use it for fuzzing. We should add a special mode in KCOV to feedback
> >> userland (syzkaller) with traces from string functions.
> >>
> >> https://github.com/google/honggfuzz/blob/bbb476eec95ad927d6d7d3d367d2b3e38eed3569/libhfuzz/memorycmp.c#L24
> > 
> > No argument from me there at all.  I think that's a great idea and was
> > looking at the interceptors in TSAN recently to see how they work.
> > 
> > Andrew
> > 
> 
> My note was not about switching away from ASM functions for certain
> functions, but rather giving an option to disable them under a sanitizer
> and adding an interceptor that can be useful for feedbacking a fuzzer.
> It's still not clear whether we will create such interface in KCOV as it
> has to be coordinated with Google+Linux as we would like to have a
> compatible interface for syzkaller.
> 
> TSAN - do you mean the userland ones?

Right, the userland one.
 
> BTW. There is a work-in-progress MKSANITIZER support for TSan, but it
> used to create unkillable processes (kernel bug). Basically when using a
> TSanitized userland applications, you will quickly end up with such
> processes (from AFAIR calling ls(1) and other simple applications are
> enough).
> 
> If you are interested, I can share a reproducer.

Yes please.  Is the setup difficult?  I plan to look at some of the
remaining issues noted on syzbot over the next while too.

Cheers,
Andrew


Re: CVS commit: src/lib/libpthread

2020-06-04 Thread Andrew Doran
Hi,

On Thu, Jun 04, 2020 at 06:26:07PM +, Martin Husemann wrote:

> On Wed, Jun 03, 2020 at 10:10:24PM +0000, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Wed Jun  3 22:10:24 UTC 2020
> > 
> > Modified Files:
> > src/lib/libpthread: pthread.c pthread_cond.c pthread_mutex.c
> > 
> > Log Message:
> > Deal with a couple of problems with threads being awoken early due to
> > timeouts or cancellation where:
> 
> Not sure if it is caused by this commit or joergs TSD/malloc, but today
> most of the libpthread tests time out on aarch64, while everything but
> a few minor nits was fine on May 30.

I'll try to reproduce it over the weekend.

Andrew
 
> Martin
> 
> (this is on a hummingboard pulse 4-core board)
> lib/libpthread/t_barrier (398/848): 1 test cases
> barrier: [300.034098s] Failed: Test case timed out after 300 seconds
> [300.034568s]
> 
> lib/libpthread/t_cond (399/848): 9 test cases
> bogus_timedwaits: [300.016677s] Failed: Test case timed out after 300 
> seconds
> broadcast: [300.023040s] Failed: Test case timed out after 300 seconds
> cond_timedwait_race: [300.034567s] Failed: Test case timed out after 300 
> seconds
> condattr: [0.013756s] Passed.
> destroy_after_cancel: [300.025649s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock: [300.023630s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock_static_init: [300.022909s] Failed: Test case timed 
> out after 300 seconds
> signal_delay_wait: [300.031310s] Failed: Test case timed out after 300 
> seconds
> signal_wait_race: [300.025552s] Failed: Test case timed out after 300 
> seconds
> [2400.221956s]
> 
> lib/libpthread/t_condwait (400/848): 2 test cases
> cond_wait_mono: [2.023083s] Passed.
> cond_wait_real: [2.017972s] Passed.
> [4.042078s]
> 
> lib/libpthread/t_detach (401/848): 1 test cases
> pthread_detach: [300.021242s] Failed: Test case timed out after 300 
> seconds
> [300.021708s]
> 
> lib/libpthread/t_equal (402/848): 1 test cases
> pthread_equal: [300.027646s] Failed: Test case timed out after 300 seconds
> [300.028295s]
> 
> lib/libpthread/t_fork (403/848): 1 test cases
> fork: [300.018077s] Failed: Test case timed out after 300 seconds
> [300.018975s]
> 
> lib/libpthread/t_fpu (404/848): 1 test cases
> fpu: [0.020578s] Passed.
> [0.021314s]
> 
> lib/libpthread/t_join (405/848): 1 test cases
> pthread_join: [0.021632s] Passed.
> [0.022411s]
> 
> lib/libpthread/t_kill (406/848): 1 test cases
> simple: [300.033759s] Failed: Test case timed out after 300 seconds
> [300.034528s]
> 
> lib/libpthread/t_mutex (407/848): 7 test cases
> mutex1: [300.017542s] Failed: Test case timed out after 300 seconds
> mutex2: [300.023064s] Failed: Test case timed out after 300 seconds
> mutex3: [300.023259s] Failed: Test case timed out after 300 seconds
> mutex4: [300.023015s] Failed: Test case timed out after 300 seconds
> mutex5: [300.023495s] Failed: Test case timed out after 300 seconds
> mutexattr1: [0.013769s] Passed.
> mutexattr2: [0.025483s] Passed.
> [1500.154160s]
> 
> lib/libpthread/t_name (408/848): 1 test cases
> name: [0.014262s] Passed.
> [0.014951s]
> 
> ...


Re: [stos, again] Re: CVS commit: src/sys/arch/amd64

2020-06-03 Thread Andrew Doran
Maxime,

I read your e-mail carefully and conclude that the best way forward here is
put this one to core@ for a technical decision.

Cheers,
Andrew

On Wed, Jun 03, 2020 at 08:25:32AM +0200, Maxime Villard wrote:
> Le 03/06/2020 ? 01:49, Andrew Doran a ?crit?:
> > On Tue, Jun 02, 2020 at 08:41:53AM +0200, Maxime Villard wrote:
> > 
> > > Le 02/06/2020 ? 00:58, Andrew Doran a ?crit?:
> > > > Module Name:src
> > > > Committed By:   ad
> > > > Date:   Mon Jun  1 22:58:06 UTC 2020
> > > > 
> > > > Modified Files:
> > > > src/sys/arch/amd64/amd64: cpufunc.S
> > > > src/sys/arch/amd64/include: frameasm.h
> > > > 
> > > > Log Message:
> > > > Reported-by: syzbot+6dd5a230d19f0cbc7...@syzkaller.appspotmail.com
> > > > 
> > > > Instrument STOS/MOVS for KMSAN to unbreak it.
> > > > 
> > > > 
> > > > To generate a diff of this commit:
> > > > cvs rdiff -u -r1.58 -r1.59 src/sys/arch/amd64/amd64/cpufunc.S
> > > > cvs rdiff -u -r1.49 -r1.50 src/sys/arch/amd64/include/frameasm.h
> > > > 
> > > > Please note that diffs are not public domain; they are subject to the
> > > > copyright notices on the relevant files.
> > > 
> > > Can you just stop ignoring the remarks that are made?
> > 
> > That's up to you Maxime.  If you habitually make it difficult for people to
> > come to a reasonable compromise with you, then you're asking to not be taken
> > seriously and will find yourself being ignored.
> 
> You are confused. I asked for KMSAN to be unbroken, and proposed an 
> alternative,
> which is atomic_load_relaxed. You were free to explain why my point was a bad
> idea or why it didn't matter, but you refused, and just added a hack on top of
> another. I'm afraid that's up to you.
> 
> But whatever, it doesn't matter. Thanks for reverting the pmap.c changes, it
> is more correct now than before.
> 
> > > I said explicitly
> > > that adding manual instrumentation here is _wrong_.
> > 
> > I don't share your assessment in the general sense.  It should be possible
> > to instrument assembly code because KMSAN is useful AND we can't get by
> > without assembly - there are some things that C just can't do (or do well
> > enough).
> > 
> > On the assembly thing recall that recently you expressed a desire to remove
> > all of the amd64 assembly string functions from libc because of sanitizers -
> > I invested my time to do up a little demo to try and show you why that's not
> > a good idea:
> > 
> > http://mail-index.netbsd.org/port-amd64/2020/04/19/msg003226.html
> 
> I saw, yes. I answered explaining that a conversation with Ryo Shimizu had
> changed my mind a bit, and seeing your results (which as far as I can tell
> do not indicate a performance improvement significant enough to not be
> considered as noise), I asked you whether it was that relevant. You didn't
> follow up though.
> 
> > The system is a balancing act.  There are lots of factors to be taken into
> > account: maintainability, tooling like KMSAN, user's varying needs, the
> > capabilites of different machines, performance, feature set and so on, and
> > dare I say it even the enjoyment of the people working on the project is
> > important too.  Myopic focus on one factor only to the detriment of others
> > is no good.
> 
> I am well aware of that.
> 
> > > The kMSan ASM fixups
> > > are limited to args/returns, and that is part of a sensical policy that
> > > _should not_ be changed without a good reason.
> > > 
> > > x86_movs/x86_stos have strictly _no reason_ to exist. Of the 18 
> > > conversions
> > > you made to them in pmap.c, not one is justified. memcpy/memset were all
> > > correct.
> > 
> > I introduced these functions as a compromise because you were unhappy with
> > use of memcpy() to copy PDEs.  See:
> > 
> > http://mail-index.netbsd.org/port-amd64/2020/05/23/msg003280.html
> > 
> > You wrote:
> > 
> > In the [XXX] window, the PTEs could be used by userland.  If you
> > copied them using memcpy(), some parts of the bytes could contain
> > stale values.
> 
> Sure, I was explicitly referring to SVS, which has an unusual way of
> accessing PTEs (contrary to pmap), which is why it needs special atomic
> care that other places do not.
> 
> > Live PDEs are also copied in pmap.c so I made a change there too.  After
> > that I decided "why not" and used the

Re: [stos, again] Re: CVS commit: src/sys/arch/amd64

2020-06-03 Thread Andrew Doran
On Wed, Jun 03, 2020 at 02:03:22AM +0200, Kamil Rytarowski wrote:

> On 03.06.2020 01:49, Andrew Doran wrote:
> > On the assembly thing recall that recently you expressed a desire to remove
> > all of the amd64 assembly string functions from libc because of sanitizers -
> > I invested my time to do up a little demo to try and show you why that's not
> > a good idea:
> > 
> > http://mail-index.netbsd.org/port-amd64/2020/04/19/msg003226.html
> 
> Please note that interceptors for string functions are not just some
> extra burden, but also very useful approach to feedback a fuzzer through
> additional coverage.
>
> At least libFuzzer and honggfuzz wrap many kinds of string functions and
> use it for fuzzing. We should add a special mode in KCOV to feedback
> userland (syzkaller) with traces from string functions.
> 
> https://github.com/google/honggfuzz/blob/bbb476eec95ad927d6d7d3d367d2b3e38eed3569/libhfuzz/memorycmp.c#L24

No argument from me there at all.  I think that's a great idea and was
looking at the interceptors in TSAN recently to see how they work.

Andrew


Re: [stos, again] Re: CVS commit: src/sys/arch/amd64

2020-06-02 Thread Andrew Doran
On Tue, Jun 02, 2020 at 08:41:53AM +0200, Maxime Villard wrote:

> Le 02/06/2020 ? 00:58, Andrew Doran a ?crit?:
> > Module Name:src
> > Committed By:   ad
> > Date:   Mon Jun  1 22:58:06 UTC 2020
> > 
> > Modified Files:
> > src/sys/arch/amd64/amd64: cpufunc.S
> > src/sys/arch/amd64/include: frameasm.h
> > 
> > Log Message:
> > Reported-by: syzbot+6dd5a230d19f0cbc7...@syzkaller.appspotmail.com
> > 
> > Instrument STOS/MOVS for KMSAN to unbreak it.
> > 
> > 
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.58 -r1.59 src/sys/arch/amd64/amd64/cpufunc.S
> > cvs rdiff -u -r1.49 -r1.50 src/sys/arch/amd64/include/frameasm.h
> > 
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> 
> Can you just stop ignoring the remarks that are made?

That's up to you Maxime.  If you habitually make it difficult for people to
come to a reasonable compromise with you, then you're asking to not be taken
seriously and will find yourself being ignored.

> I said explicitly
> that adding manual instrumentation here is _wrong_.

I don't share your assessment in the general sense.  It should be possible
to instrument assembly code because KMSAN is useful AND we can't get by
without assembly - there are some things that C just can't do (or do well
enough).

On the assembly thing recall that recently you expressed a desire to remove
all of the amd64 assembly string functions from libc because of sanitizers -
I invested my time to do up a little demo to try and show you why that's not
a good idea:

http://mail-index.netbsd.org/port-amd64/2020/04/19/msg003226.html

The system is a balancing act.  There are lots of factors to be taken into
account: maintainability, tooling like KMSAN, user's varying needs, the
capabilites of different machines, performance, feature set and so on, and
dare I say it even the enjoyment of the people working on the project is
important too.  Myopic focus on one factor only to the detriment of others
is no good.

> The kMSan ASM fixups
> are limited to args/returns, and that is part of a sensical policy that
> _should not_ be changed without a good reason.
>
> x86_movs/x86_stos have strictly _no reason_ to exist. Of the 18 conversions
> you made to them in pmap.c, not one is justified. memcpy/memset were all
> correct.

I introduced these functions as a compromise because you were unhappy with
use of memcpy() to copy PDEs.  See:

http://mail-index.netbsd.org/port-amd64/2020/05/23/msg003280.html

You wrote:

In the [XXX] window, the PTEs could be used by userland.  If you
copied them using memcpy(), some parts of the bytes could contain
stale values.

Live PDEs are also copied in pmap.c so I made a change there too.  After
that I decided "why not" and used the new functions everywhere PDEs/PTEs or
pages are block zeroed / copied.  But I'm also happy with memcpy()/memset(). 
Either way will work.  In fairness I do work too fast sometimes.

> The only reason you made these big unneeded changes is for SVS not to take
> the bus lock,

There is no bus lock on x86 (not since the 1990s anyway).

> but as was said already, atomic_load_relaxed will do what
> you want without the need for these functions.
>
> Please revert _both changes now_, this one and the previous one which
> introduced both functions, and let's use atomic_load_relaxed.

You're focusing on only one factor.  I'll explain in detail.  Here is the
original code I replaced:

685 static inline pt_entry_t
686 svs_pte_atomic_read(struct pmap *pmap, size_t idx)
687 {
688 /*
689  * XXX: We don't have a basic atomic_fetch_64 function?
690  */
691 return atomic_cas_64(>pm_pdir[idx], 666, 666);
692 }
...
717 /* User slots. */
718 for (i = 0; i < PDIR_SLOT_USERLIM; i++) {
719 pte = svs_pte_atomic_read(pmap, i);
720 ci->ci_svs_updir[i] = pte;
721 }

There's no need for an atomic op there because fetches on x86 are by
definition atomic, and it does nothing to alter the memory ordering in this
case.  There are side effects to the atomic op: it's serializing and always
generates an unbuffered writeback, even in the failure case.  So the source
is being copied into itself, as well as into the destination, and the CPU's
store buffer is rendered ineffective.

A cut-n-paste replacement to use the relaxed functions predictably ties the
compiler in knots and the generated code is bad.

/* User slots. */
for (i = 0; i < PDIR_SLOT_USERLIM; i++) {
pte = atomic_load_relaxed(>pm_pdir[i]);
atomic_store_relaxed(>ci_svs_updir[i], pte);
}

Re: [stos] Re: CVS commit: src/sys/arch

2020-05-28 Thread Andrew Doran
On Thu, May 28, 2020 at 07:06:04PM +0200, Maxime Villard wrote:
> Le 27/05/2020 ? 21:58, Maxime Villard a ?crit?:
> > Le 27/05/2020 ? 21:33, Andrew Doran a ?crit?:
> > > Module Name:??? src
> > > Committed By:??? ad
> > > Date:??? Wed May 27 19:33:40 UTC 2020
> > > 
> > > Modified Files:
> > > src/sys/arch/amd64/amd64: cpufunc.S locore.S
> > > src/sys/arch/i386/i386: cpufunc.S locore.S
> > > src/sys/arch/x86/include: pmap.h
> > > src/sys/arch/x86/x86: pmap.c
> > > 
> > > Log Message:
> > > - Add a couple of wrapper functions around STOS and MOVS and use them to 
> > > zero
> > > ?? and copy PTEs in preference to memset()/memcpy().
> > > 
> > > - Remove related SSE / pageidlezero stuff.
> > > 
> > > 
> > > To generate a diff of this commit:
> > > cvs rdiff -u -r1.56 -r1.57 src/sys/arch/amd64/amd64/cpufunc.S
> > > cvs rdiff -u -r1.208 -r1.209 src/sys/arch/amd64/amd64/locore.S
> > > cvs rdiff -u -r1.42 -r1.43 src/sys/arch/i386/i386/cpufunc.S
> > > cvs rdiff -u -r1.184 -r1.185 src/sys/arch/i386/i386/locore.S
> > > cvs rdiff -u -r1.121 -r1.122 src/sys/arch/x86/include/pmap.h
> > > cvs rdiff -u -r1.395 -r1.396 src/sys/arch/x86/x86/pmap.c
> > > 
> > > Please note that diffs are not public domain; they are subject to the
> > > copyright notices on the relevant files.
> > 
> >  -END(x86_stos)
> >  +END(x86_movs)
> > 
> > The naming convention should also be more explicit I think, because movs
> > does not say if it is b/w/l/q. Don't have anything meaningful to suggest
> > though
> 
> I see that syzbot-kMSan is failing because of this change. Looking at it more
> closely:
> 
>  - Having MI ASM copy functions is unwanted, because the sanitizers won't be
>able to sanitize the accesses. kMSan misses the initialization and reports
>false positives. As well kASan will miss memory corruptions and kCSan will
>miss races.
> 
>  - It appears you replaced memcpy/memset by x86_movs/x86_stos in places where
>it is unnecessary and unwanted. Eg the majority of the changes in pmap.c
>are unwanted and should remain memcpy/memset.
>
> Please revert this change promptly in order to unbreak kMSan. We really need
> to have a clear policy on which function should be in ASM, and not introduce
> wild MI things like that which not only are rarely justified but also are a
> big PITA when sanitizers come into play.

Very good.  I see a function in subr_msan.c that looks like it does the
needful here:

void __msan_instrument_asm_store(void *addr, size_t size)

I don't see any uses in tree.  Is there a reason for that?

Thanks,
Andrew


Re: CVS commit: src/sys/arch

2020-05-21 Thread Andrew Doran
On Thu, May 21, 2020 at 11:19:49PM +0200, Joerg Sonnenberger wrote:
> On Thu, May 21, 2020 at 09:12:31PM +0000, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Thu May 21 21:12:31 UTC 2020
> > 
> > Modified Files:
> > src/sys/arch/x86/acpi: acpi_wakeup.c
> > src/sys/arch/x86/include: i82489var.h
> > src/sys/arch/x86/x86: cpu.c lapic.c x86_machdep.c
> > src/sys/arch/xen/x86: cpu.c
> > src/sys/arch/xen/xen: hypervisor.c xen_clock.c
> > 
> > Log Message:
> > - Recalibrate the APIC timer using the TSC, once the TSC has in turn been
> >   recalibrated using the HPET.  This gets the clock interrupt firing more
> >   closely to HZ.
> 
> Why using the TSC and not the HPET directly? For systems with HPET, same
> question with the ACPI / PCI timer.

The idea I'm going with is:

The TSC is precise and very quick to read, far more so than the other timers
where it's at least 0.5-1us to pull a value from the counter register.  To
get around slowness of the other timers the TSC is calibrated with respect
to the HPET over a long span (seconds).  I have no objection to revisiting
it if we can do better, but this is at least much better than where we were
before.

kern.timecounter.choice = TSC(q=3000, f=10 Hz) lapic(q=-100, 
f=2 Hz) clockinterrupt(q=0, f=100 Hz) hpet0(q=2000, f=14318180 Hz) 
ACPI-Fast(q=1000, f=3579545 Hz) i8254(q=100, f=1193182 Hz) dummy(q=-100, 
f=100 Hz)
kern.timecounter.choice = TSC(q=3000, f=25 Hz) lapic(q=-100, 
f=1 Hz) clockinterrupt(q=0, f=100 Hz) ichlpcib0(q=1000, f=3579545 Hz) 
hpet0(q=2000, f=14318180 Hz) ACPI-Fast(q=1000, f=3579545 Hz) i8254(q=100, 
f=1193182 Hz) dummy(q=-100, f=100 Hz)

Andrew


Re: CVS commit: src/sys/fs/tmpfs

2020-05-19 Thread Andrew Doran
On Tue, May 19, 2020 at 11:09:07PM +, Andrew Doran wrote:

> vnode locks are not held during getpages/putpages.

^ for fault handling, anyway.  for read/write they are held by the caller to
ubc_uiomove().

Andrew


Re: CVS commit: src/sys/fs/tmpfs

2020-05-19 Thread Andrew Doran
On Sun, May 17, 2020 at 11:49:52PM +, m...@netbsd.org wrote:

> On Sun, May 17, 2020 at 09:47:50PM +, m...@netbsd.org wrote:
> > On Sun, May 17, 2020 at 07:39:15PM +0000, Andrew Doran wrote:
> > > Module Name:  src
> > > Committed By: ad
> > > Date: Sun May 17 19:39:15 UTC 2020
> > > 
> > > Modified Files:
> > >   src/sys/fs/tmpfs: tmpfs.h tmpfs_subr.c tmpfs_vnops.c
> > > 
> > > Log Message:
> > > PR kern/55268: tmpfs is slow
> > > 
> > > tmpfs_getpages(): ...implement lazy update of atime/mtime.
> > > 
> > 
> > I'm confused about how this makes sense. Can you elaborate?
> > Presumably RAM is as fast as other RAM.
> 
> riastradh responded to this elsewhere: avoid atomic ops

Right, also to:

- avoid taking a mutex to serialize the update to the time fields in the
  tmpfs node.  vnode locks are not held during getpages/putpages.  the
  vmobjlock is held here and often read-held so there can be lots of
  concurrent access.

- avoid doing a writeback on the tmpfs_node, which if it's heavily shared
  (consider for example ld.so or libc.so) involves lots of cache coherency
  overhead.

- avoid querying the clock.

Andrew


Re: CVS commit: src/sys

2020-05-12 Thread Andrew Doran
On Sun, May 10, 2020 at 03:15:22PM -, Christos Zoulas wrote:
> In article <20200508220155.446eef...@cvs.netbsd.org>,
> Andrew Doran  wrote:
> >-=-=-=-=-=-
> >
> >Module Name: src
> >Committed By:ad
> >Date:Fri May  8 22:01:55 UTC 2020
> >
> >Modified Files:
> > src/sys/arch/x86/include: cpu_counter.h
> > src/sys/arch/x86/x86: cpu.c tsc.c
> > src/sys/dev/ic: hpet.c hpetvar.h
> >
> >Log Message:
> >Fix the TSC timecounter (on the systems I have access to):
> >
> >- Make the early i8254-based calculation of frequency a bit more accurate.
> >
> >- Keep track of how far the HPET & TSC advance between HPET attach and
> >  secondary CPU boot, and use to compute an accurate value before attaching
> >  the timecounter.  Initial idea from joerg@.
> >
> >- When determining skew and drift between CPUs, make each measurement 1000
> >  times and pick the lowest observed value.  Increase the error threshold to
> >  1000 clock cycles.
> >
> >- Use the frequency computed on the boot CPU for secondary CPUs too.
> >
> >- Remove cpu_counter_serializing().
> 
> The TSC is still faster than it is supposed to be so ntpd does not sync
> (it diverges). It is better than before but not good enough to keep time.

I suspect this problem is related to the MSR-based freq determination rather
than the calibration so hopefully in combo with msaitoh@'s follow-up change
it should be fixed.

Andrew


Re: CVS commit: src/sys

2020-04-29 Thread Andrew Doran
On Wed, Apr 29, 2020 at 09:58:55PM +, Andrew Doran wrote:

> On Wed, Apr 29, 2020 at 02:32:39PM +0900, Tetsuya Isaki wrote:
> > At Wed, 29 Apr 2020 12:22:01 +0900,
> 
> > Tetsuya Isaki wrote:
> > > > i would just put it in types.h called __AUDIO_BLK_MS,
> > > > and leave a default used in the code if unset.
> > > It sounds nice.
> > > I commit once here, and then I will try it.
> > 
> > How about this diff?
> > The old platforms are the same as before, hppa, m68k, sh3, sparc(!64)
> > and vax.
> 
> machine/param.h seems more natural to me.  I don't have a strong opinion
> though.

.. I mean, if it's a "tuneable" value like this rather than a constant like
__HAVE_SLOW_COMPUTER. :-)

Andrew

> 
> Andrew
> 
>  
> > Thanks,
> > 
> > --- a/sys/arch/hppa/include/types.h
> > +++ b/sys/arch/hppa/include/types.h
> > @@ -103,4 +103,8 @@ extern const char __CONCAT(name,_ras_start[]), 
> > __CONCAT(name,_ras_end[])
> >  #define__HAVE_MM_MD_DIRECT_MAPPED_PHYS
> >  #define__HAVE_MM_MD_KERNACC
> >  
> > +#if defined(_KERNEL)
> > +#define__AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
> > +#endif
> > +
> >  #endif /* _HPPA_TYPES_H_ */
> > diff --git a/sys/arch/m68k/include/types.h b/sys/arch/m68k/include/types.h
> > index d8b1347ae..0a581dff0 100644
> > --- a/sys/arch/m68k/include/types.h
> > +++ b/sys/arch/m68k/include/types.h
> > @@ -80,6 +80,7 @@ typedef int   __register_t;
> >  
> >  #if defined(_KERNEL)
> >  #define__HAVE_RAS
> > +#define__AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
> >  #endif
> >  
> >  #endif /* !_M68K_TYPES_H_ */
> > diff --git a/sys/arch/sh3/include/types.h b/sys/arch/sh3/include/types.h
> > index 9a8b247be..f0a8e92d7 100644
> > --- a/sys/arch/sh3/include/types.h
> > +++ b/sys/arch/sh3/include/types.h
> > @@ -79,6 +79,7 @@ typedef   int __register_t;
> >  
> >  #if defined(_KERNEL)
> >  #define__HAVE_RAS
> > +#define__AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
> >  #endif
> >  
> >  #define__HAVE_CPU_LWP_SETPRIVATE
> > diff --git a/sys/arch/sparc/include/types.h b/sys/arch/sparc/include/types.h
> > index 01af19775..360bb069a 100644
> > --- a/sys/arch/sparc/include/types.h
> > +++ b/sys/arch/sparc/include/types.h
> > @@ -141,4 +141,10 @@ typedef unsigned long int  __register_t;
> >  #define__HAVE_TLS_VARIANT_II
> >  #define__HAVE_COMMON___TLS_GET_ADDR
> >  
> > +#if defined(_KERNEL)
> > +#if !defined(__arch64__)
> > +#define__AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
> > +#endif
> > +#endif
> > +
> >  #endif /* _MACHTYPES_H_ */
> > diff --git a/sys/arch/vax/include/types.h b/sys/arch/vax/include/types.h
> > index e49c9db96..8ab482051 100644
> > --- a/sys/arch/vax/include/types.h
> > +++ b/sys/arch/vax/include/types.h
> > @@ -82,6 +82,7 @@ typedef int   __register_t;
> >  #define__HAVE_OLD_DISKLABEL
> >  #ifdef _KERNEL
> >  #define__HAVE_RAS
> > +#define__AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
> >  #endif
> >  
> >  #define__HAVE___LWP_GETPRIVATE_FAST
> > diff --git a/sys/dev/audio/audio.c b/sys/dev/audio/audio.c
> > index 13386ccfb..20a0c6c10 100644
> > --- a/sys/dev/audio/audio.c
> > +++ b/sys/dev/audio/audio.c
> > @@ -183,6 +183,7 @@ __KERNEL_RCSID(0, "$NetBSD: audio.c,v 1.41 2020/01/11 
> > 04:53:10 isaki Exp $");
> >  #include 
> >  
> >  #include 
> > +#include  /* for __AUDIO_BLK_MS */
> >  
> >  #include 
> >  
> > @@ -454,38 +455,25 @@ audio_track_bufstat(audio_track_t *track, struct 
> > audio_track_debugbuf *buf)
> >  /*
> >   * Default hardware blocksize in msec.
> >   *
> > - * We use 10 msec for most platforms.  This period is good enough to play
> > - * audio and video synchronizely.
> > + * We use 10 msec for most modern platforms.  This period is good enough to
> > + * play audio and video synchronizely.
> >   * In contrast, for very old platforms, this is usually too short and too
> >   * severe.  Also such platforms usually can not play video confortably, so
> > - * it's not so important to make the blocksize shorter.
> > + * it's not so important to make the blocksize shorter.  If the platform
> > + * defines its own value as __AUDIO_BLK_MS in its , it
> > + * uses this instead.
> > + *
> >   * In either case, you can overwrite AUDIO

Re: CVS commit: src/sys

2020-04-29 Thread Andrew Doran
On Wed, Apr 29, 2020 at 02:32:39PM +0900, Tetsuya Isaki wrote:
> At Wed, 29 Apr 2020 12:22:01 +0900,

> Tetsuya Isaki wrote:
> > > i would just put it in types.h called __AUDIO_BLK_MS,
> > > and leave a default used in the code if unset.
> > It sounds nice.
> > I commit once here, and then I will try it.
> 
> How about this diff?
> The old platforms are the same as before, hppa, m68k, sh3, sparc(!64)
> and vax.

machine/param.h seems more natural to me.  I don't have a strong opinion
though.

Andrew

 
> Thanks,
> 
> --- a/sys/arch/hppa/include/types.h
> +++ b/sys/arch/hppa/include/types.h
> @@ -103,4 +103,8 @@ extern const char __CONCAT(name,_ras_start[]), 
> __CONCAT(name,_ras_end[])
>  #define  __HAVE_MM_MD_DIRECT_MAPPED_PHYS
>  #define  __HAVE_MM_MD_KERNACC
>  
> +#if defined(_KERNEL)
> +#define  __AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
> +#endif
> +
>  #endif   /* _HPPA_TYPES_H_ */
> diff --git a/sys/arch/m68k/include/types.h b/sys/arch/m68k/include/types.h
> index d8b1347ae..0a581dff0 100644
> --- a/sys/arch/m68k/include/types.h
> +++ b/sys/arch/m68k/include/types.h
> @@ -80,6 +80,7 @@ typedef int __register_t;
>  
>  #if defined(_KERNEL)
>  #define  __HAVE_RAS
> +#define  __AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
>  #endif
>  
>  #endif   /* !_M68K_TYPES_H_ */
> diff --git a/sys/arch/sh3/include/types.h b/sys/arch/sh3/include/types.h
> index 9a8b247be..f0a8e92d7 100644
> --- a/sys/arch/sh3/include/types.h
> +++ b/sys/arch/sh3/include/types.h
> @@ -79,6 +79,7 @@ typedef int __register_t;
>  
>  #if defined(_KERNEL)
>  #define  __HAVE_RAS
> +#define  __AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
>  #endif
>  
>  #define  __HAVE_CPU_LWP_SETPRIVATE
> diff --git a/sys/arch/sparc/include/types.h b/sys/arch/sparc/include/types.h
> index 01af19775..360bb069a 100644
> --- a/sys/arch/sparc/include/types.h
> +++ b/sys/arch/sparc/include/types.h
> @@ -141,4 +141,10 @@ typedef unsigned long int__register_t;
>  #define  __HAVE_TLS_VARIANT_II
>  #define  __HAVE_COMMON___TLS_GET_ADDR
>  
> +#if defined(_KERNEL)
> +#if !defined(__arch64__)
> +#define  __AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
> +#endif
> +#endif
> +
>  #endif   /* _MACHTYPES_H_ */
> diff --git a/sys/arch/vax/include/types.h b/sys/arch/vax/include/types.h
> index e49c9db96..8ab482051 100644
> --- a/sys/arch/vax/include/types.h
> +++ b/sys/arch/vax/include/types.h
> @@ -82,6 +82,7 @@ typedef int __register_t;
>  #define  __HAVE_OLD_DISKLABEL
>  #ifdef _KERNEL
>  #define  __HAVE_RAS
> +#define  __AUDIO_BLK_MS (40) /* See sys/dev/audio/audio.c */
>  #endif
>  
>  #define  __HAVE___LWP_GETPRIVATE_FAST
> diff --git a/sys/dev/audio/audio.c b/sys/dev/audio/audio.c
> index 13386ccfb..20a0c6c10 100644
> --- a/sys/dev/audio/audio.c
> +++ b/sys/dev/audio/audio.c
> @@ -183,6 +183,7 @@ __KERNEL_RCSID(0, "$NetBSD: audio.c,v 1.41 2020/01/11 
> 04:53:10 isaki Exp $");
>  #include 
>  
>  #include 
> +#include/* for __AUDIO_BLK_MS */
>  
>  #include 
>  
> @@ -454,38 +455,25 @@ audio_track_bufstat(audio_track_t *track, struct 
> audio_track_debugbuf *buf)
>  /*
>   * Default hardware blocksize in msec.
>   *
> - * We use 10 msec for most platforms.  This period is good enough to play
> - * audio and video synchronizely.
> + * We use 10 msec for most modern platforms.  This period is good enough to
> + * play audio and video synchronizely.
>   * In contrast, for very old platforms, this is usually too short and too
>   * severe.  Also such platforms usually can not play video confortably, so
> - * it's not so important to make the blocksize shorter.
> + * it's not so important to make the blocksize shorter.  If the platform
> + * defines its own value as __AUDIO_BLK_MS in its , it
> + * uses this instead.
> + *
>   * In either case, you can overwrite AUDIO_BLK_MS by your kernel
>   * configuration file if you wish.
> - *
> - * 40 msec was initially choosen for the following reason:
> - * (1 / 40ms) = 25 = 5^2.  Thus, the frequency is factored by 5.
> - * In this case, the number of frames in a block can be an integer
> - * even if the frequency is a multiple of 100 (44100, 48000, etc),
> - * or even if 15625Hz (vs(4)).
>   */
> -#if defined(__hppa__)|| \
> -defined(__m68k__)|| \
> -defined(__sh3__) || \
> -(defined(__sparc__) && !defined(__sparc64__))|| \
> -defined(__vax__)
> -#define AUDIO_TOO_SLOW_ARCHS 1
> -#endif
> -
>  #if !defined(AUDIO_BLK_MS)
> -# if defined(AUDIO_TOO_SLOW_ARCHS)
> -#  define AUDIO_BLK_MS 40
> +# if defined(__AUDIO_BLK_MS)
> +#  define AUDIO_BLK_MS __AUDIO_BLK_MS
>  # else
> -#  define AUDIO_BLK_MS 10
> +#  define AUDIO_BLK_MS (10)
>  # endif
>  #endif
>  
> -#undef AUDIO_TOO_SLOW_ARCHS
> -
>  /* Device timeout in msec */
>  #define AUDIO_TIMEOUT(3000)
> 
> ---
> Tetsuya Isaki 
>  


Re: CVS commit: src/sys/kern

2020-04-13 Thread Andrew Doran
On Mon, Apr 13, 2020 at 04:34:48PM +0100, Roy Marples wrote:
> On 13/04/2020 16:31, Andrew Doran wrote:
> > Hi Roy.
> > 
> > On Sat, Apr 11, 2020 at 02:13:06AM +0100, Roy Marples wrote:
> > > On 10/04/2020 23:34, Andrew Doran wrote:
> > > > Module Name:src
> > > > Committed By:   ad
> > > > Date:   Fri Apr 10 22:34:36 UTC 2020
> > > > 
> > > > Modified Files:
> > > > src/sys/kern: vfs_mount.c
> > > > 
> > > > Log Message:
> > > > vfs_mountroot(): don't needlessly grab a second reference to the root 
> > > > vnode
> > > > (the kernel never chdirs) nor a lock on it.
> > > 
> > > So the kernel chrooting to sysctl init.root is still ok?
> > 
> > How is that accomplished?  I couldn't find the code and don't recall seeing
> > it.
> 
> sysctl -w init.root=/altroot
> 
> See the ZFS Root ramdisk:
> https://nxr.netbsd.org/xref/src/distrib/common/zfsroot.rc#33
> 
> I tested kernel from yesterdays sources and it still works fine, so I guess
> nothing was needed here.

I had a look and it's init that chroots, not the kernel, so no issue.  The
kernel chrooting would be evil.

Andrew


Re: CVS commit: src/sys/kern

2020-04-13 Thread Andrew Doran
Hi Roy.

On Sat, Apr 11, 2020 at 02:13:06AM +0100, Roy Marples wrote:
> On 10/04/2020 23:34, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Fri Apr 10 22:34:36 UTC 2020
> > 
> > Modified Files:
> > src/sys/kern: vfs_mount.c
> > 
> > Log Message:
> > vfs_mountroot(): don't needlessly grab a second reference to the root vnode
> > (the kernel never chdirs) nor a lock on it.
> 
> So the kernel chrooting to sysctl init.root is still ok?

How is that accomplished?  I couldn't find the code and don't recall seeing
it.

Andrew


Re: [vfs_cache] Re: CVS commit: src/sys/kern

2020-04-05 Thread Andrew Doran
On Sun, Mar 29, 2020 at 11:41:23AM +0200, Maxime Villard wrote:
> Le 23/03/2020 ? 21:02, Andrew Doran a ?crit?:
> > Module Name:src
> > Committed By:   ad
> > Date:   Mon Mar 23 20:02:14 UTC 2020
> > 
> > Modified Files:
> > src/sys/kern: vfs_cache.c
> > 
> > Log Message:
> > cache_remove(): remove from the vnode list first, so cache_revlookup()
> > doesn't try to re-activate an entry no longer on the LRU list.
> > 
> > 
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.133 -r1.134 src/sys/kern/vfs_cache.c
> > 
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> 
> It appears that your recent changes in vfs_cache.c have introduced a
> use-after-free. Booting KASAN gives:
> 
>   ASan: Unauthorized Access In 0x...: Addr 0x... [1 byte, read, 
> PoolUseAfterFree]
> 
> It seems that the problem is here:
> 
> 557   if (nameiop == CREATE && (cnflags & ISLASTCN) != 0) {
> 558   /*
> 559* Last component and we are preparing to create
> 560* the named object, so flush the negative cache
> 561* entry.
> 562*/
> 563   COUNT(ncs_badhits);
> 564   cache_remove(ncp, true);< HERE
> 565   hit = false;
> 566   } else {
> 567   COUNT(ncs_neghits);
> 568   SDT_PROBE(vfs, namecache, lookup, hit, dvp, name,
> 569   namelen, 0, 0);
> 570   /* found neg entry; vn is already null from above */
> 571   hit = true;
> 572   }
> 573   if (iswht_ret != NULL) {
> 574   /*
> 575* Restore the ISWHITEOUT flag saved earlier.
> 576*/
> 577   *iswht_ret = ncp->nc_whiteout;  <-- ouch
> 578   } else {
> 579   KASSERT(!ncp->nc_whiteout);  <-- ouch
> 580   }
> 
> cache_remove() frees 'ncp', and then 'ncp->nc_whiteout' is read.

Fixed with vfs_cache.c 1.136.  Thank you for bringing it to my attention.

Andrew


Re: CVS commit: src/external/gpl3

2020-03-26 Thread Andrew Doran
On Thu, Mar 26, 2020 at 06:07:54PM +0100, Kamil Rytarowski wrote:

> On 26.03.2020 17:49, Robert Elz wrote:
> > Date:Thu, 26 Mar 2020 16:28:18 +0100
> > From:Kamil Rytarowski 
> > Message-ID:  <84460ebb-b4bf-f3ee-e51b-e27d0b6e2...@gmx.com>
> >
> >
> >   | That is negligible cost of getting TMPDIR propagated to most programs.
> >
> > Sure, it isn't much, for one program, but when you're running tens of
> > thousands, it all adds up.
> >
> 
> It's still negligible for hundreds of millions calls.
> 
> Modern CPUs like Ryzen Threadripper 3990X can execute that extra amount
> of instructions (around 600) in a single clock cycle.

NetBSD-current would probably take longer to build on that AMD chip, than
Research Unix took to build on a PDP-11/70 in the late 1970s.  It still
counts.

https://www.bell-labs.com/usr/dmr/www/retro.pdf

Andrew


Re: CVS commit: src/sys/arch/x86/x86

2020-03-17 Thread Andrew Doran
On Tue, Mar 17, 2020 at 10:38:14PM +, Andrew Doran wrote:

> Log Message:
> - Change some expensive checks DEBUG -> DIAGNOSTIC.

That was meant to be the other way around, oops.

Andrew


Re: CVS commit: src/sys/arch/sparc/sparc

2020-03-15 Thread Andrew Doran
On Sun, Mar 15, 2020 at 11:38:03AM +1100, matthew green wrote:
> "Andrew Doran" writes:
> > Module Name:src
> > Committed By:   ad
> > Date:   Sat Mar 14 13:34:44 UTC 2020
> > 
> > Modified Files:
> > src/sys/arch/sparc/sparc: intr.c
> > 
> > Log Message:
> > sparc cpu_intr_p(): try to work around l_cpu not being set early on by
> > using curcpu().
> 
> ah, good idea.  this will involve one fewer deref, curcpu()
> is mapped at a fixed address, so it might even be faster now
> than it used to be.

Oh, hm..  Given that it's a fixed address is there a way the compiler can
screw this up in the face of a kernel preemption (theoretical right now for
sparc*)?  I don't think so..  None of the fanciness is required then.

bool
cpu_intr_p(void)
{

return curcpu()->ci_idepth != 0;
}

Andrew


Re: CVS commit: src/sys/kern

2020-03-08 Thread Andrew Doran
On Sun, Mar 08, 2020 at 08:34:29AM +0100, Maxime Villard wrote:
> Le 08/03/2020 ? 01:31, Andrew Doran a ?crit?:
> > Module Name:src
> > Committed By:   ad
> > Date:   Sun Mar  8 00:31:19 UTC 2020
> > 
> > Modified Files:
> > src/sys/kern: subr_kmem.c
> > 
> > Log Message:
> > KMEM_SIZE: append the size_t to the allocated buffer, rather than
> > prepending, so it doesn't screw up the alignment of the buffer.
> > 
> > Reported-by: syzbot+c024c50570cccac51...@syzkaller.appspotmail.com
> > 
> > 
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.78 -r1.79 src/sys/kern/subr_kmem.c
> > 
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> 
> This is wrong. The point of KMEM_SIZE is to store at a reliable location
> the size of the buffer, in order to then verify that the 'size' argument
> given to kmem_free() is correct.
> 
> Here, you are using that size argument to compute the location, which
> breaks the whole point of the check.

Sure I understand the frustration, but it's still correct according to
the parameters I set for it 10+ years ago, which were for it to be a
quick-n-dirty diagnostic aid.

> Also it probably collides with the KASAN shadow.

Hmm, is that purely an alignment issue then, because it works in 8 byte
cells?  Otherwise it sounds like this should not be enabled with KASAN at
all.

Andrew
 
> Please revert this change.
>
> As said previously, if cacheline alignment is expected this way, then it
> should clearly be part of the kmem contract, and documented to be so.


Re: CVS commit: src

2020-03-07 Thread Andrew Doran
On Sat, Mar 07, 2020 at 12:24:21PM +0100, Maxime Villard wrote:

> Can we revert the "__aligned(COHERENCY_UNIT)" for now? There is no particular
> hurry to fix this bug, however the KUBSAN instance has been down for more than
> two months because of this, and it needs to be addressed.

That should be quelled now.

> Similarly, the KASAN instance is currently crashing hard on:
>   
> https://syzkaller.appspot.com/bug?id=1aa3f789d356bf04644bcef632bf8c2373398ba2
> Dozens of thousands of times each day. This has been the case for two weeks,
> and it too needs to be addressed.

That's been there since I started looking last year.

I guess it's a false positive because the sanitiser probably thinks objects
are gone once pool_cache_put() is called, but the actual point of disposal
is the pool_cache dtor.

Andrew


Re: CVS commit: src/sys/kern

2020-03-03 Thread Andrew Doran
On Tue, Mar 03, 2020 at 02:55:16PM -0500, Christos Zoulas wrote:

> Module Name:  src
> Committed By: christos
> Date: Tue Mar  3 19:55:16 UTC 2020
> 
> Modified Files:
>   src/sys/kern: vfs_syscalls.c
> 
> Log Message:
> don't skip the rdir check for the lazy case; breaks chroot df(1) hiding.

That's likely my mistake.  Thank you.

Andrew


Re: CVS commit: src

2020-02-23 Thread Andrew Doran
On Fri, Feb 21, 2020 at 02:14:31PM +0100, Kamil Rytarowski wrote:

> On 22.12.2019 20:47, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Sun Dec 22 19:47:35 UTC 2019
> > 
> > Modified Files:
> > src/external/cddl/osnet/dist/uts/common/fs/zfs: zfs_ctldir.c
> > src/sys/kern: vfs_mount.c vfs_subr.c vfs_syscalls.c
> > src/sys/miscfs/genfs: genfs_vfsops.c
> > src/sys/nfs: nfs_export.c
> > src/sys/sys: mount.h vnode.h vnode_impl.h
> > src/sys/ufs/lfs: ulfs_vfsops.c
> > src/sys/ufs/ufs: ufs_vfsops.c ufs_wapbl.c
> > 
> > Log Message:
> > Make mntvnode_lock per-mount, and address false sharing of struct mount.
> > 
> 
> This change broke kUBSan syzbot.
> 
> The sanitizer is now very noisy as struct mount requires 64 byte alignment.
> 
> http://netbsd.org/~kamil/kubsan/mount-alignment.txt

I had a look this weekend.  This is down to KMEM_SIZE messing up the
alignment, so is a DIAGNOSTIC thing.  The align_offset parameter to
pool_cache() would be a nice easy way to solve this but it seems someone
killed that off, so I'll need to give this some more thought.

Andrew

> > diff -u src/sys/sys/mount.h:1.234 src/sys/sys/mount.h:1.235
> > --- src/sys/sys/mount.h:1.234   Tue Jan  1 10:06:54 2019
> > +++ src/sys/sys/mount.h Sun Dec 22 19:47:34 2019
> > @@ -1,4 +1,4 @@
> > -/* $NetBSD: mount.h,v 1.234 2019/01/01 10:06:54 hannken Exp $  */
> > +/* $NetBSD: mount.h,v 1.235 2019/12/22 19:47:34 ad Exp $   */
> >  
> >  /*
> >   * Copyright (c) 1989, 1991, 1993
> > @@ -133,29 +133,38 @@ struct vattr;
> >   * array of operations and an instance record.
> >   */
> >  struct mount {
> > -   TAILQ_HEAD(, vnode_impl) mnt_vnodelist; /* list of vnodes this mount */
> > +   /*
> > +* Mostly stable data.
> > +*/
> > +   kmutex_t*mnt_vnodelock; /* lock on mnt_vnodelist */
> > struct vfsops   *mnt_op;/* operations on fs */
> > struct vnode*mnt_vnodecovered;  /* vnode we mounted on */
> > struct mount*mnt_lower; /* fs mounted on */
> > -   int mnt_synclist_slot;  /* synclist slot index */
> > void*mnt_transinfo; /* for FS-internal use */
> > void*mnt_data;  /* private data */
> > -   kmutex_tmnt_renamelock; /* per-fs rename lock */
> > -   int mnt_refcnt; /* ref count on this structure 
> > */
> > +   kmutex_t*mnt_renamelock;/* per-fs rename lock */
> > int mnt_flag;   /* flags */
> > int mnt_iflag;  /* internal flags */
> > int mnt_fs_bshift;  /* offset shift for lblkno */
> > int mnt_dev_bshift; /* shift for device sectors */
> > -   struct statvfs  mnt_stat;   /* cache of filesystem stats */
> > specificdata_reference
> > mnt_specdataref;/* subsystem specific data */
> > -   kmutex_tmnt_updating;   /* to serialize updates */
> > +   kmutex_t*mnt_updating;  /* to serialize updates */
> > const struct wapbl_ops
> > *mnt_wapbl_op;  /* logging ops */
> > struct wapbl*mnt_wapbl; /* log info */
> > struct wapbl_replay
> > *mnt_wapbl_replay;  /* replay support XXX: what? */
> > uint64_tmnt_gen;
> > +
> > +   /*
> > +* Volatile data: pad to keep away from the stable items.
> > +*/
> > +   int mnt_refcnt  /* ref count on this structure 
> > */
> > +   __aligned(COHERENCY_UNIT);
> > +   int mnt_synclist_slot;  /* synclist slot index */
> > +   TAILQ_HEAD(, vnode_impl) mnt_vnodelist; /* list of vnodes this mount */
> > +   struct statvfs  mnt_stat;   /* cache of filesystem stats */
> >  };
> >  
> >  #endif /* defined(_KERNEL) || defined(__EXPOSE_MOUNT) */
> > 
> 
> 
> 
> 





Re: CVS commit: src/share/man/man9

2020-02-23 Thread Andrew Doran
On Sun, Feb 23, 2020 at 08:57:44AM +, matthew green wrote:
> Module Name:  src
> Committed By: mrg
> Date: Sun Feb 23 08:57:44 UTC 2020
> 
> Modified Files:
>   src/share/man/man9: Makefile
> 
> Log Message:
> install rw_lock_op link too.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.447 -r1.448 src/share/man/man9/Makefile
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.

Oops, forgot to commit the Makefile.  Thank you.

Andrew


Re: CVS commit: src/sys/kern

2020-02-17 Thread Andrew Doran
Hi,

On Thu, Feb 06, 2020 at 06:33:55PM +0100, J. Hannken-Illjes wrote:

> > On 12. Jan 2020, at 18:49, Andrew Doran  wrote:
> > 
> > Module Name:src
> > Committed By:   ad
> > Date:   Sun Jan 12 17:49:17 UTC 2020
> > 
> > Modified Files:
> > src/sys/kern: vfs_vnode.c
> > 
> > Log Message:
> > vput(): don't drop the vnode lock, carry the hold over into vrelel() which
> > might need it anyway.
> > 
> > 
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.106 -r1.107 src/sys/kern/vfs_vnode.c
> > 
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> > 
> 
>  vput(vnode_t *vp)
>  {
> +   int lktype;
> 
> -   VOP_UNLOCK(vp);
> -   vrele(vp);
> +   if ((vp->v_vflag & VV_LOCKSWORK) == 0) {
> +   lktype = LK_EXCLUSIVE;
> +   } else {
> +   lktype = VOP_ISLOCKED(vp);
> +   KASSERT(lktype != LK_NONE);
> +   }
> +   mutex_enter(vp->v_interlock);
> +   vrelel(vp, 0, lktype);
>  }
> 
> This is quite wrong, from the manual:
> 
>  VOP_ISLOCKED(vp)
>   Test if the vnode vp is locked.  Possible return values are
>   LK_EXCLUSIVE, LK_SHARED or 0 for lock held exclusively by the
>   calling thread, shared lock held by anyone or unlocked,
>   respectively.
> 
>   This function must never be used to make locking decisions at
>   run time: it is provided only for diagnostic purposes.
> 
> I suppose you cannot determine if the current thread holds
> a shared lock.

The intention of that last sentence was to dissuade people from doing error
prone, complicated stuff with locks.  To my mind that idea of the locking
primitives is to take something difficult (concurrency) and package it up in
a way that's much easier to think about and work with - and yes it's still
complicated.

There are limited cases when I think it makes sense to infer lock ownership,
for example when known for sure that a RW lock is held but the type of hold
needs to be known - like this case.  The failure case there is the lock
being unheld, which would be caught by LOCKDEBUG in this case.  Consider
that a rw_exit() with the lock unheld by the caller will produce what you
might charitably call "undefined behaviour" in a non-LOCKDEBUG kernel.

Andrew


Re: CVS commit: src/sys/arch/powerpc/powerpc

2020-02-17 Thread Andrew Doran
On Wed, Feb 05, 2020 at 12:46:57PM +0900, Rin Okuyama wrote:

> Hi,
> 
> On 2019/12/06 5:55, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Thu Dec  5 20:55:24 UTC 2019
> > 
> > Modified Files:
> > src/sys/arch/powerpc/powerpc: powerpc_machdep.c
> > 
> > Log Message:
> > Need to call userret() from cpu_ast().
> > 
> > 
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.74 -r1.75 src/sys/arch/powerpc/powerpc/powerpc_machdep.c
> > 
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> > 
> > 
> > Modified files:
> > 
> > Index: src/sys/arch/powerpc/powerpc/powerpc_machdep.c
> > diff -u src/sys/arch/powerpc/powerpc/powerpc_machdep.c:1.74 
> > src/sys/arch/powerpc/powerpc/powerpc_machdep.c:1.75
> > --- src/sys/arch/powerpc/powerpc/powerpc_machdep.c:1.74 Sat Nov 23 
> > 19:40:36 2019
> > +++ src/sys/arch/powerpc/powerpc/powerpc_machdep.c  Thu Dec  5 20:55:24 2019
> > @@ -1,4 +1,4 @@
> > -/* $NetBSD: powerpc_machdep.c,v 1.74 2019/11/23 19:40:36 ad Exp $  */
> > +/* $NetBSD: powerpc_machdep.c,v 1.75 2019/12/05 20:55:24 ad Exp $  */
> >   /*
> >* Copyright (C) 1995, 1996 Wolfgang Solfrank.
> > @@ -32,7 +32,7 @@
> >*/
> >   #include 
> > -__KERNEL_RCSID(0, "$NetBSD: powerpc_machdep.c,v 1.74 2019/11/23 19:40:36 
> > ad Exp $");
> > +__KERNEL_RCSID(0, "$NetBSD: powerpc_machdep.c,v 1.75 2019/12/05 20:55:24 
> > ad Exp $");
> >   #include "opt_altivec.h"
> >   #include "opt_ddb.h"
> > @@ -373,6 +373,8 @@ void
> >   cpu_ast(struct lwp *l, struct cpu_info *ci)
> >   {
> > l->l_md.md_astpending = 0;  /* we are about to do it */
> > +   __insn_barrier();
> > +   userret(l, l->l_md.md_utf);
> > if (l->l_pflag & LP_OWEUPC) {
> > l->l_pflag &= ~LP_OWEUPC;
> > @@ -400,7 +402,7 @@ cpu_need_resched(struct cpu_info *ci, st
> > cpu_send_ipi(cpu_index(ci), IPI_AST);
> >   #endif
> > } else {
> > -   l->l_md.md_astpending = 1;  /* force call to ast() 
> > */
> > +   l->l_md.md_astpending = 1;  /* force call to cpu_ast() */
> > }
> >   }
> > 
> 
> This commit makes userret() called twice with AST; cpu_ast() is
> invoked from
> 
> booke/trap.c,
> https://nxr.netbsd.org/xref/src/sys/arch/powerpc/booke/trap.c#815
> 
> ibm4xx/trap.c, and
> https://nxr.netbsd.org/xref/src/sys/arch/powerpc/ibm4xx/trap.c#276
> 
> powerpc/trap.c.
> https://nxr.netbsd.org/xref/src/sys/arch/powerpc/powerpc/trap.c#348
> 
> For all cases, userret() is called afterward. (Precisely speaking,
> for ibm4xx, mi_userret(9) is used instead. This should probably be
> replaced by userret().)
> 
> Also, other ports test (l->l_pflag & LP_OWEUPC) before mi_userret(9).

I corrected the cpu_ast() case.  Yes it's curious why ibm4xx calls
mi_userret() directly; that seems wrong (I have not changed it though).  I
think it definitely makes more sense to deal with OWEUPC before userret().

Andrew
 
> Thanks,
> rin


Re: CVS commit: src/sys

2020-02-08 Thread Andrew Doran
On Thu, Feb 06, 2020 at 11:30:20PM +, Jason R Thorpe wrote:

> Modified Files:
>   src/sys/net: if.c if.h
>   src/sys/netinet: ip_carp.c
> 
> Log Message:
> Perform link state change processing on a work queue, rather than in a
> softint.

Thinking out loud more than anything else:

Bit concerning that we're growing a ton of kthreads in the network stack (my
test system now has something like 700 kthreads total) but I'm less worried
about that and moreso that a lot of these seem to be in the kernel RT range
and will therefore cause kernel preemptions when triggered.

For something like a link state change that's no biggie but I'm not fond of
the idea of bulk wire & protcol processing needing mi_switch() + kpreempt(). 
Way back when we made a concious decision not copy the design pattern our
sister project went with on this front.  Maybe it is the right way to go now
but I think that should also be a concious design decision.

Andrew


Re: CVS commit: src/lib/libpthread

2020-02-01 Thread Andrew Doran
Hmm.  Was there not originally an environment variable to control this
behaviour, since many applications are buggy?

Andrew

On Sun, Feb 02, 2020 at 01:01:49AM +0900, Ryo ONODERA wrote:
> Hi,
> 
> pthread__error()s in pthread_equal() cause segfault
> during start of pkgsrc/www/firefox-72.0.2.
> 
> Without pthread__error()s, www/firefox works fine
> like as follows.
> However I have no idea why I get segfaults.
> 
> Could you take a look at this problem?
> 
> Index: lib/libpthread/pthread.c
> ===
> RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
> retrieving revision 1.162
> diff -u -r1.162 pthread.c
> --- lib/libpthread/pthread.c  29 Jan 2020 17:11:57 -  1.162
> +++ lib/libpthread/pthread.c  1 Feb 2020 15:58:03 -
> @@ -770,11 +770,13 @@
>   if (__predict_false(__uselibcstub))
>   return __libc_thr_equal_stub(t1, t2);
>  
> +#if 0
>   pthread__error(EINVAL, "Invalid thread",
>   t1->pt_magic == PT_MAGIC);
>  
>   pthread__error(EINVAL, "Invalid thread",
>   t2->pt_magic == PT_MAGIC);
> +#endif
>  
>   /* Nothing special here. */
>   return (t1 == t2);
> @@ -1108,7 +1110,7 @@
>  {
>   char buf[1024];
>   size_t len;
> - 
> +
>   if (pthread__diagassert == 0)
>   return;
>  
> 
> 
> "Kamil Rytarowski"  writes:
> 
> > Module Name:src
> > Committed By:   kamil
> > Date:   Wed Jan 29 16:03:44 UTC 2020
> >
> > Modified Files:
> > src/lib/libpthread: pthread.c pthread_getcpuclockid.c
> >
> > Log Message:
> > Chack thread->pt_magic with PT_MAGIC promptly
> >
> > Rearrange some checks to avoid verifying pthread_t after using it.
> >
> >
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
> > cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_getcpuclockid.c
> >
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> >
> 
> -- 
> Ryo ONODERA // r...@tetera.org
> PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3


Re: CVS commit: src/common/lib/libc/arch

2020-02-01 Thread Andrew Doran
On Sat, Feb 01, 2020 at 03:02:02PM +, m...@netbsd.org wrote:
> On Mon, Jan 27, 2020 at 10:09:21PM +0000, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Mon Jan 27 22:09:21 UTC 2020
> > 
> > Removed Files:
> > src/common/lib/libc/arch/i386/string: memcmp.S
> > src/common/lib/libc/arch/x86_64/string: bcmp.S memcmp.S
> > 
> > Log Message:
> > x86 uses the C versions of bcmp() and memcmp() now.
> > 
> 
> Why?

REP CMPS is very slow on the modern CPUs I have access to.  The updated C
version is 1.5-10x faster in the configurations I have tried, from tiny
strings up to N-megabyte strings.  MOVS and STOS are very good though.

Andrew


Re: CVS commit: src/sys

2020-02-01 Thread Andrew Doran
On Sat, Feb 01, 2020 at 02:23:24AM +, Taylor R Campbell wrote:

> Module Name:  src
> Committed By: riastradh
> Date: Sat Feb  1 02:23:23 UTC 2020
> 
> Modified Files:
>   src/sys/ddb: db_xxx.c
>   src/sys/kern: kern_descrip.c kern_sig.c subr_exec_fd.c uipc_socket2.c
>   uipc_usrreq.c
> 
> Log Message:
> Load struct fdfile::ff_file with atomic_load_consume.
> 
> Exceptions: when we're only testing whether it's there, not about to
> dereference it.
> 
> Note: We do not use atomic_store_release to set it because the
> preceding mutex_exit should be enough.
> 
> (That said, it's not clear the mutex_enter/exit is needed unless
> refcnt > 0 already, in which case maybe it would be a win to switch
> from the membar implied by mutex_enter to the membar implied by
> atomic_store_release -- which I would generally expect to be much
> cheaper.  And a little clearer without a long comment.)

Likely procfs and sysctl since they go the strongly-locked route.

Andrew


Re: CVS commit: src/lib/libpthread

2020-01-31 Thread Andrew Doran
On Fri, Jan 31, 2020 at 06:55:00PM -, Christos Zoulas wrote:

> In article <724af477-010b-9ddf-6ece-e23d7cf59...@gmx.com>,
> Kamil Rytarowski   wrote:
> >-=-=-=-=-=-
> >-=-=-=-=-=-
> >
> >On 31.01.2020 03:38, Christos Zoulas wrote:
> >> And it is fixed now.
> >> 
> >> christos
> >> 
> >
> >OK. I am going to submit a bug report upstream and get some feedback
> >what is the way forward here, delaying initialization.
> 
> I think that the way forward (on our side) is to do away with libpthread,
> merge it with libc and kill all the stub nonsense.

Agreed.

pthread__init() does some expensive stuff like _lwp_ctl().  I think we can
safely & without hacks defer a lot of that till the first pthread_create().

Andrew


Re: CVS commit: src/sys/sys

2020-01-29 Thread Andrew Doran
On Wed, Jan 29, 2020 at 12:58:52AM +0700, Robert Elz wrote:

> Date:Tue, 28 Jan 2020 16:40:27 +
> From:    "Andrew Doran" 
> Message-ID:  <20200128164027.8558bf...@cvs.netbsd.org>
> 
>   | Log Message:
>   | Put pri_t back to an int.  It looks like there might be a sign extension
>   | issue somewhere but it's not worth the hassle trying to find it.
> 
> This changes the kernel internal ABI doesn't it?   It would have needed
> a kernel version bump when the reverted change was made, and now needs
> one it has been removed, doesn't it?

Yes it does.  I made one later yesterday.  My changes today need another but
Jason already bumped the version earlier in the day, and I think one a day
is enough!

Andrew


Re: CVS commit: src/sys/uvm

2020-01-22 Thread Andrew Doran
On Wed, Jan 22, 2020 at 10:08:16AM +1100, matthew green wrote:

> Andrew Doran writes:
> > I also recommend disabling ACPI idle, at least until it can be made less
> > aggressive by default.  It causes a significant slowdown.  It can be done
> > with detaching all acpicpu devices using "drvctl -d" on each.
> 
> this disables cpufreq support, doesn't it?  that seems like
> an unfortunate side effect..

I dunno, I assume so.  It seems like a tricky one to solve.

I think we could do with a heuristic that does a "fast idle" used when the
system is busy and a "low power idle" when things have gotten quiet.  And
the heuristic likely needs to be controllable with sysctl to suit
preferences (power consumption, performance).

We probably also want to take a CPU in low power idle out of
kcpuset_running, but that imposes its own penalty on x86 because it then
goes from using broadcast IPIs to sending directed IPIs to every CPU;
needs to be tried.

Also I don't know what to do about HyperThreading; it'd be nice if ACPI had
the smarts to handle it.

My problem with this one is I don't know what I'm missing here...  Maybe
time for a mail.

Andrew


Re: CVS commit: src/sys/uvm

2020-01-21 Thread Andrew Doran
On Fri, Jan 10, 2020 at 10:21:25PM +, Andrew Doran wrote:
> Hi Frank,
> 
> On Fri, Jan 10, 2020 at 01:10:02PM +0100, Frank Kardel wrote:
> 
> > Hi !
> > 
> > With this state of January 2nd we ran some tests for robustness and timing
> > with our database setup:
> > 
> > Machine:
> > 
> > Mainboard: S2600WFT
> > 
> > CPU: 2 x Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
> > 
> > machdep.spectre_v1.mitigated = 0
> > machdep.spectre_v2.hwmitigated = 1
> > machdep.spectre_v2.swmitigated = 1
> > machdep.spectre_v2.method = [GCC retpoline] + [Intel IBRS]
> > machdep.spectre_v4.mitigated = 0
> > machdep.spectre_v4.method = (none)
> > machdep.mds.mitigated = 0
> > machdep.mds.method = (none)
> > machdep.taa.mitigated = 0
> > machdep.taa.method = [MDS]
> > 
> > Memory:
> > 
> > hw.physmem64 = 549446447104
> > hw.usermem64 = 549438365696
> > 
> > This machine is/has been a challenge to NetBSD as it has 0.5Tb Memory and 32
> > cores.
> > 
> > Testcase is restoring a 1Tb Postgresql-11 database with varying degres of
> > Postresql pg_restore parallelism.
> > 
> > Why did we do the tests? The machine was installed with 8.99.24 as that
> > supported the memory setup.
> > 
> > The machine was not able to reliably copy with many db/restore processes and
> > large memory - see
> > 
> >  PR kern/54209: NetBSD 8 large memory performance extremely low
> >  PR kern/54210: NetBSD-8 processes presumably not exiting
> > 
> > for details.
> > 
> > With Andrew Doran's work on the vm system we restarted the tests.
> > 
> > The baseline is 8.99.24 from around  Sep  3 04:10:20 UTC 2018:
> > TEST 1
> > FRESH BOOT
> > time pg_restore -Upgsql -p5433 -Fd -d db -j5 20200103-db.dmpdir
> > 1826.599u 1752.878s 10:36:03.83 9.3%0+0k 397+0io 1789pf+0w
> > 
> > Higher levels of parallelism lead to a higher probability for catatonic
> > systems with increasing restore parallelism.
> > Trouble starts around -j8 and gets worse at higher levels.
> > 
> > TEST 2
> > 9.99.33 from around Fri Jan  3 16:14:02 CET 2020
> > FRESH BOOT
> > time pg_restore -Upgsql -p5433 -Fd -d db -j28 20200103-db.dmpdir
> > 2047.925u 1191.878s 14:24:15.23 6.2%0+0k 0+0io 5784pf+0w
> > 
> > This survived a -j28 run that was not possible with 8.99.24 - this a a big
> > step forward, but ~4h slower real time.
> > 
> > TEST 3
> > FRESH BOOT
> > 9.99.34 from around Mon Jan  6 14:43:01
> > time pg_restore -Upgsql -p5433 -Fd -d db -j28 20200103-db.dmpdir
> > 1816.348u 1792.530s 10:56:02.56 9.1%0+0k 395+0io 5620pf+0w
> > 
> > -j5 run to compare to 9.99.33 - big improvement in real run time though
> > system time went up.
> > 
> > TEST 4
> > State after TEST 3 run to compare to 8.99.24
> > time pg_restore -Upgsql -p5433 -Fd -d db -j5 20200103-db.dmpdir
> > 1706.548u 1748.623s 11:26:38.87 8.3%0+0k 0+0io 1420pf+0w
> > 
> > This ran faster that -j28 - probably due to less contention, but 50 min
> > slower that 8.99.24 after fresh boot.
> > 
> > TEST 5:
> > re-run TEST 4 with fresh boot for 8.99.24 comparison
> > time pg_restore -Upgsql -p5433 -Fd -d db -j5 20200103-db.dmpdir
> > 1710.665u 1611.083s 9:14:56.86 9.9% 0+0k 398+0io 1504pf+0w
> > 
> > better the 8.99.24 for real time.
> > 
> > There seems no big difference in system time between 8.99.24 and 9.99.34,
> > but a big improvement in robustness.
> > The lockups don't seem to happen any more and there are a fewer short term
> > system freezes and the systems remains
> > responsive with 9.99.34.
> > 
> > The big differences in real time are interesting but the cause for that may
> > not be easy to pinpoint. The database
> > runs on an nvme:
> > nvme0 at pci10 dev 0 function 0: Intel SSD DC P4500 (rev. 0x00)
> > nvme0: NVMe 1.2
> > nvme0: for admin queue interrupting at msix4 vec 0
> > nvme0: INTEL SSDPE2KX040T8, firmware VDV10131, serial ...
> > nvme0: for io queue 1 interrupting at msix4 vec 1 affinity to cpu0
> > [...]
> > nvme0: for io queue 32 interrupting at msix4 vec 32 affinity to cpu31
> > ld0 at nvme0 nsid 1
> > ld0: 3726 GB, 486401 cyl, 255 head, 63 sec, 512 bytes/sect x 7814037168
> > sectors
> > 
> > And we are seeing transfer rates up to 300Mb/s and up 80% busy on the
> > complex I/O (load) and CPU (build index) workload.
> > 
> > So in summary we a a big step forward in robustness.
> > 
> > Thanks to Andrew for t

Re: CVS commit: src/sys [freeze on boot]

2020-01-20 Thread Andrew Doran
Fix committed with sys/kern/kern_rwlock.c rev 1.62.  I didn't see the
problem as I am running with LOCKDEBUG.

Apologies for the disruption.

Andrew


Re: CVS commit: src/sys [freeze on boot]

2020-01-20 Thread Andrew Doran
On Mon, Jan 20, 2020 at 09:28:32AM -0800, Paul Goyette wrote:

> On Mon, 20 Jan 2020, Patrick Welche wrote:
> 
> > On Mon, Jan 20, 2020 at 12:51:00PM +0000, Andrew Doran wrote:
> > > This also happened the last time I touched rw_downgrade(), and I backed 
> > > out
> > > the change then, but both times I don't see the bug.  I have some 
> > > questions:
> > > 
> > > - Are you running DIAGNOSTIC and/or LOCKDEBUG?  I would be very interested
> > >   to see what happens with a LOCKDEBUG kernel here.
> 
> Hmmm, at least on x86, in the LOCKDEBUG case we don't use the assembler
> stubs;  we simply use the C versions.
> 
> On IRC/ICB, mlelstv has indicated there's something wrong in the stubs,
> but I don't see it.

Yup, I think it's the stubs choking on the RW_NODEBUG flag being set. 
Testing a change for that now.

Andrew

> 
> > One worked with the addition of LOCKDEBUG. The other didn't, but it seems
> > to be unrelated:
> 
> Yeah, that backtrace looks unrelated.
> 
> 
> ++--+---+
> | Paul Goyette   | PGP Key fingerprint: | E-mail addresses: |
> | (Retired)  | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com |
> | Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org   |
> ++--+---+


Re: CVS commit: src/sys [freeze on boot]

2020-01-20 Thread Andrew Doran
Thanks.  I can reproduce a hang on boot in qemu.  It's hanging starting
init, waiting on "needbuf".  Investigating now.

Andrew

On Mon, Jan 20, 2020 at 04:12:45PM +, Patrick Welche wrote:
> On Mon, Jan 20, 2020 at 12:51:00PM +, Andrew Doran wrote:
> > This also happened the last time I touched rw_downgrade(), and I backed out
> > the change then, but both times I don't see the bug.  I have some questions:
> > 
> > - Are you running DIAGNOSTIC and/or LOCKDEBUG?  I would be very interested
> >   to see what happens with a LOCKDEBUG kernel here.
> 
> One worked with the addition of LOCKDEBUG. The other didn't, but it seems
> to be unrelated:
> 
> db{0}> show panic
> Panic string: mutex_vector_enter,510: uninitialized lock 
> (lock=0xbd012366609
> 0, from=8033dc9d)
> bt
> breakpoint() at netbsd:breakpoint+0x5
> vpanic() at netbsd:vpanic+0x178
> snprintf() at netbsd:snprintf
> lockdebug_wantlock() at netbsd:lockdebug_wantlock+0x166
> mutex_enter() at netbsd:mutex_enter+0x37c
> ixgbe_getext() at netbsd:ixgbe_getext+0x1d
> ixgbe_jcl_freeall.isra.0() at netbsd:ixgbe_jcl_freeall.isra.0+0xd6
> ixgbe_jcl_destroy() at netbsd:ixgbe_jcl_destroy+0x14
> ixgbe_free_receive_structures() at netbsd:ixgbe_free_receive_structures+0x11b
> ixgbe_attach() at netbsd:ixgbe_attach+0x2b0a
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> pci_probe_device() at netbsd:pci_probe_device+0x586
> pci_enumerate_bus() at netbsd:pci_enumerate_bus+0x1b7
> pcirescan() at netbsd:pcirescan+0x4e
> pciattach() at netbsd:pciattach+0x186
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> ppbattach() at netbsd:ppbattach+0x1c5
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> pci_probe_device() at netbsd:pci_probe_device+0x586
> pci_enumerate_bus() at netbsd:pci_enumerate_bus+0x1b7
> pcirescan() at netbsd:pcirescan+0x4e
> pciattach() at netbsd:pciattach+0x186
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> ppbattach() at netbsd:ppbattach+0x1c5
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> pci_probe_device() at netbsd:pci_probe_device+0x586
> pci_enumerate_bus() at netbsd:pci_enumerate_bus+0x1b7
> pcirescan() at netbsd:pcirescan+0x4e
> pciattach() at netbsd:pciattach+0x186
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> ppbattach() at netbsd:ppbattach+0x1c5
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> pci_probe_device() at netbsd:pci_probe_device+0x586
> pci_enumerate_bus() at netbsd:pci_enumerate_bus+0x1b7
> pcirescan() at netbsd:pcirescan+0x4e
> pciattach() at netbsd:pciattach+0x186
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> config_found_sm_loc() at netbsd:config_found_sm_loc+0x4d
> mp_pci_scan() at netbsd:mp_pci_scan+0xa4
> amd64_mainbus_attach() at netbsd:amd64_mainbus_attach+0x237
> mainbus_attach() at netbsd:mainbus_attach+0x70
> config_attach_loc() at netbsd:config_attach_loc+0x1a8
> cpu_configure() at netbsd:cpu_configure+0x2b
> main() at netbsd:main+0x311
> 
> Cheers,
> 
> Patrick


Re: CVS commit: src/sys [freeze on boot]

2020-01-20 Thread Andrew Doran
Hi,

This also happened the last time I touched rw_downgrade(), and I backed out
the change then, but both times I don't see the bug.  I have some questions:

- Are you running DIAGNOSTIC and/or LOCKDEBUG?  I would be very interested
  to see what happens with a LOCKDEBUG kernel here.
- Do you have an ATI Radeon graphics chip?
- Are you using ZFS?

Thanks,
Andrew


On Mon, Jan 20, 2020 at 12:41:37PM +0900, Ryo ONODERA wrote:
> Hi,
> 
> After this commit, the kernel stalls just before root file system
> will be found on my NetBSD/amd64 laptop.
> 
> Reverting
> src/sys/kern/kern_rwlock.c to r1.60
> and
> src/sys/sys/rwlock.h to r1.12
> in latest -current tree and I can get the kernel that works like
> before.
> 
> And on another laptop, the problematic kernel stalls before root file
> system detection like my laptop.
> 
> It may be universal problem.
> 
> Could you take a look at this problem?
> 
> Thank you.
> 
> "Andrew Doran"  writes:
> 
> > Module Name:src
> > Committed By:   ad
> > Date:   Sun Jan 19 18:34:24 UTC 2020
> >
> > Modified Files:
> > src/sys/kern: kern_rwlock.c
> > src/sys/sys: rwlock.h
> >
> > Log Message:
> > Tidy rwlocks a bit, no functional change intended.  Mainly:
> >
> > - rw_downgrade(): do it in a for () loop like all the others.
> > - Explicitly carry around RW_NODEBUG - don't be lazy.
> > - Remove pointless macros.
> > - Don't make assertions conditional on LOCKDEBUG, there's no reason.
> > - Make space for a new flag bit (not added yet).
> >
> >
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.60 -r1.61 src/sys/kern/kern_rwlock.c
> > cvs rdiff -u -r1.12 -r1.13 src/sys/sys/rwlock.h
> >
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> >
> > Modified files:
> >
> > Index: src/sys/kern/kern_rwlock.c
> > diff -u src/sys/kern/kern_rwlock.c:1.60 src/sys/kern/kern_rwlock.c:1.61
> > --- src/sys/kern/kern_rwlock.c:1.60 Sun Jan 12 18:37:10 2020
> > +++ src/sys/kern/kern_rwlock.c  Sun Jan 19 18:34:24 2020
> > @@ -1,4 +1,4 @@
> > -/* $NetBSD: kern_rwlock.c,v 1.60 2020/01/12 18:37:10 ad Exp $  */
> > +/* $NetBSD: kern_rwlock.c,v 1.61 2020/01/19 18:34:24 ad Exp $  */
> >  
> >  /*-
> >   * Copyright (c) 2002, 2006, 2007, 2008, 2009, 2019, 2020
> > @@ -39,7 +39,9 @@
> >   */
> >  
> >  #include 
> > -__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.60 2020/01/12 18:37:10 ad 
> > Exp $");
> > +__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.61 2020/01/19 18:34:24 ad 
> > Exp $");
> > +
> > +#include "opt_lockdebug.h"
> >  
> >  #define__RWLOCK_PRIVATE
> >  
> > @@ -63,58 +65,32 @@ __KERNEL_RCSID(0, "$NetBSD: kern_rwlock.
> >   * LOCKDEBUG
> >   */
> >  
> > -#if defined(LOCKDEBUG)
> > -
> > -#defineRW_WANTLOCK(rw, op) 
> > \
> > -   LOCKDEBUG_WANTLOCK(RW_DEBUG_P(rw), (rw),\
> > -   (uintptr_t)__builtin_return_address(0), op == RW_READER);
> > -#defineRW_LOCKED(rw, op)   
> > \
> > -   LOCKDEBUG_LOCKED(RW_DEBUG_P(rw), (rw), NULL,\
> > -   (uintptr_t)__builtin_return_address(0), op == RW_READER);
> > -#defineRW_UNLOCKED(rw, op) 
> > \
> > -   LOCKDEBUG_UNLOCKED(RW_DEBUG_P(rw), (rw),\
> > -   (uintptr_t)__builtin_return_address(0), op == RW_READER);
> > -#defineRW_DASSERT(rw, cond)
> > \
> > -do {   
> > \
> > -   if (__predict_false(!(cond)))   \
> > -   rw_abort(__func__, __LINE__, rw, "assertion failed: " #cond);\
> > -} while (/* CONSTCOND */ 0);
> > -
> > -#else  /* LOCKDEBUG */
> > -
> > -#defineRW_WANTLOCK(rw, op) /* nothing */
> > -#defineRW_LOCKED(rw, op)   /* nothing */
> > -#defineRW_UNLOCKED(rw, op) /* nothing */
> > -#defineRW_DASSERT(rw, cond)/* nothing */
> > +#defineRW_DEBUG_P(rw)  (((rw)->rw_owner & RW_NODEBUG) == 0)
> >  
> > -#endif /* LOCKDEBUG */
> > +#defineRW_WANTLOCK(rw, op) \
> > +LOCKDEBUG_WANTLOCK(RW_DEBUG_P(rw), (rw), \
> > +(uintptr_t)__b

Re: CVS commit: src/common/lib/libc/arch/x86_64/string

2020-01-16 Thread Andrew Doran
Hi,

Change backed out.  Sorry about the disruption.

Andrew

On Thu, Jan 16, 2020 at 05:30:20PM +0900, Ryo ONODERA wrote:
> Hi,
> 
> pkgsrc/www/firefox and mail/notmuch are also
> broken after this commit.
> 
> 
> On January 16, 2020 5:23:47 PM GMT+09:00, Kamil Rytarowski  
> wrote:
> >On 15.01.2020 11:56, Andrew Doran wrote:
> >> Module Name:   src
> >> Committed By:  ad
> >> Date:  Wed Jan 15 10:56:49 UTC 2020
> >> 
> >> Modified Files:
> >>src/common/lib/libc/arch/x86_64/string: bcmp.S memcmp.S
> >> 
> >> Log Message:
> >> Rewrite bcmp() & memcmp() to not use REP CMPS.  Seems about 5-10x
> >faster for
> >> small strings on modern hardware.
> >> 
> >> 
> >> To generate a diff of this commit:
> >> cvs rdiff -u -r1.3 -r1.4
> >src/common/lib/libc/arch/x86_64/string/bcmp.S \
> >> src/common/lib/libc/arch/x86_64/string/memcmp.S
> >> 
> >> Please note that diffs are not public domain; they are subject to the
> >> copyright notices on the relevant files.
> >> 
> >
> >This exact change broke git:
> >
> >$ git clone g...@github.com:anphsw/memtest86.git
> >Cloning into 'memtest86'...
> >remote: Enumerating objects: 3, done.
> >remote: Counting objects: 100% (3/3), done.
> >remote: Compressing objects: 100% (3/3), done.
> >remote: Total 477 (delta 0), reused 1 (delta 0), pack-reused 474
> >Receiving objects: 100% (477/477), 312.55 KiB | 984.00 KiB/s, done.
> >Resolving deltas: 100% (290/290), done.
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >error: wrong index v2 file size in
> >/public/memtest86/.git/objects/pack/pack-3baed7d30e4536c1173b6c083d81de217e9c829a.idx
> >fatal: bad object 9da11d5d7eaa9f1a6a26c04c22b5422c886699a0
> >fatal: remote did not send all necessary objects
> >
> >Userland is now broken.
> >
> >Please fix.
> 
> -- 
> Ryo ONODERA // r...@tetera.org
> PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB FD1B F404 27FA C7D1 15F3


Re: CVS commit: src/sys

2020-01-13 Thread Andrew Doran
On Mon, Jan 13, 2020 at 06:54:33AM -0800, Jason Thorpe wrote:

> > On Jan 12, 2020, at 10:20 PM, Kamil Rytarowski  wrote:
> > 
> > While there, could we garbage collect unused defines from sys/param.h?
> > 
> > I'm thinking in particular about:
> 
> As long as we still have tsleep(9) and friends, we can't rid ourselves of 
> these historical defines.
> 
> Perhaps we should make a push to rid ourselves of those legacy interfaces?

Agree.  They could probably be wrapped in _KERNEL for now though.  Any user
program still relying could do with catching up.

Andrew


Re: CVS commit: src/sys/arch/x86

2020-01-12 Thread Andrew Doran
On Sun, Jan 12, 2020 at 08:25:27PM +0100, Joerg Sonnenberger wrote:
> On Sun, Jan 12, 2020 at 01:01:12PM +0000, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Sun Jan 12 13:01:12 UTC 2020
> > 
> > Modified Files:
> > src/sys/arch/x86/include: pmap.h pmap_pv.h
> > src/sys/arch/x86/x86: pmap.c vm_machdep.c x86_tlb.c
> > 
> > Log Message:
> > x86 pmap:
> > 
> > - It turns out that every page the pmap frees is necessarily zeroed.  Tell
> >   the VM system about this and use the pmap as a source of pre-zeroed pages.
> 
> Would it make sense to actually verify this under DEBUG? I fear the
> debugging we have to do if a chance ever violates this assumption.

Yup we already do exactly this!

Andrew


Re: CVS commit: src/sys/sys

2020-01-12 Thread Andrew Doran
On Sun, Jan 12, 2020 at 01:30:57PM +, Nick Hudson wrote:
> On 12/01/2020 13:19, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Sun Jan 12 13:19:32 UTC 2020
> > 
> > Modified Files:
> > src/sys/sys: param.h
> > 
> > Log Message:
> > Bump MIN_LWP_ALIGNMENT to 64.
> 
> Should aarch64/mips define MIN_LWP_ALIGNMENT as 128 as they have CPUs
> that have this cache line size?

Good point - fixed!

Cheers,
Andrew


Re: [x86 pmap changes] CVS commit: src/sys/arch

2020-01-11 Thread Andrew Doran
On Fri, Jan 10, 2020 at 06:56:29PM +, Andrew Doran wrote:

> On Thu, Jan 09, 2020 at 05:46:13PM +0100, Maxime Villard wrote:
> 
> > This FPU issue should be fixed in the latest nvmm_x86_vmx.c, we still have
> > STTS/CLTS (not needed but for debugging) as part of context switches, and
> > when overhauling the FPU code I overlooked that VMX needs special CR0_TS
> > care that SVM doesn't need.
> > 
> > Note that dropping STTS/CLTS would probably increase cswitch performance,
> > because updating cr0 is costly.
> > 
> > Having said that, I am still hitting a KASSERT related to pmap:
> > 
> > kernel diagnostic assertion "ptp->wire_count == 1" failed file
> > ".../x86/x86/pmap.c", line 1969
> > pmap_freepages
> > pmap_ept_free_ptp
> > pmap_ept_remove
> > pmap_remove
> > uvm_unmap_remove
> > uvmspace_free
> > nvmm_ioctl
> > sys_ioctl
> 
> Taking a look.  I think it's probably related to the changes I made to
> further defer freeing PTPs (ugly but necessary for yamt-pagecache).  I think
> I see a more elegant way to handle that.

Yes that's the problem. I have a fix and will commit soon.

Andrew


Re: CVS commit: src/sys/uvm

2020-01-10 Thread Andrew Doran
Hi Frank,

On Fri, Jan 10, 2020 at 01:10:02PM +0100, Frank Kardel wrote:

> Hi !
> 
> With this state of January 2nd we ran some tests for robustness and timing
> with our database setup:
> 
> Machine:
> 
> Mainboard: S2600WFT
> 
> CPU: 2 x Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
> 
> machdep.spectre_v1.mitigated = 0
> machdep.spectre_v2.hwmitigated = 1
> machdep.spectre_v2.swmitigated = 1
> machdep.spectre_v2.method = [GCC retpoline] + [Intel IBRS]
> machdep.spectre_v4.mitigated = 0
> machdep.spectre_v4.method = (none)
> machdep.mds.mitigated = 0
> machdep.mds.method = (none)
> machdep.taa.mitigated = 0
> machdep.taa.method = [MDS]
> 
> Memory:
> 
> hw.physmem64 = 549446447104
> hw.usermem64 = 549438365696
> 
> This machine is/has been a challenge to NetBSD as it has 0.5Tb Memory and 32
> cores.
> 
> Testcase is restoring a 1Tb Postgresql-11 database with varying degres of
> Postresql pg_restore parallelism.
> 
> Why did we do the tests? The machine was installed with 8.99.24 as that
> supported the memory setup.
> 
> The machine was not able to reliably copy with many db/restore processes and
> large memory - see
> 
>  PR kern/54209: NetBSD 8 large memory performance extremely low
>  PR kern/54210: NetBSD-8 processes presumably not exiting
> 
> for details.
> 
> With Andrew Doran's work on the vm system we restarted the tests.
> 
> The baseline is 8.99.24 from around  Sep  3 04:10:20 UTC 2018:
> TEST 1
> FRESH BOOT
> time pg_restore -Upgsql -p5433 -Fd -d db -j5 20200103-db.dmpdir
> 1826.599u 1752.878s 10:36:03.83 9.3%0+0k 397+0io 1789pf+0w
> 
> Higher levels of parallelism lead to a higher probability for catatonic
> systems with increasing restore parallelism.
> Trouble starts around -j8 and gets worse at higher levels.
> 
> TEST 2
> 9.99.33 from around Fri Jan  3 16:14:02 CET 2020
> FRESH BOOT
> time pg_restore -Upgsql -p5433 -Fd -d db -j28 20200103-db.dmpdir
> 2047.925u 1191.878s 14:24:15.23 6.2%0+0k 0+0io 5784pf+0w
> 
> This survived a -j28 run that was not possible with 8.99.24 - this a a big
> step forward, but ~4h slower real time.
> 
> TEST 3
> FRESH BOOT
> 9.99.34 from around Mon Jan  6 14:43:01
> time pg_restore -Upgsql -p5433 -Fd -d db -j28 20200103-db.dmpdir
> 1816.348u 1792.530s 10:56:02.56 9.1%0+0k 395+0io 5620pf+0w
> 
> -j5 run to compare to 9.99.33 - big improvement in real run time though
> system time went up.
> 
> TEST 4
> State after TEST 3 run to compare to 8.99.24
> time pg_restore -Upgsql -p5433 -Fd -d db -j5 20200103-db.dmpdir
> 1706.548u 1748.623s 11:26:38.87 8.3%0+0k 0+0io 1420pf+0w
> 
> This ran faster that -j28 - probably due to less contention, but 50 min
> slower that 8.99.24 after fresh boot.
> 
> TEST 5:
> re-run TEST 4 with fresh boot for 8.99.24 comparison
> time pg_restore -Upgsql -p5433 -Fd -d db -j5 20200103-db.dmpdir
> 1710.665u 1611.083s 9:14:56.86 9.9% 0+0k 398+0io 1504pf+0w
> 
> better the 8.99.24 for real time.
> 
> There seems no big difference in system time between 8.99.24 and 9.99.34,
> but a big improvement in robustness.
> The lockups don't seem to happen any more and there are a fewer short term
> system freezes and the systems remains
> responsive with 9.99.34.
> 
> The big differences in real time are interesting but the cause for that may
> not be easy to pinpoint. The database
> runs on an nvme:
> nvme0 at pci10 dev 0 function 0: Intel SSD DC P4500 (rev. 0x00)
> nvme0: NVMe 1.2
> nvme0: for admin queue interrupting at msix4 vec 0
> nvme0: INTEL SSDPE2KX040T8, firmware VDV10131, serial ...
> nvme0: for io queue 1 interrupting at msix4 vec 1 affinity to cpu0
> [...]
> nvme0: for io queue 32 interrupting at msix4 vec 32 affinity to cpu31
> ld0 at nvme0 nsid 1
> ld0: 3726 GB, 486401 cyl, 255 head, 63 sec, 512 bytes/sect x 7814037168
> sectors
> 
> And we are seeing transfer rates up to 300Mb/s and up 80% busy on the
> complex I/O (load) and CPU (build index) workload.
> 
> So in summary we a a big step forward in robustness.
> 
> Thanks to Andrew for the big improvements here.

Thank you for the detailed testing, and report.

Many of the changes to the VM system came from Takashi Yamamoto's
yamt-pagecache branch, so it's not all my work.

I'm glad to hear that this has worked well for you.  There are a couple of
things that, time permitting, I would like to get in place over the next few
weeks which should help a little with this workload (and then I am done, for
now).

The first is enabling Jaromir Dolecek's vm.ubc_direct by default, which may
help with such a high I/O rate.  There is a possible deadlock condition with
this that needs to be fixed first.

The second is pulling in efficient tracking of page clean/dirty status from
the yamt-pagecache branch.  This reduces the amount of work fsync() needs to
do, which should be of benefit to the DBMS.

Cheers,
Andrew


Re: [x86 pmap changes] CVS commit: src/sys/arch

2020-01-10 Thread Andrew Doran
On Thu, Jan 09, 2020 at 05:46:13PM +0100, Maxime Villard wrote:

> This FPU issue should be fixed in the latest nvmm_x86_vmx.c, we still have
> STTS/CLTS (not needed but for debugging) as part of context switches, and
> when overhauling the FPU code I overlooked that VMX needs special CR0_TS
> care that SVM doesn't need.
> 
> Note that dropping STTS/CLTS would probably increase cswitch performance,
> because updating cr0 is costly.
> 
> Having said that, I am still hitting a KASSERT related to pmap:
> 
>   kernel diagnostic assertion "ptp->wire_count == 1" failed file
>   ".../x86/x86/pmap.c", line 1969
>   pmap_freepages
>   pmap_ept_free_ptp
>   pmap_ept_remove
>   pmap_remove
>   uvm_unmap_remove
>   uvmspace_free
>   nvmm_ioctl
>   sys_ioctl

Taking a look.  I think it's probably related to the changes I made to
further defer freeing PTPs (ugly but necessary for yamt-pagecache).  I think
I see a more elegant way to handle that.

Andrew


Re: [x86 pmap changes] CVS commit: src/sys/arch

2020-01-08 Thread Andrew Doran
On Tue, Jan 07, 2020 at 09:39:22AM +0100, Maxime Villard wrote:

> > Module Name:src
> > Committed By:   ad
> > Date:   Sat Jan  4 22:49:20 UTC 2020
> > 
> > Modified Files:
> > src/sys/arch/x86/include: pmap.h pmap_pv.h
> > src/sys/arch/x86/x86: pmap.c
> > src/sys/arch/xen/x86: xen_pmap.c
> > 
> > Log Message:
> > x86 pmap improvements, reducing system time during a build by about 15% on
> > my test machine:
> 
> This breaks nvmm-intel. I have only given a quick glance, but this change
> already is wrong:
> 
> - old_pp->pp_attrs |= pmap_ept_to_pp_attrs(opte);
> + old_pp->pp_attrs |= pmap_pte_to_pp_attrs(opte);
> 
> This is an EPT function handling EPT PTEs, so "ept" was correct. Fixing
> this bug is not sufficient, so it seems that there are more bugs.
> 
> Reverting the whole change puts nvmm-intel back in a functional state.
> 
> You can test with this on an Intel CPU:
> 
>   # modload nvmm
>   # /usr/tests/lib/libnvmm/./h_mem_assist
> 
> This currently gives random crashes.

With a couple of typos fixed (PTE -> EPT, now checked in) I see the same FPU
DNA exception that Chavdar reports on current-users (in his case with a
kernel which doesn't have these pmap changes).  It's coming from:

vmx_vcpu_guest_fpu_leave() -> fpu_area_save() -> fxsave()

What I can tell you is that the fxsave area is definitely writable and
correctly aligned but beyond that I have no idea what's causing it.  Any
suggestions?

Cheers,
Andrew


Re: [x86 pmap changes] CVS commit: src/sys/arch

2020-01-07 Thread Andrew Doran
On Tue, Jan 07, 2020 at 09:39:22AM +0100, Maxime Villard wrote:

> You can test with this on an Intel CPU:
> 
>   # modload nvmm
>   # /usr/tests/lib/libnvmm/./h_mem_assist
> 
> This currently gives random crashes.

I'll dig into it.  There were also reports of NVMM failures on the lists but
unrelated to this set of changes.

Andrew


Re: [x86 pmap changes] CVS commit: src/sys/arch

2020-01-07 Thread Andrew Doran
On Tue, Jan 07, 2020 at 09:50:42AM +0100, Maxime Villard wrote:

> Also unrelated remark:
> 
> + kmutex_t pm_lock/* locks for pm_objs */
> + __aligned(64);  /* give lock own cache line */
> 
> x86 CPUs will soon have 128-byte cache lines, and to ease the upgrade,
> it is better to use COHERENCY_UNIT rather than hardcoded values.

I totally agree.  However in this instance it looked like it could leave me
in dependency hell.  I don't have much time left to get my changes in and
tested, so left it to come back to some other time.

Andrew


Re: Gallium build broken on evbarm...

2020-01-06 Thread Andrew Doran
On Mon, Jan 06, 2020 at 11:14:18AM +0100, Martin Husemann wrote:

> On Mon, Jan 06, 2020 at 10:09:25AM +, Martin Husemann wrote:
> > > Log Message:
> > > Build fix.  Add back inclusion of , which was previously
> > > included via .
> 
> Ah - easy one: the  include was in a
> 
> #if defined(_KERNEL) || defined(_KMEMUSER)
> 
> section - will fix.

Thanks!

Andrew


Re: CVS commit: src/sys/uvm

2019-12-24 Thread Andrew Doran
On Tue, Dec 24, 2019 at 03:22:54AM +, Taylor R Campbell wrote:

> > Module Name:src
> > Committed By:   ad
> > Date:   Sat Dec 21 14:41:44 UTC 2019
> > 
> > - Add inlines to set/get locator values in the unused lower bits of
> >   pg->phys_addr.  Begin by using it to cache the freelist index, because
> >   computing it is expensive and that shows up during profiling.  Discussed
> >   on tech-kern.
> 
> So I guess we won't be switching pg->phys_addr from paddr to pfn?
> Speaking of which, any plans for expanding to >32-bit (or >31-bit, if
> signed) pfns in the rest of uvm?

That's not part of my current plans for UVM, which right now extend only as
far as breaking the back of the performance problems with builds.  31 bit
pfns get us into the terabyte range I think.  I have a couple of thoughts
here, firstly SVR4 or Solaris has a pfn_t and pgcnt_t or something like that
and it would would be nice to have an analogue.  Kind of surprised we didn't
inherit something like that from Mach.  Secondly in the comments above those
functions:

 * None of this is set in stone; it can be adjusted as needed.

What I have put in there is only a means to an end, to get some extra fields
without putting too much pressure on old systems with small memory.

> Can you use __BITS/__SHIFTIN/__SHIFTOUT for this instead of magic hex
> masks?

Good suggestion, I'll make that change.

Cheers,
Andrew


Re: CVS commit: src/sys/sys

2019-12-22 Thread Andrew Doran
On Sat, Dec 21, 2019 at 05:23:23PM +, Alexander Nasonov wrote:

> Andrew Doran wrote:
> > Log Message:
> > NetBSD 9.99.28 - cpu_data & UVM changes.
> 
> Wow, you bump versions faster than I compile new releases. At this
> pace, we'll get to 9.99.99 in a month or two ;-)

There are quite a few people using modules and I don't want to screw them
over.  We got into the MP game properly with NetBSD 5.0 but we're lagging
badly and as someone else observed we've been in need of many of these
changes for 10 years now.

Andrew


Re: CVS commit: src/sys

2019-12-22 Thread Andrew Doran
Hi Joerg,

On Sun, Dec 22, 2019 at 01:27:44AM +0100, Joerg Sonnenberger wrote:
> On Fri, Dec 20, 2019 at 09:05:34PM +0000, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Fri Dec 20 21:05:34 UTC 2019
> > 
> > Modified Files:
> > src/sys/arch/aarch64/aarch64: cpu.c cpufunc.c
> > src/sys/arch/arm/arm32: arm32_boot.c cpu.c
> > src/sys/arch/macppc/macppc: cpu.c
> > src/sys/arch/mips/mips: cpu_subr.c
> > src/sys/arch/x86/x86: cpu.c cpu_topology.c identcpu.c
> > src/sys/kern: kern_cpu.c
> > src/sys/sys: cpu.h cpu_data.h sched.h
> > 
> > Log Message:
> > Some more CPU topology stuff:
> > 
> > - Use cegger@'s ACPI SRAT parsing code to figure out NUMA node ID for each
> >   CPU as it is attached.
> 
> My build system panics on boot with this due to zero sized allocations
> in acpisrat_refresh.

I missed this last night, oops.

Rev 1.7 of src/sys/dev/acpi/acpi_srat.c should fix.

Cheers,
Andrew


Re: CVS commit: src/sys/uvm

2019-12-21 Thread Andrew Doran
On Sat, Dec 21, 2019 at 03:08:18PM +0100, Christoph Badura wrote:

> On Sat, Dec 21, 2019 at 12:58:26PM +0000, Andrew Doran wrote:
> > Modified Files:
> > src/sys/uvm: uvm_extern.h uvm_page.c
> > Log Message:
> > Add uvm_free(): returns number of free pages in system.
> 
> Can you rename this to a more descriptive name? Say uvm_free_pages() or
> something.
> 
> Also, we traditionally use xxx_free() to actually free some object and not
> return statistics.  It would be nice to spare future readers the
> confusion.

Yes that's fair enough, I see your point.  May not be today but I'll come
back to it soon.

Andrew


Re: CVS commit: src/sys/kern

2019-12-10 Thread Andrew Doran
On Wed, Dec 11, 2019 at 09:06:33AM +1100, matthew green wrote:

> "Andrew Doran" writes:
> > Module Name:src
> > Committed By:   ad
> > Date:   Mon Dec  9 21:02:10 UTC 2019
> > 
> > Modified Files:
> > src/sys/kern: kern_rwlock.c
> > 
> > Log Message:
> > Expunge the panicstr checks, we don't need them.
> 
> can you explain why?

Sure, I have developed a bit of a feel for it after years off watching the
thing panic.  The checks to not go too hard on the assertions once panicstr
are set are pretty good in my experience - we don't want that to snowball.

The ones that disable locking were of some kind of use (at least to me) back
in 2007 before we had a decent LOCKDEBUG setup for the newlock2 primitives. 
So it sprang out of development requirements and an uncertainty about all
how this new synchronization stuff was going to behave in practice more than
a desire to do the right thing.

On a uniprocessor or dual core box back then the panicstr checks didn't
really seem to have many bad effects, but with more CPUs it often seems to
make the crash much worse than it needs to be and I keep bumping into the
effects of it.  Here's a craptacular example:

http://www.netbsd.org/~ad/

That's kind of amusing to look at and it's only my framebuffer memory but I
worry that it could just as well be inodes or mbufs or anything else that
belongs to the user, and I think that until the CPUs are locked up and
activity stopped, the thing needs to keep working as properly as it can.

> what sort of crash-time testing did you perform?

That the system can be debugged and reset cleanly.  If we've got code in DDB
that hangs up or crashdups don't work then that's something we should fix.
I've not run into any in a long time, they seem to get fixed.

Do you have a particular concern or something else in mind?

Cheers,
Andrew


Re: CVS commit: src/sys

2019-12-06 Thread Andrew Doran
This is experimental: we use l_cpu to mean a number of things and this
involves changing it to a different value than curcpu() in a running LWP,
which we have not done before.  I'll see about making it more robust.

Andrew

On Fri, Dec 06, 2019 at 09:36:11PM +, Andrew Doran wrote:

> Module Name:  src
> Committed By: ad
> Date: Fri Dec  6 21:36:11 UTC 2019
> 
> Modified Files:
>   src/sys/kern: kern_exec.c kern_exit.c kern_idle.c kern_lwp.c kern_sig.c
>   kern_sleepq.c kern_softint.c kern_synch.c
>   src/sys/sys: sched.h
> 
> Log Message:
> Make it possible to call mi_switch() and immediately switch to another CPU.
> This seems to take about 3us on my Intel system.  Two changes required:
> 
> - Have the caller to mi_switch() be responsible for calling spc_lock().
> - Avoid using l->l_cpu in mi_switch().
> 
> While here:
> 
> - Add a couple of calls to membar_enter()
> - Have the idle LWP set itself to LSIDL, to match softint_thread().
> - Remove unused return value from mi_switch().
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.484 -r1.485 src/sys/kern/kern_exec.c
> cvs rdiff -u -r1.277 -r1.278 src/sys/kern/kern_exit.c
> cvs rdiff -u -r1.27 -r1.28 src/sys/kern/kern_idle.c
> cvs rdiff -u -r1.216 -r1.217 src/sys/kern/kern_lwp.c
> cvs rdiff -u -r1.380 -r1.381 src/sys/kern/kern_sig.c
> cvs rdiff -u -r1.53 -r1.54 src/sys/kern/kern_sleepq.c
> cvs rdiff -u -r1.54 -r1.55 src/sys/kern/kern_softint.c
> cvs rdiff -u -r1.328 -r1.329 src/sys/kern/kern_synch.c
> cvs rdiff -u -r1.79 -r1.80 src/sys/sys/sched.h
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
> 


Re: CVS commit: src/sys/arch

2019-12-06 Thread Andrew Doran
On Sat, Dec 07, 2019 at 07:12:10AM +1100, matthew green wrote:
> > > > Why? I consider this totaly useless bloat, what was wrong with the 
> > > > boot.cfg
> > > > solution?
> > > 
> > > policy:  no default modules in the installation unless licenses
> > > issues force such, until module+kernel solution.
> > 
> > OK, but this is gone awry. The boot.cfg solution is great if anoyne
> > really needs this dmesg sugar, everyone else can go on using pcictl 
> > later instead.
> > 
> > So now the (reasonable) policy forces a (relatively harmless, as easily
> > overidable but still useless) change into bloat for everyone. Can we
> > just revert to the state before the initial boot.cfg change instead?
> 
> i'm not really a fan of this solution.
> 
> this code was removed from GENERIC back when ad was pushing
> hard on modules and reducing memory consumption using them,
> but we've got problems still to resolve about their safe
> usage.  (i really can't agree with your "harmless" line
> above.  modules are dangerous even with careful usage
> currently.)
> 
> so, this change is really about reverting a bad choice made
> 11 years ago, that still has not been properly resolved.
> we lost that functionality in netbsd 5.0 GENERIC with the
> idea it would be resolved "soon".  it's been a long, long
> "soon"...

Right.  And since then the kernel has doubled in size.  I don't like the
bloat, very frustrating, and I still think modules are the way to go.  But
whatever about the correct direction technically, people really seem to like
PCIVERBOSE so I'm throwing my hands up and saying that in retrospect, given
the way things have panned out, removing it was the wrong choice.

Andrew


Re: CVS commit: src/sys/arch/x86/x86

2019-12-03 Thread Andrew Doran
On Tue, Dec 03, 2019 at 01:14:14PM +0100, Kamil Rytarowski wrote:

> On 03.12.2019 12:50, Juergen Hannken-Illjes wrote:
> > Module Name:src
> > Committed By:   hannken
> > Date:   Tue Dec  3 11:50:45 UTC 2019
> > 
> > Modified Files:
> > src/sys/arch/x86/x86: x86_machdep.c
> > 
> > Log Message:
> > Make sure the assignment to "idepth" is done inside the loop to prevent
> > preemption between loop end and dereference of "l_cpu->ci_depth".
> > 
> > 
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.131 -r1.132 src/sys/arch/x86/x86/x86_machdep.c
> > 
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> > 
> > 
> > Modified files:
> > 
> > Index: src/sys/arch/x86/x86/x86_machdep.c
> > diff -u src/sys/arch/x86/x86/x86_machdep.c:1.131 
> > src/sys/arch/x86/x86/x86_machdep.c:1.132
> > --- src/sys/arch/x86/x86/x86_machdep.c:1.131Tue Dec  3 11:50:16 2019
> > +++ src/sys/arch/x86/x86/x86_machdep.c  Tue Dec  3 11:50:45 2019
> > @@ -1,4 +1,4 @@
> > -/* $NetBSD: x86_machdep.c,v 1.131 2019/12/03 11:50:16 hannken Exp $
> > */
> > +/* $NetBSD: x86_machdep.c,v 1.132 2019/12/03 11:50:45 hannken Exp $
> > */
> >  
> >  /*-
> >   * Copyright (c) 2002, 2006, 2007 YAMAMOTO Takashi,
> > @@ -31,7 +31,7 @@
> >   */
> >  
> >  #include 
> > -__KERNEL_RCSID(0, "$NetBSD: x86_machdep.c,v 1.131 2019/12/03 11:50:16 
> > hannken Exp $");
> > +__KERNEL_RCSID(0, "$NetBSD: x86_machdep.c,v 1.132 2019/12/03 11:50:45 
> > hannken Exp $");
> >  
> >  #include "opt_modular.h"
> >  #include "opt_physmem.h"
> > @@ -350,7 +350,7 @@ bool
> >  cpu_intr_p(void)
> >  {
> > uint64_t ncsw;
> > -   int idepth;
> > +   volatile int idepth;
> > lwp_t *l;
> >  
> > l = curlwp;
> > 
> 
> Thanks!
> 
> This looks like to be in need to be propagated to:
> src/sys/arch/arm/arm/arm_machdep.c, src/sys/arch/mips/mips/cpu_subr.c,
> src/sys/arch/sparc/sparc/intr.c, src/sys/arch/sparc64/sparc64/machdep.c,
> src/sys/arch/usermode/dev/cpu.c,

I will take care of that later today.

Andrew




Re: CVS commit: src/sys/uvm

2019-12-01 Thread Andrew Doran
Hi,

On Sun, Dec 01, 2019 at 08:19:09AM +, Maxime Villard wrote:

> Modified Files:
>   src/sys/uvm: uvm_fault.c
> 
> Log Message:
> Use atomic_{load,store}_relaxed() on global counters.

If you would be so kind, please don't do any more of the UVM counters.  I
have a patch to make these CPU local which will be going out for review in
the near future.

Cheers,
Andrew

<<< uvm_fault.c
uvm_stat_inc(UVM_STAT_FLTANGET);
===
atomic_store_relaxed(,
atomic_load_relaxed() + 1);
>>> 1.210


CVS commit: src/sys/dev/onewire

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 23:06:52 UTC 2019

Modified Files:
src/sys/dev/onewire: owtemp.c

Log Message:
Make owtemp reliable for me:

- Don't do the calculation if there is a CRC error.
- If we get any kind of error during a refresh, retry up to three times.
- Add event counters to report what's going on.


To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.19 src/sys/dev/onewire/owtemp.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/dev/onewire

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 23:06:52 UTC 2019

Modified Files:
src/sys/dev/onewire: owtemp.c

Log Message:
Make owtemp reliable for me:

- Don't do the calculation if there is a CRC error.
- If we get any kind of error during a refresh, retry up to three times.
- Add event counters to report what's going on.


To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.19 src/sys/dev/onewire/owtemp.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/dev/onewire/owtemp.c
diff -u src/sys/dev/onewire/owtemp.c:1.18 src/sys/dev/onewire/owtemp.c:1.19
--- src/sys/dev/onewire/owtemp.c:1.18	Fri Oct 25 16:25:14 2019
+++ src/sys/dev/onewire/owtemp.c	Sat Nov 30 23:06:52 2019
@@ -1,6 +1,35 @@
-/*	$NetBSD: owtemp.c,v 1.18 2019/10/25 16:25:14 martin Exp $	*/
+/*	$NetBSD: owtemp.c,v 1.19 2019/11/30 23:06:52 ad Exp $	*/
 /*	$OpenBSD: owtemp.c,v 1.1 2006/03/04 16:27:03 grange Exp $	*/
 
+/*-
+ * Copyright (c) 2019 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * This code is derived from software contributed to The NetBSD Foundation
+ * by Andrew Doran.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
 /*
  * Copyright (c) 2006 Alexander Yurchenko 
  *
@@ -22,7 +51,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: owtemp.c,v 1.18 2019/10/25 16:25:14 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: owtemp.c,v 1.19 2019/11/30 23:06:52 ad Exp $");
 
 #include 
 #include 
@@ -51,14 +80,20 @@ struct owtemp_softc {
 	uint32_t			(*sc_owtemp_decode)(const uint8_t *);
 
 	intsc_dying;
+
+	struct evcnt			sc_ev_update;
+	struct evcnt			sc_ev_rsterr;
+	struct evcnt			sc_ev_crcerr;
 };
 
 static int	owtemp_match(device_t, cfdata_t, void *);
 static void	owtemp_attach(device_t, device_t, void *);
 static int	owtemp_detach(device_t, int);
 static int	owtemp_activate(device_t, enum devact);
-
-static void	owtemp_update(void *);
+static bool	owtemp_update(struct owtemp_softc *sc, uint32_t *temp);
+static void	owtemp_refresh(struct sysmon_envsys *, envsys_data_t *);
+static uint32_t	owtemp_decode_ds18b20(const uint8_t *);
+static uint32_t	owtemp_decode_ds1920(const uint8_t *);
 
 CFATTACH_DECL_NEW(owtemp, sizeof(struct owtemp_softc),
 	owtemp_match, owtemp_attach, owtemp_detach, owtemp_activate);
@@ -71,10 +106,7 @@ static const struct onewire_matchfam owt
 	{ ONEWIRE_FAMILY_DS1822 },
 };
 
-static void	owtemp_refresh(struct sysmon_envsys *, envsys_data_t *);
-
-static uint32_t	owtemp_decode_ds18b20(const uint8_t *);
-static uint32_t	owtemp_decode_ds1920(const uint8_t *);
+int	owtemp_retries = 3;
 
 static int
 owtemp_match(device_t parent, cfdata_t match, void *aux)
@@ -110,6 +142,13 @@ owtemp_attach(device_t parent, device_t 
 		break;
 	}
 
+	evcnt_attach_dynamic(>sc_ev_update, EVCNT_TYPE_MISC, NULL,
+	   device_xname(self), "update");
+	evcnt_attach_dynamic(>sc_ev_rsterr, EVCNT_TYPE_MISC, NULL,
+	   device_xname(self), "reset error");
+	evcnt_attach_dynamic(>sc_ev_crcerr, EVCNT_TYPE_MISC, NULL,
+	   device_xname(self), "crc error");
+
 	sc->sc_sme = sysmon_envsys_create();
 
 	/* Initialize sensor */
@@ -144,6 +183,9 @@ owtemp_detach(device_t self, int flags)
 	struct owtemp_softc *sc = device_private(self);
 
 	sysmon_envsys_unregister(sc->sc_sme);
+	evcnt_detach(>sc_ev_rsterr);
+	evcnt_detach(>sc_ev_update);
+	evcnt_detach(>sc_ev_crcerr);
 
 	return 0;
 }
@@ -162,18 +204,12 @@ owtemp_activate(device_t self, enum deva
 	}
 }
 
-static void
-owtemp_update(void *arg)
+static bool
+owtemp_update(struct owtemp_softc *sc, uint32_t *temp)
 {
-	struct o

CVS commit: src/sys/dev

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 23:04:12 UTC 2019

Modified Files:
src/sys/dev/gpio: gpioow.c
src/sys/dev/onewire: onewire.c onewire_bitbang.c onewirevar.h

Log Message:
onewire:

- Re-do the signalling to be a little more forgiving and efficient.
- If bus reset fails during probe, try a second time.
- Spread out kernel threads for many busses to avoid thundering herd effect.


To generate a diff of this commit:
cvs rdiff -u -r1.15 -r1.16 src/sys/dev/gpio/gpioow.c
cvs rdiff -u -r1.17 -r1.18 src/sys/dev/onewire/onewire.c
cvs rdiff -u -r1.1 -r1.2 src/sys/dev/onewire/onewire_bitbang.c
cvs rdiff -u -r1.5 -r1.6 src/sys/dev/onewire/onewirevar.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/dev/gpio/gpioow.c
diff -u src/sys/dev/gpio/gpioow.c:1.15 src/sys/dev/gpio/gpioow.c:1.16
--- src/sys/dev/gpio/gpioow.c:1.15	Sat Oct 28 04:53:56 2017
+++ src/sys/dev/gpio/gpioow.c	Sat Nov 30 23:04:12 2019
@@ -1,4 +1,4 @@
-/* $NetBSD: gpioow.c,v 1.15 2017/10/28 04:53:56 riastradh Exp $ */
+/* $NetBSD: gpioow.c,v 1.16 2019/11/30 23:04:12 ad Exp $ */
 /*	$OpenBSD: gpioow.c,v 1.1 2006/03/04 16:27:03 grange Exp $	*/
 
 /*
@@ -18,7 +18,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: gpioow.c,v 1.15 2017/10/28 04:53:56 riastradh Exp $");
+__KERNEL_RCSID(0, "$NetBSD: gpioow.c,v 1.16 2019/11/30 23:04:12 ad Exp $");
 
 /*
  * 1-Wire bus bit-banging through GPIO pin.
@@ -57,7 +57,8 @@ int	gpioow_detach(device_t, int);
 int	gpioow_activate(device_t, enum devact);
 
 int	gpioow_ow_reset(void *);
-int	gpioow_ow_bit(void *, int);
+int	gpioow_ow_read_bit(void *);
+void	gpioow_ow_write_bit(void *, int);
 
 void	gpioow_bb_rx(void *);
 void	gpioow_bb_tx(void *);
@@ -143,7 +144,8 @@ gpioow_attach(device_t parent, device_t 
 	/* Attach 1-Wire bus */
 	sc->sc_ow_bus.bus_cookie = sc;
 	sc->sc_ow_bus.bus_reset = gpioow_ow_reset;
-	sc->sc_ow_bus.bus_bit = gpioow_ow_bit;
+	sc->sc_ow_bus.bus_read_bit = gpioow_ow_read_bit;
+	sc->sc_ow_bus.bus_write_bit = gpioow_ow_write_bit;
 
 	memset(, 0, sizeof(oba));
 	oba.oba_bus = >sc_ow_bus;
@@ -193,9 +195,15 @@ gpioow_ow_reset(void *arg)
 }
 
 int
-gpioow_ow_bit(void *arg, int value)
+gpioow_ow_read_bit(void *arg)
 {
-	return (onewire_bb_bit(_bbops, arg, value));
+	return (onewire_bb_read_bit(_bbops, arg));
+}
+
+void
+gpioow_ow_write_bit(void *arg, int value)
+{
+	onewire_bb_write_bit(_bbops, arg, value);
 }
 
 void

Index: src/sys/dev/onewire/onewire.c
diff -u src/sys/dev/onewire/onewire.c:1.17 src/sys/dev/onewire/onewire.c:1.18
--- src/sys/dev/onewire/onewire.c:1.17	Fri Oct 25 16:25:14 2019
+++ src/sys/dev/onewire/onewire.c	Sat Nov 30 23:04:12 2019
@@ -1,6 +1,35 @@
-/*	$NetBSD: onewire.c,v 1.17 2019/10/25 16:25:14 martin Exp $	*/
+/*	$NetBSD: onewire.c,v 1.18 2019/11/30 23:04:12 ad Exp $	*/
 /*	$OpenBSD: onewire.c,v 1.1 2006/03/04 16:27:03 grange Exp $	*/
 
+/*-
+ * Copyright (c) 2019 The NetBSD Foundation, Inc.
+ * All rights reserved.
+ *
+ * This code is derived from software contributed to The NetBSD Foundation
+ * by Andrew Doran.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
 /*
  * Copyright (c) 2006 Alexander Yurchenko 
  *
@@ -18,7 +47,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: onewire.c,v 1.17 2019/10/25 16:25:14 martin Exp $");
+__KERNEL_RCSID(0, "$NetBSD: onewire.c,v 1.18 2019/11/30 23:04:12 ad Exp $");
 
 /*
  * 1-Wire bus driver.
@@ -202,14 +231,25 @@ onewire_reset(void *arg)
 }
 
 int
-onewire_bit(void *arg, int value)
+onewire_read_bit(void *arg)
 {
 	struct onewire_softc *sc = arg;
 	struct onewire_bus *bus = sc-&

CVS commit: src/sys/dev

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 23:04:12 UTC 2019

Modified Files:
src/sys/dev/gpio: gpioow.c
src/sys/dev/onewire: onewire.c onewire_bitbang.c onewirevar.h

Log Message:
onewire:

- Re-do the signalling to be a little more forgiving and efficient.
- If bus reset fails during probe, try a second time.
- Spread out kernel threads for many busses to avoid thundering herd effect.


To generate a diff of this commit:
cvs rdiff -u -r1.15 -r1.16 src/sys/dev/gpio/gpioow.c
cvs rdiff -u -r1.17 -r1.18 src/sys/dev/onewire/onewire.c
cvs rdiff -u -r1.1 -r1.2 src/sys/dev/onewire/onewire_bitbang.c
cvs rdiff -u -r1.5 -r1.6 src/sys/dev/onewire/onewirevar.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/kern

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 20:45:49 UTC 2019

Modified Files:
src/sys/kern: tty_ptm.c

Log Message:
VOP_UNLOCK + vrele -> vput


To generate a diff of this commit:
cvs rdiff -u -r1.40 -r1.41 src/sys/kern/tty_ptm.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/kern

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 20:45:49 UTC 2019

Modified Files:
src/sys/kern: tty_ptm.c

Log Message:
VOP_UNLOCK + vrele -> vput


To generate a diff of this commit:
cvs rdiff -u -r1.40 -r1.41 src/sys/kern/tty_ptm.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/tty_ptm.c
diff -u src/sys/kern/tty_ptm.c:1.40 src/sys/kern/tty_ptm.c:1.41
--- src/sys/kern/tty_ptm.c:1.40	Fri Mar  1 11:06:57 2019
+++ src/sys/kern/tty_ptm.c	Sat Nov 30 20:45:49 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: tty_ptm.c,v 1.40 2019/03/01 11:06:57 pgoyette Exp $	*/
+/*	$NetBSD: tty_ptm.c,v 1.41 2019/11/30 20:45:49 ad Exp $	*/
 
 /*-
  * Copyright (c) 2004 The NetBSD Foundation, Inc.
@@ -27,7 +27,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: tty_ptm.c,v 1.40 2019/03/01 11:06:57 pgoyette Exp $");
+__KERNEL_RCSID(0, "$NetBSD: tty_ptm.c,v 1.41 2019/11/30 20:45:49 ad Exp $");
 
 #ifdef _KERNEL_OPT
 #include "opt_compat_netbsd.h"
@@ -234,8 +234,7 @@ pty_grant_slave(struct lwp *l, dev_t dev
 		error = VOP_SETATTR(vp, , lwp0.l_cred);
 		if (error) {
 			DPRINTF(("setattr %d\n", error));
-			VOP_UNLOCK(vp);
-			vrele(vp);
+			vput(vp);
 			return error;
 		}
 	}



CVS commit: src/sys/sys

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 17:49:03 UTC 2019

Modified Files:
src/sys/sys: userret.h

Log Message:
Avoid false sharing: only update spc_curpriority if value has changed.


To generate a diff of this commit:
cvs rdiff -u -r1.30 -r1.31 src/sys/sys/userret.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/sys/userret.h
diff -u src/sys/sys/userret.h:1.30 src/sys/sys/userret.h:1.31
--- src/sys/sys/userret.h:1.30	Fri Nov 22 23:38:15 2019
+++ src/sys/sys/userret.h	Sat Nov 30 17:49:03 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: userret.h,v 1.30 2019/11/22 23:38:15 ad Exp $	*/
+/*	$NetBSD: userret.h,v 1.31 2019/11/30 17:49:03 ad Exp $	*/
 
 /*-
  * Copyright (c) 1998, 2000, 2003, 2006, 2008, 2019 The NetBSD Foundation, Inc.
@@ -100,14 +100,20 @@ mi_userret(struct lwp *l)
 		KPREEMPT_DISABLE(l);
 		ci = l->l_cpu;
 	}
-	l->l_kpriority = false;
 	/*
 	 * lwp_eprio() is too involved to use here unlocked.  At this point
 	 * it only matters for PTHREAD_PRIO_PROTECT; setting a too low value
 	 * is OK because the scheduler will find out the true value if we
 	 * end up in mi_switch().
+	 *
+	 * This is being called on every syscall and trap, and remote CPUs
+	 * regularly look at ci_schedstate.  Keep the cache line in the
+	 * SHARED state by only updating spc_curpriority if it has changed.
 	 */
-	ci->ci_schedstate.spc_curpriority = l->l_priority;
+	l->l_kpriority = false;
+	if (ci->ci_schedstate.spc_curpriority != l->l_priority) {
+		ci->ci_schedstate.spc_curpriority = l->l_priority;
+	}
 	KPREEMPT_ENABLE(l);
 
 	LOCKDEBUG_BARRIER(NULL, 0);



CVS commit: src/sys/sys

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 17:49:03 UTC 2019

Modified Files:
src/sys/sys: userret.h

Log Message:
Avoid false sharing: only update spc_curpriority if value has changed.


To generate a diff of this commit:
cvs rdiff -u -r1.30 -r1.31 src/sys/sys/userret.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/sys

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 17:46:27 UTC 2019

Modified Files:
src/sys/sys: sched.h

Log Message:
Mark spc_curpriority volatile.


To generate a diff of this commit:
cvs rdiff -u -r1.77 -r1.78 src/sys/sys/sched.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/sys

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 17:46:27 UTC 2019

Modified Files:
src/sys/sys: sched.h

Log Message:
Mark spc_curpriority volatile.


To generate a diff of this commit:
cvs rdiff -u -r1.77 -r1.78 src/sys/sys/sched.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/sys/sched.h
diff -u src/sys/sys/sched.h:1.77 src/sys/sys/sched.h:1.78
--- src/sys/sys/sched.h:1.77	Sat Nov 23 19:42:52 2019
+++ src/sys/sys/sched.h	Sat Nov 30 17:46:27 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: sched.h,v 1.77 2019/11/23 19:42:52 ad Exp $	*/
+/*	$NetBSD: sched.h,v 1.78 2019/11/30 17:46:27 ad Exp $	*/
 
 /*-
  * Copyright (c) 1999, 2000, 2001, 2002, 2007, 2008, 2019
@@ -154,16 +154,13 @@ __END_DECLS
  * c:	cpu_lock
  */
 struct schedstate_percpu {
-	/* First set of data is likely to be accessed by other CPUs. */
 	kmutex_t	*spc_mutex;	/* (: lock on below, runnable LWPs */
 	kmutex_t	*spc_lwplock;	/* (: general purpose lock for LWPs */
 	struct lwp	*spc_migrating;	/* (: migrating LWP */
-	pri_t		spc_curpriority;/* m: usrpri of curlwp */
+	volatile pri_t	spc_curpriority;/* m: usrpri of curlwp */
 	pri_t		spc_maxpriority;/* m: highest priority queued */
 	psetid_t	spc_psid;	/* c: processor-set ID */
 	time_t		spc_lastmod;	/* c: time of last cpu state change */
-
-	/* For the most part, this set of data is CPU-private. */
 	void		*spc_sched_info;/* (: scheduler-specific structure */
 	volatile int	spc_flags;	/* s: flags; see below */
 	u_int		spc_schedticks;	/* s: ticks for schedclock() */



CVS commit: src/sys/sys

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 17:45:54 UTC 2019

Modified Files:
src/sys/sys: lwp.h

Log Message:
Mark the context switch counters volatile (because preemption).


To generate a diff of this commit:
cvs rdiff -u -r1.190 -r1.191 src/sys/sys/lwp.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/sys/lwp.h
diff -u src/sys/sys/lwp.h:1.190 src/sys/sys/lwp.h:1.191
--- src/sys/sys/lwp.h:1.190	Sat Nov 23 19:42:52 2019
+++ src/sys/sys/lwp.h	Sat Nov 30 17:45:54 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: lwp.h,v 1.190 2019/11/23 19:42:52 ad Exp $	*/
+/*	$NetBSD: lwp.h,v 1.191 2019/11/30 17:45:54 ad Exp $	*/
 
 /*
  * Copyright (c) 2001, 2006, 2007, 2008, 2009, 2010, 2019
@@ -112,8 +112,8 @@ struct lwp {
 	pri_t		l_auxprio;	/* l: max(inherit,protect) priority */
 	int		l_protectdepth;	/* l: for PTHREAD_PRIO_PROTECT */
 	SLIST_HEAD(, turnstile) l_pi_lenders; /* l: ts lending us priority */
-	uint64_t	l_ncsw;		/* l: total context switches */
-	uint64_t	l_nivcsw;	/* l: involuntary context switches */
+	volatile uint64_t l_ncsw;	/* l: total context switches */
+	volatile uint64_t l_nivcsw;	/* l: involuntary context switches */
 	u_int		l_cpticks;	/* (: Ticks of CPU time */
 	fixpt_t		l_pctcpu;	/* p: %cpu during l_swtime */
 	fixpt_t		l_estcpu;	/* l: cpu time for SCHED_4BSD */



CVS commit: src/sys/sys

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 17:45:54 UTC 2019

Modified Files:
src/sys/sys: lwp.h

Log Message:
Mark the context switch counters volatile (because preemption).


To generate a diff of this commit:
cvs rdiff -u -r1.190 -r1.191 src/sys/sys/lwp.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch/sh3

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 15:53:36 UTC 2019

Modified Files:
src/sys/arch/sh3/include: userret.h
src/sys/arch/sh3/sh3: exception.c

Log Message:
Revert previous.  Looks like it requires a more extensive fix.


To generate a diff of this commit:
cvs rdiff -u -r1.15 -r1.16 src/sys/arch/sh3/include/userret.h
cvs rdiff -u -r1.69 -r1.70 src/sys/arch/sh3/sh3/exception.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/arch/sh3/include/userret.h
diff -u src/sys/arch/sh3/include/userret.h:1.15 src/sys/arch/sh3/include/userret.h:1.16
--- src/sys/arch/sh3/include/userret.h:1.15	Fri Nov 29 18:27:32 2019
+++ src/sys/arch/sh3/include/userret.h	Sat Nov 30 15:53:36 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: userret.h,v 1.15 2019/11/29 18:27:32 ad Exp $	*/
+/*	$NetBSD: userret.h,v 1.16 2019/11/30 15:53:36 ad Exp $	*/
 
 /*
  * Copyright (c) 1988 University of Utah.
@@ -52,16 +52,7 @@ userret(struct lwp *l)
 {
 
 	/* Invoke MI userret code */
-	do {
-		//curcpu()->ci_data.cpu_nast++;
-		l->l_md.md_astpending = 0;
-		mi_userret(l);
-	} while (l->l_md.md_astpending);
-
-	if (l->l_pflag & LP_OWEUPC) {
-		l->l_pflag &= ~LP_OWEUPC;
-		ADDUPROF(l);
-	}
+	mi_userret(l);
 
 #ifdef PTRACE_HOOKS
 	/* Check if lwp is being PT_STEP'ed */

Index: src/sys/arch/sh3/sh3/exception.c
diff -u src/sys/arch/sh3/sh3/exception.c:1.69 src/sys/arch/sh3/sh3/exception.c:1.70
--- src/sys/arch/sh3/sh3/exception.c:1.69	Fri Nov 29 18:27:33 2019
+++ src/sys/arch/sh3/sh3/exception.c	Sat Nov 30 15:53:36 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: exception.c,v 1.69 2019/11/29 18:27:33 ad Exp $	*/
+/*	$NetBSD: exception.c,v 1.70 2019/11/30 15:53:36 ad Exp $	*/
 
 /*-
  * Copyright (c) 2002 The NetBSD Foundation, Inc. All rights reserved.
@@ -79,7 +79,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: exception.c,v 1.69 2019/11/29 18:27:33 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: exception.c,v 1.70 2019/11/30 15:53:36 ad Exp $");
 
 #include "opt_ddb.h"
 #include "opt_kgdb.h"
@@ -471,5 +471,15 @@ ast(struct lwp *l, struct trapframe *tf)
 	KDASSERT(l != NULL);
 	KDASSERT(l->l_md.md_regs == tf);
 
-	userret(l);
+	while (l->l_md.md_astpending) {
+		//curcpu()->ci_data.cpu_nast++;
+		l->l_md.md_astpending = 0;
+
+		if (l->l_pflag & LP_OWEUPC) {
+			l->l_pflag &= ~LP_OWEUPC;
+			ADDUPROF(l);
+		}
+
+		userret(l);
+	}
 }



CVS commit: src/sys/arch/sh3

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 15:53:36 UTC 2019

Modified Files:
src/sys/arch/sh3/include: userret.h
src/sys/arch/sh3/sh3: exception.c

Log Message:
Revert previous.  Looks like it requires a more extensive fix.


To generate a diff of this commit:
cvs rdiff -u -r1.15 -r1.16 src/sys/arch/sh3/include/userret.h
cvs rdiff -u -r1.69 -r1.70 src/sys/arch/sh3/sh3/exception.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



Re: CVS commit: src/sys/kern

2019-11-30 Thread Andrew Doran
Hi,

On Sat, Nov 30, 2019 at 04:52:38PM +0900, Rin Okuyama wrote:
> On 2019/11/30 5:50, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Fri Nov 29 20:50:54 UTC 2019
> > 
> > Modified Files:
> > src/sys/kern: kern_rwlock.c
> > 
> > Log Message:
> > A couple more tweaks to avoid reading the lock word.
> > 
> > 
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.56 -r1.57 src/sys/kern/kern_rwlock.c
> > 
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> 
> Hi,
> 
> After this commit, GENERIC64 kernel on evbarm-aarch64 does no longer
> boot multiuser anymore with rwlock related panics:
> 
>   panic: kernel diagnostic assertion "vm_map_locked_p(map)" failed: file 
> "../../../../uvm/uvm_map.c", line 1315
> 
> or
> 
>   panic: kernel diagnostic assertion "rw_write_held(>lock)" failed: 
> file "../../../../uvm/uvm_map.c", line 701
> 
> By reverting kern_rwlock.c to rev 1.56, panics disappear as far as
> I can see.

Hmm, it works fine on amd64 and looks OK but me, but I have backed it out
for the time being.

Thank you,
Andrew


CVS commit: src/sys/kern

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 14:21:16 UTC 2019

Modified Files:
src/sys/kern: kern_rwlock.c

Log Message:
Back out previous.  It works on amd64 under stress test but not
evbarm-aarch64 for some reason.  Will revisit.


To generate a diff of this commit:
cvs rdiff -u -r1.57 -r1.58 src/sys/kern/kern_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/kern

2019-11-30 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sat Nov 30 14:21:16 UTC 2019

Modified Files:
src/sys/kern: kern_rwlock.c

Log Message:
Back out previous.  It works on amd64 under stress test but not
evbarm-aarch64 for some reason.  Will revisit.


To generate a diff of this commit:
cvs rdiff -u -r1.57 -r1.58 src/sys/kern/kern_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/kern_rwlock.c
diff -u src/sys/kern/kern_rwlock.c:1.57 src/sys/kern/kern_rwlock.c:1.58
--- src/sys/kern/kern_rwlock.c:1.57	Fri Nov 29 20:50:54 2019
+++ src/sys/kern/kern_rwlock.c	Sat Nov 30 14:21:16 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: kern_rwlock.c,v 1.57 2019/11/29 20:50:54 ad Exp $	*/
+/*	$NetBSD: kern_rwlock.c,v 1.58 2019/11/30 14:21:16 ad Exp $	*/
 
 /*-
  * Copyright (c) 2002, 2006, 2007, 2008, 2009, 2019 The NetBSD Foundation, Inc.
@@ -38,7 +38,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.57 2019/11/29 20:50:54 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.58 2019/11/30 14:21:16 ad Exp $");
 
 #define	__RWLOCK_PRIVATE
 
@@ -417,11 +417,10 @@ rw_vector_enter(krwlock_t *rw, const krw
 		 * No need for a memory barrier because of context switch.
 		 * If not handed the lock, then spin again.
 		 */
-		if (op == RW_READER)
+		if (op == RW_READER || (rw->rw_owner & RW_THREAD) == curthread)
 			break;
+
 		owner = rw->rw_owner;
-		if ((owner & RW_THREAD) == curthread)
-			break;
 	}
 	KPREEMPT_ENABLE(curlwp);
 
@@ -477,13 +476,14 @@ rw_vector_exit(krwlock_t *rw)
 	 * lock would become unowned.
 	 */
 	RW_MEMBAR_EXIT();
-	for (;; owner = next) {
+	for (;;) {
 		newown = (owner - decr);
 		if ((newown & (RW_THREAD | RW_HAS_WAITERS)) == RW_HAS_WAITERS)
 			break;
 		next = rw_cas(rw, owner, newown);
 		if (__predict_true(next == owner))
 			return;
+		owner = next;
 	}
 
 	/*
@@ -568,15 +568,15 @@ rw_vector_tryenter(krwlock_t *rw, const 
 		need_wait = RW_WRITE_LOCKED | RW_THREAD;
 	}
 
-	for (owner = 0;; owner = next) {
+	for (owner = rw->rw_owner;; owner = next) {
+		if (__predict_false((owner & need_wait) != 0))
+			return 0;
 		next = rw_cas(rw, owner, owner + incr);
 		if (__predict_true(next == owner)) {
 			/* Got it! */
 			RW_MEMBAR_ENTER();
 			break;
 		}
-		if (__predict_false((owner & need_wait) != 0))
-			return 0;
 	}
 
 	RW_WANTLOCK(rw, op);



CVS commit: src/sys/kern

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 20:50:54 UTC 2019

Modified Files:
src/sys/kern: kern_rwlock.c

Log Message:
A couple more tweaks to avoid reading the lock word.


To generate a diff of this commit:
cvs rdiff -u -r1.56 -r1.57 src/sys/kern/kern_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/kern_rwlock.c
diff -u src/sys/kern/kern_rwlock.c:1.56 src/sys/kern/kern_rwlock.c:1.57
--- src/sys/kern/kern_rwlock.c:1.56	Fri Nov 29 20:04:54 2019
+++ src/sys/kern/kern_rwlock.c	Fri Nov 29 20:50:54 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: kern_rwlock.c,v 1.56 2019/11/29 20:04:54 riastradh Exp $	*/
+/*	$NetBSD: kern_rwlock.c,v 1.57 2019/11/29 20:50:54 ad Exp $	*/
 
 /*-
  * Copyright (c) 2002, 2006, 2007, 2008, 2009, 2019 The NetBSD Foundation, Inc.
@@ -38,7 +38,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.56 2019/11/29 20:04:54 riastradh Exp $");
+__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.57 2019/11/29 20:50:54 ad Exp $");
 
 #define	__RWLOCK_PRIVATE
 
@@ -417,10 +417,11 @@ rw_vector_enter(krwlock_t *rw, const krw
 		 * No need for a memory barrier because of context switch.
 		 * If not handed the lock, then spin again.
 		 */
-		if (op == RW_READER || (rw->rw_owner & RW_THREAD) == curthread)
+		if (op == RW_READER)
 			break;
-
 		owner = rw->rw_owner;
+		if ((owner & RW_THREAD) == curthread)
+			break;
 	}
 	KPREEMPT_ENABLE(curlwp);
 
@@ -476,14 +477,13 @@ rw_vector_exit(krwlock_t *rw)
 	 * lock would become unowned.
 	 */
 	RW_MEMBAR_EXIT();
-	for (;;) {
+	for (;; owner = next) {
 		newown = (owner - decr);
 		if ((newown & (RW_THREAD | RW_HAS_WAITERS)) == RW_HAS_WAITERS)
 			break;
 		next = rw_cas(rw, owner, newown);
 		if (__predict_true(next == owner))
 			return;
-		owner = next;
 	}
 
 	/*
@@ -568,15 +568,15 @@ rw_vector_tryenter(krwlock_t *rw, const 
 		need_wait = RW_WRITE_LOCKED | RW_THREAD;
 	}
 
-	for (owner = rw->rw_owner;; owner = next) {
-		if (__predict_false((owner & need_wait) != 0))
-			return 0;
+	for (owner = 0;; owner = next) {
 		next = rw_cas(rw, owner, owner + incr);
 		if (__predict_true(next == owner)) {
 			/* Got it! */
 			RW_MEMBAR_ENTER();
 			break;
 		}
+		if (__predict_false((owner & need_wait) != 0))
+			return 0;
 	}
 
 	RW_WANTLOCK(rw, op);



CVS commit: src/sys/kern

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 20:50:54 UTC 2019

Modified Files:
src/sys/kern: kern_rwlock.c

Log Message:
A couple more tweaks to avoid reading the lock word.


To generate a diff of this commit:
cvs rdiff -u -r1.56 -r1.57 src/sys/kern/kern_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/kern

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 19:44:59 UTC 2019

Modified Files:
src/sys/kern: kern_mutex.c

Log Message:
Get rid of MUTEX_RECEIVE/MUTEX_GIVE.


To generate a diff of this commit:
cvs rdiff -u -r1.79 -r1.80 src/sys/kern/kern_mutex.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/kern

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 19:44:59 UTC 2019

Modified Files:
src/sys/kern: kern_mutex.c

Log Message:
Get rid of MUTEX_RECEIVE/MUTEX_GIVE.


To generate a diff of this commit:
cvs rdiff -u -r1.79 -r1.80 src/sys/kern/kern_mutex.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/kern_mutex.c
diff -u src/sys/kern/kern_mutex.c:1.79 src/sys/kern/kern_mutex.c:1.80
--- src/sys/kern/kern_mutex.c:1.79	Thu May  9 05:00:31 2019
+++ src/sys/kern/kern_mutex.c	Fri Nov 29 19:44:59 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: kern_mutex.c,v 1.79 2019/05/09 05:00:31 ozaki-r Exp $	*/
+/*	$NetBSD: kern_mutex.c,v 1.80 2019/11/29 19:44:59 ad Exp $	*/
 
 /*-
  * Copyright (c) 2002, 2006, 2007, 2008 The NetBSD Foundation, Inc.
@@ -40,7 +40,7 @@
 #define	__MUTEX_PRIVATE
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: kern_mutex.c,v 1.79 2019/05/09 05:00:31 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: kern_mutex.c,v 1.80 2019/11/29 19:44:59 ad Exp $");
 
 #include 
 #include 
@@ -168,6 +168,17 @@ do {	\
 } while (/* CONSTCOND */ 0)
 
 /*
+ * Memory barriers.
+ */
+#ifdef __HAVE_ATOMIC_AS_MEMBAR
+#define	MUTEX_MEMBAR_ENTER()
+#define	MUTEX_MEMBAR_EXIT()
+#else
+#define	MUTEX_MEMBAR_ENTER()		membar_enter()
+#define	MUTEX_MEMBAR_EXIT()		membar_exit()
+#endif
+
+/*
  * For architectures that provide 'simple' mutexes: they provide a
  * CAS function that is either MP-safe, or does not need to be MP
  * safe.  Adaptive mutexes on these architectures do not require an
@@ -225,7 +236,7 @@ MUTEX_ACQUIRE(kmutex_t *mtx, uintptr_t c
 	MUTEX_INHERITDEBUG(oldown, mtx->mtx_owner);
 	MUTEX_INHERITDEBUG(newown, oldown);
 	rv = MUTEX_CAS(>mtx_owner, oldown, newown);
-	MUTEX_RECEIVE(mtx);
+	MUTEX_MEMBAR_ENTER();
 	return rv;
 }
 
@@ -234,7 +245,7 @@ MUTEX_SET_WAITERS(kmutex_t *mtx, uintptr
 {
 	int rv;
 	rv = MUTEX_CAS(>mtx_owner, owner, owner | MUTEX_BIT_WAITERS);
-	MUTEX_RECEIVE(mtx);
+	MUTEX_MEMBAR_ENTER();
 	return rv;
 }
 
@@ -243,7 +254,7 @@ MUTEX_RELEASE(kmutex_t *mtx)
 {
 	uintptr_t newown;
 
-	MUTEX_GIVE(mtx);
+	MUTEX_MEMBAR_EXIT();
 	newown = 0;
 	MUTEX_INHERITDEBUG(newown, mtx->mtx_owner);
 	mtx->mtx_owner = newown;



CVS commit: src/sys/kern

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 18:29:45 UTC 2019

Modified Files:
src/sys/kern: sched_4bsd.c

Log Message:
Don't try to kpreempt a CPU hog unless __HAVE_PREEMPTION (oops).


To generate a diff of this commit:
cvs rdiff -u -r1.37 -r1.38 src/sys/kern/sched_4bsd.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/sched_4bsd.c
diff -u src/sys/kern/sched_4bsd.c:1.37 src/sys/kern/sched_4bsd.c:1.38
--- src/sys/kern/sched_4bsd.c:1.37	Sat Nov 23 22:35:08 2019
+++ src/sys/kern/sched_4bsd.c	Fri Nov 29 18:29:45 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: sched_4bsd.c,v 1.37 2019/11/23 22:35:08 ad Exp $	*/
+/*	$NetBSD: sched_4bsd.c,v 1.38 2019/11/29 18:29:45 ad Exp $	*/
 
 /*
  * Copyright (c) 1999, 2000, 2004, 2006, 2007, 2008, 2019
@@ -69,7 +69,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: sched_4bsd.c,v 1.37 2019/11/23 22:35:08 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: sched_4bsd.c,v 1.38 2019/11/29 18:29:45 ad Exp $");
 
 #include "opt_ddb.h"
 #include "opt_lockdebug.h"
@@ -128,8 +128,12 @@ sched_tick(struct cpu_info *ci)
 		break;
 	case SCHED_RR:
 		/* Force it into mi_switch() to look for other jobs to run. */
+#ifdef __HAVE_PREEMPTION
 		atomic_or_uint(>l_dopreempt, DOPREEMPT_ACTIVE);
 		cpu_need_resched(ci, l, RESCHED_KPREEMPT);
+#else
+		cpu_need_resched(ci, l, RESCHED_UPREEMPT);
+#endif
 		break;
 	default:
 		if (spc->spc_flags & SPCF_SHOULDYIELD) {
@@ -138,8 +142,12 @@ sched_tick(struct cpu_info *ci)
 			 * due to buggy or inefficient code.  Force a
 			 * kernel preemption.
 			 */
+#ifdef __HAVE_PREEMPTION
 			atomic_or_uint(>l_dopreempt, DOPREEMPT_ACTIVE);
 			cpu_need_resched(ci, l, RESCHED_KPREEMPT);
+#else
+			cpu_need_resched(ci, l, RESCHED_UPREEMPT);
+#endif
 		} else if (spc->spc_flags & SPCF_SEENRR) {
 			/*
 			 * The process has already been through a roundrobin



CVS commit: src/sys/kern

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 18:29:45 UTC 2019

Modified Files:
src/sys/kern: sched_4bsd.c

Log Message:
Don't try to kpreempt a CPU hog unless __HAVE_PREEMPTION (oops).


To generate a diff of this commit:
cvs rdiff -u -r1.37 -r1.38 src/sys/kern/sched_4bsd.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 18:27:33 UTC 2019

Modified Files:
src/sys/arch/hppa/hppa: trap.c
src/sys/arch/sh3/include: userret.h
src/sys/arch/sh3/sh3: exception.c
src/sys/arch/sparc/include: userret.h
src/sys/arch/usermode/usermode: trap.c

Log Message:
PR port-sparc/54718 (sparc install hangs since recent scheduler changes)

- userret() must be called every time we return to user, it's not optional.
- If clearing the AST with interrupts off, you must loop over userret().


To generate a diff of this commit:
cvs rdiff -u -r1.112 -r1.113 src/sys/arch/hppa/hppa/trap.c
cvs rdiff -u -r1.14 -r1.15 src/sys/arch/sh3/include/userret.h
cvs rdiff -u -r1.68 -r1.69 src/sys/arch/sh3/sh3/exception.c
cvs rdiff -u -r1.10 -r1.11 src/sys/arch/sparc/include/userret.h
cvs rdiff -u -r1.71 -r1.72 src/sys/arch/usermode/usermode/trap.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch

2019-11-29 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Fri Nov 29 18:27:33 UTC 2019

Modified Files:
src/sys/arch/hppa/hppa: trap.c
src/sys/arch/sh3/include: userret.h
src/sys/arch/sh3/sh3: exception.c
src/sys/arch/sparc/include: userret.h
src/sys/arch/usermode/usermode: trap.c

Log Message:
PR port-sparc/54718 (sparc install hangs since recent scheduler changes)

- userret() must be called every time we return to user, it's not optional.
- If clearing the AST with interrupts off, you must loop over userret().


To generate a diff of this commit:
cvs rdiff -u -r1.112 -r1.113 src/sys/arch/hppa/hppa/trap.c
cvs rdiff -u -r1.14 -r1.15 src/sys/arch/sh3/include/userret.h
cvs rdiff -u -r1.68 -r1.69 src/sys/arch/sh3/sh3/exception.c
cvs rdiff -u -r1.10 -r1.11 src/sys/arch/sparc/include/userret.h
cvs rdiff -u -r1.71 -r1.72 src/sys/arch/usermode/usermode/trap.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/arch/hppa/hppa/trap.c
diff -u src/sys/arch/hppa/hppa/trap.c:1.112 src/sys/arch/hppa/hppa/trap.c:1.113
--- src/sys/arch/hppa/hppa/trap.c:1.112	Thu Nov 21 19:24:00 2019
+++ src/sys/arch/hppa/hppa/trap.c	Fri Nov 29 18:27:32 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: trap.c,v 1.112 2019/11/21 19:24:00 ad Exp $	*/
+/*	$NetBSD: trap.c,v 1.113 2019/11/29 18:27:32 ad Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002 The NetBSD Foundation, Inc.
@@ -58,7 +58,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: trap.c,v 1.112 2019/11/21 19:24:00 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: trap.c,v 1.113 2019/11/29 18:27:32 ad Exp $");
 
 /* #define INTRDEBUG */
 /* #define TRAPDEBUG */
@@ -202,10 +202,11 @@ userret(struct lwp *l, register_t pc, u_
 {
 	struct proc *p = l->l_proc;
 
-	l->l_md.md_astpending = 0;
-	//curcpu()->ci_data.cpu_nast++;
-
-	mi_userret(l);
+	do {
+		l->l_md.md_astpending = 0;
+		//curcpu()->ci_data.cpu_nast++;
+		mi_userret(l);
+	} while (l->l_md.md_astpending);
 
 	/*
 	 * If profiling, charge recent system time to the trapped pc.

Index: src/sys/arch/sh3/include/userret.h
diff -u src/sys/arch/sh3/include/userret.h:1.14 src/sys/arch/sh3/include/userret.h:1.15
--- src/sys/arch/sh3/include/userret.h:1.14	Wed Nov  2 00:11:59 2016
+++ src/sys/arch/sh3/include/userret.h	Fri Nov 29 18:27:32 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: userret.h,v 1.14 2016/11/02 00:11:59 pgoyette Exp $	*/
+/*	$NetBSD: userret.h,v 1.15 2019/11/29 18:27:32 ad Exp $	*/
 
 /*
  * Copyright (c) 1988 University of Utah.
@@ -52,7 +52,16 @@ userret(struct lwp *l)
 {
 
 	/* Invoke MI userret code */
-	mi_userret(l);
+	do {
+		//curcpu()->ci_data.cpu_nast++;
+		l->l_md.md_astpending = 0;
+		mi_userret(l);
+	} while (l->l_md.md_astpending);
+
+	if (l->l_pflag & LP_OWEUPC) {
+		l->l_pflag &= ~LP_OWEUPC;
+		ADDUPROF(l);
+	}
 
 #ifdef PTRACE_HOOKS
 	/* Check if lwp is being PT_STEP'ed */

Index: src/sys/arch/sh3/sh3/exception.c
diff -u src/sys/arch/sh3/sh3/exception.c:1.68 src/sys/arch/sh3/sh3/exception.c:1.69
--- src/sys/arch/sh3/sh3/exception.c:1.68	Thu Nov 21 19:24:01 2019
+++ src/sys/arch/sh3/sh3/exception.c	Fri Nov 29 18:27:33 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: exception.c,v 1.68 2019/11/21 19:24:01 ad Exp $	*/
+/*	$NetBSD: exception.c,v 1.69 2019/11/29 18:27:33 ad Exp $	*/
 
 /*-
  * Copyright (c) 2002 The NetBSD Foundation, Inc. All rights reserved.
@@ -79,7 +79,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: exception.c,v 1.68 2019/11/21 19:24:01 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: exception.c,v 1.69 2019/11/29 18:27:33 ad Exp $");
 
 #include "opt_ddb.h"
 #include "opt_kgdb.h"
@@ -471,15 +471,5 @@ ast(struct lwp *l, struct trapframe *tf)
 	KDASSERT(l != NULL);
 	KDASSERT(l->l_md.md_regs == tf);
 
-	while (l->l_md.md_astpending) {
-		//curcpu()->ci_data.cpu_nast++;
-		l->l_md.md_astpending = 0;
-
-		if (l->l_pflag & LP_OWEUPC) {
-			l->l_pflag &= ~LP_OWEUPC;
-			ADDUPROF(l);
-		}
-
-		userret(l);
-	}
+	userret(l);
 }

Index: src/sys/arch/sparc/include/userret.h
diff -u src/sys/arch/sparc/include/userret.h:1.10 src/sys/arch/sparc/include/userret.h:1.11
--- src/sys/arch/sparc/include/userret.h:1.10	Sat Nov 23 16:50:39 2019
+++ src/sys/arch/sparc/include/userret.h	Fri Nov 29 18:27:33 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: userret.h,v 1.10 2019/11/23 16:50:39 ad Exp $ */
+/*	$NetBSD: userret.h,v 1.11 2019/11/29 18:27:33 ad Exp $ */
 
 /*
  * Copyright (c) 1996
@@ -63,13 +63,14 @@ userret(struct lwp *l, int pc, u_quad_t 
 {
 	struct proc *p = l->l_proc;
 
-	while (cpuinfo.ci_want_ast) {
+	do {
 		cpuinfo.ci_want_ast = 0;
 		mi_userret(l);
-		if (l->l_pflag & LP_OWEUPC) {
-			l->l_pflag &= ~LP_OWEUPC;
-			ADDUPROF(l);
-		}
+	} while (cpuinfo.ci_want_ast);
+
+	if (l->l_pflag & LP_OWEUPC) {
+		l->l_pflag &= ~LP_OWEUPC;
+		ADDUPROF(l);
 	}
 
 	/*

Index: src/sys/arch/usermode/usermode/trap.c
diff -u src/sys/arch/usermode/usermode/trap.c:1.71 src/sys/arch/usermode/usermode/trap.c:1.72
--- 

CVS commit: src/sys/kern

2019-11-27 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Wed Nov 27 20:31:13 UTC 2019

Modified Files:
src/sys/kern: kern_runq.c

Log Message:
Don't try to IPI other CPUs early on.  Fixes a crash on sparc64.  Thanks
to martin@ for diagnosing.


To generate a diff of this commit:
cvs rdiff -u -r1.49 -r1.50 src/sys/kern/kern_runq.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/kern

2019-11-27 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Wed Nov 27 20:31:13 UTC 2019

Modified Files:
src/sys/kern: kern_runq.c

Log Message:
Don't try to IPI other CPUs early on.  Fixes a crash on sparc64.  Thanks
to martin@ for diagnosing.


To generate a diff of this commit:
cvs rdiff -u -r1.49 -r1.50 src/sys/kern/kern_runq.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/kern_runq.c
diff -u src/sys/kern/kern_runq.c:1.49 src/sys/kern/kern_runq.c:1.50
--- src/sys/kern/kern_runq.c:1.49	Sat Nov 23 22:35:08 2019
+++ src/sys/kern/kern_runq.c	Wed Nov 27 20:31:13 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: kern_runq.c,v 1.49 2019/11/23 22:35:08 ad Exp $	*/
+/*	$NetBSD: kern_runq.c,v 1.50 2019/11/27 20:31:13 ad Exp $	*/
 
 /*-
  * Copyright (c) 2019 The NetBSD Foundation, Inc.
@@ -56,7 +56,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: kern_runq.c,v 1.49 2019/11/23 22:35:08 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: kern_runq.c,v 1.50 2019/11/27 20:31:13 ad Exp $");
 
 #include "opt_dtrace.h"
 
@@ -368,7 +368,7 @@ sched_resched_cpu(struct cpu_info *ci, p
 	 * If the priority level we're evaluating wouldn't cause a new LWP
 	 * to be run on the CPU, then we have nothing to do.
 	 */
-	if (pri <= spc->spc_curpriority) {
+	if (pri <= spc->spc_curpriority || !mp_online) {
 		if (__predict_true(unlock)) {
 			spc_unlock(ci);
 		}



CVS commit: src/sys/kern

2019-11-25 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Mon Nov 25 20:16:23 UTC 2019

Modified Files:
src/sys/kern: kern_rwlock.c

Log Message:
Remove some unneeded memory barriers and reads of the lock word.
Prodded by Mateusz Guzik.


To generate a diff of this commit:
cvs rdiff -u -r1.54 -r1.55 src/sys/kern/kern_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/kern

2019-11-25 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Mon Nov 25 20:16:23 UTC 2019

Modified Files:
src/sys/kern: kern_rwlock.c

Log Message:
Remove some unneeded memory barriers and reads of the lock word.
Prodded by Mateusz Guzik.


To generate a diff of this commit:
cvs rdiff -u -r1.54 -r1.55 src/sys/kern/kern_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/kern_rwlock.c
diff -u src/sys/kern/kern_rwlock.c:1.54 src/sys/kern/kern_rwlock.c:1.55
--- src/sys/kern/kern_rwlock.c:1.54	Thu May  9 05:00:31 2019
+++ src/sys/kern/kern_rwlock.c	Mon Nov 25 20:16:22 2019
@@ -1,7 +1,7 @@
-/*	$NetBSD: kern_rwlock.c,v 1.54 2019/05/09 05:00:31 ozaki-r Exp $	*/
+/*	$NetBSD: kern_rwlock.c,v 1.55 2019/11/25 20:16:22 ad Exp $	*/
 
 /*-
- * Copyright (c) 2002, 2006, 2007, 2008, 2009 The NetBSD Foundation, Inc.
+ * Copyright (c) 2002, 2006, 2007, 2008, 2009, 2019 The NetBSD Foundation, Inc.
  * All rights reserved.
  *
  * This code is derived from software contributed to The NetBSD Foundation
@@ -38,7 +38,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.54 2019/05/09 05:00:31 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: kern_rwlock.c,v 1.55 2019/11/25 20:16:22 ad Exp $");
 
 #define	__RWLOCK_PRIVATE
 
@@ -112,6 +112,19 @@ do {	\
 #define	RW_INHERITDEBUG(n, o)		/* nothing */
 #endif /* defined(LOCKDEBUG) */
 
+/*
+ * Memory barriers.
+ */
+#ifdef __HAVE_ATOMIC_AS_MEMBAR
+#define	RW_MEMBAR_ENTER()
+#define	RW_MEMBAR_EXIT()
+#define	RW_MEMBAR_PRODUCER()
+#else
+#define	RW_MEMBAR_ENTER()		membar_enter()
+#define	RW_MEMBAR_EXIT()		membar_exit()
+#define	RW_MEMBAR_PRODUCER()		membar_producer()
+#endif
+
 static void	rw_abort(const char *, size_t, krwlock_t *, const char *);
 static void	rw_dump(const volatile void *, lockop_printer_t);
 static lwp_t	*rw_owner(wchan_t);
@@ -321,7 +334,7 @@ rw_vector_enter(krwlock_t *rw, const krw
 	LOCKSTAT_ENTER(lsflag);
 
 	KPREEMPT_DISABLE(curlwp);
-	for (owner = rw->rw_owner; ;) {
+	for (owner = rw->rw_owner;;) {
 		/*
 		 * Read the lock owner field.  If the need-to-wait
 		 * indicator is clear, then try to acquire the lock.
@@ -331,7 +344,7 @@ rw_vector_enter(krwlock_t *rw, const krw
 			~RW_WRITE_WANTED);
 			if (__predict_true(next == owner)) {
 /* Got it! */
-membar_enter();
+RW_MEMBAR_ENTER();
 break;
 			}
 
@@ -460,7 +473,7 @@ rw_vector_exit(krwlock_t *rw)
 	 * proceed to do direct handoff if there are waiters, and if the
 	 * lock would become unowned.
 	 */
-	membar_exit();
+	RW_MEMBAR_EXIT();
 	for (;;) {
 		newown = (owner - decr);
 		if ((newown & (RW_THREAD | RW_HAS_WAITERS)) == RW_HAS_WAITERS)
@@ -554,13 +567,12 @@ rw_vector_tryenter(krwlock_t *rw, const 
 	}
 
 	for (owner = rw->rw_owner;; owner = next) {
-		owner = rw->rw_owner;
 		if (__predict_false((owner & need_wait) != 0))
 			return 0;
 		next = rw_cas(rw, owner, owner + incr);
 		if (__predict_true(next == owner)) {
 			/* Got it! */
-			membar_enter();
+			RW_MEMBAR_ENTER();
 			break;
 		}
 	}
@@ -576,7 +588,8 @@ rw_vector_tryenter(krwlock_t *rw, const 
 /*
  * rw_downgrade:
  *
- *	Downgrade a write lock to a read lock.
+ *	Downgrade a write lock to a read lock.  Optimise memory accesses for
+ *	the uncontended case.
  */
 void
 rw_downgrade(krwlock_t *rw)
@@ -594,24 +607,20 @@ rw_downgrade(krwlock_t *rw)
 	__USE(curthread);
 #endif
 
-
-	membar_producer();
-	owner = rw->rw_owner;
-	if ((owner & RW_HAS_WAITERS) == 0) {
-		/*
-		 * There are no waiters, so we can do this the easy way.
-		 * Try swapping us down to one read hold.  If it fails, the
-		 * lock condition has changed and we most likely now have
-		 * waiters.
-		 */
-		next = rw_cas(rw, owner, RW_READ_INCR);
-		if (__predict_true(next == owner)) {
-			RW_LOCKED(rw, RW_READER);
-			RW_DASSERT(rw, (rw->rw_owner & RW_WRITE_LOCKED) == 0);
-			RW_DASSERT(rw, RW_COUNT(rw) != 0);
-			return;
-		}
-		owner = next;
+	/*
+	 * If there are no waiters, so we can do this the easy way.
+	 * Try swapping us down to one read hold.  If it fails, the
+	 * lock condition has changed and we most likely now have
+	 * waiters.
+	 */
+	RW_MEMBAR_PRODUCER();
+	owner = curthread | RW_WRITE_LOCKED;
+	next = rw_cas(rw, owner, RW_READ_INCR);
+	if (__predict_true(next == owner)) {
+		RW_LOCKED(rw, RW_READER);
+		RW_DASSERT(rw, (rw->rw_owner & RW_WRITE_LOCKED) == 0);
+		RW_DASSERT(rw, RW_COUNT(rw) != 0);
+		return;
 	}
 
 	/*
@@ -619,7 +628,8 @@ rw_downgrade(krwlock_t *rw)
 	 * on the sleep queue.  Once we have that, we can adjust the
 	 * waiter bits.
 	 */
-	for (;; owner = next) {
+	for (;;) {
+		owner = next;
 		ts = turnstile_lookup(rw);
 		RW_DASSERT(rw, ts != NULL);
 
@@ -670,8 +680,8 @@ rw_downgrade(krwlock_t *rw)
 /*
  * rw_tryupgrade:
  *
- *	Try to upgrade a read lock to a write lock.  We must be the
- *	only reader.
+ *	Try to upgrade a read lock to a write lock.  We must be the only
+ *	reader.  Optimise 

CVS commit: src/sys/kern

2019-11-25 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Mon Nov 25 17:24:59 UTC 2019

Modified Files:
src/sys/kern: kern_softint.c

Log Message:
port-sparc/54718 (sparc install hangs since recent scheduler changes)


To generate a diff of this commit:
cvs rdiff -u -r1.50 -r1.51 src/sys/kern/kern_softint.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/kern/kern_softint.c
diff -u src/sys/kern/kern_softint.c:1.50 src/sys/kern/kern_softint.c:1.51
--- src/sys/kern/kern_softint.c:1.50	Sat Nov 23 19:42:52 2019
+++ src/sys/kern/kern_softint.c	Mon Nov 25 17:24:59 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: kern_softint.c,v 1.50 2019/11/23 19:42:52 ad Exp $	*/
+/*	$NetBSD: kern_softint.c,v 1.51 2019/11/25 17:24:59 ad Exp $	*/
 
 /*-
  * Copyright (c) 2007, 2008, 2019 The NetBSD Foundation, Inc.
@@ -170,7 +170,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: kern_softint.c,v 1.50 2019/11/23 19:42:52 ad Exp $");
+__KERNEL_RCSID(0, "$NetBSD: kern_softint.c,v 1.51 2019/11/25 17:24:59 ad Exp $");
 
 #include 
 #include 
@@ -689,14 +689,14 @@ softint_trigger(uintptr_t machdep)
 	struct cpu_info *ci;
 	lwp_t *l;
 
-	l = curlwp;
-	ci = l->l_cpu;
+	ci = curcpu();
 	ci->ci_data.cpu_softints |= machdep;
+	l = ci->ci_data.cpu_onproc;
 	if (l == ci->ci_data.cpu_idlelwp) {
 		atomic_or_uint(>ci_want_resched, RESCHED_UPREEMPT);
 	} else {
 		/* MI equivalent of aston() */
-		lwp_need_userret(l);
+		cpu_signotify(l);
 	}
 }
 



CVS commit: src/sys/kern

2019-11-25 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Mon Nov 25 17:24:59 UTC 2019

Modified Files:
src/sys/kern: kern_softint.c

Log Message:
port-sparc/54718 (sparc install hangs since recent scheduler changes)


To generate a diff of this commit:
cvs rdiff -u -r1.50 -r1.51 src/sys/kern/kern_softint.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch/sun68k/include

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:53:47 UTC 2019

Modified Files:
src/sys/arch/sun68k/include: cpu.h

Log Message:
Correction to previous.


To generate a diff of this commit:
cvs rdiff -u -r1.25 -r1.26 src/sys/arch/sun68k/include/cpu.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/arch/sun68k/include/cpu.h
diff -u src/sys/arch/sun68k/include/cpu.h:1.25 src/sys/arch/sun68k/include/cpu.h:1.26
--- src/sys/arch/sun68k/include/cpu.h:1.25	Sat Nov 23 19:40:37 2019
+++ src/sys/arch/sun68k/include/cpu.h	Sun Nov 24 15:53:47 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: cpu.h,v 1.25 2019/11/23 19:40:37 ad Exp $	*/
+/*	$NetBSD: cpu.h,v 1.26 2019/11/24 15:53:47 ad Exp $	*/
 
 /*
  * Copyright (c) 1982, 1990 The Regents of the University of California.
@@ -132,7 +132,7 @@ extern int astpending;	 /* need to trap 
  * Preempt the current process if in interrupt from user mode,
  * or after the current trap/syscall if in system mode.
  */
-#define	cpu_need_resched(ci,flags)	do {	\
+#define	cpu_need_resched(ci,l,flags)	do {	\
 	__USE(flags); \
 	aston();\
 } while (/*CONSTCOND*/0)



CVS commit: src/sys/arch/sun68k/include

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:53:47 UTC 2019

Modified Files:
src/sys/arch/sun68k/include: cpu.h

Log Message:
Correction to previous.


To generate a diff of this commit:
cvs rdiff -u -r1.25 -r1.26 src/sys/arch/sun68k/include/cpu.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch/powerpc/pic

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:49:12 UTC 2019

Modified Files:
src/sys/arch/powerpc/pic: ipi.c ipivar.h

Log Message:
Add IPI_AST.


To generate a diff of this commit:
cvs rdiff -u -r1.12 -r1.13 src/sys/arch/powerpc/pic/ipi.c
cvs rdiff -u -r1.9 -r1.10 src/sys/arch/powerpc/pic/ipivar.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/arch/powerpc/pic/ipi.c
diff -u src/sys/arch/powerpc/pic/ipi.c:1.12 src/sys/arch/powerpc/pic/ipi.c:1.13
--- src/sys/arch/powerpc/pic/ipi.c:1.12	Fri Jan 23 07:27:05 2015
+++ src/sys/arch/powerpc/pic/ipi.c	Sun Nov 24 15:49:12 2019
@@ -1,4 +1,4 @@
-/* $NetBSD: ipi.c,v 1.12 2015/01/23 07:27:05 nonaka Exp $ */
+/* $NetBSD: ipi.c,v 1.13 2019/11/24 15:49:12 ad Exp $ */
 /*-
  * Copyright (c) 2007 The NetBSD Foundation, Inc.
  * All rights reserved.
@@ -29,7 +29,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: ipi.c,v 1.12 2015/01/23 07:27:05 nonaka Exp $");
+__KERNEL_RCSID(0, "$NetBSD: ipi.c,v 1.13 2019/11/24 15:49:12 ad Exp $");
 
 #include "opt_multiprocessor.h"
 #include "opt_pic.h"
@@ -78,6 +78,9 @@ ipi_intr(void *v)
 	if (ipi & IPI_SUSPEND)
 		cpu_pause(NULL);
 
+	if (ipi & IPI_AST)
+		ci->ci_data.cpu_onproc->l_md.md_astpending = 1;
+
 	if (ipi & IPI_HALT) {
 		struct cpuset_info * const csi = _info;
 		aprint_normal("halting CPU %d\n", cpu_id);

Index: src/sys/arch/powerpc/pic/ipivar.h
diff -u src/sys/arch/powerpc/pic/ipivar.h:1.9 src/sys/arch/powerpc/pic/ipivar.h:1.10
--- src/sys/arch/powerpc/pic/ipivar.h:1.9	Thu Apr 19 21:50:07 2018
+++ src/sys/arch/powerpc/pic/ipivar.h	Sun Nov 24 15:49:12 2019
@@ -1,4 +1,4 @@
-/* $NetBSD: ipivar.h,v 1.9 2018/04/19 21:50:07 christos Exp $ */
+/* $NetBSD: ipivar.h,v 1.10 2019/11/24 15:49:12 ad Exp $ */
 /*-
  * Copyright (c) 2007 The NetBSD Foundation, Inc.
  * All rights reserved.
@@ -29,7 +29,7 @@
  */
 
 #include 
-__KERNEL_RCSID(0, "$NetBSD: ipivar.h,v 1.9 2018/04/19 21:50:07 christos Exp $");
+__KERNEL_RCSID(0, "$NetBSD: ipivar.h,v 1.10 2019/11/24 15:49:12 ad Exp $");
 
 #ifndef _IPI_VAR_H_
 #define _IPI_VAR_H_
@@ -56,6 +56,7 @@ struct ipi_ops {
 #define IPI_KPREEMPT		0x0004
 #define IPI_GENERIC		0x0008
 #define IPI_SUSPEND		0x0010
+#define	IPI_AST			0x0020
 
 /* OpenPIC */
 void setup_openpic_ipi(void);



CVS commit: src/sys/arch/powerpc/pic

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:49:12 UTC 2019

Modified Files:
src/sys/arch/powerpc/pic: ipi.c ipivar.h

Log Message:
Add IPI_AST.


To generate a diff of this commit:
cvs rdiff -u -r1.12 -r1.13 src/sys/arch/powerpc/pic/ipi.c
cvs rdiff -u -r1.9 -r1.10 src/sys/arch/powerpc/pic/ipivar.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch/ia64/include

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:45:41 UTC 2019

Modified Files:
src/sys/arch/ia64/include: cpu.h

Log Message:
Make ci_want_resched a u_int.


To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.19 src/sys/arch/ia64/include/cpu.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/arch/ia64/include/cpu.h
diff -u src/sys/arch/ia64/include/cpu.h:1.18 src/sys/arch/ia64/include/cpu.h:1.19
--- src/sys/arch/ia64/include/cpu.h:1.18	Sat Nov 23 19:40:35 2019
+++ src/sys/arch/ia64/include/cpu.h	Sun Nov 24 15:45:41 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: cpu.h,v 1.18 2019/11/23 19:40:35 ad Exp $	*/
+/*	$NetBSD: cpu.h,v 1.19 2019/11/24 15:45:41 ad Exp $	*/
 
 /*-
  * Copyright (c) 2006 The NetBSD Foundation, Inc.
@@ -107,7 +107,8 @@ struct cpu_info {
 	struct lwp *ci_fpcurlwp;	/* current owner of the FPU */
 	paddr_t ci_curpcb;		/* PA of current HW PCB */
 	struct pcb *ci_idle_pcb;	/* our idle PCB */
-	u_long ci_want_resched;		/* preempt current process */
+	u_int ci_want_resched;		/* preempt current process */
+	u_int ci_unused;		/* unused */
 	u_long ci_intrdepth;		/* interrupt trap depth */
 	struct trapframe *ci_db_regs;	/* registers for debuggers */
 	uint64_t ci_clock;		/* clock counter */



CVS commit: src/sys/arch/ia64/include

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:45:41 UTC 2019

Modified Files:
src/sys/arch/ia64/include: cpu.h

Log Message:
Make ci_want_resched a u_int.


To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.19 src/sys/arch/ia64/include/cpu.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch/alpha/include

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:40:24 UTC 2019

Modified Files:
src/sys/arch/alpha/include: cpu.h

Log Message:
Make ci_want_resched a u_int.


To generate a diff of this commit:
cvs rdiff -u -r1.84 -r1.85 src/sys/arch/alpha/include/cpu.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



CVS commit: src/sys/arch/alpha/include

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:40:24 UTC 2019

Modified Files:
src/sys/arch/alpha/include: cpu.h

Log Message:
Make ci_want_resched a u_int.


To generate a diff of this commit:
cvs rdiff -u -r1.84 -r1.85 src/sys/arch/alpha/include/cpu.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/sys/arch/alpha/include/cpu.h
diff -u src/sys/arch/alpha/include/cpu.h:1.84 src/sys/arch/alpha/include/cpu.h:1.85
--- src/sys/arch/alpha/include/cpu.h:1.84	Wed Aug 22 01:05:21 2018
+++ src/sys/arch/alpha/include/cpu.h	Sun Nov 24 15:40:24 2019
@@ -1,4 +1,4 @@
-/* $NetBSD: cpu.h,v 1.84 2018/08/22 01:05:21 msaitoh Exp $ */
+/* $NetBSD: cpu.h,v 1.85 2019/11/24 15:40:24 ad Exp $ */
 
 /*-
  * Copyright (c) 1998, 1999, 2000, 2001 The NetBSD Foundation, Inc.
@@ -121,7 +121,8 @@ struct cpu_info {
 	struct mchkinfo ci_mcinfo;	/* machine check info */
 	cpuid_t ci_cpuid;		/* our CPU ID */
 	struct cpu_softc *ci_softc;	/* pointer to our device */
-	u_long ci_want_resched;		/* preempt current process */
+	u_int ci_want_resched;		/* preempt current process */
+	u_int ci_unused;		/* unused */
 	u_long ci_intrdepth;		/* interrupt trap depth */
 	struct trapframe *ci_db_regs;	/* registers for debuggers */
 	uint64_t ci_pcc_freq;		/* cpu cycles/second */



CVS commit: src/sys/arch/mips/mips

2019-11-24 Thread Andrew Doran
Module Name:src
Committed By:   ad
Date:   Sun Nov 24 15:37:39 UTC 2019

Modified Files:
src/sys/arch/mips/mips: cpu_subr.c

Log Message:
Typo.


To generate a diff of this commit:
cvs rdiff -u -r1.36 -r1.37 src/sys/arch/mips/mips/cpu_subr.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.



  1   2   >