Re: [PATCH] microoptimize locking primitives by introducing randomized delay between atomic ops

[Mateusz Guzik]

On Sun, Jul 10, 2016 at 08:32:01AM -0600, Ian Lepore wrote:
> On Sun, 2016-07-10 at 13:13 +0200, Mateusz Guzik wrote:
> > If the lock is contended, primitives like __mtx_lock_sleep will spin
> > checking if the owner is running or the lock was freed. The problem
> > is
> > that once it is discovered that the lock is free, multiple CPUs are
> > likely to try to do the atomic op which will make it more costly for
> > everyone and throughput suffers.
> > 
> > The standard thing to do is to have some sort of a randomized delay
> > so
> > that this kind of behaviour is reduced.
> > 
> > As such, below is a trivial hack which takes cpu_ticks() into account
> > and performs % 2048, which in my testing gives reasonbly good
> > results.
> > 
> > Please note there is definitely way more room for improvement in
> > general.
> > 
> > In terms of results, there was no statistically significant change in
> > -j 40 buildworld nor buildkernel.
> > 
> > However, a 40-way find on a ports tree placed on tmpfs yielded the
> > following:
> > 
> > x vanilla
> > + patched
> > +
> > +
> > > + x
> > >  x x x  |
> > > + ++++  +  + ++   +   + x   x 
> > >  x   x x x|
> > >|_MA__| 
> > >  |AM__| |
> > +
> > +
> > N   Min   MaxMedian   Avg   
> >  Stddev
> > x  2012.43115.95214.897   14.7444   
> >  0.74241657
> > +  20 8.10311.8639.0135   9.44565
> >  1.0059484
> > Difference at 95.0% confidence
> > -5.29875 +/- 0.565836
> > -35.9374% +/- 3.83764%
> > (Student's t, pooled s = 0.884057)
> > 
> > The patch:
> [...]
> 
> What about platforms that don't have a useful implementation of
> cpu_ticks()?
> 

Do we have such platforms and do they have smp?

> What about platforms that don't suffer the large expense for atomic ops
> that x86 apparently does?
> 

The current state of locking primitives already seems to be x86-centric.
Postponing of atomic ops is implemented in some parts and this patch
only extends it (in a different form).

That said, if we have platforms where this kind of stuff is detrimental
to performance, machine-specific primitives should be introduced.

Meanwhile, courtesy of andrew@ I tested the patch on cavium (48-way
arm64) and saw great improvement.


x vanilla
+ patched
++
|+  
 |
|+  
 |
|+  
 |
|+  
 |
|+  
 |
|+  
 x   |
|++ 
xxx  |
|++ 
   xx|
|A| 
|A_| |
++
N   Min   MaxMedian   AvgStddev
x  10 17.2517.849 17.48   17.49680.19581556
+  10  6.56 6.679 6.5866.6011   0.038013009
Difference at 95.0% confidence
-10.8957 +/- 0.132528
-62.2725% +/- 0.757439%
(Student's t, pooled s = 0.141047)

Note: find does open+close a lot. close results in exclusive vnode locking if
the fs does not have the MNTK_EXTENDED_SHARED flag set, which is the case on
tmpfs. On this machine it contributed to a major slowdown. The flag was set
locally. I'm not sure yet how safe the change in terms of general use. It is
definitely fine enough for the benchmark.

That said, I would like to commit this next week unless there are objections.

-- 
Mateusz Guzik 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Jenkins build is back to normal : FreeBSD_HEAD #448

[jenkins-admin]

See 

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Build failed in Jenkins: FreeBSD_HEAD #447

[jenkins-admin]

See 

--
[...truncated 164336 lines...]
--- ventel.o ---
--- all_subdir_usr.sbin ---
--- iasl.full ---
--- all_subdir_usr.bin ---
cc  -O2 -pipe -I/builds/workspace/FreeBSD_HEAD/src/usr.bin/tip/tip -DDEFBR=9600 
-DDEFFS=BUFSIZ -DACULOG -DPRISTINE  -DCONNECT -DV831 -DVENTEL -DHAYES -DCOURIER 
-DT3000   -g -MD  -MF.depend.ventel.o -MTventel.o -std=gnu99 
-fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k 
-Wno-uninitialized -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int 
-Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value 
-Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion 
-Wno-unused-local-typedef -Wno-switch -Wno-switch-enum 
-Wno-knr-promoted-parameter  -Qunused-arguments  -c 
/builds/workspace/FreeBSD_HEAD/src/usr.bin/tip/tip/../libacu/ventel.c -o 
ventel.o
--- all_subdir_usr.sbin ---
cc -O2 -pipe -DACPI_ASL_COMPILER -I. 
-I/builds/workspace/FreeBSD_HEAD/src/usr.sbin/acpi/iasl/../../../sys -g 
-std=gnu99 -fstack-protector-strong -Wsystem-headers -Werror -Wall 
-Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -Wno-empty-body 
-Wno-string-plus-int -Wno-unused-const-variable -Wno-tautological-compare 
-Wno-unused-value -Wno-parentheses-equality -Wno-unused-function 
-Wno-enum-conversion -Wno-unused-local-typedef -Wno-switch -Wno-switch-enum 
-Wno-knr-promoted-parameter -Qunused-arguments  -o iasl.full acfileio.o 
adfile.o adisasm.o adwalk.o ahids.o ahpredef.o ahtable.o ahuuids.o cmfsize.o 
dmextern.o dmrestag.o dmtable.o dmtables.o dmtbdump.o dmtbinfo.o getopt.o 
aslanalyze.o aslascii.o aslbtypes.o aslcodegen.o aslcompile.o aslcompilerlex.o 
aslcompilerparse.o asldebug.o aslerror.o aslexternal.o aslfileio.o aslfiles.o 
aslfold.o aslhex.o asllength.o asllisting.o asllistsup.o aslload.o asllookup.o 
aslmain.o aslmap.o aslmapenter.o aslmapoutput.o aslmaputils.o aslmessages.o asl
 method.o aslnamesp.o asloffset.o aslopcodes.o asloperands.o aslopt.o 
asloptions.o aslpld.o aslpredef.o aslprepkg.o aslprintf.o aslprune.o 
aslresource.o aslrestype1.o aslrestype1i.o aslrestype2.o aslrestype2d.o 
aslrestype2e.o aslrestype2q.o aslrestype2s.o aslrestype2w.o aslstartup.o 
aslstubs.o asltransform.o asltree.o aslutils.o asluuid.o aslwalks.o aslxref.o 
aslxrefout.o dtcompile.o dtexpress.o dtfield.o dtio.o dtparserlex.o 
dtparserparse.o dtsubtable.o dttable.o dttable1.o dttable2.o dttemplate.o 
dtutils.o prexpress.o prmacros.o prparserlex.o prparserparse.o prscan.o 
prutils.o dbfileio.o dmbuffer.o dmcstyle.o dmdeferred.o dmnames.o dmopcode.o 
dmresrc.o dmresrcl.o dmresrcl2.o dmresrcs.o dmutils.o dmwalk.o dsargs.o 
dscontrol.o dsfield.o dsobject.o dsopcode.o dsutils.o dswexec.o dswload.o 
dswload2.o dswscope.o dswstate.o exconcat.o exconvrt.o excreate.o exdump.o 
exmisc.o exmutex.o exnames.o exoparg1.o exoparg2.o exoparg3.o exoparg6.o 
exprep.o exresnte.o exresolv.o exresop.o exstore.o 
 exstoren.o exstorob.o exsystem.o exutils.o psargs.o psloop.o psobject.o 
psopcode.o psopinfo.o psparse.o psscope.o pstree.o psutils.o pswalk.o 
nsaccess.o nsalloc.o nsdump.o nsnames.o nsobject.o nsparse.o nssearch.o 
nsutils.o nswalk.o tbdata.o tbfadt.o tbinstal.o tbprint.o tbutils.o tbxface.o 
tbxfload.o utaddress.o utalloc.o utascii.o utbuffer.o utcache.o utcopy.o 
utdebug.o utdecode.o utdelete.o uterror.o utexcep.o utglobal.o uthex.o utinit.o 
utlock.o utmath.o utmisc.o utmutex.o utnonansi.o utobject.o utownerid.o 
utpredef.o utprint.o utresrc.o utstate.o utstring.o utuuid.o utxface.o 
utxferror.o oslibcfs.o osunixxf.o  
--- all_subdir_lib ---
--- name_match.po ---
cc  -pg  -O2 -pipe   -D_XOPEN_SOURCE_EXTENDED -DENABLE_WIDEC -I. 
-I/builds/workspace/FreeBSD_HEAD/obj/builds/workspace/FreeBSD_HEAD/src/lib/ncurses/ncursesw/../ncursesw
 -I/builds/workspace/FreeBSD_HEAD/src/lib/ncurses/ncursesw/../ncursesw 
-I/builds/workspace/FreeBSD_HEAD/src/lib/ncurses/ncursesw/../ncurses 
-I/builds/workspace/FreeBSD_HEAD/src/lib/ncurses/ncursesw/../../../contrib/ncurses/include
 
-I/builds/workspace/FreeBSD_HEAD/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses
 -Wall -DNDEBUG -DHAVE_CONFIG_H -DFREEBSD_NATIVE -DTERMIOS -MD  
-MF.depend.name_match.po -MTname_match.po -std=gnu99 -fstack-protector-strong 
-Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter 
-Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized 
-Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int 
-Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value 
-Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion 
-Wno-unused-local-typedef
   -Qunused-arguments  -c 
/builds/workspace/FreeBSD_HEAD/src/lib/ncurses/ncursesw/../../../contrib/ncurses/ncurses/tinfo/name_match.c
 -o name_match.po
--- obsolete.po ---
cc  -pg  -O2 -pipe   -D_XOPEN_SOURCE_EXTENDED -DENABLE_WIDEC -I. 
-I/builds/workspace/FreeBSD_HEAD/obj/builds/workspace/FreeBSD_HEAD/src/lib/ncurses/ncursesw

Re: ath (AR9460) no longer works after going to 11-STABLE r302483

[Wolfgang Zenker]

Hi,

[sorry, forgot to Cc current first]

* Adrian Chadd  [160715 22:40]:
> On 15 July 2016 at 13:28, Wolfgang Zenker  wrote:
>> * Adrian Chadd  [160715 00:00]:
>>> On 14 July 2016 at 14:37, Wolfgang Zenker  wrote:
 * Adrian Chadd  [160710 21:47]:
> Since you've reverted the ath driver directories without success, I'm
> mostly out of simple ideas. I think you need to bisect the whole
> kernel version until you find the commit that broke things.

 done. The commit is 11-STABLE r302410. AFAICS the only change here
 is the removal of debugging options from the GENERIC kernel config:

 https://svnweb.freebsd.org/base/stable/11/sys/amd64/conf/GENERIC?r1=302408&r2=302410

>>> ... loool, okay. Let me see.

>>> Try INVARIANTS and INVARIANT_SUPPORT. Maybe something in the ath
>>> driver needs it.. oops!

>> Nope, wlan0 still works after disabling INVARIANTS and INVARIANT_SUPPORT.
>> Any suggestions for next try?

> Just try disabling the others and see what happens.

commenting out DEADLKRES and MALLOC_DEBUG_MAXZONES in addition to 
INVARIANTS and INVARIANT_SUPPORT did not change the situation: ath
still worked. Re-enabling all of the above and disabling WITNESS
resulted in ath failing.

Wolfgang
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: ptrace attach in multi-threaded processes

[Konstantin Belousov]

On Fri, Jul 15, 2016 at 11:01:59AM -0700, Mark Johnston wrote:
> Thanks, this seems to give the desired behaviour in the single-threaded
> case. I'll write a test case for the multi-threaded case next.
> 
> Am I correct in thinking that r302179 could be reverted if your change
> is committed?
I suspect that it is not.

Suppose that we have a single-threaded process which only thread
is right on the syscall exit path when the debugger is attached,
and debugger requested PTRACE_SCX stops. Then the debuggee reaches
the ptracestop(SIGTRAP) stop point before cursig(9) is ever called.
So despite the patch, first reported signal is SIGTRAP and not the
attaching STOP.  If debugger detaches right after that, the process
should still be left in stopped state.

You may object that gcore(1) does not request SCX, but my point is that
even with the patch, first reported signal could be other than the
STOP.  I suspect that some other ptracestop() call might cause that
behaviour either now or in future even with the default event mask.

So I would left the r302179 in place.

Below is the patch with reverted next_xthread() bits. I reverted them
because Peter found the changes to require much more work to not cause
regressions.

diff --git a/sys/kern/kern_sig.c b/sys/kern/kern_sig.c
index 1da4b99..9e1a494 100644
--- a/sys/kern/kern_sig.c
+++ b/sys/kern/kern_sig.c
@@ -2726,7 +2726,20 @@ issignal(struct thread *td)
SIG_STOPSIGMASK(sigpending);
if (SIGISEMPTY(sigpending)) /* no signal to send */
return (0);
-   sig = sig_ffs(&sigpending);
+   if ((p->p_flag & (P_TRACED | P_PPTRACE)) == P_TRACED &&
+   (p->p_flag2 & P2_PTRACE_FSTP) != 0 &&
+   SIGISMEMBER(sigpending, SIGSTOP)) {
+   /*
+* If debugger just attached, always consume
+* SIGSTOP from ptrace(PT_ATTACH) first, to
+* execute the debugger attach ritual in
+* order.
+*/
+   sig = SIGSTOP;
+   p->p_flag2 &= ~P2_PTRACE_FSTP;
+   } else {
+   sig = sig_ffs(&sigpending);
+   }
 
if (p->p_stops & S_SIG) {
mtx_unlock(&ps->ps_mtx);
@@ -2743,7 +2756,7 @@ issignal(struct thread *td)
sigqueue_delete(&p->p_sigqueue, sig);
continue;
}
-   if (p->p_flag & P_TRACED && (p->p_flag & P_PPTRACE) == 0) {
+   if ((p->p_flag & (P_TRACED | P_PPTRACE)) == P_TRACED) {
/*
 * If traced, always stop.
 * Remove old signal from queue before the stop.
@@ -2846,6 +2859,8 @@ issignal(struct thread *td)
mtx_unlock(&ps->ps_mtx);
WITNESS_WARN(WARN_GIANTOK | WARN_SLEEPOK,
&p->p_mtx.lock_object, "Catching SIGSTOP");
+   sigqueue_delete(&td->td_sigqueue, sig);
+   sigqueue_delete(&p->p_sigqueue, sig);
p->p_flag |= P_STOPPED_SIG;
p->p_xsig = sig;
PROC_SLOCK(p);
@@ -2853,7 +2868,7 @@ issignal(struct thread *td)
thread_suspend_switch(td, p);
PROC_SUNLOCK(p);
mtx_lock(&ps->ps_mtx);
-   break;
+   goto next;
} else if (prop & SA_IGNORE) {
/*
 * Except for SIGCONT, shouldn't get here.
@@ -2884,6 +2899,7 @@ issignal(struct thread *td)
}
sigqueue_delete(&td->td_sigqueue, sig); /* take the signal! */
sigqueue_delete(&p->p_sigqueue, sig);
+next:;
}
/* NOTREACHED */
 }
diff --git a/sys/kern/sys_process.c b/sys/kern/sys_process.c
index f1477ce..86e7c52 100644
--- a/sys/kern/sys_process.c
+++ b/sys/kern/sys_process.c
@@ -900,6 +900,7 @@ kern_ptrace(struct thread *td, int req, pid_t pid, void 
*addr, int data)
case PT_TRACE_ME:
/* set my trace flag and "owner" so it can read/write me */
p->p_flag |= P_TRACED;
+   p->p_flag2 |= P2_PTRACE_FSTP;
p->p_ptevents = PTRACE_DEFAULT;
if (p->p_flag & P_PPWAIT)
p->p_flag |= P_PPTRACE;
@@ -919,6 +920,7 @@ kern_ptrace(struct thread *td, int req, pid_t pid, void 
*addr, int data)
 * on a "detach".
 */
p->p_flag |= P_TRACED;
+   p->p_flag2 |= P2_PTRACE_FSTP;
p->p_ptevents = PTRACE_DEFAULT;
p->p_oppid = p->p_p

Re: 4.6 DRM/i915 update CFT (Sandy Bridge?)/IvyBridge/Haswell/Broadwell/SkyLake/KabyLake supported

[O. Hartmann]

Am Fri, 15 Jul 2016 13:19:04 -0700
Adrian Chadd  schrieb:

> Heh, I'll get mmacy to poke me some more so I can start landing more
> stuff in -head to enable this.
> 
> 
> 
> -adrian

That is nice to hear. Thank you.

Regards,
Oliver


pgpFhjjRY6eik.pgp
Description: OpenPGP digital signature

Re: CURRENT: frequent crashes if mpd5 is running

[Oleg V. Nauman]

On Thursday 14 July 2016 14:38:15 Allan Jude wrote:
> On 2016-07-14 13:13, Oleg V. Nauman wrote:
> >  I'm experiencing frequent CURRENT ( 12.0-CURRENT r302535 amd64 ) crashes
> > 
> > triggered by mpd5:
> > 
> > Fatal trap 12: page fault while in kernel mode
[skip]
> 
> There is a patch for this issue:
> 
> https://reviews.freebsd.org/D7209
> 
> You might try seeing if it solves your problem, and reporting that to
> the author

 It was stable during 24 hours after applying this patch, thank you.
I have updated a patch review with comment that it solves issue for me.

Thank you

signature.asc
Description: This is a digitally signed message part.