CVS commit: src/lib/libpthread

[Christos Zoulas]

Module Name:src
Committed By:   christos
Date:   Fri Jan 19 19:55:03 UTC 2024

Modified Files:
src/lib/libpthread: pthread_cancelstub.c

Log Message:
Add missing headers


To generate a diff of this commit:
cvs rdiff -u -r1.44 -r1.45 src/lib/libpthread/pthread_cancelstub.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_cancelstub.c
diff -u src/lib/libpthread/pthread_cancelstub.c:1.44 src/lib/libpthread/pthread_cancelstub.c:1.45
--- src/lib/libpthread/pthread_cancelstub.c:1.44	Fri Jul 28 14:19:00 2023
+++ src/lib/libpthread/pthread_cancelstub.c	Fri Jan 19 14:55:03 2024
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_cancelstub.c,v 1.44 2023/07/28 18:19:00 christos Exp $	*/
+/*	$NetBSD: pthread_cancelstub.c,v 1.45 2024/01/19 19:55:03 christos Exp $	*/
 
 /*-
  * Copyright (c) 2002, 2007 The NetBSD Foundation, Inc.
@@ -33,7 +33,7 @@
 #undef _FORTIFY_SOURCE
 
 #include 
-__RCSID("$NetBSD: pthread_cancelstub.c,v 1.44 2023/07/28 18:19:00 christos Exp $");
+__RCSID("$NetBSD: pthread_cancelstub.c,v 1.45 2024/01/19 19:55:03 christos Exp $");
 
 /* Need to use libc-private names for atomic operations. */
 #include "../../common/lib/libc/atomic/atomic_op_namespace.h"
@@ -73,12 +73,14 @@ __RCSID("$NetBSD: pthread_cancelstub.c,v
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 

CVS commit: src/lib/libpthread

[Christos Zoulas]

Module Name:src
Committed By:   christos
Date:   Fri Jan 19 19:55:03 UTC 2024

Modified Files:
src/lib/libpthread: pthread_cancelstub.c

Log Message:
Add missing headers


To generate a diff of this commit:
cvs rdiff -u -r1.44 -r1.45 src/lib/libpthread/pthread_cancelstub.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Thu Dec  7 16:55:01 UTC 2023

Modified Files:
src/lib/libpthread: pthread_attr_getguardsize.3 pthread_attr_getstack.3

Log Message:
pthread: Document the setstack vs setguardsize bug.

Suggest the safe, compatible workaround.


To generate a diff of this commit:
cvs rdiff -u -r1.5 -r1.6 src/lib/libpthread/pthread_attr_getguardsize.3
cvs rdiff -u -r1.8 -r1.9 src/lib/libpthread/pthread_attr_getstack.3

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Thu Dec  7 16:55:01 UTC 2023

Modified Files:
src/lib/libpthread: pthread_attr_getguardsize.3 pthread_attr_getstack.3

Log Message:
pthread: Document the setstack vs setguardsize bug.

Suggest the safe, compatible workaround.


To generate a diff of this commit:
cvs rdiff -u -r1.5 -r1.6 src/lib/libpthread/pthread_attr_getguardsize.3
cvs rdiff -u -r1.8 -r1.9 src/lib/libpthread/pthread_attr_getstack.3

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_attr_getguardsize.3
diff -u src/lib/libpthread/pthread_attr_getguardsize.3:1.5 src/lib/libpthread/pthread_attr_getguardsize.3:1.6
--- src/lib/libpthread/pthread_attr_getguardsize.3:1.5	Sun Oct 22 16:37:24 2017
+++ src/lib/libpthread/pthread_attr_getguardsize.3	Thu Dec  7 16:55:01 2023
@@ -1,4 +1,4 @@
-.\"	$NetBSD: pthread_attr_getguardsize.3,v 1.5 2017/10/22 16:37:24 abhinav Exp $
+.\"	$NetBSD: pthread_attr_getguardsize.3,v 1.6 2023/12/07 16:55:01 riastradh Exp $
 .\"
 .\" Copyright (c) 2010 Jukka Ruohonen 
 .\" All rights reserved.
@@ -123,3 +123,27 @@ There was insufficient memory.
 .Sh STANDARDS
 Both functions conform to
 .St -p1003.1-2008 .
+.Sh BUGS
+Older versions of
+.Nx ,
+prior to 10.0, 9.4, and 8.3, incorrectly adjust the stack address by
+the guard size in threads configured with
+.Xr pthread_attr_setstack 3 ,
+instead of ignoring the guard size in that case as
+.Tn POSIX
+prescribes
+.Po
+see
+.Lk https://gnats.NetBSD.org/57721 "PR lib/57721"
+.Pc .
+.Pp
+Even if you didn't set a nonzero guard size with
+.Fn pthread_attr_setguardsize ,
+the system will choose a nonzero default guard size.
+.Pp
+To work around this in applications that run on older and newer
+versions of
+.Nx ,
+as well as on other operating systems, you can safely set the guard
+size to zero:
+.Dl "pthread_attr_setguardsize(, 0);"

Index: src/lib/libpthread/pthread_attr_getstack.3
diff -u src/lib/libpthread/pthread_attr_getstack.3:1.8 src/lib/libpthread/pthread_attr_getstack.3:1.9
--- src/lib/libpthread/pthread_attr_getstack.3:1.8	Mon Oct 23 01:03:23 2017
+++ src/lib/libpthread/pthread_attr_getstack.3	Thu Dec  7 16:55:01 2023
@@ -1,4 +1,4 @@
-.\"	$NetBSD: pthread_attr_getstack.3,v 1.8 2017/10/23 01:03:23 wiz Exp $
+.\"	$NetBSD: pthread_attr_getstack.3,v 1.9 2023/12/07 16:55:01 riastradh Exp $
 .\"
 .\" Copyright (c) 2010 Jukka Ruohonen 
 .\" All rights reserved.
@@ -177,3 +177,27 @@ and
 were however removed from the specification in the
 .St -p1003.1-2008
 revision.
+.Sh BUGS
+Older versions of
+.Nx ,
+prior to 10.0, 9.4, and 8.3, incorrectly adjust the stack address by
+the guard size in threads configured with
+.Fn pthread_attr_setstack ,
+instead of ignoring the guard size in that case as
+.Tn POSIX
+prescribes
+.Po
+see
+.Lk https://gnats.NetBSD.org/57721 "PR lib/57721"
+.Pc .
+.Pp
+Even if you didn't set a nonzero guard size with
+.Xr pthread_attr_setguardsize 3 ,
+the system will choose a nonzero default guard size.
+.Pp
+To work around this in applications that run on older and newer
+versions of
+.Nx ,
+as well as on other operating systems, you can safely set the guard
+size to zero:
+.Dl "pthread_attr_setguardsize(, 0);"

CVS commit: src/lib/libpthread

[Rin Okuyama]

Module Name:src
Committed By:   rin
Date:   Thu Sep 14 03:17:02 UTC 2023

Modified Files:
src/lib/libpthread: Makefile

Log Message:
libpthread/Makefile: trailing whitespace


To generate a diff of this commit:
cvs rdiff -u -r1.97 -r1.98 src/lib/libpthread/Makefile

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/Makefile
diff -u src/lib/libpthread/Makefile:1.97 src/lib/libpthread/Makefile:1.98
--- src/lib/libpthread/Makefile:1.97	Fri Jul 14 23:28:01 2023
+++ src/lib/libpthread/Makefile	Thu Sep 14 03:17:02 2023
@@ -1,4 +1,4 @@
-#	$NetBSD: Makefile,v 1.97 2023/07/14 23:28:01 lukem Exp $
+#	$NetBSD: Makefile,v 1.98 2023/09/14 03:17:02 rin Exp $
 #
 
 NOSANITIZER=	# defined
@@ -17,7 +17,7 @@ ARCHSUBDIR=	${PTHREAD_MACHINE_ARCH}
 ARCHSUBDIR=	${PTHREAD_MACHINE_CPU}
 .elif exists(${.CURDIR}/arch/${MACHINE_ARCH})
 ARCHSUBDIR=	${MACHINE_ARCH}
-.elif exists(${.CURDIR}/arch/${MACHINE_CPU}) 
+.elif exists(${.CURDIR}/arch/${MACHINE_CPU})
 ARCHSUBDIR=	${MACHINE_CPU}
 .else
 .BEGIN:
@@ -53,7 +53,7 @@ PTHREAD_MAKELWP?=	pthread_makelwp_netbsd
 # gets a reference to a symbol in that file.  Otherwise, Unix's stupid static
 # library semantics will end up discarding potentially important objects.
 #
-SRCS=	pthread.c 
+SRCS=	pthread.c
 SRCS+=	pthread_attr.c
 SRCS+=	pthread_barrier.c
 # used by rumprun-posix to work around symbol collisions
@@ -62,7 +62,7 @@ SRCS+=	pthread_cancelstub.c
 .endif
 SRCS+=	pthread_cond.c
 SRCS+=	pthread_getcpuclockid.c
-SRCS+=	pthread_lock.c 
+SRCS+=	pthread_lock.c
 SRCS+=	${PTHREAD_MAKELWP}
 SRCS+=	pthread_misc.c
 SRCS+=	pthread_mutex.c
@@ -195,7 +195,7 @@ MLINKS+=	pthread_mutex.3 pthread_mutex_t
 MLINKS+=	pthread_mutex.3 pthread_mutex_unlock.3
 MLINKS+=	pthread_mutex.3 pthread_mutex_timedlock.3
 MLINKS+=	pthread_mutex.3	pthread_mutex_getprioceiling.3
-MLINKS+=	pthread_mutex.3	pthread_mutex_setprioceiling.3 
+MLINKS+=	pthread_mutex.3	pthread_mutex_setprioceiling.3
 
 MLINKS+=	pthread_mutexattr.3 pthread_mutexattr_init.3
 MLINKS+=	pthread_mutexattr.3 pthread_mutexattr_destroy.3
@@ -206,7 +206,7 @@ MLINKS+=	pthread_mutexattr.3 pthread_mut
 MLINKS+=	pthread_mutexattr.3 pthread_mutexattr_getprotocol.3
 MLINKS+=	pthread_mutexattr.3 pthread_mutexattr_setprotocol.3
 MLINKS+=	pthread_mutexattr.3 pthread_mutexattr_getprioceiling.3
-MLINKS+=	pthread_mutexattr.3 pthread_mutexattr_setprioceiling.3	
+MLINKS+=	pthread_mutexattr.3 pthread_mutexattr_setprioceiling.3
 
 MLINKS+=	pthread_rwlock.3 pthread_rwlock_init.3
 MLINKS+=	pthread_rwlock.3 pthread_rwlock_destroy.3

CVS commit: src/lib/libpthread

[Rin Okuyama]

Module Name:src
Committed By:   rin
Date:   Thu Sep 14 03:17:02 UTC 2023

Modified Files:
src/lib/libpthread: Makefile

Log Message:
libpthread/Makefile: trailing whitespace


To generate a diff of this commit:
cvs rdiff -u -r1.97 -r1.98 src/lib/libpthread/Makefile

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Andrew Doran]

Module Name:src
Committed By:   ad
Date:   Thu Sep  7 19:59:20 UTC 2023

Modified Files:
src/lib/libpthread: pthread.c

Log Message:
raise() has understood threads for a long time, don't reimplment it.


To generate a diff of this commit:
cvs rdiff -u -r1.182 -r1.183 src/lib/libpthread/pthread.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread.c
diff -u src/lib/libpthread/pthread.c:1.182 src/lib/libpthread/pthread.c:1.183
--- src/lib/libpthread/pthread.c:1.182	Fri Mar 24 14:18:18 2023
+++ src/lib/libpthread/pthread.c	Thu Sep  7 19:59:20 2023
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread.c,v 1.182 2023/03/24 14:18:18 joerg Exp $	*/
+/*	$NetBSD: pthread.c,v 1.183 2023/09/07 19:59:20 ad Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
@@ -31,7 +31,7 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread.c,v 1.182 2023/03/24 14:18:18 joerg Exp $");
+__RCSID("$NetBSD: pthread.c,v 1.183 2023/09/07 19:59:20 ad Exp $");
 
 #define	__EXPOSE_STACK	1
 
@@ -1125,7 +1125,7 @@ pthread__errorfunc(const char *file, int
 		syslog(LOG_DEBUG | LOG_USER, "%s", buf);
 
 	if (pthread__diagassert & DIAGASSERT_ABORT) {
-		(void)_lwp_kill(_lwp_self(), SIGABRT);
+		(void)raise(SIGABRT);
 		_exit(1);
 	}
 }

CVS commit: src/lib/libpthread

[Andrew Doran]

Module Name:src
Committed By:   ad
Date:   Thu Sep  7 19:59:20 UTC 2023

Modified Files:
src/lib/libpthread: pthread.c

Log Message:
raise() has understood threads for a long time, don't reimplment it.


To generate a diff of this commit:
cvs rdiff -u -r1.182 -r1.183 src/lib/libpthread/pthread.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Thu May 25 14:30:03 UTC 2023

Modified Files:
src/lib/libpthread: pthread_int.h pthread_spin.c
src/lib/libpthread/arch/aarch64: pthread_md.h
src/lib/libpthread/arch/arm: pthread_md.h
src/lib/libpthread/arch/i386: pthread_md.h
src/lib/libpthread/arch/x86_64: pthread_md.h

Log Message:
libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:

- wfe, wait for event -- if event register is clear, enter low power
  mode and wait until event register is set; then exit low power mode
  and clear event register

- sev, signal event -- sets event register on all CPUs (other
  circumstances like interrupts also set the event register and cause
  wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely.  Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile.  So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386.  No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.

PR port-arm/57437

XXX pullup-10


To generate a diff of this commit:
cvs rdiff -u -r1.110 -r1.111 src/lib/libpthread/pthread_int.h
cvs rdiff -u -r1.10 -r1.11 src/lib/libpthread/pthread_spin.c
cvs rdiff -u -r1.1 -r1.2 src/lib/libpthread/arch/aarch64/pthread_md.h
cvs rdiff -u -r1.12 -r1.13 src/lib/libpthread/arch/arm/pthread_md.h
cvs rdiff -u -r1.20 -r1.21 src/lib/libpthread/arch/i386/pthread_md.h
cvs rdiff -u -r1.12 -r1.13 src/lib/libpthread/arch/x86_64/pthread_md.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_int.h
diff -u src/lib/libpthread/pthread_int.h:1.110 src/lib/libpthread/pthread_int.h:1.111
--- src/lib/libpthread/pthread_int.h:1.110	Thu May 25 14:29:45 2023
+++ src/lib/libpthread/pthread_int.h	Thu May 25 14:30:03 2023
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_int.h,v 1.110 2023/05/25 14:29:45 riastradh Exp $	*/
+/*	$NetBSD: pthread_int.h,v 1.111 2023/05/25 14:30:03 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
@@ -303,6 +303,9 @@ int	pthread__add_specific(pthread_t, pth
 #ifndef pthread__smt_pause
 #define	pthread__smt_pause()	__nothing
 #endif
+#ifndef pthread__smt_wait
+#define	pthread__smt_wait()	__nothing
+#endif
 #ifndef pthread__smt_wake
 #define	pthread__smt_wake()	__nothing
 #endif

Index: src/lib/libpthread/pthread_spin.c
diff -u src/lib/libpthread/pthread_spin.c:1.10 src/lib/libpthread/pthread_spin.c:1.11
--- src/lib/libpthread/pthread_spin.c:1.10	Sun Apr 10 10:38:33 2022
+++ src/lib/libpthread/pthread_spin.c	Thu May 25 14:30:03 2023
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_spin.c,v 1.10 2022/04/10 10:38:33 riastradh Exp $	*/
+/*	$NetBSD: pthread_spin.c,v 1.11 2023/05/25 14:30:03 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2006, 2007 The NetBSD Foundation, Inc.
@@ -34,7 +34,7 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread_spin.c,v 1.10 2022/04/10 10:38:33 riastradh Exp $");
+__RCSID("$NetBSD: pthread_spin.c,v 1.11 2023/05/25 14:30:03 riastradh Exp $");
 
 /* Need to use libc-private names for atomic operations. */
 #include "../../common/lib/libc/atomic/atomic_op_namespace.h"
@@ -98,7 +98,7 @@ pthread_spin_lock(pthread_spinlock_t *lo
 
 	self = pthread__self();
 	while (pthread__spintrylock(self, >pts_spin) == 0) {
-		pthread__smt_pause();
+		pthread__smt_wait();
 	}
 
 	return 0;

Index: src/lib/libpthread/arch/aarch64/pthread_md.h
diff -u src/lib/libpthread/arch/aarch64/pthread_md.h:1.1 src/lib/libpthread/arch/aarch64/pthread_md.h:1.2
--- src/lib/libpthread/arch/aarch64/pthread_md.h:1.1	Sun Aug 10 05:47:37 2014
+++ src/lib/libpthread/arch/aarch64/pthread_md.h	Thu May 25 14:30:02 2023
@@ -1,4 +1,4 @@
-/* $NetBSD: pthread_md.h,v 1.1 2014/08/10 05:47:37 matt Exp $ */
+/* $NetBSD: pthread_md.h,v 1.2 2023/05/25 14:30:02 riastradh Exp $ */
 
 /*-
  * Copyright (c) 2014 The NetBSD Foundation, Inc.
@@ -42,7 +42,7 @@ pthread__sp(void)
 	return ret;
 }
 
-#define pthread__smt_pause()	__asm __volatile("wfe") /* wfe */
+#define pthread__smt_wait()	__asm 

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Thu May 25 14:30:03 UTC 2023

Modified Files:
src/lib/libpthread: pthread_int.h pthread_spin.c
src/lib/libpthread/arch/aarch64: pthread_md.h
src/lib/libpthread/arch/arm: pthread_md.h
src/lib/libpthread/arch/i386: pthread_md.h
src/lib/libpthread/arch/x86_64: pthread_md.h

Log Message:
libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:

- wfe, wait for event -- if event register is clear, enter low power
  mode and wait until event register is set; then exit low power mode
  and clear event register

- sev, signal event -- sets event register on all CPUs (other
  circumstances like interrupts also set the event register and cause
  wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely.  Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile.  So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386.  No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.

PR port-arm/57437

XXX pullup-10


To generate a diff of this commit:
cvs rdiff -u -r1.110 -r1.111 src/lib/libpthread/pthread_int.h
cvs rdiff -u -r1.10 -r1.11 src/lib/libpthread/pthread_spin.c
cvs rdiff -u -r1.1 -r1.2 src/lib/libpthread/arch/aarch64/pthread_md.h
cvs rdiff -u -r1.12 -r1.13 src/lib/libpthread/arch/arm/pthread_md.h
cvs rdiff -u -r1.20 -r1.21 src/lib/libpthread/arch/i386/pthread_md.h
cvs rdiff -u -r1.12 -r1.13 src/lib/libpthread/arch/x86_64/pthread_md.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Thu May 25 14:29:45 UTC 2023

Modified Files:
src/lib/libpthread: pthread_int.h
src/lib/libpthread/arch/arm: pthread_md.h

Log Message:
libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.

PR port-arm/57437 (pthread__smt_pause/wake issue)

XXX pullup-10


To generate a diff of this commit:
cvs rdiff -u -r1.109 -r1.110 src/lib/libpthread/pthread_int.h
cvs rdiff -u -r1.11 -r1.12 src/lib/libpthread/arch/arm/pthread_md.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_int.h
diff -u src/lib/libpthread/pthread_int.h:1.109 src/lib/libpthread/pthread_int.h:1.110
--- src/lib/libpthread/pthread_int.h:1.109	Tue Apr 19 20:32:17 2022
+++ src/lib/libpthread/pthread_int.h	Thu May 25 14:29:45 2023
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_int.h,v 1.109 2022/04/19 20:32:17 rillig Exp $	*/
+/*	$NetBSD: pthread_int.h,v 1.110 2023/05/25 14:29:45 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
@@ -301,10 +301,10 @@ int	pthread__checkpri(int) PTHREAD_HIDE;
 int	pthread__add_specific(pthread_t, pthread_key_t, const void *) PTHREAD_HIDE;
 
 #ifndef pthread__smt_pause
-#define	pthread__smt_pause()	/* nothing */
+#define	pthread__smt_pause()	__nothing
 #endif
 #ifndef pthread__smt_wake
-#define	pthread__smt_wake()	/* nothing */
+#define	pthread__smt_wake()	__nothing
 #endif
 
 /*

Index: src/lib/libpthread/arch/arm/pthread_md.h
diff -u src/lib/libpthread/arch/arm/pthread_md.h:1.11 src/lib/libpthread/arch/arm/pthread_md.h:1.12
--- src/lib/libpthread/arch/arm/pthread_md.h:1.11	Thu Nov 22 20:38:59 2018
+++ src/lib/libpthread/arch/arm/pthread_md.h	Thu May 25 14:29:45 2023
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_md.h,v 1.11 2018/11/22 20:38:59 skrll Exp $	*/
+/*	$NetBSD: pthread_md.h,v 1.12 2023/05/25 14:29:45 riastradh Exp $	*/
 
 /*
  * Copyright (c) 2001 Wasabi Systems, Inc.
@@ -56,8 +56,8 @@ pthread__sp(void)
 #define pthread__smt_pause()	__asm __volatile(".inst 0xe320f002") /* wfe */
 #define pthread__smt_wake()	__asm __volatile(".inst 0xe320f004") /* sev */
 #else
-#define pthread__smt_pause()
-#define pthread__smt_wake()
+#define pthread__smt_pause()	__nothing
+#define pthread__smt_wake()	__nothing
 #endif
 
 #define	pthread__uc_sp(ucp)	((ucp)->uc_mcontext.__gregs[_REG_SP])

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Thu May 25 14:29:45 UTC 2023

Modified Files:
src/lib/libpthread: pthread_int.h
src/lib/libpthread/arch/arm: pthread_md.h

Log Message:
libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.

PR port-arm/57437 (pthread__smt_pause/wake issue)

XXX pullup-10


To generate a diff of this commit:
cvs rdiff -u -r1.109 -r1.110 src/lib/libpthread/pthread_int.h
cvs rdiff -u -r1.11 -r1.12 src/lib/libpthread/arch/arm/pthread_md.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Valeriy E. Ushakov]

Module Name:src
Committed By:   uwe
Date:   Sat Apr 29 21:37:07 UTC 2023

Modified Files:
src/lib/libpthread: pthread_create.3

Log Message:
pthread_create(3): minor markup tweaks


To generate a diff of this commit:
cvs rdiff -u -r1.8 -r1.9 src/lib/libpthread/pthread_create.3

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_create.3
diff -u src/lib/libpthread/pthread_create.3:1.8 src/lib/libpthread/pthread_create.3:1.9
--- src/lib/libpthread/pthread_create.3:1.8	Sat Apr 29 20:51:45 2023
+++ src/lib/libpthread/pthread_create.3	Sat Apr 29 21:37:07 2023
@@ -1,4 +1,4 @@
-.\" $NetBSD: pthread_create.3,v 1.8 2023/04/29 20:51:45 jschauma Exp $
+.\" $NetBSD: pthread_create.3,v 1.9 2023/04/29 21:37:07 uwe Exp $
 .\"
 .\" Copyright (c) 2002 The NetBSD Foundation, Inc.
 .\" All rights reserved.
@@ -64,8 +64,15 @@
 .Lb libpthread
 .Sh SYNOPSIS
 .In pthread.h
+.
 .Ft int
-.Fn pthread_create "pthread_t * restrict thread" "const pthread_attr_t * restrict attr" "void *(*start_routine)(void *)" "void * restrict arg"
+.Fo pthread_create
+.Fa "pthread_t * restrict thread"
+.Fa "const pthread_attr_t * restrict attr"
+.Fa "void *(*start_routine)(void *)"
+.Fa "void * restrict arg"
+.Fc
+.
 .Sh DESCRIPTION
 The
 .Fn pthread_create
@@ -103,7 +110,7 @@ as its sole argument.
 If the
 .Fa start_routine
 returns, the effect is as if there was an implicit call to
-.Fn pthread_exit
+.Xr pthread_exit 3
 using the return value of
 .Fa start_routine
 as the exit status.
@@ -113,7 +120,7 @@ was originally invoked differs from this
 When it returns from
 .Fn main ,
 the effect is as if there was an implicit call to
-.Fn exit
+.Xr exit 3
 using the return value of
 .Fn main
 as the exit status.

CVS commit: src/lib/libpthread

[Valeriy E. Ushakov]

Module Name:src
Committed By:   uwe
Date:   Sat Apr 29 21:37:07 UTC 2023

Modified Files:
src/lib/libpthread: pthread_create.3

Log Message:
pthread_create(3): minor markup tweaks


To generate a diff of this commit:
cvs rdiff -u -r1.8 -r1.9 src/lib/libpthread/pthread_create.3

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Jan Schaumann]

Module Name:src
Committed By:   jschauma
Date:   Sat Apr 29 20:51:45 UTC 2023

Modified Files:
src/lib/libpthread: pthread_create.3

Log Message:
delete spurious space


To generate a diff of this commit:
cvs rdiff -u -r1.7 -r1.8 src/lib/libpthread/pthread_create.3

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_create.3
diff -u src/lib/libpthread/pthread_create.3:1.7 src/lib/libpthread/pthread_create.3:1.8
--- src/lib/libpthread/pthread_create.3:1.7	Fri Jul  9 08:51:28 2010
+++ src/lib/libpthread/pthread_create.3	Sat Apr 29 20:51:45 2023
@@ -1,4 +1,4 @@
-.\" $NetBSD: pthread_create.3,v 1.7 2010/07/09 08:51:28 jruoho Exp $
+.\" $NetBSD: pthread_create.3,v 1.8 2023/04/29 20:51:45 jschauma Exp $
 .\"
 .\" Copyright (c) 2002 The NetBSD Foundation, Inc.
 .\" All rights reserved.
@@ -126,7 +126,7 @@ The signal mask is inherited from the cr
 The set of signals pending for the new thread is empty.
 .El
 .Sh RETURN VALUES
-If successful,  the
+If successful, the
 .Fn pthread_create
 function will return zero.
 Otherwise an error number will be returned to

CVS commit: src/lib/libpthread

[Jan Schaumann]

Module Name:src
Committed By:   jschauma
Date:   Sat Apr 29 20:51:45 UTC 2023

Modified Files:
src/lib/libpthread: pthread_create.3

Log Message:
delete spurious space


To generate a diff of this commit:
cvs rdiff -u -r1.7 -r1.8 src/lib/libpthread/pthread_create.3

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

Module Name:src
Committed By:   joerg
Date:   Fri Mar 24 14:18:19 UTC 2023

Modified Files:
src/lib/libpthread: pthread.c

Log Message:
Use snprintf_ss in pthread__assertfunc and update comment in
pthread__errorfunc. snprintf can use locks in some code paths and we
only care about the restricted subset here.


To generate a diff of this commit:
cvs rdiff -u -r1.181 -r1.182 src/lib/libpthread/pthread.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

Module Name:src
Committed By:   joerg
Date:   Fri Mar 24 14:18:19 UTC 2023

Modified Files:
src/lib/libpthread: pthread.c

Log Message:
Use snprintf_ss in pthread__assertfunc and update comment in
pthread__errorfunc. snprintf can use locks in some code paths and we
only care about the restricted subset here.


To generate a diff of this commit:
cvs rdiff -u -r1.181 -r1.182 src/lib/libpthread/pthread.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread.c
diff -u src/lib/libpthread/pthread.c:1.181 src/lib/libpthread/pthread.c:1.182
--- src/lib/libpthread/pthread.c:1.181	Tue May 31 14:23:39 2022
+++ src/lib/libpthread/pthread.c	Fri Mar 24 14:18:18 2023
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread.c,v 1.181 2022/05/31 14:23:39 riastradh Exp $	*/
+/*	$NetBSD: pthread.c,v 1.182 2023/03/24 14:18:18 joerg Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
@@ -31,7 +31,7 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread.c,v 1.181 2022/05/31 14:23:39 riastradh Exp $");
+__RCSID("$NetBSD: pthread.c,v 1.182 2023/03/24 14:18:18 joerg Exp $");
 
 #define	__EXPOSE_STACK	1
 
@@ -1073,10 +1073,10 @@ pthread__assertfunc(const char *file, in
 	int len;
 
 	/*
-	 * snprintf should not acquire any locks, or we could
+	 * snprintf_ss should not acquire any locks, or we could
 	 * end up deadlocked if the assert caller held locks.
 	 */
-	len = snprintf(buf, 1024,
+	len = snprintf_ss(buf, 1024,
 	"assertion \"%s\" failed: file \"%s\", line %d%s%s%s\n",
 	expr, file, line,
 	function ? ", function \"" : "",
@@ -1106,7 +1106,7 @@ pthread__errorfunc(const char *file, int
 	va_end(ap);
 
 	/*
-	 * snprintf should not acquire any locks, or we could
+	 * snprintf_ss should not acquire any locks, or we could
 	 * end up deadlocked if the assert caller held locks.
 	 */
 	len = snprintf_ss(buf, sizeof(buf),

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Tue May 31 14:23:39 UTC 2022

Modified Files:
src/lib/libpthread: pthread.c

Log Message:
libpthread(3): Fix a marvellous interaction with rtld.

Patch from chs@.  Comment explaining the story by me.  This patch may
not be optimal -- maybe it would be better in pthread__init, or
better for rtld to call _lwp_unpark after _lwp_park in the contened
case -- but we've tested this version and it's annoying to reproduce,
so let's take this version and worry about testing improvements
later.


To generate a diff of this commit:
cvs rdiff -u -r1.180 -r1.181 src/lib/libpthread/pthread.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread.c
diff -u src/lib/libpthread/pthread.c:1.180 src/lib/libpthread/pthread.c:1.181
--- src/lib/libpthread/pthread.c:1.180	Sat Feb 12 14:59:32 2022
+++ src/lib/libpthread/pthread.c	Tue May 31 14:23:39 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread.c,v 1.180 2022/02/12 14:59:32 riastradh Exp $	*/
+/*	$NetBSD: pthread.c,v 1.181 2022/05/31 14:23:39 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
@@ -31,7 +31,7 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread.c,v 1.180 2022/02/12 14:59:32 riastradh Exp $");
+__RCSID("$NetBSD: pthread.c,v 1.181 2022/05/31 14:23:39 riastradh Exp $");
 
 #define	__EXPOSE_STACK	1
 
@@ -414,6 +414,34 @@ pthread_create(pthread_t *thread, const 
 	else
 		return EINVAL;
 
+	if (!pthread__started) {
+		/*
+		 * Force the _lwp_park symbol to be resolved before we
+		 * begin any activity that might rely on concurrent
+		 * wakeups.
+		 *
+		 * This is necessary because rtld itself uses _lwp_park
+		 * and _lwp_unpark internally for its own locking: If
+		 * we wait to resolve _lwp_park until there is an
+		 * _lwp_unpark from another thread pending in the
+		 * current lwp (for example, pthread_mutex_unlock or
+		 * pthread_cond_signal), rtld's internal use of
+		 * _lwp_park might consume the pending unpark.  The
+		 * result is a deadlock where libpthread and rtld have
+		 * both correctly used _lwp_park and _lwp_unpark for
+		 * themselves, but rtld has consumed the wakeup meant
+		 * for libpthread so it is lost to libpthread.
+		 *
+		 * For the very first thread, before pthread__started
+		 * is set to true, pthread__self()->pt_lid should have
+		 * been initialized in pthread__init by the time we get
+		 * here to the correct lid so we go to sleep and wake
+		 * ourselves at the same time as a no-op.
+		 */
+		_lwp_park(CLOCK_REALTIME, 0, NULL, pthread__self()->pt_lid,
+		NULL, NULL);
+	}
+
 	pthread__started = 1;
 
 	/* Fetch misc. attributes from the attr structure. */

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Tue May 31 14:23:39 UTC 2022

Modified Files:
src/lib/libpthread: pthread.c

Log Message:
libpthread(3): Fix a marvellous interaction with rtld.

Patch from chs@.  Comment explaining the story by me.  This patch may
not be optimal -- maybe it would be better in pthread__init, or
better for rtld to call _lwp_unpark after _lwp_park in the contened
case -- but we've tested this version and it's annoying to reproduce,
so let's take this version and worry about testing improvements
later.


To generate a diff of this commit:
cvs rdiff -u -r1.180 -r1.181 src/lib/libpthread/pthread.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread/arch/hppa

[Nick Hudson]

Module Name:src
Committed By:   skrll
Date:   Thu May 19 07:10:15 UTC 2022

Modified Files:
src/lib/libpthread/arch/hppa: pthread_md.S

Log Message:
Use RAS_{START,END}_ASM_HIDDEN


To generate a diff of this commit:
cvs rdiff -u -r1.1 -r1.2 src/lib/libpthread/arch/hppa/pthread_md.S

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/arch/hppa/pthread_md.S
diff -u src/lib/libpthread/arch/hppa/pthread_md.S:1.1 src/lib/libpthread/arch/hppa/pthread_md.S:1.2
--- src/lib/libpthread/arch/hppa/pthread_md.S:1.1	Sat May 16 22:20:40 2009
+++ src/lib/libpthread/arch/hppa/pthread_md.S	Thu May 19 07:10:15 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_md.S,v 1.1 2009/05/16 22:20:40 ad Exp $	*/
+/*	$NetBSD: pthread_md.S,v 1.2 2022/05/19 07:10:15 skrll Exp $	*/
 
 /*
  * Copyright (c) 2001 The NetBSD Foundation, Inc.
@@ -29,6 +29,7 @@
  * POSSIBILITY OF SUCH DAMAGE.
  */
 
+#include 
 #include 
 
 LEAF_ENTRY(pthread__ras_simple_lock_init)
@@ -47,10 +48,10 @@ LEAF_ENTRY(pthread__ras_simple_lock_try)
 	ldo	15(%arg0),%arg0
 	depi	0,31,4,%arg0
 
-pthread__lock_ras_start:
+RAS_START_ASM_HIDDEN(pthread__lock)
 	ldw	0(%arg0),%ret0
 	stw	%r0,0(%arg0)		/* 0 == locked */
-pthread__lock_ras_end:
+RAS_END_ASM_HIDDEN(pthread__lock)
 
 	comiclr,= 0,%ret0,%ret0		/* if locked return 0 */
 	ldi	1,%ret0			/* else return 1 */

CVS commit: src/lib/libpthread/arch/hppa

[Nick Hudson]

Module Name:src
Committed By:   skrll
Date:   Thu May 19 07:10:15 UTC 2022

Modified Files:
src/lib/libpthread/arch/hppa: pthread_md.S

Log Message:
Use RAS_{START,END}_ASM_HIDDEN


To generate a diff of this commit:
cvs rdiff -u -r1.1 -r1.2 src/lib/libpthread/arch/hppa/pthread_md.S

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Sun Apr 10 10:38:33 UTC 2022

Modified Files:
src/lib/libpthread: pthread_attr.c pthread_cancelstub.c pthread_lock.c
pthread_misc.c pthread_mutex.c pthread_queue.h pthread_specific.c
pthread_spin.c pthread_tsd.c pthread_types.h

Log Message:
pthread: Nix trailing whitespace.


To generate a diff of this commit:
cvs rdiff -u -r1.20 -r1.21 src/lib/libpthread/pthread_attr.c
cvs rdiff -u -r1.41 -r1.42 src/lib/libpthread/pthread_cancelstub.c
cvs rdiff -u -r1.35 -r1.36 src/lib/libpthread/pthread_lock.c
cvs rdiff -u -r1.18 -r1.19 src/lib/libpthread/pthread_misc.c
cvs rdiff -u -r1.82 -r1.83 src/lib/libpthread/pthread_mutex.c
cvs rdiff -u -r1.5 -r1.6 src/lib/libpthread/pthread_queue.h
cvs rdiff -u -r1.27 -r1.28 src/lib/libpthread/pthread_specific.c
cvs rdiff -u -r1.9 -r1.10 src/lib/libpthread/pthread_spin.c
cvs rdiff -u -r1.24 -r1.25 src/lib/libpthread/pthread_tsd.c
cvs rdiff -u -r1.26 -r1.27 src/lib/libpthread/pthread_types.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_attr.c
diff -u src/lib/libpthread/pthread_attr.c:1.20 src/lib/libpthread/pthread_attr.c:1.21
--- src/lib/libpthread/pthread_attr.c:1.20	Sat Feb 12 14:59:32 2022
+++ src/lib/libpthread/pthread_attr.c	Sun Apr 10 10:38:33 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_attr.c,v 1.20 2022/02/12 14:59:32 riastradh Exp $	*/
+/*	$NetBSD: pthread_attr.c,v 1.21 2022/04/10 10:38:33 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2008 The NetBSD Foundation, Inc.
@@ -30,7 +30,7 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread_attr.c,v 1.20 2022/02/12 14:59:32 riastradh Exp $");
+__RCSID("$NetBSD: pthread_attr.c,v 1.21 2022/04/10 10:38:33 riastradh Exp $");
 
 /* Need to use libc-private names for atomic operations. */
 #include "../../common/lib/libc/atomic/atomic_op_namespace.h"
@@ -113,7 +113,7 @@ pthread_attr_get_np(pthread_t thread, pt
 	if (p == NULL)
 		return ENOMEM;
 
-	attr->pta_flags = thread->pt_flags & 
+	attr->pta_flags = thread->pt_flags &
 	(PT_FLAG_DETACHED | PT_FLAG_SCOPE_SYSTEM | PT_FLAG_EXPLICIT_SCHED);
 
 	p->ptap_namearg = thread->pt_name;
@@ -309,7 +309,7 @@ pthread_attr_getschedparam(const pthread
 	p = attr->pta_private;
 	if (p == NULL)
 		memset(param, 0, sizeof(*param));
-	else 
+	else
 		*param = p->ptap_sp;
 	return 0;
 }

Index: src/lib/libpthread/pthread_cancelstub.c
diff -u src/lib/libpthread/pthread_cancelstub.c:1.41 src/lib/libpthread/pthread_cancelstub.c:1.42
--- src/lib/libpthread/pthread_cancelstub.c:1.41	Sat Feb 12 14:59:32 2022
+++ src/lib/libpthread/pthread_cancelstub.c	Sun Apr 10 10:38:33 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_cancelstub.c,v 1.41 2022/02/12 14:59:32 riastradh Exp $	*/
+/*	$NetBSD: pthread_cancelstub.c,v 1.42 2022/04/10 10:38:33 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2002, 2007 The NetBSD Foundation, Inc.
@@ -33,7 +33,7 @@
 #undef _FORTIFY_SOURCE
 
 #include 
-__RCSID("$NetBSD: pthread_cancelstub.c,v 1.41 2022/02/12 14:59:32 riastradh Exp $");
+__RCSID("$NetBSD: pthread_cancelstub.c,v 1.42 2022/04/10 10:38:33 riastradh Exp $");
 
 /* Need to use libc-private names for atomic operations. */
 #include "../../common/lib/libc/atomic/atomic_op_namespace.h"
@@ -160,7 +160,7 @@ accept(int s, struct sockaddr *addr, soc
 	TESTCANCEL(self);
 	retval = _sys_accept(s, addr, addrlen);
 	TESTCANCEL(self);
-	
+
 	return retval;
 }
 
@@ -205,7 +205,7 @@ clock_nanosleep(clockid_t clock_id, int 
 	TESTCANCEL(self);
 	retval = _sys_clock_nanosleep(clock_id, flags, rqtp, rmtp);
 	TESTCANCEL(self);
-	
+
 	return retval;
 }
 
@@ -219,7 +219,7 @@ close(int d)
 	TESTCANCEL(self);
 	retval = _sys_close(d);
 	TESTCANCEL(self);
-	
+
 	return retval;
 }
 
@@ -233,7 +233,7 @@ connect(int s, const struct sockaddr *ad
 	TESTCANCEL(self);
 	retval = _sys_connect(s, addr, namelen);
 	TESTCANCEL(self);
-	
+
 	return retval;
 }
 
@@ -264,7 +264,7 @@ fdatasync(int d)
 	TESTCANCEL(self);
 	retval = _sys_fdatasync(d);
 	TESTCANCEL(self);
-	
+
 	return retval;
 }
 
@@ -278,7 +278,7 @@ fsync(int d)
 	TESTCANCEL(self);
 	retval = _sys_fsync(d);
 	TESTCANCEL(self);
-	
+
 	return retval;
 }
 
@@ -492,7 +492,7 @@ pread(int d, void *buf, size_t nbytes, o
 }
 
 int
-__pselect50(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, 
+__pselect50(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
 const struct timespec *timeout, const sigset_t *sigmask)
 {
 	int retval;
@@ -594,7 +594,7 @@ recvmmsg(int s, struct mmsghdr *mmsg, un
 }
 
 int
-__select50(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, 
+__select50(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
 struct timeval *timeout)
 {
 	int retval;

Index: src/lib/libpthread/pthread_lock.c
diff -u src/lib/libpthread/pthread_lock.c:1.35 src/lib/libpthread/pthread_lock.c:1.36
--- 

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Sun Apr 10 10:38:33 UTC 2022

Modified Files:
src/lib/libpthread: pthread_attr.c pthread_cancelstub.c pthread_lock.c
pthread_misc.c pthread_mutex.c pthread_queue.h pthread_specific.c
pthread_spin.c pthread_tsd.c pthread_types.h

Log Message:
pthread: Nix trailing whitespace.


To generate a diff of this commit:
cvs rdiff -u -r1.20 -r1.21 src/lib/libpthread/pthread_attr.c
cvs rdiff -u -r1.41 -r1.42 src/lib/libpthread/pthread_cancelstub.c
cvs rdiff -u -r1.35 -r1.36 src/lib/libpthread/pthread_lock.c
cvs rdiff -u -r1.18 -r1.19 src/lib/libpthread/pthread_misc.c
cvs rdiff -u -r1.82 -r1.83 src/lib/libpthread/pthread_mutex.c
cvs rdiff -u -r1.5 -r1.6 src/lib/libpthread/pthread_queue.h
cvs rdiff -u -r1.27 -r1.28 src/lib/libpthread/pthread_specific.c
cvs rdiff -u -r1.9 -r1.10 src/lib/libpthread/pthread_spin.c
cvs rdiff -u -r1.24 -r1.25 src/lib/libpthread/pthread_tsd.c
cvs rdiff -u -r1.26 -r1.27 src/lib/libpthread/pthread_types.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Sat Feb 12 14:59:32 UTC 2022

Modified Files:
src/lib/libpthread: pthread.c pthread_attr.c pthread_barrier.c
pthread_cancelstub.c pthread_compat.c pthread_cond.c
pthread_getcpuclockid.c pthread_int.h pthread_lock.c
pthread_makelwp_netbsd.c pthread_misc.c pthread_mutex.c
pthread_once.c pthread_rwlock.c pthread_specific.c pthread_spin.c
pthread_tsd.c res_state.c

Log Message:
libpthread: Move namespacing include to top of .c files.

Stuff like libc's namespace.h, or atomic_op_namespace.h, which does
namespacing tricks like `#define atomic_cas_uint _atomic_cas_uint',
has to go at the top of each .c file.  If it goes in the middle, it
might be too late to affect the declarations, and result in compile
errors.

I tripped over this by including  in mips
.

(Maybe we should create a new pthread_namespace.h file for the
purpose, but this'll do for now.)


To generate a diff of this commit:
cvs rdiff -u -r1.179 -r1.180 src/lib/libpthread/pthread.c
cvs rdiff -u -r1.19 -r1.20 src/lib/libpthread/pthread_attr.c
cvs rdiff -u -r1.22 -r1.23 src/lib/libpthread/pthread_barrier.c
cvs rdiff -u -r1.40 -r1.41 src/lib/libpthread/pthread_cancelstub.c
cvs rdiff -u -r1.6 -r1.7 src/lib/libpthread/pthread_compat.c \
src/lib/libpthread/res_state.c
cvs rdiff -u -r1.76 -r1.77 src/lib/libpthread/pthread_cond.c
cvs rdiff -u -r1.3 -r1.4 src/lib/libpthread/pthread_getcpuclockid.c \
src/lib/libpthread/pthread_once.c
cvs rdiff -u -r1.107 -r1.108 src/lib/libpthread/pthread_int.h
cvs rdiff -u -r1.34 -r1.35 src/lib/libpthread/pthread_lock.c
cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_makelwp_netbsd.c
cvs rdiff -u -r1.17 -r1.18 src/lib/libpthread/pthread_misc.c
cvs rdiff -u -r1.81 -r1.82 src/lib/libpthread/pthread_mutex.c
cvs rdiff -u -r1.43 -r1.44 src/lib/libpthread/pthread_rwlock.c
cvs rdiff -u -r1.26 -r1.27 src/lib/libpthread/pthread_specific.c
cvs rdiff -u -r1.8 -r1.9 src/lib/libpthread/pthread_spin.c
cvs rdiff -u -r1.23 -r1.24 src/lib/libpthread/pthread_tsd.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread.c
diff -u src/lib/libpthread/pthread.c:1.179 src/lib/libpthread/pthread.c:1.180
--- src/lib/libpthread/pthread.c:1.179	Tue Apr 13 00:31:54 2021
+++ src/lib/libpthread/pthread.c	Sat Feb 12 14:59:32 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread.c,v 1.179 2021/04/13 00:31:54 mrg Exp $	*/
+/*	$NetBSD: pthread.c,v 1.180 2022/02/12 14:59:32 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
@@ -31,10 +31,13 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread.c,v 1.179 2021/04/13 00:31:54 mrg Exp $");
+__RCSID("$NetBSD: pthread.c,v 1.180 2022/02/12 14:59:32 riastradh Exp $");
 
 #define	__EXPOSE_STACK	1
 
+/* Need to use libc-private names for atomic operations. */
+#include "../../common/lib/libc/atomic/atomic_op_namespace.h"
+
 #include 
 #include 
 #include 

Index: src/lib/libpthread/pthread_attr.c
diff -u src/lib/libpthread/pthread_attr.c:1.19 src/lib/libpthread/pthread_attr.c:1.20
--- src/lib/libpthread/pthread_attr.c:1.19	Wed Jan 29 13:47:31 2020
+++ src/lib/libpthread/pthread_attr.c	Sat Feb 12 14:59:32 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_attr.c,v 1.19 2020/01/29 13:47:31 kamil Exp $	*/
+/*	$NetBSD: pthread_attr.c,v 1.20 2022/02/12 14:59:32 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2002, 2003, 2008 The NetBSD Foundation, Inc.
@@ -30,7 +30,10 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread_attr.c,v 1.19 2020/01/29 13:47:31 kamil Exp $");
+__RCSID("$NetBSD: pthread_attr.c,v 1.20 2022/02/12 14:59:32 riastradh Exp $");
+
+/* Need to use libc-private names for atomic operations. */
+#include "../../common/lib/libc/atomic/atomic_op_namespace.h"
 
 #include 
 #include 

Index: src/lib/libpthread/pthread_barrier.c
diff -u src/lib/libpthread/pthread_barrier.c:1.22 src/lib/libpthread/pthread_barrier.c:1.23
--- src/lib/libpthread/pthread_barrier.c:1.22	Sat May 16 22:53:37 2020
+++ src/lib/libpthread/pthread_barrier.c	Sat Feb 12 14:59:32 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_barrier.c,v 1.22 2020/05/16 22:53:37 ad Exp $	*/
+/*	$NetBSD: pthread_barrier.c,v 1.23 2022/02/12 14:59:32 riastradh Exp $	*/
 
 /*-
  * Copyright (c) 2001, 2003, 2006, 2007, 2009, 2020 The NetBSD Foundation, Inc.
@@ -30,7 +30,10 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread_barrier.c,v 1.22 2020/05/16 22:53:37 ad Exp $");
+__RCSID("$NetBSD: pthread_barrier.c,v 1.23 2022/02/12 14:59:32 riastradh Exp $");
+
+/* Need to use libc-private names for atomic operations. */
+#include "../../common/lib/libc/atomic/atomic_op_namespace.h"
 
 #include 
 

Index: src/lib/libpthread/pthread_cancelstub.c
diff -u src/lib/libpthread/pthread_cancelstub.c:1.40 src/lib/libpthread/pthread_cancelstub.c:1.41
--- src/lib/libpthread/pthread_cancelstub.c:1.40	Fri Oct  1 20:13:38 2021
+++ 

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Sat Feb 12 14:59:32 UTC 2022

Modified Files:
src/lib/libpthread: pthread.c pthread_attr.c pthread_barrier.c
pthread_cancelstub.c pthread_compat.c pthread_cond.c
pthread_getcpuclockid.c pthread_int.h pthread_lock.c
pthread_makelwp_netbsd.c pthread_misc.c pthread_mutex.c
pthread_once.c pthread_rwlock.c pthread_specific.c pthread_spin.c
pthread_tsd.c res_state.c

Log Message:
libpthread: Move namespacing include to top of .c files.

Stuff like libc's namespace.h, or atomic_op_namespace.h, which does
namespacing tricks like `#define atomic_cas_uint _atomic_cas_uint',
has to go at the top of each .c file.  If it goes in the middle, it
might be too late to affect the declarations, and result in compile
errors.

I tripped over this by including  in mips
.

(Maybe we should create a new pthread_namespace.h file for the
purpose, but this'll do for now.)


To generate a diff of this commit:
cvs rdiff -u -r1.179 -r1.180 src/lib/libpthread/pthread.c
cvs rdiff -u -r1.19 -r1.20 src/lib/libpthread/pthread_attr.c
cvs rdiff -u -r1.22 -r1.23 src/lib/libpthread/pthread_barrier.c
cvs rdiff -u -r1.40 -r1.41 src/lib/libpthread/pthread_cancelstub.c
cvs rdiff -u -r1.6 -r1.7 src/lib/libpthread/pthread_compat.c \
src/lib/libpthread/res_state.c
cvs rdiff -u -r1.76 -r1.77 src/lib/libpthread/pthread_cond.c
cvs rdiff -u -r1.3 -r1.4 src/lib/libpthread/pthread_getcpuclockid.c \
src/lib/libpthread/pthread_once.c
cvs rdiff -u -r1.107 -r1.108 src/lib/libpthread/pthread_int.h
cvs rdiff -u -r1.34 -r1.35 src/lib/libpthread/pthread_lock.c
cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_makelwp_netbsd.c
cvs rdiff -u -r1.17 -r1.18 src/lib/libpthread/pthread_misc.c
cvs rdiff -u -r1.81 -r1.82 src/lib/libpthread/pthread_mutex.c
cvs rdiff -u -r1.43 -r1.44 src/lib/libpthread/pthread_rwlock.c
cvs rdiff -u -r1.26 -r1.27 src/lib/libpthread/pthread_specific.c
cvs rdiff -u -r1.8 -r1.9 src/lib/libpthread/pthread_spin.c
cvs rdiff -u -r1.23 -r1.24 src/lib/libpthread/pthread_tsd.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Fri Feb 11 21:40:59 UTC 2022

Modified Files:
src/lib/libpthread: pthread_rwlock.c

Log Message:
libpthread: Fix membars around rwlocks.

1. After loading self->pt_rwlocked, membar_enter() must not be
   conditional on PTHREAD__ATOMIC_IS_MEMBAR because there is no
   atomic r/m/w operation here which could imply the acquire barrier.

   (This should maybe just be a load-acquire operation, but we don't
   have atomic_load_acquire in userland at the moment -- TBD.)

2. Before storing thread->pt_rwlocked, must issue membar_exit() so
   that this is a store-release operation -- except if we had just
   done an atomic r/m/w and PTHREAD__ATOMIC_IS_MEMBAR is set, in
   which case it can be elided.

   The second membar_exit() added here might be safely hoisted out of
   the loop but I'm not sure -- needs more analysis to prove that
   would be safe.


To generate a diff of this commit:
cvs rdiff -u -r1.42 -r1.43 src/lib/libpthread/pthread_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_rwlock.c
diff -u src/lib/libpthread/pthread_rwlock.c:1.42 src/lib/libpthread/pthread_rwlock.c:1.43
--- src/lib/libpthread/pthread_rwlock.c:1.42	Tue Jun  2 00:29:53 2020
+++ src/lib/libpthread/pthread_rwlock.c	Fri Feb 11 21:40:58 2022
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_rwlock.c,v 1.42 2020/06/02 00:29:53 joerg Exp $ */
+/*	$NetBSD: pthread_rwlock.c,v 1.43 2022/02/11 21:40:58 riastradh Exp $ */
 
 /*-
  * Copyright (c) 2002, 2006, 2007, 2008, 2020 The NetBSD Foundation, Inc.
@@ -30,7 +30,7 @@
  */
 
 #include 
-__RCSID("$NetBSD: pthread_rwlock.c,v 1.42 2020/06/02 00:29:53 joerg Exp $");
+__RCSID("$NetBSD: pthread_rwlock.c,v 1.43 2022/02/11 21:40:58 riastradh Exp $");
 
 #include 
 #include 
@@ -228,9 +228,7 @@ pthread__rwlock_rdlock(pthread_rwlock_t 
 
 		/* Did we get the lock? */
 		if (self->pt_rwlocked == _RW_LOCKED) {
-#ifndef PTHREAD__ATOMIC_IS_MEMBAR
 			membar_enter();
-#endif
 			return 0;
 		}
 		if (error != 0)
@@ -352,9 +350,7 @@ pthread__rwlock_wrlock(pthread_rwlock_t 
 
 		/* Did we get the lock? */
 		if (self->pt_rwlocked == _RW_LOCKED) {
-#ifndef PTHREAD__ATOMIC_IS_MEMBAR
 			membar_enter();
-#endif
 			return 0;
 		}
 		if (error != 0)
@@ -526,6 +522,9 @@ pthread_rwlock_unlock(pthread_rwlock_t *
 			 * by the writer that we are about to wake.
 			 */
 			(void)atomic_swap_ptr(>ptr_owner, (void *)new);
+#ifndef PTHREAD__ATOMIC_IS_MEMBAR
+			membar_exit();
+#endif
 
 			/* Wake the writer. */
 			thread->pt_rwlocked = _RW_LOCKED;
@@ -543,6 +542,7 @@ pthread_rwlock_unlock(pthread_rwlock_t *
 if (thread->pt_sleepobj == NULL)
 	continue;
 new += RW_READ_INCR;
+membar_exit();
 thread->pt_rwlocked = _RW_LOCKED;
 			}
 

CVS commit: src/lib/libpthread

[Taylor R Campbell]

Module Name:src
Committed By:   riastradh
Date:   Fri Feb 11 21:40:59 UTC 2022

Modified Files:
src/lib/libpthread: pthread_rwlock.c

Log Message:
libpthread: Fix membars around rwlocks.

1. After loading self->pt_rwlocked, membar_enter() must not be
   conditional on PTHREAD__ATOMIC_IS_MEMBAR because there is no
   atomic r/m/w operation here which could imply the acquire barrier.

   (This should maybe just be a load-acquire operation, but we don't
   have atomic_load_acquire in userland at the moment -- TBD.)

2. Before storing thread->pt_rwlocked, must issue membar_exit() so
   that this is a store-release operation -- except if we had just
   done an atomic r/m/w and PTHREAD__ATOMIC_IS_MEMBAR is set, in
   which case it can be elided.

   The second membar_exit() added here might be safely hoisted out of
   the loop but I'm not sure -- needs more analysis to prove that
   would be safe.


To generate a diff of this commit:
cvs rdiff -u -r1.42 -r1.43 src/lib/libpthread/pthread_rwlock.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Christos Zoulas]

Module Name:src
Committed By:   christos
Date:   Fri Oct  1 20:13:38 UTC 2021

Modified Files:
src/lib/libpthread: pthread_cancelstub.c

Log Message:
remove parameter names from decls.


To generate a diff of this commit:
cvs rdiff -u -r1.39 -r1.40 src/lib/libpthread/pthread_cancelstub.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/pthread_cancelstub.c
diff -u src/lib/libpthread/pthread_cancelstub.c:1.39 src/lib/libpthread/pthread_cancelstub.c:1.40
--- src/lib/libpthread/pthread_cancelstub.c:1.39	Fri Oct  1 13:13:44 2021
+++ src/lib/libpthread/pthread_cancelstub.c	Fri Oct  1 16:13:38 2021
@@ -1,4 +1,4 @@
-/*	$NetBSD: pthread_cancelstub.c,v 1.39 2021/10/01 17:13:44 christos Exp $	*/
+/*	$NetBSD: pthread_cancelstub.c,v 1.40 2021/10/01 20:13:38 christos Exp $	*/
 
 /*-
  * Copyright (c) 2002, 2007 The NetBSD Foundation, Inc.
@@ -33,7 +33,7 @@
 #undef _FORTIFY_SOURCE
 
 #include 
-__RCSID("$NetBSD: pthread_cancelstub.c,v 1.39 2021/10/01 17:13:44 christos Exp $");
+__RCSID("$NetBSD: pthread_cancelstub.c,v 1.40 2021/10/01 20:13:38 christos Exp $");
 
 #ifndef lint
 
@@ -127,11 +127,10 @@ ssize_t	_sys_recvfrom(int, void * restri
 ssize_t _sys_recvmsg(int, struct msghdr *, int);
 int _sys_recvmmsg(int, struct mmsghdr *, unsigned int, unsigned int,
 struct timespec *);
-ssize_t _sys_sendto(int s, const void *msg, size_t len, int flags,
-const struct sockaddr *to, socklen_t tolen);
-ssize_t _sys_sendmsg(int s, const struct msghdr *msg, int flags);
-int _sys_sendmmsg(int s, struct mmsghdr *mmsg, unsigned int vlen,
-unsigned int flags);
+ssize_t _sys_sendto(int, const void *, size_t, int, const struct sockaddr *,
+socklen_t);
+ssize_t _sys_sendmsg(int, const struct msghdr *, int);
+int _sys_sendmmsg(int, struct mmsghdr *, unsigned int, unsigned int);
 int	_sys___select50(int, fd_set *, fd_set *, fd_set *, struct timeval *);
 int	_sys___wait450(pid_t, int *, int, struct rusage *);
 ssize_t	_sys_write(int, const void *, size_t);

CVS commit: src/lib/libpthread

[Christos Zoulas]

Module Name:src
Committed By:   christos
Date:   Fri Oct  1 20:13:38 UTC 2021

Modified Files:
src/lib/libpthread: pthread_cancelstub.c

Log Message:
remove parameter names from decls.


To generate a diff of this commit:
cvs rdiff -u -r1.39 -r1.40 src/lib/libpthread/pthread_cancelstub.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Re: CVS commit: src/lib/libpthread

[Andrew Doran]

On Thu, Jun 04, 2020 at 06:26:07PM +, Martin Husemann wrote:

> On Wed, Jun 03, 2020 at 10:10:24PM +, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Wed Jun  3 22:10:24 UTC 2020
> > 
> > Modified Files:
> > src/lib/libpthread: pthread.c pthread_cond.c pthread_mutex.c
> > 
> > Log Message:
> > Deal with a couple of problems with threads being awoken early due to
> > timeouts or cancellation where:
> 
> Not sure if it is caused by this commit or joergs TSD/malloc, but today
> most of the libpthread tests time out on aarch64, while everything but
> a few minor nits was fine on May 30.

I tried with everything updated and it seems to be hanging on exit in
jemalloc.  Haven't looked further yet.

http://www.netbsd.org/~ad/2020/barrier.txt

Andrew


 
> Martin
> 
> (this is on a hummingboard pulse 4-core board)
> lib/libpthread/t_barrier (398/848): 1 test cases
> barrier: [300.034098s] Failed: Test case timed out after 300 seconds
> [300.034568s]
> 
> lib/libpthread/t_cond (399/848): 9 test cases
> bogus_timedwaits: [300.016677s] Failed: Test case timed out after 300 
> seconds
> broadcast: [300.023040s] Failed: Test case timed out after 300 seconds
> cond_timedwait_race: [300.034567s] Failed: Test case timed out after 300 
> seconds
> condattr: [0.013756s] Passed.
> destroy_after_cancel: [300.025649s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock: [300.023630s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock_static_init: [300.022909s] Failed: Test case timed 
> out after 300 seconds
> signal_delay_wait: [300.031310s] Failed: Test case timed out after 300 
> seconds
> signal_wait_race: [300.025552s] Failed: Test case timed out after 300 
> seconds
> [2400.221956s]
> 
> lib/libpthread/t_condwait (400/848): 2 test cases
> cond_wait_mono: [2.023083s] Passed.
> cond_wait_real: [2.017972s] Passed.
> [4.042078s]
> 
> lib/libpthread/t_detach (401/848): 1 test cases
> pthread_detach: [300.021242s] Failed: Test case timed out after 300 
> seconds
> [300.021708s]
> 
> lib/libpthread/t_equal (402/848): 1 test cases
> pthread_equal: [300.027646s] Failed: Test case timed out after 300 seconds
> [300.028295s]
> 
> lib/libpthread/t_fork (403/848): 1 test cases
> fork: [300.018077s] Failed: Test case timed out after 300 seconds
> [300.018975s]
> 
> lib/libpthread/t_fpu (404/848): 1 test cases
> fpu: [0.020578s] Passed.
> [0.021314s]
> 
> lib/libpthread/t_join (405/848): 1 test cases
> pthread_join: [0.021632s] Passed.
> [0.022411s]
> 
> lib/libpthread/t_kill (406/848): 1 test cases
> simple: [300.033759s] Failed: Test case timed out after 300 seconds
> [300.034528s]
> 
> lib/libpthread/t_mutex (407/848): 7 test cases
> mutex1: [300.017542s] Failed: Test case timed out after 300 seconds
> mutex2: [300.023064s] Failed: Test case timed out after 300 seconds
> mutex3: [300.023259s] Failed: Test case timed out after 300 seconds
> mutex4: [300.023015s] Failed: Test case timed out after 300 seconds
> mutex5: [300.023495s] Failed: Test case timed out after 300 seconds
> mutexattr1: [0.013769s] Passed.
> mutexattr2: [0.025483s] Passed.
> [1500.154160s]
> 
> lib/libpthread/t_name (408/848): 1 test cases
> name: [0.014262s] Passed.
> [0.014951s]
> 
> ...

Re: CVS commit: src/lib/libpthread

[Andrew Doran]

On Thu, Jun 04, 2020 at 09:28:09PM +, Andrew Doran wrote:

> On Thu, Jun 04, 2020 at 06:26:07PM +, Martin Husemann wrote:
> 
> > On Wed, Jun 03, 2020 at 10:10:24PM +, Andrew Doran wrote:
> > > Module Name:  src
> > > Committed By: ad
> > > Date: Wed Jun  3 22:10:24 UTC 2020
> > > 
> > > Modified Files:
> > >   src/lib/libpthread: pthread.c pthread_cond.c pthread_mutex.c
> > > 
> > > Log Message:
> > > Deal with a couple of problems with threads being awoken early due to
> > > timeouts or cancellation where:
> > 
> > Not sure if it is caused by this commit or joergs TSD/malloc, but today
> > most of the libpthread tests time out on aarch64, while everything but
> > a few minor nits was fine on May 30.
> 
> I'll try to reproduce it over the weekend.

This works OK for me on aarch64 with libpthread updated in isolation.  I'll
try updating more stuff tomorrow.

Andrew

> 
> Andrew
>  
> > Martin
> > 
> > (this is on a hummingboard pulse 4-core board)
> > lib/libpthread/t_barrier (398/848): 1 test cases
> > barrier: [300.034098s] Failed: Test case timed out after 300 seconds
> > [300.034568s]
> > 
> > lib/libpthread/t_cond (399/848): 9 test cases
> > bogus_timedwaits: [300.016677s] Failed: Test case timed out after 300 
> > seconds
> > broadcast: [300.023040s] Failed: Test case timed out after 300 seconds
> > cond_timedwait_race: [300.034567s] Failed: Test case timed out after 
> > 300 seconds
> > condattr: [0.013756s] Passed.
> > destroy_after_cancel: [300.025649s] Failed: Test case timed out after 
> > 300 seconds
> > signal_before_unlock: [300.023630s] Failed: Test case timed out after 
> > 300 seconds
> > signal_before_unlock_static_init: [300.022909s] Failed: Test case timed 
> > out after 300 seconds
> > signal_delay_wait: [300.031310s] Failed: Test case timed out after 300 
> > seconds
> > signal_wait_race: [300.025552s] Failed: Test case timed out after 300 
> > seconds
> > [2400.221956s]
> > 
> > lib/libpthread/t_condwait (400/848): 2 test cases
> > cond_wait_mono: [2.023083s] Passed.
> > cond_wait_real: [2.017972s] Passed.
> > [4.042078s]
> > 
> > lib/libpthread/t_detach (401/848): 1 test cases
> > pthread_detach: [300.021242s] Failed: Test case timed out after 300 
> > seconds
> > [300.021708s]
> > 
> > lib/libpthread/t_equal (402/848): 1 test cases
> > pthread_equal: [300.027646s] Failed: Test case timed out after 300 
> > seconds
> > [300.028295s]
> > 
> > lib/libpthread/t_fork (403/848): 1 test cases
> > fork: [300.018077s] Failed: Test case timed out after 300 seconds
> > [300.018975s]
> > 
> > lib/libpthread/t_fpu (404/848): 1 test cases
> > fpu: [0.020578s] Passed.
> > [0.021314s]
> > 
> > lib/libpthread/t_join (405/848): 1 test cases
> > pthread_join: [0.021632s] Passed.
> > [0.022411s]
> > 
> > lib/libpthread/t_kill (406/848): 1 test cases
> > simple: [300.033759s] Failed: Test case timed out after 300 seconds
> > [300.034528s]
> > 
> > lib/libpthread/t_mutex (407/848): 7 test cases
> > mutex1: [300.017542s] Failed: Test case timed out after 300 seconds
> > mutex2: [300.023064s] Failed: Test case timed out after 300 seconds
> > mutex3: [300.023259s] Failed: Test case timed out after 300 seconds
> > mutex4: [300.023015s] Failed: Test case timed out after 300 seconds
> > mutex5: [300.023495s] Failed: Test case timed out after 300 seconds
> > mutexattr1: [0.013769s] Passed.
> > mutexattr2: [0.025483s] Passed.
> > [1500.154160s]
> > 
> > lib/libpthread/t_name (408/848): 1 test cases
> > name: [0.014262s] Passed.
> > [0.014951s]
> > 
> > ...

Re: CVS commit: src/lib/libpthread

[Andrew Doran]

Hi,

On Thu, Jun 04, 2020 at 06:26:07PM +, Martin Husemann wrote:

> On Wed, Jun 03, 2020 at 10:10:24PM +, Andrew Doran wrote:
> > Module Name:src
> > Committed By:   ad
> > Date:   Wed Jun  3 22:10:24 UTC 2020
> > 
> > Modified Files:
> > src/lib/libpthread: pthread.c pthread_cond.c pthread_mutex.c
> > 
> > Log Message:
> > Deal with a couple of problems with threads being awoken early due to
> > timeouts or cancellation where:
> 
> Not sure if it is caused by this commit or joergs TSD/malloc, but today
> most of the libpthread tests time out on aarch64, while everything but
> a few minor nits was fine on May 30.

I'll try to reproduce it over the weekend.

Andrew
 
> Martin
> 
> (this is on a hummingboard pulse 4-core board)
> lib/libpthread/t_barrier (398/848): 1 test cases
> barrier: [300.034098s] Failed: Test case timed out after 300 seconds
> [300.034568s]
> 
> lib/libpthread/t_cond (399/848): 9 test cases
> bogus_timedwaits: [300.016677s] Failed: Test case timed out after 300 
> seconds
> broadcast: [300.023040s] Failed: Test case timed out after 300 seconds
> cond_timedwait_race: [300.034567s] Failed: Test case timed out after 300 
> seconds
> condattr: [0.013756s] Passed.
> destroy_after_cancel: [300.025649s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock: [300.023630s] Failed: Test case timed out after 300 
> seconds
> signal_before_unlock_static_init: [300.022909s] Failed: Test case timed 
> out after 300 seconds
> signal_delay_wait: [300.031310s] Failed: Test case timed out after 300 
> seconds
> signal_wait_race: [300.025552s] Failed: Test case timed out after 300 
> seconds
> [2400.221956s]
> 
> lib/libpthread/t_condwait (400/848): 2 test cases
> cond_wait_mono: [2.023083s] Passed.
> cond_wait_real: [2.017972s] Passed.
> [4.042078s]
> 
> lib/libpthread/t_detach (401/848): 1 test cases
> pthread_detach: [300.021242s] Failed: Test case timed out after 300 
> seconds
> [300.021708s]
> 
> lib/libpthread/t_equal (402/848): 1 test cases
> pthread_equal: [300.027646s] Failed: Test case timed out after 300 seconds
> [300.028295s]
> 
> lib/libpthread/t_fork (403/848): 1 test cases
> fork: [300.018077s] Failed: Test case timed out after 300 seconds
> [300.018975s]
> 
> lib/libpthread/t_fpu (404/848): 1 test cases
> fpu: [0.020578s] Passed.
> [0.021314s]
> 
> lib/libpthread/t_join (405/848): 1 test cases
> pthread_join: [0.021632s] Passed.
> [0.022411s]
> 
> lib/libpthread/t_kill (406/848): 1 test cases
> simple: [300.033759s] Failed: Test case timed out after 300 seconds
> [300.034528s]
> 
> lib/libpthread/t_mutex (407/848): 7 test cases
> mutex1: [300.017542s] Failed: Test case timed out after 300 seconds
> mutex2: [300.023064s] Failed: Test case timed out after 300 seconds
> mutex3: [300.023259s] Failed: Test case timed out after 300 seconds
> mutex4: [300.023015s] Failed: Test case timed out after 300 seconds
> mutex5: [300.023495s] Failed: Test case timed out after 300 seconds
> mutexattr1: [0.013769s] Passed.
> mutexattr2: [0.025483s] Passed.
> [1500.154160s]
> 
> lib/libpthread/t_name (408/848): 1 test cases
> name: [0.014262s] Passed.
> [0.014951s]
> 
> ...

Re: CVS commit: src/lib/libpthread

[Martin Husemann]

On Wed, Jun 03, 2020 at 10:10:24PM +, Andrew Doran wrote:
> Module Name:  src
> Committed By: ad
> Date: Wed Jun  3 22:10:24 UTC 2020
> 
> Modified Files:
>   src/lib/libpthread: pthread.c pthread_cond.c pthread_mutex.c
> 
> Log Message:
> Deal with a couple of problems with threads being awoken early due to
> timeouts or cancellation where:

Not sure if it is caused by this commit or joergs TSD/malloc, but today
most of the libpthread tests time out on aarch64, while everything but
a few minor nits was fine on May 30.

Martin

(this is on a hummingboard pulse 4-core board)
lib/libpthread/t_barrier (398/848): 1 test cases
barrier: [300.034098s] Failed: Test case timed out after 300 seconds
[300.034568s]

lib/libpthread/t_cond (399/848): 9 test cases
bogus_timedwaits: [300.016677s] Failed: Test case timed out after 300 
seconds
broadcast: [300.023040s] Failed: Test case timed out after 300 seconds
cond_timedwait_race: [300.034567s] Failed: Test case timed out after 300 
seconds
condattr: [0.013756s] Passed.
destroy_after_cancel: [300.025649s] Failed: Test case timed out after 300 
seconds
signal_before_unlock: [300.023630s] Failed: Test case timed out after 300 
seconds
signal_before_unlock_static_init: [300.022909s] Failed: Test case timed out 
after 300 seconds
signal_delay_wait: [300.031310s] Failed: Test case timed out after 300 
seconds
signal_wait_race: [300.025552s] Failed: Test case timed out after 300 
seconds
[2400.221956s]

lib/libpthread/t_condwait (400/848): 2 test cases
cond_wait_mono: [2.023083s] Passed.
cond_wait_real: [2.017972s] Passed.
[4.042078s]

lib/libpthread/t_detach (401/848): 1 test cases
pthread_detach: [300.021242s] Failed: Test case timed out after 300 seconds
[300.021708s]

lib/libpthread/t_equal (402/848): 1 test cases
pthread_equal: [300.027646s] Failed: Test case timed out after 300 seconds
[300.028295s]

lib/libpthread/t_fork (403/848): 1 test cases
fork: [300.018077s] Failed: Test case timed out after 300 seconds
[300.018975s]

lib/libpthread/t_fpu (404/848): 1 test cases
fpu: [0.020578s] Passed.
[0.021314s]

lib/libpthread/t_join (405/848): 1 test cases
pthread_join: [0.021632s] Passed.
[0.022411s]

lib/libpthread/t_kill (406/848): 1 test cases
simple: [300.033759s] Failed: Test case timed out after 300 seconds
[300.034528s]

lib/libpthread/t_mutex (407/848): 7 test cases
mutex1: [300.017542s] Failed: Test case timed out after 300 seconds
mutex2: [300.023064s] Failed: Test case timed out after 300 seconds
mutex3: [300.023259s] Failed: Test case timed out after 300 seconds
mutex4: [300.023015s] Failed: Test case timed out after 300 seconds
mutex5: [300.023495s] Failed: Test case timed out after 300 seconds
mutexattr1: [0.013769s] Passed.
mutexattr2: [0.025483s] Passed.
[1500.154160s]

lib/libpthread/t_name (408/848): 1 test cases
name: [0.014262s] Passed.
[0.014951s]

...

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

Kamil Rytarowski  writes:

> On 13.02.2020 20:03, Ryo ONODERA wrote:
>> Hi,
>> 
>> Kamil Rytarowski  writes:
>> 
>>> On 12.02.2020 15:01, Ryo ONODERA wrote:
 Hi,

 Kamil Rytarowski  writes:

> Hello,
>
> I will have a look at them.

 Thank you.
 Real fix is welcome.

 And multimedia/handbrake has workaround already.
 I have workaround patches for lang/mono6 (like your nspr patch).
 I will commit them after some tests.

>>>
>>> libblueray real fix patch is pending upstream.
>>>
>>> https://code.videolan.org/videolan/libbluray/merge_requests/17
>> 
>> Thank you very much!
>> I will apply this to multimedia/handbrake too.
>> 
>
> HandBrake is now patched upstream:
>
> https://github.com/HandBrake/HandBrake/commit/73e958d198aba4cab8f434be6cf0661da1a2f91b
>
>>> I will look into mono next.
>> 
>> Excellent.
>> 
> What is the reproducer for mono?
>

Remove the folowing patches:
lang/mono6/patches/patch-external_corert_src_Native_gc_env_gcenv.structs.h
lang/mono6/patches/patch-mono_metadata_w32mutex-unix.
lang/mono6/patches/patch-libgc_pthread__stop__world.c
lang/mono6/patches/patch-libgc_pthread__support.c
lang/mono6/patches/patch-mono_utils_mono-threads-posix.c

And run "make" in your pkgsrc/lang/mono6 directory.
During build, you will get SIGSEGVs.

Thank you.

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 13.02.2020 20:03, Ryo ONODERA wrote:
> Hi,
> 
> Kamil Rytarowski  writes:
> 
>> On 12.02.2020 15:01, Ryo ONODERA wrote:
>>> Hi,
>>>
>>> Kamil Rytarowski  writes:
>>>
 Hello,

 I will have a look at them.
>>>
>>> Thank you.
>>> Real fix is welcome.
>>>
>>> And multimedia/handbrake has workaround already.
>>> I have workaround patches for lang/mono6 (like your nspr patch).
>>> I will commit them after some tests.
>>>
>>
>> libblueray real fix patch is pending upstream.
>>
>> https://code.videolan.org/videolan/libbluray/merge_requests/17
> 
> Thank you very much!
> I will apply this to multimedia/handbrake too.
> 

HandBrake is now patched upstream:

https://github.com/HandBrake/HandBrake/commit/73e958d198aba4cab8f434be6cf0661da1a2f91b

>> I will look into mono next.
> 
> Excellent.
> 
What is the reproducer for mono?



signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

Kamil Rytarowski  writes:

> On 12.02.2020 15:01, Ryo ONODERA wrote:
>> Hi,
>> 
>> Kamil Rytarowski  writes:
>> 
>>> Hello,
>>>
>>> I will have a look at them.
>> 
>> Thank you.
>> Real fix is welcome.
>> 
>> And multimedia/handbrake has workaround already.
>> I have workaround patches for lang/mono6 (like your nspr patch).
>> I will commit them after some tests.
>> 
>
> libblueray real fix patch is pending upstream.
>
> https://code.videolan.org/videolan/libbluray/merge_requests/17

Thank you very much!
I will apply this to multimedia/handbrake too.

> I will look into mono next.

Excellent.

>
>>> On 12.02.2020 14:02, Ryo ONODERA wrote:
 Hi,

 Kamil Rytarowski  writes:

> Please apple workaround (same like in NSPR) for now if fixing is 
> difficult.
>
> Such bugs can have security implications.

 Adding workarounds will not improve security problems.
 And I feel that such workarounds will not be accepted by upstream.
 I will add workarounds to some packages.
 However I feel that it is not meaningful...

> On 12.02.2020 09:49, Ryo ONODERA wrote:
>> Hi,
>>
>> I have two problematic pkgsrc packages at least.
>> Of course these programs have misuses and/or bugs, however I feel that
>> dealing pt_magic in pthread_equal() is too hasty for pkgsrc.
>>
>> multimedia/handbrake (internal libbluray):
>> The invalid thread pointer is not NULL.
>> pthread_equal t1: 0x
>> pthread_equal t2: 0x7073b25e2000
>>
>> Another one is lang/mono6:
>> The invalid thread pointer is not 0x.
>> pthread_equal t1: 0x7b066d4d7800
>> pthread_equal t2: 0x60f5f000
>>
>> Of course, it is desirable to fix every misuses and bugs in pkgsrc.
>> However it is impossible for now (at least for me).
>>
>> "Kamil Rytarowski"  writes:
>>
>>> Module Name:src
>>> Committed By:   kamil
>>> Date:   Sat Feb  8 17:06:03 UTC 2020
>>>
>>> Modified Files:
>>> src/lib/libpthread: pthread.c
>>>
>>> Log Message:
>>> Change the behavior of pthread_equal()
>>>
>>> On error when not aborting, do not return EINVAL as it has a side effect
>>> of being interpreted as matching threads. For invalid threads return
>>> unmatched.
>>>
>>> Check pthreads for NULL, before accessing pt_magic field. This avoids
>>> faults on comparision with a NULL pointer.
>>>
>>> This behavior is in the scope of UB, but should be easier to deal with
>>> buggy software.
>>>
>>>
>>> To generate a diff of this commit:
>>> cvs rdiff -u -r1.163 -r1.164 src/lib/libpthread/pthread.c
>>>
>>> Please note that diffs are not public domain; they are subject to the
>>> copyright notices on the relevant files.
>>>
>>> Modified files:
>>>
>>> Index: src/lib/libpthread/pthread.c
>>> diff -u src/lib/libpthread/pthread.c:1.163 
>>> src/lib/libpthread/pthread.c:1.164
>>> --- src/lib/libpthread/pthread.c:1.163  Wed Feb  5 14:56:04 2020
>>> +++ src/lib/libpthread/pthread.cSat Feb  8 17:06:03 2020
>>> @@ -1,4 +1,4 @@
>>> -/* $NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $  
>>> */
>>> +/* $NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $  
>>> */
>>>  
>>>  /*-
>>>   * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
>>> @@ -31,7 +31,7 @@
>>>   */
>>>  
>>>  #include 
>>> -__RCSID("$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $");
>>> +__RCSID("$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $");
>>>  
>>>  #define__EXPOSE_STACK  1
>>>  
>>> @@ -770,11 +770,11 @@ pthread_equal(pthread_t t1, pthread_t t2
>>> if (__predict_false(__uselibcstub))
>>> return __libc_thr_equal_stub(t1, t2);
>>>  
>>> -   pthread__error(EINVAL, "Invalid thread",
>>> -   t1->pt_magic == PT_MAGIC);
>>> +   pthread__error(0, "Invalid thread",
>>> +   (t1 != NULL) && (t1->pt_magic == PT_MAGIC));
>>>  
>>> -   pthread__error(EINVAL, "Invalid thread",
>>> -   t2->pt_magic == PT_MAGIC);
>>> +   pthread__error(0, "Invalid thread",
>>> +   (t2 != NULL) && (t2->pt_magic == PT_MAGIC));
>>>  
>>> /* Nothing special here. */
>>> return (t1 == t2);
>>>
>>
>
>

>>>
>>>
>> 
>
>

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 12.02.2020 15:01, Ryo ONODERA wrote:
> Hi,
> 
> Kamil Rytarowski  writes:
> 
>> Hello,
>>
>> I will have a look at them.
> 
> Thank you.
> Real fix is welcome.
> 
> And multimedia/handbrake has workaround already.
> I have workaround patches for lang/mono6 (like your nspr patch).
> I will commit them after some tests.
> 

libblueray real fix patch is pending upstream.

https://code.videolan.org/videolan/libbluray/merge_requests/17

I will look into mono next.

>> On 12.02.2020 14:02, Ryo ONODERA wrote:
>>> Hi,
>>>
>>> Kamil Rytarowski  writes:
>>>
 Please apple workaround (same like in NSPR) for now if fixing is difficult.

 Such bugs can have security implications.
>>>
>>> Adding workarounds will not improve security problems.
>>> And I feel that such workarounds will not be accepted by upstream.
>>> I will add workarounds to some packages.
>>> However I feel that it is not meaningful...
>>>
 On 12.02.2020 09:49, Ryo ONODERA wrote:
> Hi,
>
> I have two problematic pkgsrc packages at least.
> Of course these programs have misuses and/or bugs, however I feel that
> dealing pt_magic in pthread_equal() is too hasty for pkgsrc.
>
> multimedia/handbrake (internal libbluray):
> The invalid thread pointer is not NULL.
> pthread_equal t1: 0x
> pthread_equal t2: 0x7073b25e2000
>
> Another one is lang/mono6:
> The invalid thread pointer is not 0x.
> pthread_equal t1: 0x7b066d4d7800
> pthread_equal t2: 0x60f5f000
>
> Of course, it is desirable to fix every misuses and bugs in pkgsrc.
> However it is impossible for now (at least for me).
>
> "Kamil Rytarowski"  writes:
>
>> Module Name: src
>> Committed By:kamil
>> Date:Sat Feb  8 17:06:03 UTC 2020
>>
>> Modified Files:
>>  src/lib/libpthread: pthread.c
>>
>> Log Message:
>> Change the behavior of pthread_equal()
>>
>> On error when not aborting, do not return EINVAL as it has a side effect
>> of being interpreted as matching threads. For invalid threads return
>> unmatched.
>>
>> Check pthreads for NULL, before accessing pt_magic field. This avoids
>> faults on comparision with a NULL pointer.
>>
>> This behavior is in the scope of UB, but should be easier to deal with
>> buggy software.
>>
>>
>> To generate a diff of this commit:
>> cvs rdiff -u -r1.163 -r1.164 src/lib/libpthread/pthread.c
>>
>> Please note that diffs are not public domain; they are subject to the
>> copyright notices on the relevant files.
>>
>> Modified files:
>>
>> Index: src/lib/libpthread/pthread.c
>> diff -u src/lib/libpthread/pthread.c:1.163 
>> src/lib/libpthread/pthread.c:1.164
>> --- src/lib/libpthread/pthread.c:1.163   Wed Feb  5 14:56:04 2020
>> +++ src/lib/libpthread/pthread.c Sat Feb  8 17:06:03 2020
>> @@ -1,4 +1,4 @@
>> -/*  $NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $  
>> */
>> +/*  $NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $  
>> */
>>  
>>  /*-
>>   * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
>> @@ -31,7 +31,7 @@
>>   */
>>  
>>  #include 
>> -__RCSID("$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $");
>> +__RCSID("$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $");
>>  
>>  #define __EXPOSE_STACK  1
>>  
>> @@ -770,11 +770,11 @@ pthread_equal(pthread_t t1, pthread_t t2
>>  if (__predict_false(__uselibcstub))
>>  return __libc_thr_equal_stub(t1, t2);
>>  
>> -pthread__error(EINVAL, "Invalid thread",
>> -t1->pt_magic == PT_MAGIC);
>> +pthread__error(0, "Invalid thread",
>> +(t1 != NULL) && (t1->pt_magic == PT_MAGIC));
>>  
>> -pthread__error(EINVAL, "Invalid thread",
>> -t2->pt_magic == PT_MAGIC);
>> +pthread__error(0, "Invalid thread",
>> +(t2 != NULL) && (t2->pt_magic == PT_MAGIC));
>>  
>>  /* Nothing special here. */
>>  return (t1 == t2);
>>
>


>>>
>>
>>
> 




signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

Kamil Rytarowski  writes:

> Hello,
>
> I will have a look at them.

Thank you.
Real fix is welcome.

And multimedia/handbrake has workaround already.
I have workaround patches for lang/mono6 (like your nspr patch).
I will commit them after some tests.

> On 12.02.2020 14:02, Ryo ONODERA wrote:
>> Hi,
>> 
>> Kamil Rytarowski  writes:
>> 
>>> Please apple workaround (same like in NSPR) for now if fixing is difficult.
>>>
>>> Such bugs can have security implications.
>> 
>> Adding workarounds will not improve security problems.
>> And I feel that such workarounds will not be accepted by upstream.
>> I will add workarounds to some packages.
>> However I feel that it is not meaningful...
>> 
>>> On 12.02.2020 09:49, Ryo ONODERA wrote:
 Hi,

 I have two problematic pkgsrc packages at least.
 Of course these programs have misuses and/or bugs, however I feel that
 dealing pt_magic in pthread_equal() is too hasty for pkgsrc.

 multimedia/handbrake (internal libbluray):
 The invalid thread pointer is not NULL.
 pthread_equal t1: 0x
 pthread_equal t2: 0x7073b25e2000

 Another one is lang/mono6:
 The invalid thread pointer is not 0x.
 pthread_equal t1: 0x7b066d4d7800
 pthread_equal t2: 0x60f5f000

 Of course, it is desirable to fix every misuses and bugs in pkgsrc.
 However it is impossible for now (at least for me).

 "Kamil Rytarowski"  writes:

> Module Name:  src
> Committed By: kamil
> Date: Sat Feb  8 17:06:03 UTC 2020
>
> Modified Files:
>   src/lib/libpthread: pthread.c
>
> Log Message:
> Change the behavior of pthread_equal()
>
> On error when not aborting, do not return EINVAL as it has a side effect
> of being interpreted as matching threads. For invalid threads return
> unmatched.
>
> Check pthreads for NULL, before accessing pt_magic field. This avoids
> faults on comparision with a NULL pointer.
>
> This behavior is in the scope of UB, but should be easier to deal with
> buggy software.
>
>
> To generate a diff of this commit:
> cvs rdiff -u -r1.163 -r1.164 src/lib/libpthread/pthread.c
>
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
>
> Modified files:
>
> Index: src/lib/libpthread/pthread.c
> diff -u src/lib/libpthread/pthread.c:1.163 
> src/lib/libpthread/pthread.c:1.164
> --- src/lib/libpthread/pthread.c:1.163Wed Feb  5 14:56:04 2020
> +++ src/lib/libpthread/pthread.c  Sat Feb  8 17:06:03 2020
> @@ -1,4 +1,4 @@
> -/*   $NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $  
> */
> +/*   $NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $  
> */
>  
>  /*-
>   * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
> @@ -31,7 +31,7 @@
>   */
>  
>  #include 
> -__RCSID("$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $");
> +__RCSID("$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $");
>  
>  #define  __EXPOSE_STACK  1
>  
> @@ -770,11 +770,11 @@ pthread_equal(pthread_t t1, pthread_t t2
>   if (__predict_false(__uselibcstub))
>   return __libc_thr_equal_stub(t1, t2);
>  
> - pthread__error(EINVAL, "Invalid thread",
> - t1->pt_magic == PT_MAGIC);
> + pthread__error(0, "Invalid thread",
> + (t1 != NULL) && (t1->pt_magic == PT_MAGIC));
>  
> - pthread__error(EINVAL, "Invalid thread",
> - t2->pt_magic == PT_MAGIC);
> + pthread__error(0, "Invalid thread",
> + (t2 != NULL) && (t2->pt_magic == PT_MAGIC));
>  
>   /* Nothing special here. */
>   return (t1 == t2);
>

>>>
>>>
>> 
>
>

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Hello,

I will have a look at them.

On 12.02.2020 14:02, Ryo ONODERA wrote:
> Hi,
> 
> Kamil Rytarowski  writes:
> 
>> Please apple workaround (same like in NSPR) for now if fixing is difficult.
>>
>> Such bugs can have security implications.
> 
> Adding workarounds will not improve security problems.
> And I feel that such workarounds will not be accepted by upstream.
> I will add workarounds to some packages.
> However I feel that it is not meaningful...
> 
>> On 12.02.2020 09:49, Ryo ONODERA wrote:
>>> Hi,
>>>
>>> I have two problematic pkgsrc packages at least.
>>> Of course these programs have misuses and/or bugs, however I feel that
>>> dealing pt_magic in pthread_equal() is too hasty for pkgsrc.
>>>
>>> multimedia/handbrake (internal libbluray):
>>> The invalid thread pointer is not NULL.
>>> pthread_equal t1: 0x
>>> pthread_equal t2: 0x7073b25e2000
>>>
>>> Another one is lang/mono6:
>>> The invalid thread pointer is not 0x.
>>> pthread_equal t1: 0x7b066d4d7800
>>> pthread_equal t2: 0x60f5f000
>>>
>>> Of course, it is desirable to fix every misuses and bugs in pkgsrc.
>>> However it is impossible for now (at least for me).
>>>
>>> "Kamil Rytarowski"  writes:
>>>
 Module Name:   src
 Committed By:  kamil
 Date:  Sat Feb  8 17:06:03 UTC 2020

 Modified Files:
src/lib/libpthread: pthread.c

 Log Message:
 Change the behavior of pthread_equal()

 On error when not aborting, do not return EINVAL as it has a side effect
 of being interpreted as matching threads. For invalid threads return
 unmatched.

 Check pthreads for NULL, before accessing pt_magic field. This avoids
 faults on comparision with a NULL pointer.

 This behavior is in the scope of UB, but should be easier to deal with
 buggy software.


 To generate a diff of this commit:
 cvs rdiff -u -r1.163 -r1.164 src/lib/libpthread/pthread.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

 Modified files:

 Index: src/lib/libpthread/pthread.c
 diff -u src/lib/libpthread/pthread.c:1.163 
 src/lib/libpthread/pthread.c:1.164
 --- src/lib/libpthread/pthread.c:1.163 Wed Feb  5 14:56:04 2020
 +++ src/lib/libpthread/pthread.c   Sat Feb  8 17:06:03 2020
 @@ -1,4 +1,4 @@
 -/*$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $  
 */
 +/*$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $  
 */
  
  /*-
   * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
 @@ -31,7 +31,7 @@
   */
  
  #include 
 -__RCSID("$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $");
 +__RCSID("$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $");
  
  #define   __EXPOSE_STACK  1
  
 @@ -770,11 +770,11 @@ pthread_equal(pthread_t t1, pthread_t t2
if (__predict_false(__uselibcstub))
return __libc_thr_equal_stub(t1, t2);
  
 -  pthread__error(EINVAL, "Invalid thread",
 -  t1->pt_magic == PT_MAGIC);
 +  pthread__error(0, "Invalid thread",
 +  (t1 != NULL) && (t1->pt_magic == PT_MAGIC));
  
 -  pthread__error(EINVAL, "Invalid thread",
 -  t2->pt_magic == PT_MAGIC);
 +  pthread__error(0, "Invalid thread",
 +  (t2 != NULL) && (t2->pt_magic == PT_MAGIC));
  
/* Nothing special here. */
return (t1 == t2);

>>>
>>
>>
> 




signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

Kamil Rytarowski  writes:

> Please apple workaround (same like in NSPR) for now if fixing is difficult.
>
> Such bugs can have security implications.

Adding workarounds will not improve security problems.
And I feel that such workarounds will not be accepted by upstream.
I will add workarounds to some packages.
However I feel that it is not meaningful...

> On 12.02.2020 09:49, Ryo ONODERA wrote:
>> Hi,
>> 
>> I have two problematic pkgsrc packages at least.
>> Of course these programs have misuses and/or bugs, however I feel that
>> dealing pt_magic in pthread_equal() is too hasty for pkgsrc.
>> 
>> multimedia/handbrake (internal libbluray):
>> The invalid thread pointer is not NULL.
>> pthread_equal t1: 0x
>> pthread_equal t2: 0x7073b25e2000
>> 
>> Another one is lang/mono6:
>> The invalid thread pointer is not 0x.
>> pthread_equal t1: 0x7b066d4d7800
>> pthread_equal t2: 0x60f5f000
>> 
>> Of course, it is desirable to fix every misuses and bugs in pkgsrc.
>> However it is impossible for now (at least for me).
>> 
>> "Kamil Rytarowski"  writes:
>> 
>>> Module Name:src
>>> Committed By:   kamil
>>> Date:   Sat Feb  8 17:06:03 UTC 2020
>>>
>>> Modified Files:
>>> src/lib/libpthread: pthread.c
>>>
>>> Log Message:
>>> Change the behavior of pthread_equal()
>>>
>>> On error when not aborting, do not return EINVAL as it has a side effect
>>> of being interpreted as matching threads. For invalid threads return
>>> unmatched.
>>>
>>> Check pthreads for NULL, before accessing pt_magic field. This avoids
>>> faults on comparision with a NULL pointer.
>>>
>>> This behavior is in the scope of UB, but should be easier to deal with
>>> buggy software.
>>>
>>>
>>> To generate a diff of this commit:
>>> cvs rdiff -u -r1.163 -r1.164 src/lib/libpthread/pthread.c
>>>
>>> Please note that diffs are not public domain; they are subject to the
>>> copyright notices on the relevant files.
>>>
>>> Modified files:
>>>
>>> Index: src/lib/libpthread/pthread.c
>>> diff -u src/lib/libpthread/pthread.c:1.163 
>>> src/lib/libpthread/pthread.c:1.164
>>> --- src/lib/libpthread/pthread.c:1.163  Wed Feb  5 14:56:04 2020
>>> +++ src/lib/libpthread/pthread.cSat Feb  8 17:06:03 2020
>>> @@ -1,4 +1,4 @@
>>> -/* $NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $  */
>>> +/* $NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $  */
>>>  
>>>  /*-
>>>   * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
>>> @@ -31,7 +31,7 @@
>>>   */
>>>  
>>>  #include 
>>> -__RCSID("$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $");
>>> +__RCSID("$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $");
>>>  
>>>  #define__EXPOSE_STACK  1
>>>  
>>> @@ -770,11 +770,11 @@ pthread_equal(pthread_t t1, pthread_t t2
>>> if (__predict_false(__uselibcstub))
>>> return __libc_thr_equal_stub(t1, t2);
>>>  
>>> -   pthread__error(EINVAL, "Invalid thread",
>>> -   t1->pt_magic == PT_MAGIC);
>>> +   pthread__error(0, "Invalid thread",
>>> +   (t1 != NULL) && (t1->pt_magic == PT_MAGIC));
>>>  
>>> -   pthread__error(EINVAL, "Invalid thread",
>>> -   t2->pt_magic == PT_MAGIC);
>>> +   pthread__error(0, "Invalid thread",
>>> +   (t2 != NULL) && (t2->pt_magic == PT_MAGIC));
>>>  
>>> /* Nothing special here. */
>>> return (t1 == t2);
>>>
>> 
>
>

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Please apple workaround (same like in NSPR) for now if fixing is difficult.

Such bugs can have security implications.

On 12.02.2020 09:49, Ryo ONODERA wrote:
> Hi,
> 
> I have two problematic pkgsrc packages at least.
> Of course these programs have misuses and/or bugs, however I feel that
> dealing pt_magic in pthread_equal() is too hasty for pkgsrc.
> 
> multimedia/handbrake (internal libbluray):
> The invalid thread pointer is not NULL.
> pthread_equal t1: 0x
> pthread_equal t2: 0x7073b25e2000
> 
> Another one is lang/mono6:
> The invalid thread pointer is not 0x.
> pthread_equal t1: 0x7b066d4d7800
> pthread_equal t2: 0x60f5f000
> 
> Of course, it is desirable to fix every misuses and bugs in pkgsrc.
> However it is impossible for now (at least for me).
> 
> "Kamil Rytarowski"  writes:
> 
>> Module Name: src
>> Committed By:kamil
>> Date:Sat Feb  8 17:06:03 UTC 2020
>>
>> Modified Files:
>>  src/lib/libpthread: pthread.c
>>
>> Log Message:
>> Change the behavior of pthread_equal()
>>
>> On error when not aborting, do not return EINVAL as it has a side effect
>> of being interpreted as matching threads. For invalid threads return
>> unmatched.
>>
>> Check pthreads for NULL, before accessing pt_magic field. This avoids
>> faults on comparision with a NULL pointer.
>>
>> This behavior is in the scope of UB, but should be easier to deal with
>> buggy software.
>>
>>
>> To generate a diff of this commit:
>> cvs rdiff -u -r1.163 -r1.164 src/lib/libpthread/pthread.c
>>
>> Please note that diffs are not public domain; they are subject to the
>> copyright notices on the relevant files.
>>
>> Modified files:
>>
>> Index: src/lib/libpthread/pthread.c
>> diff -u src/lib/libpthread/pthread.c:1.163 src/lib/libpthread/pthread.c:1.164
>> --- src/lib/libpthread/pthread.c:1.163   Wed Feb  5 14:56:04 2020
>> +++ src/lib/libpthread/pthread.c Sat Feb  8 17:06:03 2020
>> @@ -1,4 +1,4 @@
>> -/*  $NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $  */
>> +/*  $NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $  */
>>  
>>  /*-
>>   * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
>> @@ -31,7 +31,7 @@
>>   */
>>  
>>  #include 
>> -__RCSID("$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $");
>> +__RCSID("$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $");
>>  
>>  #define __EXPOSE_STACK  1
>>  
>> @@ -770,11 +770,11 @@ pthread_equal(pthread_t t1, pthread_t t2
>>  if (__predict_false(__uselibcstub))
>>  return __libc_thr_equal_stub(t1, t2);
>>  
>> -pthread__error(EINVAL, "Invalid thread",
>> -t1->pt_magic == PT_MAGIC);
>> +pthread__error(0, "Invalid thread",
>> +(t1 != NULL) && (t1->pt_magic == PT_MAGIC));
>>  
>> -pthread__error(EINVAL, "Invalid thread",
>> -t2->pt_magic == PT_MAGIC);
>> +pthread__error(0, "Invalid thread",
>> +(t2 != NULL) && (t2->pt_magic == PT_MAGIC));
>>  
>>  /* Nothing special here. */
>>  return (t1 == t2);
>>
> 




signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

I have two problematic pkgsrc packages at least.
Of course these programs have misuses and/or bugs, however I feel that
dealing pt_magic in pthread_equal() is too hasty for pkgsrc.

multimedia/handbrake (internal libbluray):
The invalid thread pointer is not NULL.
pthread_equal t1: 0x
pthread_equal t2: 0x7073b25e2000

Another one is lang/mono6:
The invalid thread pointer is not 0x.
pthread_equal t1: 0x7b066d4d7800
pthread_equal t2: 0x60f5f000

Of course, it is desirable to fix every misuses and bugs in pkgsrc.
However it is impossible for now (at least for me).

"Kamil Rytarowski"  writes:

> Module Name:  src
> Committed By: kamil
> Date: Sat Feb  8 17:06:03 UTC 2020
>
> Modified Files:
>   src/lib/libpthread: pthread.c
>
> Log Message:
> Change the behavior of pthread_equal()
>
> On error when not aborting, do not return EINVAL as it has a side effect
> of being interpreted as matching threads. For invalid threads return
> unmatched.
>
> Check pthreads for NULL, before accessing pt_magic field. This avoids
> faults on comparision with a NULL pointer.
>
> This behavior is in the scope of UB, but should be easier to deal with
> buggy software.
>
>
> To generate a diff of this commit:
> cvs rdiff -u -r1.163 -r1.164 src/lib/libpthread/pthread.c
>
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
>
> Modified files:
>
> Index: src/lib/libpthread/pthread.c
> diff -u src/lib/libpthread/pthread.c:1.163 src/lib/libpthread/pthread.c:1.164
> --- src/lib/libpthread/pthread.c:1.163Wed Feb  5 14:56:04 2020
> +++ src/lib/libpthread/pthread.c  Sat Feb  8 17:06:03 2020
> @@ -1,4 +1,4 @@
> -/*   $NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $  */
> +/*   $NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $  */
>  
>  /*-
>   * Copyright (c) 2001, 2002, 2003, 2006, 2007, 2008, 2020
> @@ -31,7 +31,7 @@
>   */
>  
>  #include 
> -__RCSID("$NetBSD: pthread.c,v 1.163 2020/02/05 14:56:04 ryoon Exp $");
> +__RCSID("$NetBSD: pthread.c,v 1.164 2020/02/08 17:06:03 kamil Exp $");
>  
>  #define  __EXPOSE_STACK  1
>  
> @@ -770,11 +770,11 @@ pthread_equal(pthread_t t1, pthread_t t2
>   if (__predict_false(__uselibcstub))
>   return __libc_thr_equal_stub(t1, t2);
>  
> - pthread__error(EINVAL, "Invalid thread",
> - t1->pt_magic == PT_MAGIC);
> + pthread__error(0, "Invalid thread",
> + (t1 != NULL) && (t1->pt_magic == PT_MAGIC));
>  
> - pthread__error(EINVAL, "Invalid thread",
> - t2->pt_magic == PT_MAGIC);
> + pthread__error(0, "Invalid thread",
> + (t2 != NULL) && (t2->pt_magic == PT_MAGIC));
>  
>   /* Nothing special here. */
>   return (t1 == t2);
>

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 03.02.2020 17:10, Ryo ONODERA wrote:
> Hi,
> 
> Kamil Rytarowski  writes:
> 
>> Please check this workaround:
>>
>> http://netbsd.org/~kamil/patch-00224-firefox-pthread_equal.txt
>>
>> It has to be applied on firefox's package.
> 
> Thank you very much.
> 
> This should be applied to pkgsrc/devel/nspr with slight change.
> (devel/nspr's C files cannot accept // comment.)
> This eliminates segfaults.
> 
> For the record, pthread_equal()'s first thread pointer, mon->owner, is
> invalid in PR_GetMonitorEntryCount() in nspr/pr/src/pthreads/ptsynch.c
> of devel/nspr.
> 

Please patch all affected packages in pkgsrc.

>>
>> The problem has to be reported upstream as a real bug.
> 
> Please file the bug report to Mozilla's bugzilla.
> 

I don't have a direct reproducer as of now myself. Please report a bug.

> Thank you very much.
> 




signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

Kamil Rytarowski  writes:

> On 03.02.2020 17:10, Ryo ONODERA wrote:
>> Hi,
>> 
>> Kamil Rytarowski  writes:
>> 
>>> Please check this workaround:
>>>
>>> http://netbsd.org/~kamil/patch-00224-firefox-pthread_equal.txt
>>>
>>> It has to be applied on firefox's package.
>> 
>> Thank you very much.
>> 
>> This should be applied to pkgsrc/devel/nspr with slight change.
>> (devel/nspr's C files cannot accept // comment.)
>> This eliminates segfaults.
>> 
>> For the record, pthread_equal()'s first thread pointer, mon->owner, is
>> invalid in PR_GetMonitorEntryCount() in nspr/pr/src/pthreads/ptsynch.c
>> of devel/nspr.
>> 
>
> Please patch all affected packages in pkgsrc.

I will commit this patch to devwl/nspr.

>>>
>>> The problem has to be reported upstream as a real bug.
>> 
>> Please file the bug report to Mozilla's bugzilla.
>> 
>
> I don't have a direct reproducer as of now myself. Please report a bug.

O.k. I will submit this to Mozilla.

Thank you very much.

>> Thank you very much.
>> 
>
>

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

Kamil Rytarowski  writes:

> Please check this workaround:
>
> http://netbsd.org/~kamil/patch-00224-firefox-pthread_equal.txt
>
> It has to be applied on firefox's package.

Thank you very much.

This should be applied to pkgsrc/devel/nspr with slight change.
(devel/nspr's C files cannot accept // comment.)
This eliminates segfaults.

For the record, pthread_equal()'s first thread pointer, mon->owner, is
invalid in PR_GetMonitorEntryCount() in nspr/pr/src/pthreads/ptsynch.c
of devel/nspr.

>
> The problem has to be reported upstream as a real bug.

Please file the bug report to Mozilla's bugzilla.

Thank you very much.

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Please check this workaround:

http://netbsd.org/~kamil/patch-00224-firefox-pthread_equal.txt

It has to be applied on firefox's package.

The problem has to be reported upstream as a real bug.

On 03.02.2020 16:01, Ryo ONODERA wrote:
> Hi,
> 
> Ryo ONODERA  writes:
> 
>> Hi,
>>
>> I had tested with PTHREAD_DIAGASSERT however it did not produce any output.
>>
>> I am building current and pkgsrc packages from scratch now.
>>
>> I will reply my situation after this rebuild.
>>
>> Thank you.
> 
> With latest toolchain, kernel and userland,
> my firefox gets sefgault every run.
> 
> The core dump generated from PTHREAD_DIAGASSERT=ae does not show
> any useful things.
> See:
> Core was generated by `firefox'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  0x743cb39860ca in _lwp_kill () from /usr/lib/libc.so.12
> [Current thread is 1 (process 4)]
> (gdb) bt
> #0  0x743cb39860ca in _lwp_kill () from /usr/lib/libc.so.12
> #1  0x743ca1943721 in ?? () from /usr/pkg/lib/firefox/libxul.so
> #2  0x743ca2172bee in ?? () from /usr/pkg/lib/firefox/libxul.so
> #3  0x743cb38b0140 in opendir () from /usr/lib/libc.so.12
> #4  0x0001000b in ?? ()
> #5  0x in ?? ()
> 
> 
> And I cannot get any text output to stdout.
> As far as I understand correctly, PTHREAD_DIAGASSERT=e enables the output
> to stdout.
> 
> 
> Still I feel that pthread_equal() is cause of my segfault.
> $ gdb /usr/pkg/lib/firefox/firefox
> GNU gdb (GDB) 8.3
> Copyright (C) 2019 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Type "show copying" and "show warranty" for details.
> This GDB was configured as "x86_64--netbsd".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> .
> Find the GDB manual and other documentation resources online at:
> .
> 
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from /usr/pkg/lib/firefox/firefox...
> (No debugging symbols found in /usr/pkg/lib/firefox/firefox)
> (gdb) r
> Starting program: /usr/pkg/lib/firefox/firefox
> [New process 18477]
> [Detaching after fork from child process 17695]
> [New LWP 2 of process 18477]
> [New LWP 3 of process 18477]
> [New LWP 4 of process 18477]
> [New LWP 5 of process 18477]
> [New LWP 6 of process 18477]
> [New LWP 7 of process 18477]
> [New LWP 8 of process 18477]
> [New LWP 9 of process 18477]
> [New LWP 10 of process 18477]
> [New LWP 11 of process 18477]
> [New LWP 12 of process 18477]
> [New LWP 13 of process 18477]
> [New LWP 14 of process 18477]
> [New LWP 15 of process 18477]
> [New LWP 16 of process 18477]
> JavaScript error: , line 0: UnknownError: The operation failed for reasons 
> unrelated to the database itself and not covered by any other error code.
> process 18477 is executing new program: /usr/pkg/lib/firefox/firefox
> [New process 18477]
> [Detaching after fork from child process 17526]
> [New LWP 2 of process 18477]
> [New LWP 3 of process 18477]
> [New LWP 4 of process 18477]
> [New LWP 5 of process 18477]
> [New LWP 6 of process 18477]
> [New LWP 7 of process 18477]
> [New LWP 8 of process 18477]
> [New LWP 9 of process 18477]
> [New LWP 10 of process 18477]
> [New LWP 11 of process 18477]
> [New LWP 12 of process 18477]
> [New LWP 13 of process 18477]
> [New LWP 14 of process 18477]
> [New LWP 15 of process 18477]
> [New LWP 16 of process 18477]
> [New LWP 17 of process 18477]
> [New LWP 18 of process 18477]
> [New LWP 19 of process 18477]
> [New LWP 20 of process 18477]
> [New LWP 21 of process 18477]
> [New LWP 22 of process 18477]
> [New LWP 23 of process 18477]
> [New LWP 24 of process 18477]
> [New LWP 25 of process 18477]
> [New LWP 26 of process 18477]
> [New LWP 27 of process 18477]
> [New LWP 28 of process 18477]
> [New LWP 29 of process 18477]
> [New LWP 14 of process 18477]
> [New LWP 30 of process 18477]
> [New LWP 31 of process 18477]
> [New LWP 32 of process 18477]
> [New LWP 33 of process 18477]
> [New LWP 34 of process 18477]
> [New LWP 35 of process 18477]
> [New LWP 36 of process 18477]
> [New LWP 37 of process 18477]
> [New LWP 38 of process 18477]
> [New LWP 39 of process 18477]
> [New process 18477]
> [Detaching after fork from child process 17454]
> [New LWP 39 of process 18477]
> [New LWP 41 of process 18477]
> [New LWP 42 of process 18477]
> [New LWP 43 of process 18477]
> [New LWP 44 of process 18477]
> [New LWP 45 of process 18477]
> [New process 18477]
> [Detaching after fork from child process 15097]
> [New LWP 47 of process 18477]
> [New LWP 48 of process 18477]
> [New LWP 49 of process 18477]
> [New LWP 50 of process 18477]
> [New LWP 51 of process 18477]
> [New LWP 45 of process 18477]

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

Ryo ONODERA  writes:

> Hi,
>
> I had tested with PTHREAD_DIAGASSERT however it did not produce any output.
>
> I am building current and pkgsrc packages from scratch now.
>
> I will reply my situation after this rebuild.
>
> Thank you.

With latest toolchain, kernel and userland,
my firefox gets sefgault every run.

The core dump generated from PTHREAD_DIAGASSERT=ae does not show
any useful things.
See:
Core was generated by `firefox'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x743cb39860ca in _lwp_kill () from /usr/lib/libc.so.12
[Current thread is 1 (process 4)]
(gdb) bt
#0  0x743cb39860ca in _lwp_kill () from /usr/lib/libc.so.12
#1  0x743ca1943721 in ?? () from /usr/pkg/lib/firefox/libxul.so
#2  0x743ca2172bee in ?? () from /usr/pkg/lib/firefox/libxul.so
#3  0x743cb38b0140 in opendir () from /usr/lib/libc.so.12
#4  0x0001000b in ?? ()
#5  0x in ?? ()


And I cannot get any text output to stdout.
As far as I understand correctly, PTHREAD_DIAGASSERT=e enables the output
to stdout.


Still I feel that pthread_equal() is cause of my segfault.
$ gdb /usr/pkg/lib/firefox/firefox
GNU gdb (GDB) 8.3
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64--netbsd".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/pkg/lib/firefox/firefox...
(No debugging symbols found in /usr/pkg/lib/firefox/firefox)
(gdb) r
Starting program: /usr/pkg/lib/firefox/firefox
[New process 18477]
[Detaching after fork from child process 17695]
[New LWP 2 of process 18477]
[New LWP 3 of process 18477]
[New LWP 4 of process 18477]
[New LWP 5 of process 18477]
[New LWP 6 of process 18477]
[New LWP 7 of process 18477]
[New LWP 8 of process 18477]
[New LWP 9 of process 18477]
[New LWP 10 of process 18477]
[New LWP 11 of process 18477]
[New LWP 12 of process 18477]
[New LWP 13 of process 18477]
[New LWP 14 of process 18477]
[New LWP 15 of process 18477]
[New LWP 16 of process 18477]
JavaScript error: , line 0: UnknownError: The operation failed for reasons 
unrelated to the database itself and not covered by any other error code.
process 18477 is executing new program: /usr/pkg/lib/firefox/firefox
[New process 18477]
[Detaching after fork from child process 17526]
[New LWP 2 of process 18477]
[New LWP 3 of process 18477]
[New LWP 4 of process 18477]
[New LWP 5 of process 18477]
[New LWP 6 of process 18477]
[New LWP 7 of process 18477]
[New LWP 8 of process 18477]
[New LWP 9 of process 18477]
[New LWP 10 of process 18477]
[New LWP 11 of process 18477]
[New LWP 12 of process 18477]
[New LWP 13 of process 18477]
[New LWP 14 of process 18477]
[New LWP 15 of process 18477]
[New LWP 16 of process 18477]
[New LWP 17 of process 18477]
[New LWP 18 of process 18477]
[New LWP 19 of process 18477]
[New LWP 20 of process 18477]
[New LWP 21 of process 18477]
[New LWP 22 of process 18477]
[New LWP 23 of process 18477]
[New LWP 24 of process 18477]
[New LWP 25 of process 18477]
[New LWP 26 of process 18477]
[New LWP 27 of process 18477]
[New LWP 28 of process 18477]
[New LWP 29 of process 18477]
[New LWP 14 of process 18477]
[New LWP 30 of process 18477]
[New LWP 31 of process 18477]
[New LWP 32 of process 18477]
[New LWP 33 of process 18477]
[New LWP 34 of process 18477]
[New LWP 35 of process 18477]
[New LWP 36 of process 18477]
[New LWP 37 of process 18477]
[New LWP 38 of process 18477]
[New LWP 39 of process 18477]
[New process 18477]
[Detaching after fork from child process 17454]
[New LWP 39 of process 18477]
[New LWP 41 of process 18477]
[New LWP 42 of process 18477]
[New LWP 43 of process 18477]
[New LWP 44 of process 18477]
[New LWP 45 of process 18477]
[New process 18477]
[Detaching after fork from child process 15097]
[New LWP 47 of process 18477]
[New LWP 48 of process 18477]
[New LWP 49 of process 18477]
[New LWP 50 of process 18477]
[New LWP 51 of process 18477]
[New LWP 45 of process 18477]
[New LWP 52 of process 18477]
[New LWP 53 of process 18477]
[New LWP 54 of process 18477]
[New LWP 55 of process 18477]
[New LWP 56 of process 18477]
[New LWP 57 of process 18477]
[New LWP 58 of process 18477]
[New LWP 59 of process 18477]
[New LWP 60 of process 18477]

Thread 21 "Socket Thread" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 4 of process 18477]
0x71a59bc0c2eb in pthread_equal () from /usr/lib/libpthread.so.1
(gdb) bt
#0  0x71a59bc0c2eb in pthread_equal 

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

I had tested with PTHREAD_DIAGASSERT however it did not produce any output.

I am building current and pkgsrc packages from scratch now.

I will reply my situation after this rebuild.

Thank you.

On February 3, 2020 7:23:33 AM GMT+09:00, Kamil Rytarowski  wrote:
>Hello,
>
>I've checked with NetBSD-current from today (2020-02-02) and
>pkgsrc-current (2020-02-02) and package firefox-72.0.2.
>
>I'm not reproducing any crash due to pthread_equal(3) misuse.
>Everything
>I tested, worked for me.
>
>Please try PTHREAD_DIAGASSERT=ae and debug the culprit crash with a
>core(5) file.
>
>On 01.02.2020 22:20, Kamil Rytarowski wrote:
>> Good idea. It could be checked quicker... however I presume that
>> t1->pt_magic + t1->pt_magic already crash on invalid t1/t2 pointers
>as
>> the argument with condition is evaluated.
>> 
>> Ryo, you might check:
>> $ export PTHREAD_DIAGASSERT=ae
>> $ firefox
>> 
>> It should create a coredump for investigation.
>> 
>> According to POSIX
>>
>(https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_equal.html)
>> passing invalid parameters is UB.
>> 
>> GLIBC, Illumos and all other BSDs (+ older NetBSD) have no sanity
>check
>> in pthread_equal(3). Apparently we are the first ones to notice the
>bug.
>> 
>> On 01.02.2020 21:18, Andrew Doran wrote:
>>> Hmm.  Was there not originally an environment variable to control
>this
>>> behaviour, since many applications are buggy?
>>>
>>> Andrew
>>>
>>> On Sun, Feb 02, 2020 at 01:01:49AM +0900, Ryo ONODERA wrote:
 Hi,

 pthread__error()s in pthread_equal() cause segfault
 during start of pkgsrc/www/firefox-72.0.2.

 Without pthread__error()s, www/firefox works fine
 like as follows.
 However I have no idea why I get segfaults.

 Could you take a look at this problem?

 Index: lib/libpthread/pthread.c
 ===
 RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
 retrieving revision 1.162
 diff -u -r1.162 pthread.c
 --- lib/libpthread/pthread.c   29 Jan 2020 17:11:57 -  1.162
 +++ lib/libpthread/pthread.c   1 Feb 2020 15:58:03 -
 @@ -770,11 +770,13 @@
if (__predict_false(__uselibcstub))
return __libc_thr_equal_stub(t1, t2);
  
 +#if 0
pthread__error(EINVAL, "Invalid thread",
t1->pt_magic == PT_MAGIC);
  
pthread__error(EINVAL, "Invalid thread",
t2->pt_magic == PT_MAGIC);
 +#endif
  
/* Nothing special here. */
return (t1 == t2);
 @@ -1108,7 +1110,7 @@
  {
char buf[1024];
size_t len;
 -  
 +
if (pthread__diagassert == 0)
return;
  


 "Kamil Rytarowski"  writes:

> Module Name:  src
> Committed By: kamil
> Date: Wed Jan 29 16:03:44 UTC 2020
>
> Modified Files:
>   src/lib/libpthread: pthread.c pthread_getcpuclockid.c
>
> Log Message:
> Chack thread->pt_magic with PT_MAGIC promptly
>
> Rearrange some checks to avoid verifying pthread_t after using it.
>
>
> To generate a diff of this commit:
> cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
> cvs rdiff -u -r1.2 -r1.3
>src/lib/libpthread/pthread_getcpuclockid.c
>
> Please note that diffs are not public domain; they are subject to
>the
> copyright notices on the relevant files.
>

 -- 
 Ryo ONODERA // r...@tetera.org
 PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1
>15F3
>> 
>> 

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Hello,

I've checked with NetBSD-current from today (2020-02-02) and
pkgsrc-current (2020-02-02) and package firefox-72.0.2.

I'm not reproducing any crash due to pthread_equal(3) misuse. Everything
I tested, worked for me.

Please try PTHREAD_DIAGASSERT=ae and debug the culprit crash with a
core(5) file.

On 01.02.2020 22:20, Kamil Rytarowski wrote:
> Good idea. It could be checked quicker... however I presume that
> t1->pt_magic + t1->pt_magic already crash on invalid t1/t2 pointers as
> the argument with condition is evaluated.
> 
> Ryo, you might check:
> $ export PTHREAD_DIAGASSERT=ae
> $ firefox
> 
> It should create a coredump for investigation.
> 
> According to POSIX
> (https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_equal.html)
> passing invalid parameters is UB.
> 
> GLIBC, Illumos and all other BSDs (+ older NetBSD) have no sanity check
> in pthread_equal(3). Apparently we are the first ones to notice the bug.
> 
> On 01.02.2020 21:18, Andrew Doran wrote:
>> Hmm.  Was there not originally an environment variable to control this
>> behaviour, since many applications are buggy?
>>
>> Andrew
>>
>> On Sun, Feb 02, 2020 at 01:01:49AM +0900, Ryo ONODERA wrote:
>>> Hi,
>>>
>>> pthread__error()s in pthread_equal() cause segfault
>>> during start of pkgsrc/www/firefox-72.0.2.
>>>
>>> Without pthread__error()s, www/firefox works fine
>>> like as follows.
>>> However I have no idea why I get segfaults.
>>>
>>> Could you take a look at this problem?
>>>
>>> Index: lib/libpthread/pthread.c
>>> ===
>>> RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
>>> retrieving revision 1.162
>>> diff -u -r1.162 pthread.c
>>> --- lib/libpthread/pthread.c29 Jan 2020 17:11:57 -  1.162
>>> +++ lib/libpthread/pthread.c1 Feb 2020 15:58:03 -
>>> @@ -770,11 +770,13 @@
>>> if (__predict_false(__uselibcstub))
>>> return __libc_thr_equal_stub(t1, t2);
>>>  
>>> +#if 0
>>> pthread__error(EINVAL, "Invalid thread",
>>> t1->pt_magic == PT_MAGIC);
>>>  
>>> pthread__error(EINVAL, "Invalid thread",
>>> t2->pt_magic == PT_MAGIC);
>>> +#endif
>>>  
>>> /* Nothing special here. */
>>> return (t1 == t2);
>>> @@ -1108,7 +1110,7 @@
>>>  {
>>> char buf[1024];
>>> size_t len;
>>> -   
>>> +
>>> if (pthread__diagassert == 0)
>>> return;
>>>  
>>>
>>>
>>> "Kamil Rytarowski"  writes:
>>>
 Module Name:   src
 Committed By:  kamil
 Date:  Wed Jan 29 16:03:44 UTC 2020

 Modified Files:
src/lib/libpthread: pthread.c pthread_getcpuclockid.c

 Log Message:
 Chack thread->pt_magic with PT_MAGIC promptly

 Rearrange some checks to avoid verifying pthread_t after using it.


 To generate a diff of this commit:
 cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
 cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_getcpuclockid.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>>>
>>> -- 
>>> Ryo ONODERA // r...@tetera.org
>>> PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3
> 
> 




signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Good idea. It could be checked quicker... however I presume that
t1->pt_magic + t1->pt_magic already crash on invalid t1/t2 pointers as
the argument with condition is evaluated.

Ryo, you might check:
$ export PTHREAD_DIAGASSERT=ae
$ firefox

It should create a coredump for investigation.

According to POSIX
(https://pubs.opengroup.org/onlinepubs/9699919799/functions/pthread_equal.html)
passing invalid parameters is UB.

GLIBC, Illumos and all other BSDs (+ older NetBSD) have no sanity check
in pthread_equal(3). Apparently we are the first ones to notice the bug.

On 01.02.2020 21:18, Andrew Doran wrote:
> Hmm.  Was there not originally an environment variable to control this
> behaviour, since many applications are buggy?
> 
> Andrew
> 
> On Sun, Feb 02, 2020 at 01:01:49AM +0900, Ryo ONODERA wrote:
>> Hi,
>>
>> pthread__error()s in pthread_equal() cause segfault
>> during start of pkgsrc/www/firefox-72.0.2.
>>
>> Without pthread__error()s, www/firefox works fine
>> like as follows.
>> However I have no idea why I get segfaults.
>>
>> Could you take a look at this problem?
>>
>> Index: lib/libpthread/pthread.c
>> ===
>> RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
>> retrieving revision 1.162
>> diff -u -r1.162 pthread.c
>> --- lib/libpthread/pthread.c 29 Jan 2020 17:11:57 -  1.162
>> +++ lib/libpthread/pthread.c 1 Feb 2020 15:58:03 -
>> @@ -770,11 +770,13 @@
>>  if (__predict_false(__uselibcstub))
>>  return __libc_thr_equal_stub(t1, t2);
>>  
>> +#if 0
>>  pthread__error(EINVAL, "Invalid thread",
>>  t1->pt_magic == PT_MAGIC);
>>  
>>  pthread__error(EINVAL, "Invalid thread",
>>  t2->pt_magic == PT_MAGIC);
>> +#endif
>>  
>>  /* Nothing special here. */
>>  return (t1 == t2);
>> @@ -1108,7 +1110,7 @@
>>  {
>>  char buf[1024];
>>  size_t len;
>> -
>> +
>>  if (pthread__diagassert == 0)
>>  return;
>>  
>>
>>
>> "Kamil Rytarowski"  writes:
>>
>>> Module Name:src
>>> Committed By:   kamil
>>> Date:   Wed Jan 29 16:03:44 UTC 2020
>>>
>>> Modified Files:
>>> src/lib/libpthread: pthread.c pthread_getcpuclockid.c
>>>
>>> Log Message:
>>> Chack thread->pt_magic with PT_MAGIC promptly
>>>
>>> Rearrange some checks to avoid verifying pthread_t after using it.
>>>
>>>
>>> To generate a diff of this commit:
>>> cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
>>> cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_getcpuclockid.c
>>>
>>> Please note that diffs are not public domain; they are subject to the
>>> copyright notices on the relevant files.
>>>
>>
>> -- 
>> Ryo ONODERA // r...@tetera.org
>> PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3




signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Andrew Doran]

Hmm.  Was there not originally an environment variable to control this
behaviour, since many applications are buggy?

Andrew

On Sun, Feb 02, 2020 at 01:01:49AM +0900, Ryo ONODERA wrote:
> Hi,
> 
> pthread__error()s in pthread_equal() cause segfault
> during start of pkgsrc/www/firefox-72.0.2.
> 
> Without pthread__error()s, www/firefox works fine
> like as follows.
> However I have no idea why I get segfaults.
> 
> Could you take a look at this problem?
> 
> Index: lib/libpthread/pthread.c
> ===
> RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
> retrieving revision 1.162
> diff -u -r1.162 pthread.c
> --- lib/libpthread/pthread.c  29 Jan 2020 17:11:57 -  1.162
> +++ lib/libpthread/pthread.c  1 Feb 2020 15:58:03 -
> @@ -770,11 +770,13 @@
>   if (__predict_false(__uselibcstub))
>   return __libc_thr_equal_stub(t1, t2);
>  
> +#if 0
>   pthread__error(EINVAL, "Invalid thread",
>   t1->pt_magic == PT_MAGIC);
>  
>   pthread__error(EINVAL, "Invalid thread",
>   t2->pt_magic == PT_MAGIC);
> +#endif
>  
>   /* Nothing special here. */
>   return (t1 == t2);
> @@ -1108,7 +1110,7 @@
>  {
>   char buf[1024];
>   size_t len;
> - 
> +
>   if (pthread__diagassert == 0)
>   return;
>  
> 
> 
> "Kamil Rytarowski"  writes:
> 
> > Module Name:src
> > Committed By:   kamil
> > Date:   Wed Jan 29 16:03:44 UTC 2020
> >
> > Modified Files:
> > src/lib/libpthread: pthread.c pthread_getcpuclockid.c
> >
> > Log Message:
> > Chack thread->pt_magic with PT_MAGIC promptly
> >
> > Rearrange some checks to avoid verifying pthread_t after using it.
> >
> >
> > To generate a diff of this commit:
> > cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
> > cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_getcpuclockid.c
> >
> > Please note that diffs are not public domain; they are subject to the
> > copyright notices on the relevant files.
> >
> 
> -- 
> Ryo ONODERA // r...@tetera.org
> PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Kamil Rytarowski  writes:

> On 01.02.2020 17:01, Ryo ONODERA wrote:
>> Hi,
>> 
>> pthread__error()s in pthread_equal() cause segfault
>> during start of pkgsrc/www/firefox-72.0.2.
>> 
>> Without pthread__error()s, www/firefox works fine
>> like as follows.
>> However I have no idea why I get segfaults.
>> 
>> Could you take a look at this problem?
>> 
>> Index: lib/libpthread/pthread.c
>> ===
>> RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
>> retrieving revision 1.162
>> diff -u -r1.162 pthread.c
>> --- lib/libpthread/pthread.c 29 Jan 2020 17:11:57 -  1.162
>> +++ lib/libpthread/pthread.c 1 Feb 2020 15:58:03 -
>> @@ -770,11 +770,13 @@
>>  if (__predict_false(__uselibcstub))
>>  return __libc_thr_equal_stub(t1, t2);
>>  
>> +#if 0
>>  pthread__error(EINVAL, "Invalid thread",
>>  t1->pt_magic == PT_MAGIC);
>>  
>>  pthread__error(EINVAL, "Invalid thread",
>>  t2->pt_magic == PT_MAGIC);
>> +#endif
>>  
>>  /* Nothing special here. */
>>  return (t1 == t2);
>> @@ -1108,7 +1110,7 @@
>>  {
>>  char buf[1024];
>>  size_t len;
>> -
>> +
>>  if (pthread__diagassert == 0)
>>  return;
>>  
>> 
>> 
>> "Kamil Rytarowski"  writes:
>> 
>>> Module Name:src
>>> Committed By:   kamil
>>> Date:   Wed Jan 29 16:03:44 UTC 2020
>>>
>>> Modified Files:
>>> src/lib/libpthread: pthread.c pthread_getcpuclockid.c
>>>
>>> Log Message:
>>> Chack thread->pt_magic with PT_MAGIC promptly
>>>
>>> Rearrange some checks to avoid verifying pthread_t after using it.
>>>
>>>
>>> To generate a diff of this commit:
>>> cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
>>> cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_getcpuclockid.c
>>>
>>> Please note that diffs are not public domain; they are subject to the
>>> copyright notices on the relevant files.
>>>
>> 
>
> I will have a look, but it will take a while to build firefox for me.

Thank you.
Of course I can wait.

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 01.02.2020 17:01, Ryo ONODERA wrote:
> Hi,
> 
> pthread__error()s in pthread_equal() cause segfault
> during start of pkgsrc/www/firefox-72.0.2.
> 
> Without pthread__error()s, www/firefox works fine
> like as follows.
> However I have no idea why I get segfaults.
> 
> Could you take a look at this problem?
> 
> Index: lib/libpthread/pthread.c
> ===
> RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
> retrieving revision 1.162
> diff -u -r1.162 pthread.c
> --- lib/libpthread/pthread.c  29 Jan 2020 17:11:57 -  1.162
> +++ lib/libpthread/pthread.c  1 Feb 2020 15:58:03 -
> @@ -770,11 +770,13 @@
>   if (__predict_false(__uselibcstub))
>   return __libc_thr_equal_stub(t1, t2);
>  
> +#if 0
>   pthread__error(EINVAL, "Invalid thread",
>   t1->pt_magic == PT_MAGIC);
>  
>   pthread__error(EINVAL, "Invalid thread",
>   t2->pt_magic == PT_MAGIC);
> +#endif
>  
>   /* Nothing special here. */
>   return (t1 == t2);
> @@ -1108,7 +1110,7 @@
>  {
>   char buf[1024];
>   size_t len;
> - 
> +
>   if (pthread__diagassert == 0)
>   return;
>  
> 
> 
> "Kamil Rytarowski"  writes:
> 
>> Module Name: src
>> Committed By:kamil
>> Date:Wed Jan 29 16:03:44 UTC 2020
>>
>> Modified Files:
>>  src/lib/libpthread: pthread.c pthread_getcpuclockid.c
>>
>> Log Message:
>> Chack thread->pt_magic with PT_MAGIC promptly
>>
>> Rearrange some checks to avoid verifying pthread_t after using it.
>>
>>
>> To generate a diff of this commit:
>> cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
>> cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_getcpuclockid.c
>>
>> Please note that diffs are not public domain; they are subject to the
>> copyright notices on the relevant files.
>>
> 

I will have a look, but it will take a while to build firefox for me.



signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Ryo ONODERA]

Hi,

pthread__error()s in pthread_equal() cause segfault
during start of pkgsrc/www/firefox-72.0.2.

Without pthread__error()s, www/firefox works fine
like as follows.
However I have no idea why I get segfaults.

Could you take a look at this problem?

Index: lib/libpthread/pthread.c
===
RCS file: /cvsroot/src/lib/libpthread/pthread.c,v
retrieving revision 1.162
diff -u -r1.162 pthread.c
--- lib/libpthread/pthread.c29 Jan 2020 17:11:57 -  1.162
+++ lib/libpthread/pthread.c1 Feb 2020 15:58:03 -
@@ -770,11 +770,13 @@
if (__predict_false(__uselibcstub))
return __libc_thr_equal_stub(t1, t2);
 
+#if 0
pthread__error(EINVAL, "Invalid thread",
t1->pt_magic == PT_MAGIC);
 
pthread__error(EINVAL, "Invalid thread",
t2->pt_magic == PT_MAGIC);
+#endif
 
/* Nothing special here. */
return (t1 == t2);
@@ -1108,7 +1110,7 @@
 {
char buf[1024];
size_t len;
-   
+
if (pthread__diagassert == 0)
return;
 


"Kamil Rytarowski"  writes:

> Module Name:  src
> Committed By: kamil
> Date: Wed Jan 29 16:03:44 UTC 2020
>
> Modified Files:
>   src/lib/libpthread: pthread.c pthread_getcpuclockid.c
>
> Log Message:
> Chack thread->pt_magic with PT_MAGIC promptly
>
> Rearrange some checks to avoid verifying pthread_t after using it.
>
>
> To generate a diff of this commit:
> cvs rdiff -u -r1.160 -r1.161 src/lib/libpthread/pthread.c
> cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/pthread_getcpuclockid.c
>
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
>

-- 
Ryo ONODERA // r...@tetera.org
PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Actually it happened that modifiying pthread_atfork() to stop
malloc()ing is enough to address the problem.

I have landed the changes and removed '#if 0' kludge.

Thanks!

On 01.02.2020 13:59, Kamil Rytarowski wrote:
> On 31.01.2020 22:10, Andrew Doran wrote:
>> On Fri, Jan 31, 2020 at 06:55:00PM -, Christos Zoulas wrote:
>>
>>> In article <724af477-010b-9ddf-6ece-e23d7cf59...@gmx.com>,
>>> Kamil Rytarowski   wrote:
 -=-=-=-=-=-
 -=-=-=-=-=-

 On 31.01.2020 03:38, Christos Zoulas wrote:
> And it is fixed now.
>
> christos
>

 OK. I am going to submit a bug report upstream and get some feedback
 what is the way forward here, delaying initialization.
>>>
>>> I think that the way forward (on our side) is to do away with libpthread,
>>> merge it with libc and kill all the stub nonsense.
>>
>> Agreed.
>>
>> pthread__init() does some expensive stuff like _lwp_ctl().  I think we can
>> safely & without hacks defer a lot of that till the first pthread_create().
>>
>> Andrew
>>
> 
> This libc-libpthread split/merge is a red herring.
> 
> The problem here is with a mutual dependencies between POSIX threads
> library and malloc library.
> 
> I did some investigation and here are my findings:
> 
> 1. jemalloc abuses initialization and initializes self very early, with
> a constructor:
> 
> /*
>  * If an application creates a thread before doing any allocation in the
> main
>  * thread, then calls fork(2) in the main thread followed by memory
> allocation
>  * in the child process, a race can occur that results in deadlock
> within the
>  * child: the main thread may have forked while the created thread had
>  * partially initialized the allocator.  Ordinarily jemalloc prevents
>  * fork/malloc races via the following functions it registers during
>  * initialization using pthread_atfork(), but of course that does no good if
>  * the allocator isn't fully initialized at fork time.  The following
> library
>  * constructor is a partial solution to this problem.  It may still be
> possible
>  * to trigger the deadlock described above, but doing so would involve
> forking
>  * via a library constructor that runs before jemalloc's runs.
>  */
> #ifndef JEMALLOC_JET
> JEMALLOC_ATTR(constructor)
> static void
> 
> jemalloc_constructor(void) {
> malloc_init();
> }
> #endif
> 
> Relevant commit:
> 
> commit 20f1fc95adb35ea63dc61f47f2b0ffbd37d39f32
> Author: Jason Evans 
> Date:   Tue Oct 9 14:46:22 2012 -0700
> 
> Fix fork(2)-related deadlocks.
> 
> Add a library constructor for jemalloc that initializes the allocator.
> This fixes a race that could occur if threads were created by the main
> thread prior to any memory allocation, followed by fork(2), and then
> memory allocation in the child process.
> 
> Fix the prefork/postfork functions to acquire/release the ctl, prof, and
> rtree mutexes.  This fixes various fork() child process deadlocks, but
> one possible deadlock remains (intentionally) unaddressed: prof
> backtracing can acquire runtime library mutexes, so deadlock is still
> possible if heap profiling is enabled during fork().  This deadlock is
> known to be a real issue in at least the case of libgcc-based
> backtracing.
> 
> Reported by tfengjun.
> 
> 2. FreeBSD added a hack and an internal pthread_mutex_init() version
> called: _pthread_mutex_init_calloc_cb().. it passes a callback pointer
> to jemalloc's tiny calloc().
> 
> This is very ugly and I consider it as the wrong way of boostraping malloc.
> 
> 3. There is a problem inside libpthread. It as designed to not malloc()
> early to not trigger malloc initialization, however it was broken as it
> calls at_fork functions:
> 
> #0  malloc (size=size@entry=16) at
> /usr/src/external/bsd/jemalloc/lib/../dist/src/jemalloc.c:2052
> #1  0x776b71d71a44 in af_alloc () at
> /usr/src/lib/libc/gen/pthread_atfork.c:80
> #2  af_alloc () at /usr/src/lib/libc/gen/pthread_atfork.c:74
> #3  _pthread_atfork (prepare=prepare@entry=0x0, parent=parent@entry=0x0,
> child=child@entry=0x776b7220b3e5 )
> at /usr/src/lib/libc/gen/pthread_atfork.c:121
> #4  0x776b7220cacd in pthread__init () at
> /usr/src/lib/libpthread/pthread.c:260
> #5  0x776b71d7a585 in _libc_init () at
> /usr/src/lib/libc/misc/initfini.c:128
> 
> These at_fork routines caused also issues with false-positives in Leak
> Sanitizer. I had to pacify the sanitizer and disable tracking of its
> allocations.
> 
> 
> This patch removes '#if 0' hack from src/lib/libpthread and switches
> at_fork to mmap()+munmap().
> 
> http://netbsd.org/~kamil/patch-00219-libpthread-libc-jemalloc.txt
> 
> This test disabled the constructor hack:
> 
> http://netbsd.org/~kamil/patch-00220-jemalloc-disable-constructor.txt
> 
> With these changes everything seems to work.
> 
> In order to avoid the FreeBSD specific hack with the constructor and
> initialize jemalloc always during libc bootstrap I propose the 

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 31.01.2020 22:10, Andrew Doran wrote:
> On Fri, Jan 31, 2020 at 06:55:00PM -, Christos Zoulas wrote:
> 
>> In article <724af477-010b-9ddf-6ece-e23d7cf59...@gmx.com>,
>> Kamil Rytarowski   wrote:
>>> -=-=-=-=-=-
>>> -=-=-=-=-=-
>>>
>>> On 31.01.2020 03:38, Christos Zoulas wrote:
 And it is fixed now.

 christos

>>>
>>> OK. I am going to submit a bug report upstream and get some feedback
>>> what is the way forward here, delaying initialization.
>>
>> I think that the way forward (on our side) is to do away with libpthread,
>> merge it with libc and kill all the stub nonsense.
> 
> Agreed.
> 
> pthread__init() does some expensive stuff like _lwp_ctl().  I think we can
> safely & without hacks defer a lot of that till the first pthread_create().
> 
> Andrew
> 

This libc-libpthread split/merge is a red herring.

The problem here is with a mutual dependencies between POSIX threads
library and malloc library.

I did some investigation and here are my findings:

1. jemalloc abuses initialization and initializes self very early, with
a constructor:

/*
 * If an application creates a thread before doing any allocation in the
main
 * thread, then calls fork(2) in the main thread followed by memory
allocation
 * in the child process, a race can occur that results in deadlock
within the
 * child: the main thread may have forked while the created thread had
 * partially initialized the allocator.  Ordinarily jemalloc prevents
 * fork/malloc races via the following functions it registers during
 * initialization using pthread_atfork(), but of course that does no good if
 * the allocator isn't fully initialized at fork time.  The following
library
 * constructor is a partial solution to this problem.  It may still be
possible
 * to trigger the deadlock described above, but doing so would involve
forking
 * via a library constructor that runs before jemalloc's runs.
 */
#ifndef JEMALLOC_JET
JEMALLOC_ATTR(constructor)
static void

jemalloc_constructor(void) {
malloc_init();
}
#endif

Relevant commit:

commit 20f1fc95adb35ea63dc61f47f2b0ffbd37d39f32
Author: Jason Evans 
Date:   Tue Oct 9 14:46:22 2012 -0700

Fix fork(2)-related deadlocks.

Add a library constructor for jemalloc that initializes the allocator.
This fixes a race that could occur if threads were created by the main
thread prior to any memory allocation, followed by fork(2), and then
memory allocation in the child process.

Fix the prefork/postfork functions to acquire/release the ctl, prof, and
rtree mutexes.  This fixes various fork() child process deadlocks, but
one possible deadlock remains (intentionally) unaddressed: prof
backtracing can acquire runtime library mutexes, so deadlock is still
possible if heap profiling is enabled during fork().  This deadlock is
known to be a real issue in at least the case of libgcc-based
backtracing.

Reported by tfengjun.

2. FreeBSD added a hack and an internal pthread_mutex_init() version
called: _pthread_mutex_init_calloc_cb().. it passes a callback pointer
to jemalloc's tiny calloc().

This is very ugly and I consider it as the wrong way of boostraping malloc.

3. There is a problem inside libpthread. It as designed to not malloc()
early to not trigger malloc initialization, however it was broken as it
calls at_fork functions:

#0  malloc (size=size@entry=16) at
/usr/src/external/bsd/jemalloc/lib/../dist/src/jemalloc.c:2052
#1  0x776b71d71a44 in af_alloc () at
/usr/src/lib/libc/gen/pthread_atfork.c:80
#2  af_alloc () at /usr/src/lib/libc/gen/pthread_atfork.c:74
#3  _pthread_atfork (prepare=prepare@entry=0x0, parent=parent@entry=0x0,
child=child@entry=0x776b7220b3e5 )
at /usr/src/lib/libc/gen/pthread_atfork.c:121
#4  0x776b7220cacd in pthread__init () at
/usr/src/lib/libpthread/pthread.c:260
#5  0x776b71d7a585 in _libc_init () at
/usr/src/lib/libc/misc/initfini.c:128

These at_fork routines caused also issues with false-positives in Leak
Sanitizer. I had to pacify the sanitizer and disable tracking of its
allocations.


This patch removes '#if 0' hack from src/lib/libpthread and switches
at_fork to mmap()+munmap().

http://netbsd.org/~kamil/patch-00219-libpthread-libc-jemalloc.txt

This test disabled the constructor hack:

http://netbsd.org/~kamil/patch-00220-jemalloc-disable-constructor.txt

With these changes everything seems to work.

In order to avoid the FreeBSD specific hack with the constructor and
initialize jemalloc always during libc bootstrap I propose the following
approach:

 - add __libc_malloc_init() call in _libc_init()
 - redirect __libc_malloc_init() to jemalloc__init() with jemalloc
 - otherwise redirect to an empty stub

Here is a patch that does everything and works fine for me.

http://netbsd.org/~kamil/patch-00222-jemalloc-enhancements.txt

There are no longer jemalloc calls before being ready and jemalloc is
still initialized always but in its proper time.

(gdb) r
The program being 

Re: CVS commit: src/lib/libpthread

[Andrew Doran]

On Fri, Jan 31, 2020 at 06:55:00PM -, Christos Zoulas wrote:

> In article <724af477-010b-9ddf-6ece-e23d7cf59...@gmx.com>,
> Kamil Rytarowski   wrote:
> >-=-=-=-=-=-
> >-=-=-=-=-=-
> >
> >On 31.01.2020 03:38, Christos Zoulas wrote:
> >> And it is fixed now.
> >> 
> >> christos
> >> 
> >
> >OK. I am going to submit a bug report upstream and get some feedback
> >what is the way forward here, delaying initialization.
> 
> I think that the way forward (on our side) is to do away with libpthread,
> merge it with libc and kill all the stub nonsense.

Agreed.

pthread__init() does some expensive stuff like _lwp_ctl().  I think we can
safely & without hacks defer a lot of that till the first pthread_create().

Andrew

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 31.01.2020 19:55, Christos Zoulas wrote:
> In article <724af477-010b-9ddf-6ece-e23d7cf59...@gmx.com>,
> Kamil Rytarowski   wrote:
>> -=-=-=-=-=-
>> -=-=-=-=-=-
>>
>> On 31.01.2020 03:38, Christos Zoulas wrote:
>>> And it is fixed now.
>>>
>>> christos
>>>
>>
>> OK. I am going to submit a bug report upstream and get some feedback
>> what is the way forward here, delaying initialization.
> 
> I think that the way forward (on our side) is to do away with libpthread,
> merge it with libc and kill all the stub nonsense.
> 
> christos
> 

I was thinking about implementing custom mutexes inside jemalloc with
futexes. A potential implementation could be really thin. There is just
a problem that futexes still need some more time to land the sources.

https://locklessinc.com/articles/mutex_cv_futex/

My thread: https://github.com/jemalloc/jemalloc/issues/1753

I have got no opinion on stubs and separated libpthread.



signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

In article <724af477-010b-9ddf-6ece-e23d7cf59...@gmx.com>,
Kamil Rytarowski   wrote:
>-=-=-=-=-=-
>-=-=-=-=-=-
>
>On 31.01.2020 03:38, Christos Zoulas wrote:
>> And it is fixed now.
>> 
>> christos
>> 
>
>OK. I am going to submit a bug report upstream and get some feedback
>what is the way forward here, delaying initialization.

I think that the way forward (on our side) is to do away with libpthread,
merge it with libc and kill all the stub nonsense.

christos

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 31.01.2020 03:38, Christos Zoulas wrote:
> And it is fixed now.
> 
> christos
> 

OK. I am going to submit a bug report upstream and get some feedback
what is the way forward here, delaying initialization.



signature.asc
Description: OpenPGP digital signature

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

And it is fixed now.

christos


signature.asc
Description: Message signed with OpenPGP

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

At this point I'd say it is simpler to just initialize the mutexattr_t fields 
in a new
libc stub for the attribute init.

christos

> On Jan 30, 2020, at 8:05 PM, Kamil Rytarowski  wrote:
> 
> Signed PGP part
> On 05.03.2019 23:49, Christos Zoulas wrote:
>> Module Name: src
>> Committed By:christos
>> Date:Tue Mar  5 22:49:38 UTC 2019
>> 
>> Modified Files:
>>  src/lib/libpthread: pthread_mutex.c
>> 
>> Log Message:
>> Jemalloc initializes mutexes before we become threaded and expects to use
>> them later.
>> 
>> 
>> To generate a diff of this commit:
>> cvs rdiff -u -r1.64 -r1.65 src/lib/libpthread/pthread_mutex.c
>> 
>> Please note that diffs are not public domain; they are subject to the
>> copyright notices on the relevant files.
>> 
>> 
>> Modified files:
>> 
>> Index: src/lib/libpthread/pthread_mutex.c
>> diff -u src/lib/libpthread/pthread_mutex.c:1.64 
>> src/lib/libpthread/pthread_mutex.c:1.65
>> --- src/lib/libpthread/pthread_mutex.c:1.64  Fri Dec  8 04:24:31 2017
>> +++ src/lib/libpthread/pthread_mutex.c   Tue Mar  5 17:49:38 2019
>> @@ -1,4 +1,4 @@
>> -/*  $NetBSD: pthread_mutex.c,v 1.64 2017/12/08 09:24:31 kre Exp $   */
>> +/*  $NetBSD: pthread_mutex.c,v 1.65 2019/03/05 22:49:38 christos Exp $  
>> */
>> 
>> /*-
>>  * Copyright (c) 2001, 2003, 2006, 2007, 2008 The NetBSD Foundation, Inc.
>> @@ -47,7 +47,7 @@
>>  */
>> 
>> #include 
>> -__RCSID("$NetBSD: pthread_mutex.c,v 1.64 2017/12/08 09:24:31 kre Exp $");
>> +__RCSID("$NetBSD: pthread_mutex.c,v 1.65 2019/03/05 22:49:38 christos Exp 
>> $");
>> 
>> #include 
>> #include 
>> @@ -122,8 +122,14 @@ pthread_mutex_init(pthread_mutex_t *ptm,
>> {
>>  uintptr_t type, proto, val, ceil;
>> 
>> +#if 0
>> +/*
>> + * Always initialize the mutex structure, maybe be used later
>> + * and the cost should be minimal.
>> + */
>>  if (__predict_false(__uselibcstub))
>>  return __libc_mutex_init_stub(ptm, attr);
>> +#endif
>> 
>>  if (attr == NULL) {
>>  type = PTHREAD_MUTEX_NORMAL;
>> 
> 
> This change looks questionable.
> 
> Inside external/bsd/jemalloc/lib/../dist/src/mutex.c:
> 
>pthread_mutexattr_t attr;
> 
>if (pthread_mutexattr_init() != 0) {
>return true;
> 
>}
>pthread_mutexattr_settype(, MALLOC_MUTEX_TYPE);
>if (pthread_mutex_init(>lock, ) != 0) {
>pthread_mutexattr_destroy();
>return true;
>}
>pthread_mutexattr_destroy();
> 
> 
> We initialize attr with garbage as we use libc stubs and later pass
> garbage to pthread_mutex_init().
> 
> I want to add this sanity check inside pthread_mutex_init()
> 
> http://netbsd.org/~kamil/patch-00218-pthread_mutex_init-check-attr.txt
> 
> Unfortunately this causes jemalloc to deadlock.
> 
> (gdb) bt
> #0  pthread_rwlock_destroy (ptr=0x0) at
> /usr/src/lib/libpthread/pthread_rwlock.c:112
> #1  0x0246 in ?? ()
> #2  0x7f7fe600 in ?? ()
> #3  0x7f7ff7d0 in ?? ()
> #4  0x7f7ff72d94a0 in je_malloc_mutex_init (mutex=0x7f7fe600,
> mutex@entry=0x7f7ff7d0, name=name@entry=0x7f7ff7387106 "base",
>rank=rank@entry=18,
> lock_order=lock_order@entry=malloc_mutex_rank_exclusive) at
> /usr/src/external/bsd/jemalloc/lib/../dist/src/mutex.c:167
> #5  0x7f7ff731834f in je_base_new (tsdn=tsdn@entry=0x0,
> ind=ind@entry=0, extent_hooks=0x7f7ff75ca120 )
>at /usr/src/external/bsd/jemalloc/lib/../dist/src/base.c:366
> #6  0x7f7ff73185d2 in je_base_boot (tsdn=tsdn@entry=0x0) at
> /usr/src/external/bsd/jemalloc/lib/../dist/src/base.c:512
> #7  0x7f7ff731011d in malloc_init_hard_a0_locked () at
> /usr/src/external/bsd/jemalloc/lib/../dist/src/jemalloc.c:1327
> #8  0x7f7ff73107f5 in malloc_init_hard () at
> /usr/src/external/bsd/jemalloc/lib/../dist/src/jemalloc.c:1549
> #9  0x7f7ff723f924 in ?? () from /usr/lib/libc.so.12
> #10 0x7f7ff7ef9800 in ?? ()
> #11 0x7f7ff723ac09 in _init () from /usr/lib/libc.so.12
> #12 0x in ?? ()
> 
> Could we please fix it properly?
> 
> I am not sure what is the proper way here. We probably need to delay
> usage of pthread(3) but it is not that trivial.
> 
> Personally, I preferred the old jemalloc...
> 
> 
> 



signature.asc
Description: Message signed with OpenPGP

Re: CVS commit: src/lib/libpthread

[Kamil Rytarowski]

On 05.03.2019 23:49, Christos Zoulas wrote:
> Module Name:  src
> Committed By: christos
> Date: Tue Mar  5 22:49:38 UTC 2019
> 
> Modified Files:
>   src/lib/libpthread: pthread_mutex.c
> 
> Log Message:
> Jemalloc initializes mutexes before we become threaded and expects to use
> them later.
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.64 -r1.65 src/lib/libpthread/pthread_mutex.c
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
> 
> 
> Modified files:
> 
> Index: src/lib/libpthread/pthread_mutex.c
> diff -u src/lib/libpthread/pthread_mutex.c:1.64 
> src/lib/libpthread/pthread_mutex.c:1.65
> --- src/lib/libpthread/pthread_mutex.c:1.64   Fri Dec  8 04:24:31 2017
> +++ src/lib/libpthread/pthread_mutex.cTue Mar  5 17:49:38 2019
> @@ -1,4 +1,4 @@
> -/*   $NetBSD: pthread_mutex.c,v 1.64 2017/12/08 09:24:31 kre Exp $   */
> +/*   $NetBSD: pthread_mutex.c,v 1.65 2019/03/05 22:49:38 christos Exp $  
> */
>  
>  /*-
>   * Copyright (c) 2001, 2003, 2006, 2007, 2008 The NetBSD Foundation, Inc.
> @@ -47,7 +47,7 @@
>   */
>  
>  #include 
> -__RCSID("$NetBSD: pthread_mutex.c,v 1.64 2017/12/08 09:24:31 kre Exp $");
> +__RCSID("$NetBSD: pthread_mutex.c,v 1.65 2019/03/05 22:49:38 christos Exp 
> $");
>  
>  #include 
>  #include 
> @@ -122,8 +122,14 @@ pthread_mutex_init(pthread_mutex_t *ptm,
>  {
>   uintptr_t type, proto, val, ceil;
>  
> +#if 0
> + /*
> +  * Always initialize the mutex structure, maybe be used later
> +  * and the cost should be minimal.
> +  */
>   if (__predict_false(__uselibcstub))
>   return __libc_mutex_init_stub(ptm, attr);
> +#endif
>  
>   if (attr == NULL) {
>   type = PTHREAD_MUTEX_NORMAL;
> 

This change looks questionable.

Inside external/bsd/jemalloc/lib/../dist/src/mutex.c:

pthread_mutexattr_t attr;

if (pthread_mutexattr_init() != 0) {
return true;

}
pthread_mutexattr_settype(, MALLOC_MUTEX_TYPE);
if (pthread_mutex_init(>lock, ) != 0) {
pthread_mutexattr_destroy();
return true;
}
pthread_mutexattr_destroy();


We initialize attr with garbage as we use libc stubs and later pass
garbage to pthread_mutex_init().

I want to add this sanity check inside pthread_mutex_init()

http://netbsd.org/~kamil/patch-00218-pthread_mutex_init-check-attr.txt

Unfortunately this causes jemalloc to deadlock.

(gdb) bt
#0  pthread_rwlock_destroy (ptr=0x0) at
/usr/src/lib/libpthread/pthread_rwlock.c:112
#1  0x0246 in ?? ()
#2  0x7f7fe600 in ?? ()
#3  0x7f7ff7d0 in ?? ()
#4  0x7f7ff72d94a0 in je_malloc_mutex_init (mutex=0x7f7fe600,
mutex@entry=0x7f7ff7d0, name=name@entry=0x7f7ff7387106 "base",
rank=rank@entry=18,
lock_order=lock_order@entry=malloc_mutex_rank_exclusive) at
/usr/src/external/bsd/jemalloc/lib/../dist/src/mutex.c:167
#5  0x7f7ff731834f in je_base_new (tsdn=tsdn@entry=0x0,
ind=ind@entry=0, extent_hooks=0x7f7ff75ca120 )
at /usr/src/external/bsd/jemalloc/lib/../dist/src/base.c:366
#6  0x7f7ff73185d2 in je_base_boot (tsdn=tsdn@entry=0x0) at
/usr/src/external/bsd/jemalloc/lib/../dist/src/base.c:512
#7  0x7f7ff731011d in malloc_init_hard_a0_locked () at
/usr/src/external/bsd/jemalloc/lib/../dist/src/jemalloc.c:1327
#8  0x7f7ff73107f5 in malloc_init_hard () at
/usr/src/external/bsd/jemalloc/lib/../dist/src/jemalloc.c:1549
#9  0x7f7ff723f924 in ?? () from /usr/lib/libc.so.12
#10 0x7f7ff7ef9800 in ?? ()
#11 0x7f7ff723ac09 in _init () from /usr/lib/libc.so.12
#12 0x in ?? ()

Could we please fix it properly?

I am not sure what is the proper way here. We probably need to delay
usage of pthread(3) but it is not that trivial.

Personally, I preferred the old jemalloc...



signature.asc
Description: OpenPGP digital signature

CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Module Name:src
Committed By:   kamil
Date:   Tue Sep 10 22:34:19 UTC 2019

Modified Files:
src/lib/libpthread: thrd.c threads.h

Log Message:
Switch back _Noreturn to __dead in C11 threads

There is an ongoing discussion to unify unreturn attribute between C and
C++, making a compatible version, shared between languages. Instead of
picking C11-only approach here, switch back to __dead that wraps the
compiler extension for the same purpose.

This change makes this header more compatible with C++ and pre-C11.

Reference:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2410.pdf


To generate a diff of this commit:
cvs rdiff -u -r1.3 -r1.4 src/lib/libpthread/thrd.c
cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/threads.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

CVS commit: src/lib/libpthread

[Kamil Rytarowski]

Module Name:src
Committed By:   kamil
Date:   Tue Sep 10 22:34:19 UTC 2019

Modified Files:
src/lib/libpthread: thrd.c threads.h

Log Message:
Switch back _Noreturn to __dead in C11 threads

There is an ongoing discussion to unify unreturn attribute between C and
C++, making a compatible version, shared between languages. Instead of
picking C11-only approach here, switch back to __dead that wraps the
compiler extension for the same purpose.

This change makes this header more compatible with C++ and pre-C11.

Reference:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2410.pdf


To generate a diff of this commit:
cvs rdiff -u -r1.3 -r1.4 src/lib/libpthread/thrd.c
cvs rdiff -u -r1.2 -r1.3 src/lib/libpthread/threads.h

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: src/lib/libpthread/thrd.c
diff -u src/lib/libpthread/thrd.c:1.3 src/lib/libpthread/thrd.c:1.4
--- src/lib/libpthread/thrd.c:1.3	Mon Apr 29 20:11:43 2019
+++ src/lib/libpthread/thrd.c	Tue Sep 10 22:34:19 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: thrd.c,v 1.3 2019/04/29 20:11:43 kamil Exp $	*/
+/*	$NetBSD: thrd.c,v 1.4 2019/09/10 22:34:19 kamil Exp $	*/
 
 /*-
  * Copyright (c) 2016 The NetBSD Foundation, Inc.
@@ -30,7 +30,7 @@
  */
 
 #include 
-__RCSID("$NetBSD: thrd.c,v 1.3 2019/04/29 20:11:43 kamil Exp $");
+__RCSID("$NetBSD: thrd.c,v 1.4 2019/09/10 22:34:19 kamil Exp $");
 
 #include 
 #include 
@@ -122,7 +122,7 @@ thrd_equal(thrd_t t1, thrd_t t2)
 	return pthread_equal(t1, t2);
 }
 
-_Noreturn void
+__dead void
 thrd_exit(int res)
 {
 

Index: src/lib/libpthread/threads.h
diff -u src/lib/libpthread/threads.h:1.2 src/lib/libpthread/threads.h:1.3
--- src/lib/libpthread/threads.h:1.2	Wed Apr 24 18:47:54 2019
+++ src/lib/libpthread/threads.h	Tue Sep 10 22:34:19 2019
@@ -1,4 +1,4 @@
-/*	$NetBSD: threads.h,v 1.2 2019/04/24 18:47:54 kamil Exp $	*/
+/*	$NetBSD: threads.h,v 1.3 2019/09/10 22:34:19 kamil Exp $	*/
 
 /*-
  * Copyright (c) 2016 The NetBSD Foundation, Inc.
@@ -106,7 +106,7 @@ int	thrd_create(thrd_t *, thrd_start_t, 
 thrd_t	thrd_current(void);
 int	thrd_detach(thrd_t);
 int	thrd_equal(thrd_t, thrd_t);
-_Noreturn void	thrd_exit(int);
+__dead void	thrd_exit(int);
 int	thrd_join(thrd_t, int *);
 int	thrd_sleep(const struct timespec *, struct timespec *);
 void	thrd_yield(void);

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Tue, May 07, 2019 at 06:14:09PM +, m...@netbsd.org wrote:
> Pre-emptively making a thread where people can call this an ugly hack

It is, please put a note into doc/HACKS at least.

Joerg

Re: CVS commit: src/lib/libpthread

[maya]

Pre-emptively making a thread where people can call this an ugly hack

On Tue, May 07, 2019 at 06:12:54PM +, Maya Rashish wrote:
> Module Name:  src
> Committed By: maya
> Date: Tue May  7 18:12:53 UTC 2019
> 
> Modified Files:
>   src/lib/libpthread: Makefile
> 
> Log Message:
> Replace the link command for libpthread.a so that we create a single section
> with all the libpthread symbols in it.
> This makes -lpthread behave like to -Wl,--whole-archive -lpthread.
> 
> This avoids a situation where threaded static binaries use some libc thread
> stubs, which are racy.
> 
> Fixes PR lib/54001: call_once2_32, call_once2_static test cases failing on
> amd64 since gcc7 import.
> 
> Suggested by Jonathan Wakely, thanks!
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.92 -r1.93 src/lib/libpthread/Makefile
> 
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
> 

> Modified files:
> 
> Index: src/lib/libpthread/Makefile
> diff -u src/lib/libpthread/Makefile:1.92 src/lib/libpthread/Makefile:1.93
> --- src/lib/libpthread/Makefile:1.92  Wed Apr 24 11:43:19 2019
> +++ src/lib/libpthread/Makefile   Tue May  7 18:12:53 2019
> @@ -1,4 +1,4 @@
> -#$NetBSD: Makefile,v 1.92 2019/04/24 11:43:19 kamil Exp $
> +#$NetBSD: Makefile,v 1.93 2019/05/07 18:12:53 maya Exp $
>  #
>  
>  NOSANITIZER= # defined
> @@ -269,6 +269,20 @@ MLINKS+= tss.3 tss_set.3
>  
>  INCS+=   threads.h
>  
> +# PR lib/54001: create libpthread.a as a single large object, with all the
> +# symbols in one section. ensures that if any libpthread function is used,
> +# you get all of them from libpthread, and not the libc stubs.
> +#
> +# This makes -lpthread equivalent to -Wl,--whole-archive -lpthread
> +
> +__archivebuild: .USE
> + ${_MKTARGET_BUILD}
> + @rm -f ${.TARGET}
> + ${LD} -r -o ${.TARGET}.o `NM=${NM} ${LORDER} ${.ALLSRC:M*o} | ${TSORT}`
> + ${AR} ${_ARFL} ${.TARGET} ${.TARGET}.o
> +
> +CLEANFILES+= ${.TARGET}.o
> +
>  .include 
>  
>  .else
> 

Re: CVS commit: src/lib/libpthread

[maya]

netbsd-8 needs a web browser too

On Tue, Aug 01, 2017 at 12:31:45PM +, Martin Husemann wrote:
> Module Name:  src
> Committed By: martin
> Date: Tue Aug  1 12:31:45 UTC 2017
> 
> Modified Files:
>   src/lib/libpthread: pthread_attr.c
> 
> Log Message:
> pthread__attr_init_private:
> malloc+memset -> calloc. Also initialize all values to the proper
> defaults.
> This fixes the "rustc panic" discussed on pkgsrc-users.
> OK: joerg
> 
> 
> To generate a diff of this commit:
> cvs rdiff -u -r1.17 -r1.18 src/lib/libpthread/pthread_attr.c

Re: CVS commit: src/lib/libpthread

[Taylor R Campbell]

> Date: Tue, 28 Mar 2017 20:10:24 +0200
> From: Joerg Sonnenberger 
> 
> On Tue, Mar 28, 2017 at 05:42:53PM +, Maya Rashish wrote:
> > Remove outdated CAVEATS.
> > 
> > Not sure everything is standards compliant, but I've been told non-default
> > values are supported and pshared exists.
> 
> pshared is not supported.

That is addressed in the BUGS section, which was preserved.

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Tue, Mar 28, 2017 at 05:42:53PM +, Maya Rashish wrote:
> Module Name:  src
> Committed By: maya
> Date: Tue Mar 28 17:42:52 UTC 2017
> 
> Modified Files:
>   src/lib/libpthread: pthread_condattr.3
> 
> Log Message:
> Remove outdated CAVEATS.
> 
> Not sure everything is standards compliant, but I've been told non-default
> values are supported and pshared exists.

pshared is not supported.

Joerg

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Mon, Aug 24, 2015 at 12:45:15PM +, Antti Kantee wrote:
 I started work on this again by doing a web search for the error, and found
 this:
 http://bugs.musicpd.org/view.php?id=4110
 
 There you argue that using constexpr for PTHREAD_MUTEX_INITIALIZER is
 broken.  So if we assume that you are correct, shouldn't we remove the
 constexprs from libc++/dist/libcxx/include/__mutex_base?

Except that the standard requires it.

Joerg

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

Returning to this pickle.

On 26/06/15 18:24, Joerg Sonnenberger wrote:

On Fri, Jun 26, 2015 at 12:49:09PM +, Antti Kantee wrote:

This is the simplest program to repeat the problem with g++ 5.1 and NetBSD's
pthread_types.h:

#include pthread.h
class foo {
pthread_mutex_t m;
public:
constexpr foo() : m(PTHREAD_MUTEX_INITIALIZER) {}
};

(plus or minus typos since I typed it by hand from the virtual machine
console)

If there's some way to say pthread_types.h is even more of a system header
than with -isystem and compilers should not generate errors, then I'd like
to hear it.


You don't understand. The system header is not pthread_types.h, it is
mutex.


Also, if this is really the correct DR, I'm not sure how a flaw in the
standard translates to gcc bug
http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1911
(especially when there's no fix for the defect in the standard)


The fix is to apply the C++14 rules, which is more relaxed.


So, I really don't see how to sweep the issue under the rug except by a)
figuring out a workaround b) labeling all C++ compilers which happen to
implement the current standard as incompatible with NetBSD.


DRs are retroactive, e.g. bug fixes and corrections. As such they are
part of the current standard (and of course, it is not the current
standard, since that would be C++14).


I started work on this again by doing a web search for the error, and 
found this:

http://bugs.musicpd.org/view.php?id=4110

There you argue that using constexpr for PTHREAD_MUTEX_INITIALIZER is 
broken.  So if we assume that you are correct, shouldn't we remove the 
constexprs from libc++/dist/libcxx/include/__mutex_base?

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Mon, Aug 24, 2015 at 02:20:55PM +, Antti Kantee wrote:
 On 24/08/15 13:26, Joerg Sonnenberger wrote:
 On Mon, Aug 24, 2015 at 12:45:15PM +, Antti Kantee wrote:
 I started work on this again by doing a web search for the error, and found
 this:
 http://bugs.musicpd.org/view.php?id=4110
 
 There you argue that using constexpr for PTHREAD_MUTEX_INITIALIZER is
 broken.  So if we assume that you are correct, shouldn't we remove the
 constexprs from libc++/dist/libcxx/include/__mutex_base?
 
 Except that the standard requires it.
 
 So are you now saying that that what you said in the above URL is wrong and
 that the NetBSD headers are broken?  Let me quote the relevant part of what
 you said:

Stop inventing things, please.

 
 Nothing in ISO C11, ISO C++11 or POSIX requires PTHREAD_MUTEX_INITIALIZER
 to qualify for constexpr.
 
 But now you are saying that the standard does require
 PTHREAD_MUTEX_INITIALIZER to qualify for constexpr.

A mutex has to be constexpr constructable. That's in the standard. How a
mutex is implemented is not. Nothing in mpd requires the constexpr
property, so requiring it just makes things more difficult for no good
reason.

Joerg

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On 24/08/15 13:26, Joerg Sonnenberger wrote:

On Mon, Aug 24, 2015 at 12:45:15PM +, Antti Kantee wrote:

I started work on this again by doing a web search for the error, and found
this:
http://bugs.musicpd.org/view.php?id=4110

There you argue that using constexpr for PTHREAD_MUTEX_INITIALIZER is
broken.  So if we assume that you are correct, shouldn't we remove the
constexprs from libc++/dist/libcxx/include/__mutex_base?


Except that the standard requires it.


So are you now saying that that what you said in the above URL is wrong 
and that the NetBSD headers are broken?  Let me quote the relevant part 
of what you said:


Nothing in ISO C11, ISO C++11 or POSIX requires 
PTHREAD_MUTEX_INITIALIZER to qualify for constexpr.


But now you are saying that the standard does require 
PTHREAD_MUTEX_INITIALIZER to qualify for constexpr.


Feel free to reply in a fashion where you actually explain your stance 
instead of just dropping some facts.  Otherwise I'll have to guess what 
you mean, and that prevents me from fixing things the right way.

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On 24/08/15 14:36, Joerg Sonnenberger wrote:

On Mon, Aug 24, 2015 at 02:20:55PM +, Antti Kantee wrote:

On 24/08/15 13:26, Joerg Sonnenberger wrote:

On Mon, Aug 24, 2015 at 12:45:15PM +, Antti Kantee wrote:

I started work on this again by doing a web search for the error, and found
this:
http://bugs.musicpd.org/view.php?id=4110

There you argue that using constexpr for PTHREAD_MUTEX_INITIALIZER is
broken.  So if we assume that you are correct, shouldn't we remove the
constexprs from libc++/dist/libcxx/include/__mutex_base?


Except that the standard requires it.


So are you now saying that that what you said in the above URL is wrong and
that the NetBSD headers are broken?  Let me quote the relevant part of what
you said:


Stop inventing things, please.


A simple yes/no[, because ...] is more effective than evading the 
question.  I'm still not sure which one it is.



Nothing in ISO C11, ISO C++11 or POSIX requires PTHREAD_MUTEX_INITIALIZER
to qualify for constexpr.

But now you are saying that the standard does require
PTHREAD_MUTEX_INITIALIZER to qualify for constexpr.


A mutex has to be constexpr constructable. That's in the standard. How a
mutex is implemented is not. Nothing in mpd requires the constexpr
property, so requiring it just makes things more difficult for no good
reason.


What I understand from that response: the same code is wrong in mpd (and 
presumably every other application) but right in libc++; it's 
standard-compliant to implement mutex in a way which does not compile.

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Mon, Aug 24, 2015 at 03:03:28PM +, Antti Kantee wrote:
 On 24/08/15 14:36, Joerg Sonnenberger wrote:
 On Mon, Aug 24, 2015 at 02:20:55PM +, Antti Kantee wrote:
 On 24/08/15 13:26, Joerg Sonnenberger wrote:
 On Mon, Aug 24, 2015 at 12:45:15PM +, Antti Kantee wrote:
 I started work on this again by doing a web search for the error, and 
 found
 this:
 http://bugs.musicpd.org/view.php?id=4110
 
 There you argue that using constexpr for PTHREAD_MUTEX_INITIALIZER is
 broken.  So if we assume that you are correct, shouldn't we remove the
 constexprs from libc++/dist/libcxx/include/__mutex_base?
 
 Except that the standard requires it.
 
 So are you now saying that that what you said in the above URL is wrong and
 that the NetBSD headers are broken?  Let me quote the relevant part of what
 you said:
 
 Stop inventing things, please.
 
 A simple yes/no[, because ...] is more effective than evading the
 question.  I'm still not sure which one it is.
 
 Nothing in ISO C11, ISO C++11 or POSIX requires PTHREAD_MUTEX_INITIALIZER
 to qualify for constexpr.
 
 But now you are saying that the standard does require
 PTHREAD_MUTEX_INITIALIZER to qualify for constexpr.
 
 A mutex has to be constexpr constructable. That's in the standard. How a
 mutex is implemented is not. Nothing in mpd requires the constexpr
 property, so requiring it just makes things more difficult for no good
 reason.
 
 What I understand from that response: the same code is wrong in mpd (and
 presumably every other application) but right in libc++; it's
 standard-compliant to implement mutex in a way which does not compile.

Whether or not all the rules apply to system headers is a question
outside the scope of what I am willing to discuss. If you build a system
with strange flags that explicitly disable the normal system header
logic, you are on your own.

The code in mpd depends on the language DR to be resolved without having
a good reason to do so. Because I don't trust a compiler to behave
certainly doesn't qualify.

There are two approaches to implement a mutex in a standard-compliant
way with a PTHREAD_MUTEX_INITIALIZER using volatile elements:
(1) Assume that the compiler has a fix for the DR.
(2) Defer the actual mutex setup until a later point.

Clang implements the DR behavior in so far as skipping the check for
system headers. I have no idea what GCC is doing. The second option
comes with a significant penalty and simple is not acceptable.

Joerg

Re: CVS commit: src/lib/libpthread

[Martin Husemann]

On Fri, Jun 26, 2015 at 01:38:45AM +, Antti Kantee wrote:
 On Fri, Jun 26, 2015 at 01:33:09AM +, Antti Kantee wrote:
  Module Name:src
  Committed By:   pooka
  Date:   Fri Jun 26 01:33:09 UTC 2015
  
  Modified Files:
  src/lib/libpthread: pthread_types.h
  
  Log Message:
  C++ (namely libc++) expects to be using PTHREAD_FOO_INITIALIZER as a
  member initializer. This does not work for volatile types. Since C++
  does not touch the guts of those types, redefine them as non-volatile.
  
  Fixes libc++ compilation with g++ 5.1, as reported in PR lib/49989.
 
 Forgot to mention: the patch is from christos (thanks!)

Please document this in doc/HACKS, it is an evil hack and IMHO working
around a g++ bug.

Martin

Re: CVS commit: src/lib/libpthread

[Nick Hudson]

On 06/26/15 02:33, Antti Kantee wrote:

+# ifdef __CPU_SIMPLE_LOCK_PAD
+#  define __pthread_spin_t unsigned char
+# else
+#  define __pthread_spin_t unsigned int
+# endif


Are you sure this works for hppa which has funky __cpu_simple_lock_t?

Nick

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On 26/06/15 06:02, Martin Husemann wrote:

On Fri, Jun 26, 2015 at 01:38:45AM +, Antti Kantee wrote:

On Fri, Jun 26, 2015 at 01:33:09AM +, Antti Kantee wrote:

Module Name:src
Committed By:   pooka
Date:   Fri Jun 26 01:33:09 UTC 2015

Modified Files:
src/lib/libpthread: pthread_types.h

Log Message:
C++ (namely libc++) expects to be using PTHREAD_FOO_INITIALIZER as a
member initializer. This does not work for volatile types. Since C++
does not touch the guts of those types, redefine them as non-volatile.

Fixes libc++ compilation with g++ 5.1, as reported in PR lib/49989.


Forgot to mention: the patch is from christos (thanks!)


Please document this in doc/HACKS, it is an evil hack and IMHO working
around a g++ bug.


Done, rev 1.160 of doc/HACKS.

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

This is the simplest program to repeat the problem with g++ 5.1 and 
NetBSD's pthread_types.h:


#include pthread.h
class foo {
pthread_mutex_t m;
public:
constexpr foo() : m(PTHREAD_MUTEX_INITIALIZER) {}
};

(plus or minus typos since I typed it by hand from the virtual machine 
console)


If there's some way to say pthread_types.h is even more of a system 
header than with -isystem and compilers should not generate errors, 
then I'd like to hear it.


Also, if this is really the correct DR, I'm not sure how a flaw in the 
standard translates to gcc bug

http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1911
(especially when there's no fix for the defect in the standard)

So, I really don't see how to sweep the issue under the rug except by a) 
figuring out a workaround b) labeling all C++ compilers which happen to 
implement the current standard as incompatible with NetBSD.

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Fri, Jun 26, 2015 at 12:07:36PM +, Antti Kantee wrote:
 On 26/06/15 11:53, Joerg Sonnenberger wrote:
 That said, have
 you verified why it doesn't happen with libstdc++ itself? I would
 somewhat suspect that the threatment of the header as system header
 hides the problem for libstdc++, I can't imagine that it can correctly
 implement the constexpr constructor without performance penalties
 otherwise...
 
 I don't speak C++, but a grep-based guess coupled with common sense suggests
 that libstdc++ is fine because it doesn't use PTHREAD_MUTEX_INITIALIZER.

It is spelled __GTHREAD_MUTEX_INIT or something like that and goes via
another layer of indirection. Without it, the constexpr version would
need another flag bit, see above about runtime penalty.

Joerg

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Fri, Jun 26, 2015 at 12:49:09PM +, Antti Kantee wrote:
 This is the simplest program to repeat the problem with g++ 5.1 and NetBSD's
 pthread_types.h:
 
 #include pthread.h
 class foo {
   pthread_mutex_t m;
 public:
   constexpr foo() : m(PTHREAD_MUTEX_INITIALIZER) {}
 };
 
 (plus or minus typos since I typed it by hand from the virtual machine
 console)
 
 If there's some way to say pthread_types.h is even more of a system header
 than with -isystem and compilers should not generate errors, then I'd like
 to hear it.

You don't understand. The system header is not pthread_types.h, it is
mutex.

 Also, if this is really the correct DR, I'm not sure how a flaw in the
 standard translates to gcc bug
 http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1911
 (especially when there's no fix for the defect in the standard)

The fix is to apply the C++14 rules, which is more relaxed.

 So, I really don't see how to sweep the issue under the rug except by a)
 figuring out a workaround b) labeling all C++ compilers which happen to
 implement the current standard as incompatible with NetBSD.

DRs are retroactive, e.g. bug fixes and corrections. As such they are
part of the current standard (and of course, it is not the current
standard, since that would be C++14).

Joerg

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On 26/06/15 07:04, Nick Hudson wrote:

On 06/26/15 02:33, Antti Kantee wrote:

+# ifdef __CPU_SIMPLE_LOCK_PAD
+#  define __pthread_spin_t unsigned char
+# else
+#  define __pthread_spin_t unsigned int
+# endif


Are you sure this works for hppa which has funky __cpu_simple_lock_t?


You're right, it doesn't, I should've checked more carefully before 
commit.  I improved the definition to not depend on 
__CPU_SIMPLE_LOCK_PAD and use sizeof+alignof instead.  That will keep 
all ports happy, regardless of how they define __cpu_simple_lock_t and 
assume it to be aligned.


Thanks for catching it.

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On 26/06/15 09:14, Joerg Sonnenberger wrote:

On Fri, Jun 26, 2015 at 01:33:09AM +, Antti Kantee wrote:

Module Name:src
Committed By:   pooka
Date:   Fri Jun 26 01:33:09 UTC 2015

Modified Files:
src/lib/libpthread: pthread_types.h

Log Message:
C++ (namely libc++) expects to be using PTHREAD_FOO_INITIALIZER as a
member initializer. This does not work for volatile types. Since C++
does not touch the guts of those types, redefine them as non-volatile.

Fixes libc++ compilation with g++ 5.1, as reported in PR lib/49989.


Please revert this. It is ugly, it breaks the abstraction of the system
headers and it is a hackaround for a GCC bug. This should be handled by
GCC and only by GCC. The relevant DR against C++11 is core issue #1911.


As long as NetBSD's goals include interoperates well with other 
systems and conforms to open systems standards as much as is 
practical, the workaround is mandated.  It fixes source level 
interoperation with a contemporary release of a major compiler (fact) 
and given that it's a few lines to a single file, it's practical 
(subjective).  Of course, I'd *like* to go back in time to fix all bugs, 
but I'm not high on hopes with that being possible.

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Fri, Jun 26, 2015 at 01:33:09AM +, Antti Kantee wrote:
 Module Name:  src
 Committed By: pooka
 Date: Fri Jun 26 01:33:09 UTC 2015
 
 Modified Files:
   src/lib/libpthread: pthread_types.h
 
 Log Message:
 C++ (namely libc++) expects to be using PTHREAD_FOO_INITIALIZER as a
 member initializer. This does not work for volatile types. Since C++
 does not touch the guts of those types, redefine them as non-volatile.
 
 Fixes libc++ compilation with g++ 5.1, as reported in PR lib/49989.

Please revert this. It is ugly, it breaks the abstraction of the system
headers and it is a hackaround for a GCC bug. This should be handled by
GCC and only by GCC. The relevant DR against C++11 is core issue #1911.

Joerg

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On 26/06/15 07:04, Nick Hudson wrote:

On 06/26/15 02:33, Antti Kantee wrote:

+# ifdef __CPU_SIMPLE_LOCK_PAD
+#  define __pthread_spin_t unsigned char
+# else
+#  define __pthread_spin_t unsigned int
+# endif


Are you sure this works for hppa which has funky __cpu_simple_lock_t?


Slightly related.  arch/arm has:

/*
 * This should have always been an 8-bit type, but since it's been exposed
 * to user-space, we don't want ABI breakage there.
 */
#if defined(_KERNEL)
typedef volatile unsigned char  __cpu_simple_lock_t;
#else
typedef volatile int__cpu_simple_lock_t;
#endif /* _KERNEL */


Should that be no ifdef  __CPU_SIMPLE_LOCK_PAD like everywhere else?

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On 26/06/15 11:53, Joerg Sonnenberger wrote:

That said, have
you verified why it doesn't happen with libstdc++ itself? I would
somewhat suspect that the threatment of the header as system header
hides the problem for libstdc++, I can't imagine that it can correctly
implement the constexpr constructor without performance penalties
otherwise...


I don't speak C++, but a grep-based guess coupled with common sense 
suggests that libstdc++ is fine because it doesn't use 
PTHREAD_MUTEX_INITIALIZER.  Not that you can use common sense to reason 
about C++, but anyway.  Also, most likely nobody has tried compiling 
libstdc++ for NetBSD with the g++ 5 series, and therefore we don't know 
if it fails or not.  Finally, I don't understand why this would be an 
issue with just libstdc++/libc++ as opposed to any C++ compiled code 
which uses PTHREAD_X_INITIALIZER (lack of understanding again based on 
common sense instead of C++ sense).

Re: CVS commit: src/lib/libpthread

[Joerg Sonnenberger]

On Fri, Jun 26, 2015 at 10:14:41AM +, Antti Kantee wrote:
 As long as NetBSD's goals include interoperates well with other systems
 and conforms to open systems standards as much as is practical, the
 workaround is mandated.  It fixes source level interoperation with a
 contemporary release of a major compiler (fact) and given that it's a few
 lines to a single file, it's practical (subjective).  Of course, I'd *like*
 to go back in time to fix all bugs, but I'm not high on hopes with that
 being possible.

We generally don't workaround standard compliance bugs in compilers,
especially .0 releases with a history of being buggy. That said, have
you verified why it doesn't happen with libstdc++ itself? I would
somewhat suspect that the threatment of the header as system header
hides the problem for libstdc++, I can't imagine that it can correctly
implement the constexpr constructor without performance penalties
otherwise...

Joerg

Re: CVS commit: src/lib/libpthread

[Antti Kantee]

On Fri, Jun 26, 2015 at 01:33:09AM +, Antti Kantee wrote:
 Module Name:  src
 Committed By: pooka
 Date: Fri Jun 26 01:33:09 UTC 2015
 
 Modified Files:
   src/lib/libpthread: pthread_types.h
 
 Log Message:
 C++ (namely libc++) expects to be using PTHREAD_FOO_INITIALIZER as a
 member initializer. This does not work for volatile types. Since C++
 does not touch the guts of those types, redefine them as non-volatile.
 
 Fixes libc++ compilation with g++ 5.1, as reported in PR lib/49989.

Forgot to mention: the patch is from christos (thanks!)

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

In article 20141217014908.359b...@cvs.netbsd.org,
Antti Kantee source-changes-d@NetBSD.org wrote:
-=-=-=-=-=-

Module Name:   src
Committed By:  pooka
Date:  Wed Dec 17 01:49:08 UTC 2014

Modified Files:
   src/lib/libpthread: pthread_makelwp_netbsd.c

Log Message:
include correct header for last minute just-in-case defensive addition
that's too trivial to check

Please rename the file pthread_makelwp.c; calling a file _netbsd.c in
a NetBSD tree for a NetBSD implementation is bogus.

christos

Re: CVS commit: src/lib/libpthread

[Izumi Tsutsui]

Will this workaround fix for PR lib/47703 be pulled to 6.1 release?
Does releng have any plan for all libpthread issue (including dlopen one)?

IMO ticket #722 (pthread_condattr_setclock(3) addition) should be
reverted before 6.1 if it requires syscall bump for the real fix.

Furthermore, was it safe to add a new pthread_condattr_setclock(3) function
without libpthread minor bump?  I'm afraid pkgsrc ruby193 (and other)
binaries built on 6.1 silently fails on 6.0.1.

Currently ruby193 built on 6.0.1 works but fails on ruby193 built
on netbsd-6 because the former doesn't use pthread_condattr_setclock(3).
---
Izumi Tsutsui

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

On Apr 1,  9:18pm, tsut...@ceres.dti.ne.jp (Izumi Tsutsui) wrote:
-- Subject: Re: CVS commit: src/lib/libpthread

| Will this workaround fix for PR lib/47703 be pulled to 6.1 release?

Yes,

| Does releng have any plan for all libpthread issue (including dlopen one)?

This is an open question to core.

| IMO ticket #722 (pthread_condattr_setclock(3) addition) should be
| reverted before 6.1 if it requires syscall bump for the real fix.

It does not.

| Furthermore, was it safe to add a new pthread_condattr_setclock(3) function
| without libpthread minor bump?  I'm afraid pkgsrc ruby193 (and other)
| binaries built on 6.1 silently fails on 6.0.1.

It was a mistake, we should have bumped.

| Currently ruby193 built on 6.0.1 works but fails on ruby193 built
| on netbsd-6 because the former doesn't use pthread_condattr_setclock(3).

Right.

christos

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

In article 20130401132606.8cc2997...@rebar.astron.com,
Christos Zoulas chris...@zoulas.com wrote:

| Furthermore, was it safe to add a new pthread_condattr_setclock(3) function
| without libpthread minor bump?  I'm afraid pkgsrc ruby193 (and other)
| binaries built on 6.1 silently fails on 6.0.1.

It was a mistake, we should have bumped.

Note, that bumping would have just changed the nature of the failure, would
not have fixed the problem.

christos

Re: CVS commit: src/lib/libpthread

[Izumi Tsutsui]

 | Furthermore, was it safe to add a new pthread_condattr_setclock(3) function
 | without libpthread minor bump?  I'm afraid pkgsrc ruby193 (and other)
 | binaries built on 6.1 silently fails on 6.0.1.
 
 It was a mistake, we should have bumped.
 
 Note, that bumping would have just changed the nature of the failure, would
 not have fixed the problem.

Is it generally acceptable to bump minor
(i.e. adding new public functions in libs)
in release branches?

I'm afraid not all third party programs use proper configure script.

---
Izumi Tsutsui

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

On Apr 1, 11:43pm, tsut...@ceres.dti.ne.jp (Izumi Tsutsui) wrote:
-- Subject: Re: CVS commit: src/lib/libpthread

|  | Furthermore, was it safe to add a new pthread_condattr_setclock(3) 
function
|  | without libpthread minor bump?  I'm afraid pkgsrc ruby193 (and other)
|  | binaries built on 6.1 silently fails on 6.0.1.
|  
|  It was a mistake, we should have bumped.
|  
|  Note, that bumping would have just changed the nature of the failure, would
|  not have fixed the problem.
| 
| Is it generally acceptable to bump minor
| (i.e. adding new public functions in libs)
| in release branches?

It depends on where we build 3rd party binaries. If we build on
the earliest branch there is no problem. But if we build in the
latest one there is. We should write the rules down, so that we
don't repeat the same problem on each release. Perhaps the best
solution is to add the missing function on all branches, but that
does not help those who did not upgrade.

| I'm afraid not all third party programs use proper configure script.

That can always cause issues.

Re: CVS commit: src/lib/libpthread

[Izumi Tsutsui]

 |  | Furthermore, was it safe to add a new pthread_condattr_setclock(3) 
 function
 |  | without libpthread minor bump?  I'm afraid pkgsrc ruby193 (and other)
 |  | binaries built on 6.1 silently fails on 6.0.1.
 |  
 |  It was a mistake, we should have bumped.
 |  
 |  Note, that bumping would have just changed the nature of the failure, 
 would
 |  not have fixed the problem.
 | 
 | Is it generally acceptable to bump minor
 | (i.e. adding new public functions in libs)
 | in release branches?
 
 It depends on where we build 3rd party binaries. If we build on
 the earliest branch there is no problem. But if we build in the
 latest one there is. We should write the rules down, so that we
 don't repeat the same problem on each release. Perhaps the best
 solution is to add the missing function on all branches, but that
 does not help those who did not upgrade.

Can't we simply revert ticket #722 (pthread_condattr_setclock(3) addition)?
Is that function really necessary for 6.1?

---
Izumi Tsutsui

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

On Apr 2,  1:22am, tsut...@ceres.dti.ne.jp (Izumi Tsutsui) wrote:
-- Subject: Re: CVS commit: src/lib/libpthread

|  It depends on where we build 3rd party binaries. If we build on
|  the earliest branch there is no problem. But if we build in the
|  latest one there is. We should write the rules down, so that we
|  don't repeat the same problem on each release. Perhaps the best
|  solution is to add the missing function on all branches, but that
|  does not help those who did not upgrade.
| 
| Can't we simply revert ticket #722 (pthread_condattr_setclock(3) addition)?
| Is that function really necessary for 6.1?

I think it is too late now.

christos

Re: CVS commit: src/lib/libpthread

[Izumi Tsutsui]

 | Can't we simply revert ticket #722 (pthread_condattr_setclock(3) addition)?
 | Is that function really necessary for 6.1?
 
 I think it is too late now.

Hmm. If core and releng think so, I have no further comment.
(but I'm afraid we need 6.1_RC4 anyway since RC3 doesn't include pthread fixes)

---
Izumi Tsutsui

Re: CVS commit: src/lib/libpthread

[Christos Zoulas]

On Apr 2,  1:46am, tsut...@ceres.dti.ne.jp (Izumi Tsutsui) wrote:
-- Subject: Re: CVS commit: src/lib/libpthread

|  I think it is too late now.
| 
| Hmm. If core and releng think so, I have no further comment.
| (but I'm afraid we need 6.1_RC4 anyway since RC3 doesn't include pthread 
fixes)

I don't know if it is too late, I think it is too late. Releng knows best.

christos