Re: dlsym(RTLD_NEXT) alternative to syscall() for hard wrappers
> Date: Tue, 15 Oct 2002 20:43:48 +0200 > From: "Ralf S. Engelschall" <[EMAIL PROTECTED]> > To: [EMAIL PROTECTED] > Subject: Re: dlsym(RTLD_NEXT) alternative to syscall() for hard wrappers > > On Tue, Oct 15, 2002, Jonathan Schilling wrote: > > > Note that for this > > to work you have to use the dynamically linked Pth libpthread, so that the > > "real" definition can be found in a downstream library (usually libc). > > [...] > > The "defined(__USLC__) || defined(__svr4__)" test is to activate it for > > SCO UNIX systems; if this were to be incorporated in mainline Pth, this > > would have to be replaced with a macro test driven from the configure > > process, since the syscall() method needs to be retained as an option for > > systems that don't have dlsym(RTLD_NEXT) and for static links. > > Are you sure that libpthread itself has to be a shared library? Wouldn't > it be also sufficient if Pth lives in libpthread.a but the application > using libpthread.a is dynamically linked against libc? So, I think it > would be sufficient if libc is dynamically linked, but Pth itself is > still allowed to be statically included in the application, right? Yes, you are right. Jonathan Schilling SCO/Caldera [EMAIL PROTECTED] __ GNU Portable Threads (Pth)http://www.gnu.org/software/pth/ User Support Mailing List[EMAIL PROTECTED] Automated List Manager (Majordomo) [EMAIL PROTECTED]
Re: Pth Manual versions out of synch
On Tue, Oct 15, 2002, Jonathan Schilling wrote: > I believe the Pth Manual versions available at > http://www.gnu.org/software/pth/ are out of synch. > > The pth-manual.html HTML version there appears to be newer > than the pth-manual.ps Postscript version there. For example, > the HTML has the fix to get rid of the obsolete references > to PTH_FLAG_NOJOIN, while the Postscript still has the references > left in. Fixed. Ralf S. Engelschall [EMAIL PROTECTED] www.engelschall.com __ GNU Portable Threads (Pth)http://www.gnu.org/software/pth/ User Support Mailing List[EMAIL PROTECTED] Automated List Manager (Majordomo) [EMAIL PROTECTED]
Pth Manual versions out of synch
I believe the Pth Manual versions available at http://www.gnu.org/software/pth/ are out of synch. The pth-manual.html HTML version there appears to be newer than the pth-manual.ps Postscript version there. For example, the HTML has the fix to get rid of the obsolete references to PTH_FLAG_NOJOIN, while the Postscript still has the references left in. Jonathan Schilling SCO/Caldera [EMAIL PROTECTED] __ GNU Portable Threads (Pth)http://www.gnu.org/software/pth/ User Support Mailing List[EMAIL PROTECTED] Automated List Manager (Majordomo) [EMAIL PROTECTED]
Re: dlsym(RTLD_NEXT) alternative to syscall() for hard wrappers
On Tue, Oct 15, 2002, Jonathan Schilling wrote: > [...] First, thanks for this brilliant idea. I'll to incorporate it into the mainline for release in Pth 1.5.0. > Note that for this > to work you have to use the dynamically linked Pth libpthread, so that the > "real" definition can be found in a downstream library (usually libc). > [...] > The "defined(__USLC__) || defined(__svr4__)" test is to activate it for > SCO UNIX systems; if this were to be incorporated in mainline Pth, this > would have to be replaced with a macro test driven from the configure > process, since the syscall() method needs to be retained as an option for > systems that don't have dlsym(RTLD_NEXT) and for static links. Are you sure that libpthread itself has to be a shared library? Wouldn't it be also sufficient if Pth lives in libpthread.a but the application using libpthread.a is dynamically linked against libc? So, I think it would be sufficient if libc is dynamically linked, but Pth itself is still allowed to be statically included in the application, right? Ralf S. Engelschall [EMAIL PROTECTED] www.engelschall.com __ GNU Portable Threads (Pth)http://www.gnu.org/software/pth/ User Support Mailing List[EMAIL PROTECTED] Automated List Manager (Majordomo) [EMAIL PROTECTED]
dlsym(RTLD_NEXT) alternative to syscall() for hard wrappers
The Pth 1.4.1 "Hard System Call Mapping" scheme makes use of the syscall(2)
function to wrap system calls. The Pth doc acknowledges that this may
hit conflicts with the vendor's header files, but there is another more
fundamental problem as well:
This approach assumes that the C library interface for a function is the
same as its underlying syscall() interface, in terms of arguments and
return values. In most cases this is true, but for some calls on some
OSes it is not. For example on SCO's UnixWare/Open UNIX OS, both
system() and waitpid() have a system call interface that uses multiple
return values via assembly language registers. These return values are
mapped to the POSIX signature by an assembly code layer within the C library.
Consequently, calling these functions in the C library and calling them via
syscall() do not produce the same results; you simply can't use syscall() for
them from application C code. Thus, the Pth hard wrapper mechanism breaks
down.
An alternative approach is to use the dlsym(RTLD_NEXT) feature, which
is specified in the Unix 98 standard and is available on many current-day
systems. This is a more robust way of coding wrapper functions for
system calls, since it's designed for that purpose. Note that for this
to work you have to use the dynamically linked Pth libpthread, so that the
"real" definition can be found in a downstream library (usually libc).
Here are the changes to use dlsym. They are to pth_lib.c and pth_syscall.c.
The changes are mostly in one place, due to good encapsulation provided by
the pth_sc macro.
--- pth_lib.c..orig Sun Jan 27 06:03:40 2002
+++ pth_lib.c Tue Oct 15 13:15:04 2002
@@ -47,6 +47,11 @@
int pth_init(void)
{
pth_attr_t t_attr;
+#if defined(__USLC__) || defined(__svr4__)
+#if PTH_SYSCALL_HARD
+extern int pth_sys_func_table_init(void);
+#endif
+#endif
/* support for implicit initialization calls
and to prevent multiple explict initialization, too */
@@ -55,6 +60,13 @@
else
pth_initialized = TRUE;
+#if defined(__USLC__) || defined(__svr4__)
+#if PTH_SYSCALL_HARD
+if (pth_sys_func_table_init() == FALSE)
+ return FALSE;
+#endif
+#endif
+
pth_debug1("pth_init: enter");
/* initialize the scheduler */
--- pth_syscall.c..orig Sun Jan 27 06:03:41 2002
+++ pth_syscall.c Tue Oct 15 11:58:31 2002
@@ -36,13 +36,78 @@
#ifdef HAVE_SYS_SOCKETCALL_H
#include
#endif
+#if defined(__USLC__) || defined(__svr4__)
+int pth_sys_func_table_init(void);
+typedef int (*pth_sys_func_t)();
+typedef struct pth_sys_func_table_t {
+char* name;
+pth_sys_func_t addr;
+} pth_sys_func_table_t;
+extern pth_sys_func_table_t pth_sys_func_table[];
+#define PTH_SFT_read 0
+#define PTH_SFT_readv 1
+#define PTH_SFT_write 2
+#define PTH_SFT_writev 3
+#define PTH_SFT_select 4
+#define PTH_SFT_connect 5
+#define PTH_SFT_accept 6
+#define PTH_SFT_recvfrom 7
+#define PTH_SFT_sendto 8
+#define PTH_SFT_fork 9
+#define PTH_SFT_sigprocmask 10
+#define PTH_SFT_waitpid 11
+#define PTH_SFT__xconnect 12
+#define PTH_SFT__xaccept 13
+#define PTH_SFT__xrecvfrom 14
+#define PTH_SFT__xsendto 15
+#define pth_sc(func) (*pth_sys_func_table[PTH_SFT_##func].addr)
+#else
#define pth_sc(func) PTH_SC_##func
+#endif
#else
#define pth_sc(func) func
#endif
#endif /* cpp */
+#if defined(__USLC__) || defined(__svr4__)
+#if PTH_SYSCALL_HARD
+#include
+
+pth_sys_func_table_t pth_sys_func_table[] = {
+ // this order must match the macro values above
+ { "read", NULL },
+ { "readv", NULL },
+ { "write", NULL },
+ { "writev", NULL },
+ { "select", NULL },
+ { "connect", NULL },
+ { "accept", NULL },
+ { "recvfrom", NULL },
+ { "sendto", NULL },
+ { "fork", NULL },
+ { "sigprocmask", NULL },
+ { "waitpid", NULL },
+ { "_xconnect", NULL },
+ { "_xaccept", NULL },
+ { "_xrecvfrom", NULL },
+ { "_xsendto", NULL },
+ { NULL, NULL },
+};
+
+int pth_sys_func_table_init(void) {
+pth_sys_func_table_t* p_sft = pth_sys_func_table;
+while (p_sft->name != NULL) {
+ p_sft->addr = (pth_sys_func_t) dlsym(RTLD_NEXT, p_sft->name);
+ // Note the above dlsym() may return NULL,
+ // e.g. the networking calls when libsocket is not linked.
+ p_sft++;
+}
+return TRUE;
+}
+#endif
+#endif
+
/* some exported variables for object layer checks */
int pth_syscall_soft = PTH_SYSCALL_SOFT;
int pth_syscall_hard = PTH_SYSCALL_HARD;
The "defined(__USLC__) || defined(__svr4__)" test is to activate it for
SCO UNIX systems; if this were to be incorporated in mainline Pth, this
would have to be replaced with a macro test driven from the configure
process, since the syscall() method needs to be retained as an option for
systems that don't have dlsym(RTLD_NEXT) and for static links.
Jonathan Schilling SCO/Caldera [EMAIL PROTECTED]
___
Re: pthread_exit() incorrect behaviour
On Tue, Oct 15, 2002, Jonathan Schilling wrote:
> [...]
> If the main thread reaches its pthread_exit() call before the two
> started threads have terminated, it should wait there until those
> two threads do terminate, then it should exit. See POSIX 9945-1:1996,
> section 16.2.5.2, lines 249 - 251 ("The process shall exit with an
> exit status of 0 after the last thread has been terminated.")
> [...]
> **Pth** SCHEDULER INTERNAL ERROR: no more thread(s) available to schedule!?!?
> [...]
> The culprit is the function pth_exit_cb(), which, in effect, won't let
> pthread_exit() finish if there are any threads on the "dead queue", which
> is where non-detached terminated threads are put. The above test case
> will work correctly with Pth if the threads are created detached. But
> there's nothing in the POSIX standard that says that detached threads are
> treated differently in terms of pthread_exit() waiting until all threads
> are terminated. Thus Pth is in error.
Ops, yes, great catch! You're right. I've already searched multiple
times why the above abortion (which theoretically should never happen)
actually occured from time to time in Pth-based applications. But I've
never looked at the pth_exit_cb() function because I though the error is
directly in the scheduler. Thank you very much for discovering this long
standing bug. I've comitted your fix to the OSSP CVS for releaase in the
soon coming GNU Pth 1.5.0. Thanks for your support.
Ralf S. Engelschall
[EMAIL PROTECTED]
www.engelschall.com
__
GNU Portable Threads (Pth)http://www.gnu.org/software/pth/
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager (Majordomo) [EMAIL PROTECTED]
Re: system() not mapped in syscall-soft mode
On Tue, Oct 15, 2002, Jonathan Schilling wrote: > The Pth 1.4.1 manual states that system() is supported in the > "Soft System Call Mapping", but in fact it is omitted (as an > oversight, I think). > > [...] > The fixes are to pthread.h.in: > [...] Thanks. Committed to the OSSP CVS for inclusion into GNU Pth 1.5.0. Ralf S. Engelschall [EMAIL PROTECTED] www.engelschall.com __ GNU Portable Threads (Pth)http://www.gnu.org/software/pth/ User Support Mailing List[EMAIL PROTECTED] Automated List Manager (Majordomo) [EMAIL PROTECTED]
pthread_exit() incorrect behaviour
I believe that Pth 1.4.1 is not correctly executing pthread_exit().
Consider this simple test:
#include "pthread.h"
#include
void* start(void* s) {
fprintf(stderr, "in start() for thread %x\n", pthread_self());
pthread_exit(0);
}
int main() {
pthread_t t1;
pthread_t t2;
fprintf(stderr, "start main\n");
pthread_create(&t1, NULL, start, NULL);
pthread_create(&t2, NULL, start, NULL);
fprintf(stderr, "leave main\n");
pthread_exit(0);
}
If the main thread reaches its pthread_exit() call before the two
started threads have terminated, it should wait there until those
two threads do terminate, then it should exit. See POSIX 9945-1:1996,
section 16.2.5.2, lines 249 - 251 ("The process shall exit with an
exit status of 0 after the last thread has been terminated.")
Thus, the expected output should be something like:
start main
leave main
in start() for thread 2
in start() for thread 3
and it is, when using the system-provided UnixWare, Solaris, and
Linux threads libraries.
However Pth 1.4.1 produces:
start main
leave main
in start() for thread 805adb0
in start() for thread 806b078
**Pth** SCHEDULER INTERNAL ERROR: no more thread(s) available to schedule!?!?
Abort(coredump)
The culprit is the function pth_exit_cb(), which, in effect, won't let
pthread_exit() finish if there are any threads on the "dead queue", which
is where non-detached terminated threads are put. The above test case
will work correctly with Pth if the threads are created detached. But
there's nothing in the POSIX standard that says that detached threads are
treated differently in terms of pthread_exit() waiting until all threads
are terminated. Thus Pth is in error.
The fix for this is the following:
--- pth_lib.c..orig Sun Jan 27 06:03:40 2002
+++ pth_lib.c.fixed Tue Oct 15 11:00:04 2002
@@ -357,7 +357,6 @@
rc += pth_pqueue_elements(&pth_RQ);
rc += pth_pqueue_elements(&pth_WQ);
rc += pth_pqueue_elements(&pth_SQ);
-rc += pth_pqueue_elements(&pth_DQ);
if (rc == 1 /* just our main thread */)
return TRUE;
else
Jonathan Schilling SCO/Caldera [EMAIL PROTECTED]
__
GNU Portable Threads (Pth)http://www.gnu.org/software/pth/
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager (Majordomo) [EMAIL PROTECTED]
system() not mapped in syscall-soft mode
The Pth 1.4.1 manual states that system() is supported in the
"Soft System Call Mapping", but in fact it is omitted (as an
oversight, I think).
Build Pth with --enable-syscall-soft, then compile this test
program with -E to show the preprocessed code:
#include "pthread.h"
#include
#include
#include
int main() {
const char* msg = "... calling write() explicitly\n";
pthread_t dummy = pthread_self();
int rc = system("/bin/echo jls");
fprintf(stderr,"... after system call, rc %d\n", rc);
write(2, msg, strlen(msg));
}
You'll see that write has become __pthread_write, but system
is still system.
The fixes are to pthread.h.in:
--- pthread.h.in..orig Sun Jan 27 06:03:41 2002
+++ pthread.h.inTue Oct 15 12:15:21 2002
@@ -503,6 +504,7 @@
#if _POSIX_THREAD_SYSCALL_SOFT && !defined(_PTHREAD_PRIVATE)
#define fork __pthread_fork
#define sleep __pthread_sleep
+#define system __pthread_system
#define sigwait__pthread_sigwait
#define waitpid__pthread_waitpid
#define connect__pthread_connect
and pthread.c:
--- pthread.c..orig Thu Sep 12 15:13:15 2002
+++ pthread.c Tue Oct 15 12:17:57 2002
@@ -1057,6 +1057,12 @@
return pth_sleep(sec);
}
+int __pthread_system(const char *cmd)
+{
+pthread_initialize();
+return pth_system(cmd);
+}
+
int __pthread_sigwait(const sigset_t *set, int *sig)
{
pthread_initialize();
Jonathan Schilling SCO/Caldera [EMAIL PROTECTED]
__
GNU Portable Threads (Pth)http://www.gnu.org/software/pth/
User Support Mailing List[EMAIL PROTECTED]
Automated List Manager (Majordomo) [EMAIL PROTECTED]
