Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
Jeff Squyres, le Thu 12 Nov 2009 09:08:45 -0800, a écrit : > Fair enough. I think we actually agree -- these emails are quite > confusing. :-) Ok :) > 1. Let's not mix thread and PIDs into a single argument like Linux does Ok, I've just done it :) but in the linux.h header only, for people who really insist on using the non-portable linux tids. > 3. Identifying threads by pthread_self() is Good (is that available on > all OS's that we care about?) Some OSes don't have an interface to bind another thread, but when they do, they provide a way to express that through a pthread_t. AIX's functions uses kernel tids, but it also provides a conversion function. > More specifically: IMHO, it would be very nice/useful to be able to get > +set the affinity of threads other than the current thread (as > identified by pthread_self()), even if that only works on some OS's. That's already the case since libtopology :) Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
On Nov 12, 2009, at 8:44 AM, Samuel Thibault wrote: I'm not saying pinning a specific thread is a horror. I'm saying expressing which thread should be bound through tids instead of pthread_t is. The only sane way I could see an application use tids is a monitoring application that looks into /proc/*/task/* . In that case, ok we can expose the hwloc_linux_set_tid_cpubind function in a hwloc/linux.h header, and we need it for complete PLPA support anyway. What I'm against is even mentioning such thing in the main hwloc.h; tid vs pid vs pthread_t has been confused enough by Linus before the 2.6 fix (to the point that I still see parallel programming courses explaining the 2.4 semantic...) Fair enough. I think we actually agree -- these emails are quite confusing. :-) 1. Let's not mix thread and PIDs into a single argument like Linux does 2. Binding individual threads is a Good/useful thing 3. Identifying threads by pthread_self() is Good (is that available on all OS's that we care about?) More specifically: IMHO, it would be very nice/useful to be able to get +set the affinity of threads other than the current thread (as identified by pthread_self()), even if that only works on some OS's. -- Jeff Squyres jsquy...@cisco.com
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
Jeff Squyres, le Thu 12 Nov 2009 08:31:50 -0800, a écrit : > On Nov 11, 2009, at 4:22 PM, Samuel Thibault wrote: > >- non-linux plpa users are restricted to what really is portable. > > > > PLPA = Linux only. :-) I know, but there's usually no reason why they shouldn't now target non-Linux too. Again, I really wonder how they'd manage to end up with tids in their hands instead of pthread_t-s. > >- we don't have to cripple the hwloc interface (i.e. document that we > > accept non-portable input) just for the linuxish interface. > > The PLPA interface for specifying specific PIDs wasn't just for > threads -- it really was for getting/setting the affinity of other > processes. E.g., htop uses PLPA for this purpose (htop.sf.net). > > Do other OS's not support that? Some do, some don't. In any case no OS other than Linux mixes thread IDs and process IDs in their API and thus confuse the user like Linux does. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
Jeff Squyres, le Thu 12 Nov 2009 08:28:32 -0800, a écrit : > One thing to not forget is that some (many?) applications don't care a > whit about portability to other OS's. You view feature X as a > portability horror; they view it as a feature. > > Hence, they may actually *want* to take advantage of the ability to > pin a specific thread (that is not the current thread). I'm not saying pinning a specific thread is a horror. I'm saying expressing which thread should be bound through tids instead of pthread_t is. The only sane way I could see an application use tids is a monitoring application that looks into /proc/*/task/* . In that case, ok we can expose the hwloc_linux_set_tid_cpubind function in a hwloc/linux.h header, and we need it for complete PLPA support anyway. What I'm against is even mentioning such thing in the main hwloc.h; tid vs pid vs pthread_t has been confused enough by Linus before the 2.6 fix (to the point that I still see parallel programming courses explaining the 2.4 semantic...) Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
On Nov 11, 2009, at 4:22 PM, Samuel Thibault wrote: Another way to go is in hwloc_plpa_sched_setaffinity put in #ifdef HWLOC_LINUX_SYS some code that calls the internal hwloc_linux_set_tid_cpubind (with a strong comment that nobody else should call it), so that - existing linux plpa users can have the same behavior, but we can document here that calling hwloc_plpa_sched_setaffinity with a pid different than zero portably means something only for single- threaded processes. - non-linux plpa users are restricted to what really is portable. PLPA = Linux only. :-) Per my prior mail, remember that some (many?) apps don't care about portability. - we don't have to cripple the hwloc interface (i.e. document that we accept non-portable input) just for the linuxish interface. The PLPA interface for specifying specific PIDs wasn't just for threads -- it really was for getting/setting the affinity of other processes. E.g., htop uses PLPA for this purpose (htop.sf.net). Do other OS's not support that? -- Jeff Squyres jsquy...@cisco.com
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
One thing to not forget is that some (many?) applications don't care a whit about portability to other OS's. You view feature X as a portability horror; they view it as a feature. Hence, they may actually *want* to take advantage of the ability to pin a specific thread (that is not the current thread). On Nov 11, 2009, at 4:03 PM, Samuel Thibault wrote: Brice Goglin, le Thu 12 Nov 2009 00:31:48 +0100, a écrit : > The problem is that our hwloc/Linux does not implement > set_proc_cpubind() so far. But it can implement one that assumes that the target process is singlethreaded, i.e. in hwloc_set_proc_cpubind distinguish between HWLOC_CPUBIND_PROCESS being set or not, or by just passing the policy flag as such to OS hooks. > * document in hwloc.h that it may bind a single thread if the > application (wrongly) passes a tid I'd really rather avoid even mentioning tids in the hwloc documentation except saying "don't use that, it's not portable, don't even ask, you'd be horrified". > * document that hwloc_plpa_sched_setaffinity now works on processes > instead of pids and that application should use thread_t and > set_thread_cpubind for local threads Or pass 0 to express "the current thread", which was already valid for plpa_sched_setaffinity, and _is_ portable (and should already have been the only thing that truly portable applications use). > * maybe return -ENOSYS on Linux if STRICT is given? I guess you mean return 0 if STRICT is not given, and mean "it's not strict because we haven't actually done it for all the threads, or even not at all"? I'd really rather not lie like this. Samuel ___ hwloc-devel mailing list hwloc-de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel -- Jeff Squyres jsquy...@cisco.com
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
Another way to go is in hwloc_plpa_sched_setaffinity put in #ifdef HWLOC_LINUX_SYS some code that calls the internal hwloc_linux_set_tid_cpubind (with a strong comment that nobody else should call it), so that - existing linux plpa users can have the same behavior, but we can document here that calling hwloc_plpa_sched_setaffinity with a pid different than zero portably means something only for single-threaded processes. - non-linux plpa users are restricted to what really is portable. - we don't have to cripple the hwloc interface (i.e. document that we accept non-portable input) just for the linuxish interface. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
Brice Goglin, le Thu 12 Nov 2009 00:31:48 +0100, a écrit : > The problem is that our hwloc/Linux does not implement > set_proc_cpubind() so far. But it can implement one that assumes that the target process is singlethreaded, i.e. in hwloc_set_proc_cpubind distinguish between HWLOC_CPUBIND_PROCESS being set or not, or by just passing the policy flag as such to OS hooks. > * document in hwloc.h that it may bind a single thread if the > application (wrongly) passes a tid I'd really rather avoid even mentioning tids in the hwloc documentation except saying "don't use that, it's not portable, don't even ask, you'd be horrified". > * document that hwloc_plpa_sched_setaffinity now works on processes > instead of pids and that application should use thread_t and > set_thread_cpubind for local threads Or pass 0 to express "the current thread", which was already valid for plpa_sched_setaffinity, and _is_ portable (and should already have been the only thing that truly portable applications use). > * maybe return -ENOSYS on Linux if STRICT is given? I guess you mean return 0 if STRICT is not given, and mean "it's not strict because we haven't actually done it for all the threads, or even not at all"? I'd really rather not lie like this. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
bgog...@osl.iu.edu, le Wed 11 Nov 2009 11:33:31 -0500, a écrit :
> +/** \brief Bind thread given by \p pid to CPU set \p cpuset.
> + *
> + * \note This function now manipulates hwloc cpusets.
> + */
> +static __inline int
> +hwloc_plpa_sched_setaffinity(hwloc_topology_t topology, hwloc_pid_t pid,
> hwloc_cpuset_t cpuset)
> +{
> + /* FIXME: should be set_thread_cpubind with a pid */
> + return hwloc_set_proc_cpubind(topology, pid, cpuset, 0);
> +}
That's one instance where the Linux interface is odd (it talks about a
pid, but it's actually a thread, and there is no way to set the affinity
mask of a whole process...) and I believe we shouldn't try to support
all the cases. I'd suggest to bind a thread only when pid is 0. If
pid is not zero, that means that either the application is calling the
linux-only gettid() or some other linux-only way to get the tid of a
specific thread, or it assumes that the target is a single-threaded
process and thus providing the pid of that process is enough to change
its cpu affinity. In that case we can use hwloc_set_proc_cpubind like
already done above. Same for getaffinity.
Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1333
bgog...@osl.iu.edu, le Wed 11 Nov 2009 11:33:31 -0500, a écrit : > + /* FIXME: should be SET_THREAD_CPUBIND given with a pid */ > + if (flags & HWLOC_SUPPORT_SET_PROC_CPUBIND) > +*api_type = HWLOC_PLPA_PROBE_OK; > + else > +*api_type = HWLOC_PLPA_PROBE_NOT_SUPPORTED; > + return 0; > +} Just to refine my thoughts: providing the interface but accepting only pid 0 (i.e. "self") is however portability-wise ok and can already be mapped to the current hwloc interface. Samuel
