Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1853
Jeff Squyres, le Fri 26 Mar 2010 12:51:53 -0400, a écrit : > Does this work with compilers that (pseudo) impersonate gcc (e.g., icc)? Yes. Samuel
Re: [hwloc-devel] Strange difference
Jeff Squyres, le Fri 26 Mar 2010 14:15:50 -0400, a écrit : > On Mar 26, 2010, at 2:01 PM, Brice Goglin wrote: > > > > I like "Proc" instead of "P" even for the non-v output. :-) > > > > I am not against it, but I don't remember the reason for the initial > > change. Maybe because "processor" is confusing for some people (logical > > vs physical socket) ? > > Oh crap. I think you're right. And I think I even asked for that. ;-) > > Hmm. > > Is it a crime to use the full word "Processor"? At least on my machine, the > output width is still far less than 80 characters, so the full word should be > no problem. But I don't know if there are other strange topologies out there > that would take up more space...? (it certainly would in the graphic > output...) The graphical output is a reason to at least have a function that returns P, yes, else the output is very large. Samuel
Re: [hwloc-devel] Attribute unsed not regognized
Bert Wesarg, le Fri 26 Mar 2010 19:39:51 +0100, a écrit : > On Fri, Mar 26, 2010 at 17:01, Samuel Thibault <samuel.thiba...@inria.fr> > wrote: > > Thanks for the idea. > > > > Bert Wesarg, le Fri 26 Mar 2010 12:33:00 +0100, a écrit : > >> +#define HWLOC_HAVE(what) (defined(HWLOC_HAVE_##what) && HWLOC_HAVE_##what) > > > > Unfortunately some compilers (such as gcc 2.95) do not accept this > > What topology information do you get on machines that have this old > compiler installed? It's not only a question of *that* compiler, but that this kind of tinkering with macros is quite borderline. I actually do wonder whether it is C99 compliant. gcc happens to be able to parse it since at least 3.0, but since 2.95 doesn't, I guess there are some other compilers that won't be able to either. > For example sysfs attributes are very unlikely. > the linux kernel should require at least gcc 3. But I haven't checked > since when. hwloc is not only about Linux ;) Samuel
Re: [hwloc-devel] Strange difference
Jeff Squyres, le Fri 26 Mar 2010 17:05:39 -0400, a écrit : > On Mar 26, 2010, at 4:16 PM, Samuel Thibault wrote: > > > > Is it a crime to use the full word "Processor"? At least on my machine, > > > the output width is still far less than 80 characters, so the full word > > > should be no problem. But I don't know if there are other strange > > > topologies out there that would take up more space...? (it certainly > > > would in the graphic output...) > > > > The graphical output is a reason to at least have a function that > > returns P, yes, else the output is very large. > > How about "Processor" in the text outputs, and a shorter version in the > graphic output? > > Shorten "processor" to: > > Prcssr > Prcsr > Procr > Procsr That's still very large. We are going toward dozens of cores on each sockets, we really need to keep them small :) Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1865
Bert Wesarg, le Sat 27 Mar 2010 15:06:43 +0100, a écrit : > > + if (needed_count <= ulongs_count) > > + return; > > + > > + while (ulongs_count < needed_count) > > + ulongs_count *= 2; Mmm, this is actually 1 << (hwloc_fls(needed_count)), isn't it? SAmuel
Re: [hwloc-devel] Strange difference
Brice Goglin, le Wed 31 Mar 2010 11:37:17 +0200, a écrit : > We might need to replace some occurences of "logical processor" in the > doc with "processing unit". Or use both from time to time to make it > clear that it's very similar (and explain the difference somewhere). I'd say keep it in the glossary section of the documentation, to make it clear that by PU we mean "logical processor", and not "core". Samuel
Re: [hwloc-devel] rc1?
Brice Goglin, le Thu 01 Apr 2010 19:13:23 +0200, a écrit : > Jeff Squyres wrote: > > There have been a few commits today -- are we ready for rc1? Give me the > > word and I'll make it. :-) > > I am done now (r1895). I'd like to fix the MISC objects so they are not ignored. Samuel, hates seeing his TODO-list so full...
Re: [hwloc-devel] comments on API changes
Fawzi Mohamed, le Fri 02 Apr 2010 17:34:20 +0200, a écrit : > would be fine with me, planning to add al lot of flags? because there > is still lot of space to grow (and one can later switch to 64 bit... :) Just switching to 64bits would break the ABI. It's better to just use chars and be done with the ABI :) > >>HWLOC_OBJ_SYSTEM seems on the way out > > > >It isn't :) > > > >>I treated it just as a MACHINE anyway, > > > >What do you mean by this? They really are not supposed to be the > >same, > >as Brice explained. > > I understand, but in the end I am just interested in building a > structure to have a sequence where you first look at neighbors, and > then further and further away, so I use both just as a way got keep > together things that have the same "distance". Mmm, I don't understand. Why would you need to have a look at the type? Can't you test the parent pointer to know whether you got at the top of the hierarchy? Samuel
Re: [hwloc-devel] Embedding: is it worth it?
Jeff Squyres, le Fri 02 Apr 2010 12:32:35 -0400, a écrit : > But to use an external PLPA/libltdl/whatever, OMPI's configure would just > call their configure ?! I'm not 100% sure what RedHat etc. do, but in Debian the policy wouldn't be to do this, but to just link against the existing installed library, i.e. use the installed hwloc.pc file only, _not_ run its configure script. I believe that's what RedHat etc. mean. Samuel
Re: [hwloc-devel] rc1?
Brice Goglin, le Sun 04 Apr 2010 00:32:24 +0200, a écrit : > As I said in the past, we're trying to address 2 > different issues. I said we could have a "Group" type to replace the > current meaning of "Misc" and keep "Misc" for user-added objects only. That's what I meant by "fixing" misc objects, and just did in r1906. > A better solution for backward compatibility would be to keep current > "Misc" objects as is, and change user-added objects to a new type such > as HWLOC_OBJ_USER or CUSTOM. Then only the later needs a quirk in the > ignore code. We have already broken the backward compatibility in a lot of ways, so I believe it's better to just name NUMA and AIX groups as "groups", and not just "misc", so misc can be used for really various things, be it user-provided, top-provided, or plugin-provided, etc. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1910
Brice Goglin, le Sun 04 Apr 2010 17:11:36 +0200, a écrit : > So, do we want something like: > 1) insert_misc_by_cpuset() to always insert below objects with same cpusets? > Might be better for some use cases, and worse for others... Having one default would be easy, it's just a matter of changing the type order. > 2) have a parameter to switch between current behavior and (1) ? That would introduce a quirk in the sorting algorithm, but why not. > 3) add a quirk to lstopo to show Misc objects last? this one would also let > us replace "+" with another character right before the PID in the non-verbose > output This sound less and less clean :) Samuel
Re: [hwloc-devel] Embedding: is it worth it?
Jeff Squyres, le Fri 02 Apr 2010 14:51:06 -0400, a écrit : > On Apr 2, 2010, at 2:25 PM, Samuel Thibault wrote: > > > > But to use an external PLPA/libltdl/whatever, OMPI's configure would just > > > call their configure > > > > I'm not 100% sure what RedHat etc. do, but in Debian the policy wouldn't > > be to do this, but to just link against the existing installed library, > > i.e. use the installed hwloc.pc file only, _not_ run its configure > > script. I believe that's what RedHat etc. mean. > > Yes. > > To be clear, if OMPI is to use the internal/embedded copy, it would just call > the $path_to_hwloc/configure. Do you mean that if OMPI had to be able to use an external copy in some cases, for the internal case it would now just use ./configure? Why is it so? Can't the main ./configure call the hwloc m4 stuff or not depending on whether the internal or the external version is used? Samuel
Re: [hwloc-devel] Embedding: is it worth it?
Jeff Squyres, le Tue 06 Apr 2010 09:05:24 -0400, a écrit : > On Apr 4, 2010, at 12:15 PM, Samuel Thibault wrote: > > Why > > is it so? Can't the main ./configure call the hwloc m4 stuff or not > > depending on whether the internal or the external version is used? > > No -- we wouldn't want to call the m4 stuff in the external case, That is what I said above (if not, please tell me, that's probably just an English language issue): if the user requests the external version to be used, do not call the m4 stuff; if he doesn't, call the m4 stuff. > because what if it came to different answers vs. how the external one is > configured? That should appear in the hwloc/config.h file, be it internally generated, or the installed external one. > (e.g., if OMPI is built using icc and the external copy was built with gcc) ABIs are supposed to be system-wide (plus multilib cases), not compiler-wide. If it's not the case we have a bug that really needs to be fixed, since we don't want to impose the use of a particular compiler. > In the external case, OMPI should just use the external's hwloc.h and > whatever decisions were already made there (sizes, types, etc.). In the internal case too, the only difference is the -I parameter to reach the hwloc.h. So I still don't see why going back to ./configure instead of using m4. That being said, we do not have embedding needs, so we do not really have an opinion one way or the other :) Samuel
Re: [hwloc-devel] Embedding: is it worth it?
Jeff Squyres, le Tue 06 Apr 2010 18:05:03 -0400, a écrit : > > So I still don't see why going back to ./configure instead of using m4. > > Mainly because: > > 1. The potential for maintenance difficulties in the future. > 2. I don't (offhand) know of anyone outside of Open MPI who would use the > functionality. Ok, so we just agree indeed :) Samuel
Re: [hwloc-devel] rc1?
Just to make it clear: I'm done concerning 1.0 features. Samuel
Re: [hwloc-devel] HWLOC_OBJ_GROUP & hwloc_topology_support
Fawzi Mohamed, le Wed 14 Apr 2010 14:05:45 +0200, a écrit : > HWLOC_OBJ_GROUP > > I suppose that those groups still form a partition, Yes. > I know that was changed also due to my comments, but I am not sure the > change really better: the structure is not really hidden, so adding a > flags it breaks binary compatibility, Not when it is added at the end of the structure. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1961
Bert Wesarg, le Tue 20 Apr 2010 17:55:10 +0200, a écrit : > > +#define GNU_SOURCE > > That should be: _GNU_SOURCE. Oops, typo indeed, thanks for the proofread. Samuel
Re: [hwloc-devel] backward API compat or not?
Brice Goglin, le Tue 20 Apr 2010 19:54:13 +0200, a écrit : > I tend to think that we should just drop the get_system_obj() and make > it clear that people have to fix everything, not everything except foo > and bar. I'd tend to agree. We should however mention the previous names in the documentation, so a mere search can find them and the corresponding newer name. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r1986
Jeff Squyres, le Wed 21 Apr 2010 09:04:11 -0400, a écrit : > On Apr 21, 2010, at 8:47 AM, Bert Wesarg wrote: > > > From that page: > > > > If you are writing a header file that must work when included in > > ISO C programs, write __typeof__ instead of typeof. See Alternate > > Keywords. > > > > > Modified: trunk/src/topology.c > > > > That does not look like a header for me. > > Right, but gcc complained when used with -std=c99 unless it was __typeof__. > I did not check to see if icc or pgcc accepted typeof. I read that text to > be "if you want portable code, use __typeof__ instead of typeof." Err, putting underscores doesn't magically makes something recognized by compilers :) The reason why the documentation tells about headers and not .c files is that while you control which standard your .c files are compiled under (e.g. -std=c99), you do not control which standard your .h files will be compiled under by other applications, that's why you need a way to tell a compiler "don't complain about these extensions I know you support but warn about because you were given -std=c99). Here, typeof is not c99, and that's why gcc complains when given -std=c99 (instead of the default -std=gnu99). Putting underscores just hides the bug... Samuel
Re: [hwloc-devel] want 1.0rc4?
Jeff Squyres, le Mon 03 May 2010 09:28:41 -0400, a écrit : > I see a flurry of windows-related activity over the weekend. > > Do you guys want me to cut rc4? I'm done with what I wanted to fix in rc3. Samuel
Re: [hwloc-devel] want 1.0rc4?
Jeff Squyres, le Mon 03 May 2010 09:57:09 -0400, a écrit : > 1.0rc4 is up. Send me a windows binary when you get a chance. http://dept-info.labri.fr/~thibault/tmp/hwloc-win32-build-1.0rc4.zip Samuel
Re: [hwloc-devel] Windows 7 problems
Jeff Squyres, le Mon 03 May 2010 15:57:37 -0500, a écrit : > Running lstopo on w7 (64 bit): > > - > C:\Temp\hwloc>lstopo > Note: GetLogicalProcessorInformationEx was never tested yet! > - Ooops, it seems I have forgotten to remove it. Now done. > Plus, the memory it reported was the currently free memory, not total memory. Yes. I haven't found a win32 function that provides the total memory. Samuel
Re: [hwloc-devel] want 1.0rc4?
Brice Goglin, le Tue 04 May 2010 07:54:47 +0200, a écrit : > line 41 of src/misc.c in hwloc_snprintf(): > > str = malloc(size); > > > I am not sure what to do about this one... Is there any value we could return > without possibly breaking the caller ? 0 seems relatively safe to me. At worse the caller allocates 0 bytes, which might also return ENOMEM, but that's up to him to handle it. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2044
bgog...@osl.iu.edu, le Tue 04 May 2010 01:32:00 -0400, a écrit :
> @@ -326,6 +330,10 @@
> if (nr_tids == max_tids) {
>max_tids += 8;
>tids = realloc(tids, max_tids*sizeof(pid_t));
> + if (!tids) {
> +errno = ENOMEM;
> +return -1;
> + }
> }
> if (!strcmp(dirent->d_name, ".") || !strcmp(dirent->d_name, ".."))
>continue;
We also need to free the original allocated array.
Samuel
Re: [hwloc-devel] 1.0rc5 is posted
Jeff Squyres, le Thu 06 May 2010 08:17:53 -0400, a écrit : > hwloc 1.0rc5 is posted. Hopefully it can be the last! :-) > > (Samuel -- can you make the windows binaries?) http://dept-info.labri.fr/~thibault/tmp/hwloc-win32-build-1.0rc5.zip Samuel
Re: [hwloc-devel] rc5 = good?
Jeff Squyres, le Thu 06 May 2010 13:39:43 -0400, a écrit : > Did you want to get those warning fixes in to v1.0? (r2074 - 2076) > > Or are they trunk-only? They concern rc5 as well, I'm just unsure they are really worth making yet another RC, as they have no impact on the behavior of hwloc at all. Samuel
Re: [hwloc-devel] Graceful abort for non-C99 compilers
Pavan Balaji, le Mon 10 May 2010 20:56:19 -0500, a écrit : > I understand that hwloc requires C99 support. However, for compilers > that don't support C99, would you be willing to gracefully abort during > configure instead of failing at make time? Right, thanks. Samuel
Re: [hwloc-devel] Graceful abort for non-C99 compilers
Jeff Squyres, le Wed 12 May 2010 12:52:01 -0400, a écrit : > I just posted rc6 with the fixes for this. I have built the windows binary, available on http://dept-info.labri.fr/~thibault/tmp/hwloc-win32-build-1.0rc6.zip Samuel
Re: [hwloc-devel] 1.0rc7
Good for me. Samuel
Re: [hwloc-devel] Cacheline sizes
Wheeler, Kyle Bruce, le Tue 25 May 2010 13:46:09 -0600, a écrit : > I agree that, inherently, cache line size has nothing to do with topology. > But on the other hand, it's particularly useful for parallel shared-memory > applications (to avoid false-sharing), The false-sharing part is where it makes as much sense as the cache size, I believe, since it's related to the fact that there are several processors in the machine. I'd thus tend to agree to include the cacheline size. Samuel
Re: [hwloc-devel] [hwloc] #35: 32 bit builds fail
hwloc, le Thu 27 May 2010 21:06:56 -, a écrit : > Anyone have any comments / suggestions about r2149 before I close this > ticket and move it over to the v1.0 branch? It looks good to me. Samuel
Re: [hwloc-devel] misc questions
Jeff Squyres, le Fri 28 May 2010 09:03:30 -0400, a écrit : > On May 28, 2010, at 9:01 AM, Samuel Thibault wrote: > > > > ...actually, I'm not seeing where we use epstopdf in our build process...? > > > > I believe it's hidden somewhere in pdflatex or such call. > > Is it our responsibility to check for it, then? I'm quite uneasy with it indeed. As usual, it doesn't hurt to be cautious ourself for other people :/ Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2168
bgog...@osl.iu.edu, le Fri 28 May 2010 11:27:47 -0400, a écrit :
> Add a backend info string, except in XML since we may not want to override
> the one that we got from the XML file
>
> + add_object_info(topology->levels[0][0], strdup("Backend=AIX"));
Mmm, this will probably need to be extendable: for instance, one may
want to use the OS support for NUMA levels and the x86 support for cache
levels, and a cisco module for network levels, etc.
Samuel
Re: [hwloc-devel] 1.0.1rc1
Jeff Squyres, le Tue 01 Jun 2010 14:03:47 -0400, a écrit : > So do we like 1.0.1rc1? I haven't had the time to test it at all. Samuel
Re: [hwloc-devel] solaris 9 fixes
Jeff Squyres, le Thu 03 Jun 2010 12:32:50 -0400, a écrit : > Were you expecting all those solaris commits to the v1.0 tree to go into the > v1.0.1 release? Or are those intended for 1.0.2? Well, I see no reason not to put them in 1.0.1 actually :), except that we need to test another rc. I'm finished with solaris 9 fixes so maybe you can upload it. Samuel
Re: [hwloc-devel] libhwloc.so version number
Jeff Squyres, le Thu 03 Jun 2010 15:02:35 -0400, a écrit : > I think that hwloc's Libtool version number should go from 0:0:0 to 0:1:0. > > Can someone sanity check to ensure that's right? I believe that's right. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2223
Jeff Squyres, le Fri 18 Jun 2010 06:56:59 -0700, a écrit : > Just curious -- what's the intent of assigning to errno like this? (I ask > because I thought that middleware/applications were not supposed to assign to > errno) To convert functions returning the error into our convention of returning the error in errno. Samuel
Re: [hwloc-devel] [hwloc-users] hwloc and rpath
Hello, I agree that rpath should be avoided. However, hwloc itself doesn't add any. Jirka Hladky, le Fri 18 Jun 2010 22:09:56 +0200, a écrit : > = > hwloc.x86_64: E: binary-or-shlib-defines-rpath /usr/bin/hwloc-distrib > ['/usr/lib64'] So apparently libtool somehow doesn't realizes that /usr/lib64 is in the standard search path. I'd tend to believe it's a bug in libtool or the distribution which don't understand each other. How does configure get invoked? What is the output of gcc -print-search-dirs? (On debian, we use --libdir=/usr/lib/x86_64-linux-gnu and that doesn't introduce any rpath). Samuel
Re: [hwloc-devel] [hwloc-users] hwloc and rpath
Samuel Thibault, le Sat 19 Jun 2010 00:03:49 +0200, a écrit : > What is the output of gcc -print-search-dirs? Ah, no, I misread the configure script, sys_lib_dlsearch_path_spec comes from ld.so.conf (that makes sense actually). Could you post your /etc/ld.so.conf (and any file that it could include)? Samuel
Re: [hwloc-devel] [hwloc-users] hwloc and rpath
Jeff Squyres, le Mon 21 Jun 2010 10:48:13 -0400, a écrit : > I still see -rpath being inserted in the final link step for libhwloc.so (SVN > build using AC 2.65, AM 1.11.1, LT 2.2.6b): > > /bin/sh ../libtool --tag=CC --mode=link gcc -std=gnu99 > -fvisibility=hidden -I/usr/include/libxml2 -std=gnu99 -fvisibility=hidden > -I/users/jsquyres/svn/hwloc/include -Wall -Wunused-parameter -Wundef > -Wno-long-long -Wsign-compare -Wmissing-prototypes -Wstrict-prototypes > -Wcomment -pedantic-no-undefined -version-number 0:0:0 -lxml2 -lz -lm > -o libhwloc.la -rpath /home/jsquyres/bogus/lib topology.lo traversal.lo > topology-synthetic.lo bind.lo cpuset.lo misc.lo topology-xml.lo > topology-linux.lo topology-x86.lo -libverbs > > But unless I'm mistaken, libtool then strips it out: > > libtool: link: gcc -shared .libs/topology.o .libs/traversal.o > .libs/topology-synthetic.o .libs/bind.o .libs/cpuset.o .libs/misc.o > .libs/topology-xml.o .libs/topology-linux.o .libs/topology-x86.o -lxml2 -lz > -lm -libverbs-Wl,-soname -Wl,libhwloc.so.0 -o .libs/libhwloc.so.0.0.0 I was also seeing that behavior with LIBS, so I don't think turning to LDADD fixed that part. Samuel
Re: [hwloc-devel] [hwloc-users] hwloc and rpath
Hello, Jirka Hladky, le Mon 21 Jun 2010 18:54:47 +0200, a écrit : > I don't have "/usr/lib64" directory listed in > /etc/ld.so.conf. However, "/usr/lib64" is considered to be the > default lib location on 64-bit system. Ok. And libtool doesn't seem to add it by itself. I believe that's where the problem resides. > 1) Add /usr/lib64 into /etc/ld.so.conf. It works like a charm. Ok. > The problem is that I cannot use this change in the build environment > (on a cluster of build servers for compilation on different > architectures) Sure. On the long term I wouldn't recommend adding it anyway: it's already being looked for by the linker. > Samuel, do you have "/usr/lib64" directory listed in /etc/ld.so.conf listed > on > your 64-bit Debian? No. Debian does not have a /usr/lib64 directory, 64bit versions are in /usr/lib. For biarch system, /usr/lib/x86_64-linux-gnu will be used. > 2) Second approach is to add > sed -i 's|^hardcode_libdir_flag_spec=.*|hardcode_libdir_flag_spec=""|g' > libtool > sed -i 's|^runpath_var=LD_RUN_PATH|runpath_var=DIE_RPATH_DIE|g' libtool > into the %configure stage in rpm specs. Well, the real fix seems to me to fix sys_lib_dlsearch_path_spec="/lib /usr/lib $lt_ld_extra" into sys_lib_dlsearch_path_spec="/lib /usr/lib /usr/lib64 $lt_ld_extra" on Fedora systems. > James, any feedback on it? I'm not sure if I should blame libtool or just > open > BZ to add "/usr/lib64" into /etc/ld.so.conf. I believe in the current state of Fedora it's libtool that should be blamed. Or Fedora and libtool should talk to each other to determine how libtool is supposed to discover that /usr/lib64 is in the standard research path. Samuel
Re: [hwloc-devel] [hwloc-users] hwloc and rpath
James Laska, le Mon 21 Jun 2010 13:15:36 -0400, a écrit : > To note, if I understand correctly, adding '/usr/lib64' > to /etc/ld.so.conf or /etc/ld.so.conf.d/* should not be needed. I agree. > Anything in the standard Fedora library locations should be recognized > and not require an additional /etc/ld.so.conf.d/conf file. Yes. > However, I think if you are storing the libraries in > "/usr/lib[64]/hwloc" Mmm, is he? I don't think he is. Samuel
Re: [hwloc-devel] [hwloc-users] hwloc and rpath
Jirka Hladky, le Mon 21 Jun 2010 22:30:36 +0200, a écrit : > I'm not sure what's wrong. It seems like libtool is not smart enough to > recognize /usr/lib64 as default library directory on 64-bit system Well, on Debian it's not needed (and might even be harmful). Samuel
Re: [hwloc-devel] hwloc will be soon available as package in Fedora
Thanks for your work! Samuel
Re: [hwloc-devel] PUs not located under cores
Samuel Thibault, le Thu 24 Jun 2010 18:28:18 +0200, a écrit : > > depending on the type of machine you're running on. > > It also depends on the OS and kernel version for instance. Not all > information is available, the only you can bet on is PUs under a system > object. At worse there is just one because the OS didn't even say how > many there are. Also, it may be worth adding that you can't know whether some layer might be introduced some day between cores and PUs, so the user shouldn't assume the former being the direct father of the latter. There might be groups of PUs for instance, for whatever reason manufacturers invent in the future and that we will want to express. Samuel
Re: [hwloc-devel] hwloc-ps on trunk doesn't seem to do anything
Jeff Squyres, le Fri 25 Jun 2010 09:50:05 -0400, a écrit : > Sounds like this is just a doc bug, right? If so, I can fix. Probably, yes, please do. Samuel
Re: [hwloc-devel] xml sample output
Jeff Squyres, le Fri 25 Jun 2010 10:03:10 -0400, a écrit : > While I'm updating the docs in this area, can you guys send XML output for > dudley, hagrid, and emmett? Here they are. Samuel
Re: [hwloc-devel] xml sample output
Jeff Squyres, le Fri 25 Jun 2010 10:46:59 -0400, a écrit : > Before I do the other two, how does this look, formatting-wise? (see > attached) This looks good to me. Samuel
[hwloc-devel] Servet and hwloc
Hello Jorge, I've just noticed Servet in the ipdps 2010 proceedings. There are probably interesting things to do between Servet and hwloc http://www.open-mpi.org/projects/hwloc/ On one hand, servet could use hwloc to get binding implementations on various OSes. Indeed, Servet version 1.0 actually doesn't even build on Debian Linux systems: you need to #define _GNU_SOURCE before including , and then use CPU_ZERO/CPU_SET instead of __CPU_ZERO/__CPU_SET, and these are specific to Linux of course. Just using hwloc for that part will provide you wide OS support. On the other hand hwloc has the following TODO item: on OSes or systems which don't know the cache size and sharing from the hardware itself, just find out from measures, which is exactly what Servet just does :) Ideally, Servet would be using the BSD licence but it's currently GPL. Maybe a compromise would be to make Servet a library which some BSD plugin of hwloc could be compiled against if the user already has Servet installed. That would need Servet made a library. What do you think? Samuel
Re: [hwloc-devel] new version of docs
Jeff Squyres, le Fri 25 Jun 2010 12:13:22 -0400, a écrit : > It's unfortunate that the PPC64 SMT image is so tall; it makes weird vertical > gaps on the prior page. :-\ Maybe you can output to xml, strip half of the numa nodes, and re-render it. That's want I actually do for Hagrid. Samuel
Re: [hwloc-devel] new version of docs
Jeff Squyres, le Fri 25 Jun 2010 12:13:22 -0400, a écrit : > Rather than sending another huge attachment, here's another copy, including > all the XML and some images from the IBM PPC64 machine in a new "portability" > subsection: > > http://www.open-mpi.org/~jsquyres/hwloc/hwloc-chap1-preview.pdf I'm uneasy with “PU#15, for example, changes location from NUMA node #0 to #1.” The location doesn't have really changed, PUs have just been renumbered. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2270
Brad Benton, le Tue 29 Jun 2010 17:13:05 -0500, a écrit : > Sure...the tarball and output files are attached. Ok, so the physical index is explicitly -1, Linux itself doesn't know it. Samuel
Re: [hwloc-devel] Servet and hwloc
Hello, Jorge Gonzalez Dominguez, le Tue 29 Jun 2010 08:30:58 +0200, a écrit : > Apologise if you receive several copies of this mail but I think the mail that > I sent yesterday was rejected. I actually had bounced it to the list > On the one hand, about the OS support, [...] > Thank you for your advice, we will try to add the Debian > support in the next version of Servet. Actually it's probably not just Debian (which doesn't have patches for that part of glibc), but mere newer versions of glibc. > On the other hand, we think that the idea of a library is fantastic and it > would be very interesting that hwloc could use it. I will start with the > development of the library as soon as I could. Are you interested in all the > benchmarks or only in the caches ones? We are mostly interested in whatever provides machine topology, i.e. e.g. memory and cache sharing. Cache size would be nice too. We do not (yet!) have use for plain memory bandwidth. > If you are only interested in the cache size and sharing I could > create a first version only with these benchmarks to provide you the > library sooner. I don't think it's really urging. > Maybe you even would like to ask something about the interface of the > library... in that case, we are very interested in your opinion... Well, ideally (i.e. the easiest way to integrate into hwloc) such library would for instance provide a way to iterate over all caches and for each of them get its size and cpus using it. Samuel
Re: [hwloc-devel] 1.0.2rc2 posted
Christopher Samuel, le Wed 21 Jul 2010 14:48:10 +1000, a écrit : > CC libhwloc_ports_la-topology-freebsd.lo > topology-freebsd.c: In function ?hwloc_freebsd_set_thread_cpubind?: > topology-freebsd.c:123: warning: passing argument 3 of > ?pthread_setaffinity_np? from incompatible pointer type > /usr/include/pthread.h:448: note: expected ?const struct cpu_set_t *? > but argument is of type ?cpuset_t *? > topology-freebsd.c: In function ?hwloc_freebsd_get_thread_cpubind?: > topology-freebsd.c:147: warning: passing argument 3 of > ?pthread_getaffinity_np? from incompatible pointer type > /usr/include/pthread.h:453: note: expected ?struct cpu_set_t *? but > argument is of type ?cpuset_t *? > > Not sure if that's important. That is expected: it's a conflict between linux and freebsd declarations. Samuel
Re: [hwloc-devel] 1.0.2rc2 posted
Christopher Samuel, le Wed 21 Jul 2010 14:29:21 +1000, a écrit : > A simple ./configure && make distcheck fails on a SuSE > SLES 9 PPC64 box I have access to: > > make[1]: Entering directory `/tmp/chris/hwloc-1.0.2rc2' > ERROR: Did not build both of the doxygen docs and README. > ERROR: This tarball is not complete! > ERROR: Cowardly refusing to complete successfully... > > Is that just meant to work from an svn export ? I've only > run across distcheck recently with Torque, and there I am > working from an svn export. To make distcheck from an svn export, you need to be able to provide a complete distribution tarball, i.e. have tools to build docs. You should rather use the released tarbal. > *** The number of sockets is unknown > *** Logical processor 0 has 0 caches totaling 0KB > PASS: hwloc-hello > > I presume that's just because it's got an ancient kernel ? Yes. Samuel
Re: [hwloc-devel] 1.0.2rc2 posted
Jeff Squyres, le Tue 20 Jul 2010 15:22:08 -0400, a écrit : > I think we're waiting for Samuel to get back from vacation to test on all the > other esoteric OS's / platforms. I have uploaded the windows build on http://dept-info.labri.fr/~thibault/tmp/hwloc-win32-build-1.0.2rc2.zip Still working on tests. Samuel
Re: [hwloc-devel] Bug report: topology strange on SGI UltraViolet
Bernd Kallies, le Wed 28 Jul 2010 18:09:28 +0200, a écrit : > > > topology is understandeable. I'm wondering about "Group4", which > > > contains the three "Group3" objects. lstopo should print "1534GB" > > > instead of "1022GB". There is only one "Group4" object, and there are no > > > other direct children of the root object. > > > > Indeed, there's something wrong. > > Can you send the output of tests/linux/gather_topology.sh so that I try > > to debug this from here? > > Is attached. Actually the Group4 object doesn't contain the three Group3 objects: € grep 'Group[34]' gather-topology-uv.tar.gz.output Group4 #0 (total=1071374336KB) Group3 #0 (total=534634496KB) Group3 #1 (total=536739840KB) Group3 #2 (total=536739840KB) You can also see it using lstopo --gridsize 2 --fontsize 5 for instance. So it seems all good to me. > We have one UV rack, which is filled with 3/4 of the max. number of > blades. According to the specs, two NUMA nodes form one "blade". This > level corresponds to "Group0" in the hwloc topology. Two blades are > cross-linked via the NUMAlink, forming "paired nodes" = "Group1". What > "Group2" might correspond to - I don't know. "Group3" corresponds to one > "chassis" or IRU. "Group4" may be an "enclosure", and "Machine" is the > "rack". Wow, it's impressive that hwloc actually finds out all this just from the distance matrix :) > From my opinion the hwloc topology for our machine should contain 2x > Group4. hwloc can not find Group4: it finds out groups from the distance matrix. Since there are no two Group3 objects to group, it doesn't know some notion of Group4 exists there. > However, when walking the topology tree via the API, then it seems to > contain correct details. Yep :) Samuel
Re: [hwloc-devel] Bug report: topology strange on SGI UltraViolet
Brice Goglin, le Thu 29 Jul 2010 13:01:10 +0200, a écrit : > > To my opinion, the job hwloc does in forming "groups" is basically OK. > > Also the group content makes sense. > > We're lucky that it somehow matches the physical ordering, > but it is really meaningless given the distance matrix. Well, it does, even if it is arbitrary, it corresponds to distances and can be useful for binding applications. It could be an optional module in hwloc. Samuel
Re: [hwloc-devel] hwloc-1.0.2 on SLES11SP2 does not honor cpuset constraints
Bernd Kallies, le Fri 06 Aug 2010 18:13:06 +0200, a écrit : > > https://svn.open-mpi.org/trac/hwloc/changeset/2360 > > > > libibverbs is only used during make check when it's available. > > There is a problem with this philosophy. We provide hwloc on our > machines in a network filesystem at a unique path. All machines have > OFED installed, but some miss ibverbs. The changes above makes only the _tests_ use ibverbs, the hwloc library won't use it any more. Samuel
[hwloc-devel] hwloc membind interface? [Was: svn:hwloc r2339]
Hello, Changing the title for more attention by programmers :) What people prefer to give as parameter for the membind interface? A set of nodes (from a combination of obj->nodeset, or built by hand using os_index of NODE objects), or a set of cpus (from obj->cpuset)? OS interfaces take a set of nodes, so that if we use a set of cpus, we'll have to convert to nodes anyway. Samuel
Re: [hwloc-devel] Support for solaris lgrp_affinity_set
Hello, Terry Dontje, le Fri 06 Aug 2010 13:11:30 -0400, a écrit : > Is anyone looking at replacing the Solaris processor_bind calls with > lgrp_affinity_set calls in hwloc? I eventually added using lgrp_affinity_set(). Not as a replacement for processor_bind, as AIUI, lgrp_affinity_set() doesn't permit to specify precise processors. Samuel
Re: [hwloc-devel] lstopo pdf weirdness
Jeff Squyres, le Thu 26 Aug 2010 16:36:41 -0400, a écrit : > Here's two images, both from the same machine, both generated via hwloc 1.0.2. > > Note that the PDF image is missing some characters here and there. E.g., > "NUMANode" on the PNG shows up as "NU ANode" on the PDF. "24MB" on the PNG > is "24 B" on the PDF. > > Any idea what causes this? I'd tend to think cairo is guilty, like a fault in font processing. That might also be some buffer overrun which ends up right in the middle of the text string that utils/lstopo-cairo.c gives to cairo_show_text. You could try to pass it through valgrind to look for overruns. Samuel
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Alexey Kardashevskiy, le Thu 16 Sep 2010 15:57:47 +1000, a écrit : > >Is the device tree linux-specific ? If so, it can stay in linux file as > >long as it's not 30k lines :) We already have both sysfs and > >/proc/cpuinfo code there anyway. > > It is powerpc-specific. It is mapped from the system firmware (aka bios) > by the powerpc kernel. However it is just a folder within /proc so it is > usual linux folder. But PowerPC might be not the only architecture which > uses the same pathname for the same thing. Apparently microblaze uses it too, but it doesn't provide the cpus/ subdirectory. > >>- may be there is a better way to detect that no cache info was > >>fetched from sysfs > >> > >That's something that's not clear to me yet. There will likely be other > >cases in the future where we will fetch some info from different > >backends, and merging them may not be easy. Do you think the device tree > >generally contains more information than sysfs? If so, we could start by > >disabling cache info from sysfs when a device-tree is found, and maybe > >have a way to change that default (we already have a hidden en variable > >to use cpuinfo when sysfs is available). > > See my note above about the system firmware :) Almost every powerpc > system has device-tree no matter which OS you run on it (sony ps3 is > probably the only exception). /proc/device-tree is the only source for > sysfs on powerpc linux. Do you perhaps happen to know where it might be on AIX? Samuel
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Alexey Kardashevskiy, le Thu 16 Sep 2010 15:57:47 +1000, a écrit : > >>- where do I put IBM-specific code? > >> > >Is the device tree linux-specific ? If so, it can stay in linux file as > >long as it's not 30k lines :) We already have both sysfs and > >/proc/cpuinfo code there anyway. > > It is powerpc-specific. It is mapped from the system firmware (aka bios) > by the powerpc kernel. However it is just a folder within /proc so it is > usual linux folder. But PowerPC might be not the only architecture which > uses the same pathname for the same thing. AIUI from google searches, it is an Open Firmware standard, also used on the OLPC, and thus would be not only for PowerPC. Since that's not really Linux-specific, it would take place somewhere else than topology-linux.c, but the standard doesn't seem to say how/where this tree is exposed. Apparently there's even some sort of text format for it, see arch/powerpc/boot/dts/*.dts. I'd thus say that all OS-independant code should go to a topology-of.c file, and topology-linux.c would provide generic tree functions to browse /proc/device-tree, i.e. of_get_str, of_get_int_arr, and generic tree browsing functions, which topology-of.c can then use to browse the tree and fetch information without knowing where it is taken from. A topology-dts.c file would provide the same functions, but this time providing data from a .dts file, and that combination could be a backend replacing the /proc and /sys parsing completely, similarly to loading a .xml file. Note that I'm not asking you to do all this, I'm just asking to rework the function interfaces a little bit to have things already cleanly separated for anybody who would feel like adding another OS support or parsing .dts files some day, I believe that shouldn't be too much work for you. >From what I can read in drivers/of/fdt.c, the binary values are passed directly from the table in memory. I wonder which endian order this is supposed to be, but since I can see be32_to_cpu all around in the code there, I guess it is assumed to be all big endian. There are thus also a few fixes to be done in your code: - Make sure you have proper sizes: code like unsigned int reg = -1; if (0 != of_get_int_arr(cpu, "reg", , sizeof(reg), root_fd)) is not really safe, you should use uint32_t to make sure of the size of the integer. - Once the read is done, values need to be swapped on little-endian machines. You could use ntohl for that. - Constant-sized buffers are never a good idea. I know there are already some in topology-linux, but that's not a reason when it should be easy to make things properly :) It's thus say that you should rather provide the following functions: static inline int of_get_int(const char *p, const char *p1, uint32_t *value, int root_fd); Gets one integer into `value'. static inline uint32_t * of_get_int_arr(const char *p, const char *p1, int root_fd); Returns the array of integers. Yes, this makes a lot of allocation/deallocation, but that should be neglectible compared to the system calls and will save us nasty length-bugs later. Samuel
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Samuel Thibault, le Thu 16 Sep 2010 16:57:01 +0200, a écrit : > I'm just asking to rework the function interfaces a little bit to > have things already cleanly separated for anybody who would feel like > adding another OS support or parsing .dts files some day, I believe > that shouldn't be too much work for you. I mean, don't even bother to write a nice structure containing the functions pointers etc, just call your linux-specific functions hwloc_of_linux_get_str, hwloc_of_linux_get_int_arr, etc. and call them as such in topology-of.c, that'll be good enough for anybody wanting to abstract this easily. Samuel
Re: [hwloc-devel] hwloc trouble with the PGI compiler
Pavan Balaji, le Sun 19 Sep 2010 14:16:26 -0500, a écrit : > Thanks Samuel. This patch seems to work. There are still some warnings > with respect to the redefinition of CPU_SET, etc. Sure, we can't fix pgcc :) > but it doesn't seem to be causing any runtime errors. Good! Samuel
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Closer look on the patch:
Alexey Kardashevskiy, le Fri 17 Sep 2010 20:01:46 +1000, a écrit :
> Index: src/topology-linux.c
> ===
> --- src/topology-linux.c (revision 2443)
> +++ src/topology-linux.c (working copy)
> @@ -1391,6 +1391,238 @@
>*found = nbnodes;
> }
>
> +static int
> +hwloc_read_str(const char *p, const char *p1, char *buf, size_t nbuf, int
> root_fd)
Same as read_uint32: callers will have the tendency to do things like
char device_type[64];
which will break sooner or later. Rather allocate data according to
the file size (extending the allocation if the stat-provided size is
actually bogus) and free it in callers. It's less simple to write
hwloc_read_str, but it makes life way easier for later.
> +{
> + char fname[256];
> + int ret = -1;
> + sprintf(fname, "%s/%s", p, p1);
Again, never use arbitrary numbers :) And never use sprintf but
snprintf.
char fname[strlen(p) + 1 + strlen(p1) + 1];
snprintf(fname, sizeof(fname), "%s/%s", p, p1);
really puts us on the safe side.
> + FILE *file = hwloc_fopen(fname, "r", root_fd);
> + if (NULL == file)
> +return ret;
> + if (NULL != fgets(buf, nbuf, file))
> +ret = 0;
> + fclose(file);
> + buf[nbuf-1] = 0;
Then you can pass nbuf-1 to fgets. Right, that's really nitpicking :)
> +static size_t
> +hwloc_read_raw(const char *p, const char *p1, void *buf, size_t nbuf,
> +int root_fd)
Same for this one, actually.
All these could indeed go to src/misc.c.
> +typedef struct {
> + unsigned int n, allocated;
> + struct {
> +hwloc_cpuset_t cpuset;
> +uint32_t ibm_phandle;
> +uint32_t l2_cache;
> +char name[64];
Again, this could just be a pointer to a strdup of the dirent->d_name,
to avoid any surprise in the future.
> + } *p;
> +} device_tree_cpus_t;
> +
> +static void
> +add_device_tree_cpus_node(device_tree_cpus_t *cpus, hwloc_cpuset_t cpuset,
> +uint32_t l2_cache, uint32_t ibm_phandle, const char *name)
> +{
> + if (cpus->n == cpus->allocated) {
> +cpus->allocated += 64;
> +cpus->p = realloc(cpus->p, cpus->allocated * sizeof(cpus->p[0]));
> + }
> + cpus->p[cpus->n].ibm_phandle = ibm_phandle;
> + cpus->p[cpus->n].cpuset = (NULL == cpuset)?NULL:hwloc_cpuset_dup(cpuset);
> + cpus->p[cpus->n].l2_cache = l2_cache;
> + strncpy(cpus->p[cpus->n].name, name, sizeof(cpus->p[0].name)-1);
strdup being here of course.
> +static int
> +look_powerpc_device_tree_discover_cache(device_tree_cpus_t *cpus,
> +uint32_t ibm_phandle, unsigned int *level, hwloc_cpuset_t cpuset)
> +{
> + int ret = -1;
> + if ((NULL == level) || (NULL == cpuset))
> +return ret;
> + for (unsigned int i = 0; i < cpus->n; ++i) {
> +if (ibm_phandle != cpus->p[i].l2_cache)
Oh, it's odd that it's still called "l2_cache" for L3 caches above L2,
too :)
> + continue;
> +if (NULL != cpus->p[i].cpuset) {
> + hwloc_cpuset_or(cpuset, cpuset, cpus->p[i].cpuset);
> + ret = 0;
> +} else {
> + ++(*level);
> + if (0 == look_powerpc_device_tree_discover_cache(cpus,
> +cpus->p[i].ibm_phandle, level, cpuset))
> +ret = 0;
> +}
> + }
> + return ret;
> +}
> +
> +static void
> +look_powerpc_device_tree(struct hwloc_topology *topology)
> +{
> + device_tree_cpus_t cpus = { 0 };
We assume that the compiler can understand
device_tree_cpus_t cpus = { .n = 0 };
which is much clearer, please use that :)
> + const char ofroot[] = "/proc/device-tree/cpus";
> +
> + int root_fd = topology->backend_params.sysfs.root_fd;
> + DIR *dt = hwloc_opendir(ofroot, root_fd);
> + if (NULL == dt)
> +return;
> +
> + struct dirent *dirent;
> + while (NULL != (dirent = readdir(dt))) {
> +
> +char cpu[256];
> +sprintf(cpu, "%s/%s", ofroot, dirent->d_name);
> +
> +char device_type[64];
> +if (0 > hwloc_read_str(cpu, "device_type", device_type,
> sizeof(device_type), root_fd))
> + continue;
> +
> +uint32_t reg = -1, l2_cache = -1, ibm_phandle = -1;
> +hwloc_read_uint32(cpu, "reg", , root_fd);
> +hwloc_read_uint32(cpu, "l2-cache", _cache, root_fd);
> +hwloc_read_uint32(cpu, "ibm,phandle", _phandle, root_fd);
> +
> +if (0 == strcmp(device_type, "cache")) {
> + add_device_tree_cpus_node(, NULL, l2_cache, ibm_phandle,
> dirent->d_name);
> +}
> +else if (0 == strcmp(device_type, "cpu")) {
> + /* Found CPU */
> + hwloc_cpuset_t cpuset = NULL;
> + uint32_t threads[128], nthreads = 0;
> + nthreads = hwloc_read_raw(cpu, "ibm,ppc-interrupt-server#s",
> + threads, sizeof(threads), root_fd) / sizeof(threads[0]);
> + if (0 != nthreads) {
> +cpuset = hwloc_cpuset_alloc();
> +for (unsigned int i = 0; i < nthreads; ++i) {
> + hwloc_cpuset_set(cpuset, threads[i]);
> +}
> + } else if ((unsigned int)-1 != reg) {
> +cpuset = hwloc_cpuset_alloc();
> +hwloc_cpuset_set(cpuset, reg);
>
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Alexey Kardashevskiy, le Thu 16 Sep 2010 14:10:08 +1000, a écrit : > 1. What should I change in my patch to have it committed into the svn? > Specifically: > - where do I put IBM-specific code? I think it's good enough as it is now, since it's just a particular case of the generic devtree parsing code. > - may be there is a better way to detect that no cache info was fetched > from sysfs It's not so bad. We'll probably have to rethink that kind of thing when we move to loadable modules. > - is the coding style ok? :-) Samuel
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Alexey Kardashevskiy, le Fri 17 Sep 2010 20:01:46 +1000, a écrit : > Regarding topology walking - there is actually nothing device-tree > special in reading strings and numbers from a device-tree, it is just > common functions which (I think) should be placed in utils/misc.c. I > named functions like hwloc_read_str. Well, there are still the opendir/readdir/etc. functions which are linux-specific and could be implemented another way on another OS. But as it is now it's probably simple enough for somebody who'd want to abstract it. > Regarding open firmware and device trees - yes, it is IEEE1275 which can > be implemented anywhere. However, in our case, there are IBM-specific > properties (ibm,phandle, ibm,ppc-interrupt-server#s) which make all this > very IBM specific. I do not know how to deal with it better. It should be fine as it is now: even if ibm,phandle and ibm,ppc-interrupt-server#s are not there, the L1 cache information should be properly detected. Samuel
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Just a last question: is it ok to include the /proc and /sys trees you have posted in the hwloc testcases? Samuel
Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
Alexey Kardashevskiy, le Wed 22 Sep 2010 02:34:12 +0200, a écrit : > What values should I use for property which is not present? cache_size = > -1 and cache_line_size = -1 or what? 0, as other backends do. Samuel
Re: [hwloc-devel] roadmap
Brice Goglin, le Wed 22 Sep 2010 10:38:38 +0200, a écrit : > * Some OS bind the process too when you bind memory. Not for all kinds of memory bindings. For now, nothing that has been commited does that, it's only the remaining TODOs. The bindings in question are policy binding, i.e. not binding some given area or explicitly allocating some given size. > + Add a flag such as HWLOC_MEMBIND_EVEN_IF_FAR_FROM_PROCESS The length of the word tells me that won't be convenient :) > so that the user can explicitly refuse memory binding if it may break > process binding > + Drop hwloc_set_membind on these OSes and add a > hwloc_set_cpumembind() to bind both That's the solution I prefer most as it directly maps to existing OS practice > + Make both process and memory binding do nothing if the STRICT flag > is given. But I'd rather not play too much with this flag. Yes. We should not put too vague semantic on this. > + Drop support for memory binding on these OS. Not all support, just setting the policy. > + Drop these OS. Nope :) > * cpuset and nodeset structures are the same, they are both manipulated > with hwloc_cpuset_foo functions. So maybe rename into hwloc_set_t and > hwloc_set_foo functions. With #define and aliases to not break API/ABIs. I'd say so. Samuel
Re: [hwloc-devel] roadmap
Jeff Squyres, le Wed 22 Sep 2010 13:37:12 +0200, a écrit : > I think we should support memory binding, even if it does weird things -- > i.e., dropping membinding support on a given OS shouldn't be an option. That's why I'd tend to keep set_cpubind and set_membind, warning that one may have impact on the other, providing a flag for those who really care, and a binding guideline for normal users. > And/or have an "atomic"-like function that sets the memory binding and > returns the process memory binding? I'm not sure to understand what this means. > It would be good to put a sunset date or version on when hwloc_cpuset_foo > will expire (e.g., 6 months from now or two major revisions form now [1.3] -- > whichever comes last...?). Ok. > I'd also prefer a typedef than a #define for types (vs. a #define). Sure. Samuel
Re: [hwloc-devel] roadmap
Brice Goglin, le Fri 24 Sep 2010 13:31:06 +0200, a écrit : > By the way, what's the proper way to do the latter? > #pragma weak hwloc_cpuset_foo = hwloc_bitmap_foo ? > use __hwloc_attribute_alias instead ? There is no proper way unfortunately: the Mach-O format used by MacOS does not support such aliases. Explicit functions need to be provided. Samuel
Re: [hwloc-devel] Strange results on itanium 2
Hello, Jirka Hladky, le Wed 27 Oct 2010 14:53:01 +0200, a écrit : > I have attached a tar file with full information. Could you also post the result of test/linux/gather-topology.sh ? Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2649
Brice Goglin, le Thu 28 Oct 2010 10:14:31 +0200, a écrit : > This raises again a question that got no answer in previous emails. > Having NULL nodeset pointers in obj->*nodeset when the machine isn't > NUMA is annoying in many places. I think I will change to a full > nodeset instead. It will simplify many things. OK, I guess we should however add an assertion(!full) to hwloc_bitmap_foreach_begin. Samuel
Re: [hwloc-devel] gather-topology.sh and rpm
Jirka Hladky, le Thu 04 Nov 2010 16:59:35 +0100, a écrit : > 3) Whenever I run lstopo using X output and I close lstopo window I will get > > $lstopo > XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0.0" > after 121 requests (121 known processed) with 0 events remaining. > > Is there something wrong with my X-server settings? Can it be suppressed? That is the usual behavior. Try with the xlogo app for instance, it'll get the same. You can close more nicely by hitting q, escape, or the close button (not kill button) if your window manager provides one. Samuel
Re: [hwloc-devel] doxygen CSS change?
Jeff Squyres, le Thu 04 Nov 2010 16:03:29 +0100, a écrit : > I notice that the v1.1 doxygen-generated HTML has a new header style: > > http://www.open-mpi.org/projects/hwloc/doc/v1.1rc1/ > vs. http://www.open-mpi.org/projects/hwloc/doc/v1.0.2/ > > Did *we* change that style, or am I using a newer version of doxygen that > changed the style? We didn't change anything, it's the latter. Samuel
Re: [hwloc-devel] Final 1.0.x release?
Jeff Squyres, le Fri 05 Nov 2010 16:29:22 +0100, a écrit : > There are a few pending bug fixes on the 1.0.x branch. Should we do a 1.0.3 > release, just to close out the 1.0.x series? There are a few API breaks between 1.0 and 1.1, so I guess 1.0.3 should be released. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2734
Jeff Squyres, le Fri 05 Nov 2010 17:55:41 +0100, a écrit : > If DIST_SUBDIRS is set, it trumps SUBDIRS during "make dist". Apfff. automake is still full of surprises after having worked with it for a few years. Samuel
Re: [hwloc-devel] 1.0.3 .so version number
Jeff Squyres, le Mon 08 Nov 2010 15:18:01 +0100, a écrit : > Here's a Trac colorized diff between the 1.0 branch from r2349 and the > current HEAD: > > https://svn.open-mpi.org/trac/hwloc/changeset?old_path=/branches/v1.0=2349_path=/branches/v1.0=HEAD): > > The only interface change I see is this: > > -hwloc_linux_set_tid_cpubind(hwloc_topology_t topology > __hwloc_attribute_unused, pid_t tid, hwloc_const_cpuset_t hwloc_set) > +hwloc_linux_set_tid_cpubind(hwloc_topology_t topology > __hwloc_attribute_unused, pid_t tid __hwloc_attribute_unused, > hwloc_const_cpuset_t hwloc_set __hwloc_attribute_unused) > > Which I don't believe impacts shared library linking (i.e., if an app used > hwloc_linux_set_tid_cpubind() and compiled against hwloc 1.0.2, I believe it > would still link successfully against the 1.0.3 libhwloc. As such, I believe > that this is a non-event, in terms of shared library versioning. I believe too. > I believe the version number should be 0:2:0. New implementation of the same interface, yes. Samuel
Re: [hwloc-devel] 1.1 .so version number
Jeff Squyres, le Mon 08 Nov 2010 15:38:31 +0100, a écrit : > The last one is what I'm not sure about, but what I'm inferring from Samuel's > statement about "API breaks". Actually you can have an API break without an ABI break. Here, old applications should work fine. They'll just not be able to reach the DMI strings, as they're now in "infos". But they won't crash since attrs will just be empty. So I believe we can use 1:0:1. That's why we have kept aliases in src/cpuset.c for, actually. Samuel
Re: [hwloc-devel] 1.1 .so version number
Brice Goglin, le Mon 08 Nov 2010 16:57:44 +0100, a écrit : > Le 08/11/2010 15:38, Jeff Squyres a écrit : > > Short version: > > -- > > > > I have not looked closely -- I *think* APIs have been added and changed > > since v1.0. As such, I *think* the libtool .so version number for 1.1 > > should be 1:0:0. > > > > I don't think any function behavior changed. I don't think either. > But the object structure has been extended, cache attributes were > extended, Extension is not a problem, provided that offsets are still the same (i.e. the old C structure appears first in the new C structure) > machine attributes were removed. That is not a problem here. The attr field of hwloc_obj will be NULL, that's all, the application won't ever read it anyway. I don't see anything else that we removed (that's why you have added cpuset aliases actually). Samuel
Re: [hwloc-devel] [hwloc-announce] Hardware locality (hwloc) v1.1rc1 released
Jirka Hladky, le Thu 11 Nov 2010 13:36:46 +0100, a écrit : > > hwloc_get_membind failed (errno 38 Function not implemented) > > Yes, you are right! > --get --pid > works on Linux. > > --get --membind --pid > will give "Function not implemented" > > $ /tmp/hwloc-1.1rc2/utils/hwloc-bind --get --membind --pid 344 > hwloc_get_membind failed (errno 38 Function not implemented) > > > It actually depends on the OS. I'll see what I can. > I see. It's getting difficult then. I believe that in this case more > explanatory > error message would be enough. Mmm, to me hwloc_get_membind failed (errno 38 Function not implemented) is already self-explanatory actually. Do you see how could it be improved? Samuel
Re: [hwloc-devel] [hwloc-announce] Hardware locality (hwloc) v1.1rc1 released
Jirka Hladky, le Thu 11 Nov 2010 14:50:46 +0100, a écrit : > "On this system function XYZ is not supported by GLIBC/KERNEL)" > > I'm missing the information: > > -which function is not implemented Well, you have it: hwloc_proc_getmembind() How it'd be called by the OS in the future is unknown of course. > -where this function belong - is it system call, glibc or hwloc's function? It's always system call or glibc function, it depends on the system and we can't know where it'd be implemented in the future. Or our lack of knowledge of which system call can provide the functionality. > Or perhaps something more user friendly like > "On this system --get does not work together with --membind" We'd have to handle a big list of combinations of parameters in that case. I'd rather add a paragraph to the documentation that just explains that not everything is available on all OSes, or hwloc just doesn't know that it got implemented. Samuel
Re: [hwloc-devel] [hwloc-announce] Hardware locality (hwloc) v1.1rc1 released
Jirka Hladky, le Thu 11 Nov 2010 20:03:20 +0100, a écrit :
> On Thursday, November 11, 2010 07:19:41 pm Samuel Thibault wrote:
> > Jirka Hladky, le Thu 11 Nov 2010 14:50:46 +0100, a écrit :
> > > "On this system function XYZ is not supported by GLIBC/KERNEL)"
> > >
> > > I'm missing the information:
> > >
> > > -which function is not implemented
> >
> > Well, you have it: hwloc_proc_getmembind()
> > How it'd be called by the OS in the future is unknown of course.
> >
> > > -where this function belong - is it system call, glibc or hwloc's
> > > function?
> >
> > It's always system call or glibc function, it depends on the system and
> > we can't know where it'd be implemented in the future. Or our lack of
> > knowledge of which system call can provide the functionality.
>
> Well, I think I have not expressed myself correctly. At the moment we have:
>
> hwloc_get_membind failed (errno 38 Function not implemented)
>
> I would like to see which glibc/system call has failed.
> Example:
>
> err = get_mempolicy(, linuxmask, max_os_index, 0, 0);
> if (err < 0) {
> perror("get_mempolicy"); <== ADD THIS LINE
> goto out_with_mask;
> }
My point is that the fix here is _not_ about get_mempolicy. Hwloc didn't
even call it. Hwloc just knows that Linux doesn't provide any function
to get the mempolicy of another process. The get_mempolicy function
doesn't take a pid, and thus will never take one, so another OS function
will have to be defined in the future by Linux people, which will wear
another name. So printing "get_mempolicy" will not actually help.
> My first impression when I saw the error message above was that function
> "hwloc_get_membind" is not implemented.
hwloc_bind should probably print "hwloc_proc_get_membind" instead when
it gives the flag, indeed. I don't think much more can be printed.
Samuel
Re: [hwloc-devel] hwloc-distrib --among
Samuel Thibault, le Tue 16 Nov 2010 22:18:54 +0100, a écrit : > Also note that currently the hwloc_distribute() function doesn't take > e.g. the number of PUs into account when splitting elements over the > hierarchy. It was more a demonstration example than something to be used > as is. We can however extend it, we just need to know what's desired. Reading your mail again, I guess that's where your issue actually lied. Samuel
Re: [hwloc-devel] hwloc-distrib --among
Brice Goglin, le Tue 16 Nov 2010 22:31:38 +0100, a écrit : > Le 16/11/2010 15:18, Samuel Thibault a écrit : > > Same here, distributing 4 elements between the PUs, thus selecting the > > first 4 PUs. > > > > I'd say that --among should indeed be the horizontal portion of the > > machine to distribute on, and the existing --among option should be > > renamed into --from, and a --to option could be added to stop the > > hierarchical distribution to a given object type. > > > > Works for me but won't have time to look at it before the end of the > week. I don't think --among was available in 1.0, so maybe we can rename > it in 1.1 and we'll add other options in 1.2 ? I have renamed --among into --from, added --to and --at, and documented a bit more. Restricting the topology to some cpuset is actually something more global, I think it should be a topology configuration option, which would also be useful for lstopo for instance. Samuel
Re: [hwloc-devel] PCI device location in hwloc
Hello, Christopher Samuel, le Thu 18 Nov 2010 23:47:20 +0100, a écrit : > Does the information occur to the right of the socket with > the closest distance to the devices ? If your case, hwloc was apparently unable to decide whether the devices where "inside" one or the other NUMA node. Could you also post the output of gather-topology? > Well at the moment that looks rather nice to me, though it > would indeed be good to see GPUs labelled to - though I've > seen your comment in the source saying: > > /* FIXME: what about gpus? could try class "drm", but >proprietary drivers won't appear there */ > > I don't have any boxes at all with GPUs in them, so I'm not > sure what to suggest there. :-( Actually we might as well just first implement the modularized version of hwloc, so that we can dynamically load a cuda-based module which will just show a proper CUDA device. In the meanwhile, we could special-case a few things. We probably don't want to emit the whole pciids identifier (e.g. 85:00.0 VGA compatible controller: nVidia Corporation GT200GL [Quadro FX 5800] (rev a1) ), but in this case we could handle the nvidia vendorId specially: parse the pciid to extract Quadro FX 5800 and display that. Samuel
Re: [hwloc-devel] ***UNCHECKED*** [WARNING: A/V UNSCANNABLE]Re: hwloc-distrib --among
Hello, Jirka Hladky, le Thu 18 Nov 2010 15:14:07 +0100, a écrit : > My goal is to distribute one job per Socket with command > hwloc-distrib --single 8 Could you try again with the current v1.1 subversion branch? I have implemented asymmetry support. Samuel
Re: [hwloc-devel] Next 1.0/1.1 RC's
I agree on 1.0.3. Brice Goglin, le Mon 22 Nov 2010 14:26:36 +0100, a écrit : > It looks like Chris' latest problem (hwloc-bind test failing in a free) > is not fixed yet. But we already have several important fixes, so maybe > we can do rc3 anyway? Good for me with the r2813 fix. Samuel
Re: [hwloc-devel] hwloc to be included in RHEL 6.1
Samuel Thibault, le Mon 22 Nov 2010 17:33:15 +0100, a écrit : > > -- using "p" is a good way to indicate "physical". But IIRC, we didn't > > like "l" (for "logical") because it looks too much like 1 (one). > > > > I think we're open to having some kind of indication to denote "logical" > > instead of "physical" -- any suggestions? Perhaps P and L (vs. p and l)? > > P/L can be better than p/l, yes. Just "PU #0" is indeed probably not > precise enough, and "PU L#0" will make people wonder why the L, and then > understand why. I guess we can try to add this to an rc4. Thinking again about it: can't we just switch only lstopo to physical numbering by default, and only for the drawn part? The textual output (lstopo -) displays both anyway. We wanted to use logical numbering by default to be coherent with other hwloc tools, but the graphical/semigraphical lstopo one is very particular (I hope nobody is crazy enough to parse its output), and in almost all cases people will want physical numbering by default, other cases can be obtained through -l. I'd even say 1.0.3 should switch too (v0.9 was only using physical numbering in lstopo). Samuel
Re: [hwloc-devel] ***UNCHECKED*** [WARNING: A/V UNSCANNABLE]Re: hwloc-distrib --among
Jirka Hladky, le Mon 22 Nov 2010 18:54:24 +0100, a écrit : > However, I was wondering if there is some way to get the old way, i.e to > distribute the jobs among NUMA nodes. I DO NOT use this mode but I don't want > anybody else to be harmed. Actually I was wondering whether it's useful at all, considering that everybody who used distribute asked for a way not to be harmed by asymmetry. > I have tried > --at numa > and > --to numa > to replicate the old behavior but without success... Sure, it'll still take into account the "weight" of the NUMA nodes. --to just tells down to which objects the distribution will be performed. Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2831
Brice Goglin, le Wed 24 Nov 2010 10:15:07 +0100, a écrit : > We should uniformize how the graphic/drawing and text outputs are called > in the manpage/usage/doc/README, it may be a bit misleading now. But I > am not sure which terms are best between graphic and drawing, and > between console and text. An important point is that you can have a text-based drawing :) Samuel
Re: [hwloc-devel] multiline legend
Jeff Squyres, le Tue 30 Nov 2010 14:45:13 +0100, a écrit : > How's this, instead? I made a few minor changes: > > - prefixed each line of the legend (the "physical IDs" line was confusing to > me without a prefix) > - fixed logic for terminating timestamp string > - moved all the legend logic inside "if (legend)" That looks good to me. Samuel
Re: [hwloc-devel] SWIG bindings
Jeff Squyres, le Tue 30 Nov 2010 17:29:15 +0100, a écrit : > Would anyone object if I take a whack at making some SWIG bindings for hwloc? I'd say it's welcome, as it'd easily bring bindings for other languages as well. I'm unsure how our pointers in the hwloc_obj_t structure will nicely map, however. Samuel
Re: [hwloc-devel] SWIG bindings
Guy Streeter, le Tue 30 Nov 2010 17:48:56 +0100, a écrit : > On 11/30/2010 10:07 AM, Jeff Squyres wrote: > >Would anyone object if I take a whack at making some SWIG bindings for > >hwloc? I'm thinking specifically for perl (because that's my scripting > >language of choice), but I could probably be convinced to look at python > >as well. > > > >(this would be for 1.2 at the earliest -- definitely not for 1.1) > > I'm currently working on a python bindings package for hwloc. I suppose I > should find out how to get my code upstream. Send a patch :) Samuel
Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2875
bgog...@osl.iu.edu, le Tue 30 Nov 2010 18:22:29 +0100, a écrit : > 1) NO_PCI shows nothing > > And I wonder if we should make (1) the default for "backward compatibility". Possibly, yes. (but probably not in lstopo) Samuel
Re: [hwloc-devel] Fwd: [hwloc-svn] svn:hwloc r2868
Jiri Hladky, le Fri 03 Dec 2010 11:32:01 +0100, a écrit : > I would recommend to replace ctime with strftime function. Please note that > ctime ignores LC_TIME. strftime on the other hand is using LC_TIME I was wondering. strftime is supported by less operating systems. But we can detect that, yes. Samuel
Re: [hwloc-devel] Fwd: [hwloc-svn] svn:hwloc r2868
Jiri Hladky, le Sat 04 Dec 2010 23:16:36 +0100, a écrit : > it's a valid objection. I use only linux with pretty recent version of kernel > and C libraries so I didn't know that strftime has portability issues. Well, I meant that strftime has appeared "only" in susv release 3, i.e. something like before 1994 :) The thing is some systems (e.g. mingw & such) may still not have it. > On the hand, autoconf should be able to detect it, right? That's what I meant in my mail and commited the other day, yes. Samuel
