Re: [hwloc-devel] [OMPI devel] 0.9.1rc2 is available

[Tony Breeds]

On Thu, Oct 22, 2009 at 10:29:36AM +1100, Chris Samuel wrote:

> Dual socket, dual core Power5 (SMT disabled) running SLES9
> (2.6.9 based kernel):
> 
> System(15GB)
>   Node#0(7744MB)
> P#0
> P#2
>   Node#1(8000MB)
> P#4
> P#6

Powerpc kernels that old do not have the topology information needed (in /sys
or /proc/cpuinfo)  So for the short term that's be best we can do.  FWIW I'm 
looking at how we can pull more (if not all) the same info from the device tree
on these kenels.

Yours Tony

[hwloc-devel] 0.9.1rc2 failures

[Pavan Balaji]

Tried building 0.9.1rc2 with gcc, icc, suncc and pgcc. Builds fine with
gcc. Lots of warnings with icc (as Jeff pointed out). Does not build
with suncc and pgcc.

With suncc (sunstudio 12):
==
source='topology-linux.c' object='topology-linux.lo' libtool=yes \
DEPDIR=.deps depmode=none /bin/bash ../config/depcomp \
/bin/bash ../libtool --tag=CC   --mode=compile suncc -DHAVE_CONFIG_H
-I. -I../include/private -I../include/hwloc  -I../include -I../include
  -g -c -o topology-linux.lo topology-linux.c
libtool: compile:  suncc -DHAVE_CONFIG_H -I. -I../include/private
-I../include/hwloc -I../include -I../include -g -c topology-linux.c
-KPIC -DPIC -o .libs/topology-linux.o
"../include/hwloc/helper.h", line 46: warning: statement not reached
"../include/hwloc/helper.h", line 69: warning: statement not reached
"../include/private/private.h", line 197: warning: argument mismatch
"topology-linux.c", line 535: warning: initializer does not fit or is
out of range: -1
"topology-linux.c", line 536: warning: initializer does not fit or is
out of range: -1
"topology-linux.c", line 782: syntax error before or at: ...
"topology-linux.c", line 782: warning: null dimension: proc_physids
"topology-linux.c", line 783: syntax error before or at: ...
"topology-linux.c", line 783: warning: null dimension: osphysids
"topology-linux.c", line 784: syntax error before or at: ...
"topology-linux.c", line 784: warning: null dimension: proc_coreids
"topology-linux.c", line 785: syntax error before or at: ...
"topology-linux.c", line 785: warning: null dimension: oscoreids
"topology-linux.c", line 786: syntax error before or at: ...
"topology-linux.c", line 786: warning: null dimension: proc_osphysids
"topology-linux.c", line 787: syntax error before or at: ...
"topology-linux.c", line 787: warning: null dimension: proc_oscoreids
"topology-linux.c", line 788: syntax error before or at: ...
"topology-linux.c", line 788: warning: null dimension: core_osphysids
"topology-linux.c", line 802: warning: argument mismatch
"topology-linux.c", line 808: warning: argument mismatch
"topology-linux.c", line 888: warning: argument mismatch
"topology-linux.c", line 903: cannot recover from previous errors
cc: acomp failed for topology-linux.c
make[1]: *** [topology-linux.lo] Error 1
make[1]: Leaving directory
`/radix-homes/balaji/tmp/hwloc/hwloc-0.9.1rc2/src'
make: *** [all-recursive] Error 1
==

With pgcc (9.0-4):
==
libtool: link: pgcc -g -o .libs/lstopo lstopo-lstopo.o
lstopo-lstopo-color.o lstopo-lstopo-text.o lstopo-lstopo-draw.o
lstopo-lstopo-fig.o lstopo-lstopo-cairo.o lstopo-lstopo-xml.o
-L/home/balaji/tmp/hwloc/hwloc-0.9.1rc2/src -lm ../src/.libs/libhwloc.so
-ltermcap
../src/.libs/libhwloc.so: undefined reference to `__CPU_SET'
../src/.libs/libhwloc.so: undefined reference to `__CPU_ZERO'
make[1]: *** [lstopo] Error 2
make[1]: Leaving directory
`/radix-homes/balaji/tmp/hwloc/hwloc-0.9.1rc2/utils'
make: *** [all-recursive] Error 1
==

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

[hwloc-devel] Fwd: [OMPI devel] 0.9.1rc2 is available

[Jeff Squyres]

Here's Chris' second results email:

Begin forwarded message:


From: "Chris Samuel" 
Date: October 21, 2009 7:37:34 PM EDT
To: "Open MPI Developers" 
Subject: Re: [OMPI devel] 0.9.1rc2 is available
Reply-To: "Open MPI Developers" 


- "Chris Samuel"  wrote:

> Some sample results below for configs not represented
> on the current website.

A final example of a more convoluted configuration with
a Torque job requesting 5 CPUs on a dual Shanghai node
and has been given a non-contiguous configuration.

[csamuel@tango069 ~]$ cat /dev/cpuset/`cat /proc/$$/cpuset`/cpus
0,4-7

[csamuel@tango069 ~]$ ~/local/hwloc/0.9.1rc2/bin/lstopo
System(31GB)
  Node#0(15GB) + Socket#0 + L3(6144KB) + L2(512KB) + L1(64KB) +  
Core#0 + P#0

  Node#1(16GB) + Socket#1 + L3(6144KB)
L2(512KB) + L1(64KB) + Core#0 + P#4
L2(512KB) + L1(64KB) + Core#1 + P#5
L2(512KB) + L1(64KB) + Core#2 + P#6
L2(512KB) + L1(64KB) + Core#3 + P#7

--
Christopher Samuel - (03) 9925 4751 - Systems Manager
 The Victorian Partnership for Advanced Computing
 P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




--
Jeff Squyres
jsquy...@cisco.com

[hwloc-devel] Fwd: [OMPI devel] 0.9.1rc2 is available

[Jeff Squyres]

Arrgh.  I posted to the "0.9.1rc2 is available" notice to the wrong  
list (stupid mail client autocomplete...).


But Chris Samuel replied with 2 mails (the first is below) with some  
results:



Begin forwarded message:


From: "Chris Samuel" 
Date: October 21, 2009 7:29:36 PM EDT
To: "Open MPI Developers" 
Subject: Re: [OMPI devel] 0.9.1rc2 is available
Reply-To: "Open MPI Developers" 


- "Jeff Squyres"  wrote:

> Give it a whirl:

Nice - built without warnings with GCC 4.4.2.

Some sample results below for configs not represented
on the current website.


Dual socket Shanghai:

System(31GB)
  Node#0(15GB) + Socket#0 + L3(6144KB)
L2(512KB) + L1(64KB) + Core#0 + P#0
L2(512KB) + L1(64KB) + Core#1 + P#1
L2(512KB) + L1(64KB) + Core#2 + P#2
L2(512KB) + L1(64KB) + Core#3 + P#3
  Node#1(16GB) + Socket#1 + L3(6144KB)
L2(512KB) + L1(64KB) + Core#0 + P#4
L2(512KB) + L1(64KB) + Core#1 + P#5
L2(512KB) + L1(64KB) + Core#2 + P#6
L2(512KB) + L1(64KB) + Core#3 + P#7


Dual socket single core Opteron:

System(3961MB)
  Node#0(2014MB) + Socket#0 + L2(1024KB) + L1(1024KB) + Core#0 + P#0
  Node#1(2017MB) + Socket#1 + L2(1024KB) + L1(1024KB) + Core#0 + P#1


Dual socket, dual core Power5 (SMT disabled) running SLES9
(2.6.9 based kernel):

System(15GB)
  Node#0(7744MB)
P#0
P#2
  Node#1(8000MB)
P#4
P#6


Inside a single CPU Torque job (using cpusets) on a dual socket  
Shanghai:


System(31GB)
  Node#0(15GB) + Socket#0 + L3(6144KB) + L2(512KB) + L1(64KB) +  
Core#0 + P#0

  Node#1(16GB)


--
Christopher Samuel - (03) 9925 4751 - Systems Manager
 The Victorian Partnership for Advanced Computing
 P.O. Box 201, Carlton South, VIC 3053, Australia
VPAC is a not-for-profit Registered Research Agency
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




--
Jeff Squyres
jsquy...@cisco.com

[hwloc-devel] Create success (hwloc r1.0a1r1216)

[MPI Team]

Creating nightly hwloc snapshot SVN tarball was a success.

Snapshot:   hwloc 1.0a1r1216
Start time: Wed Oct 21 21:01:01 EDT 2009
End time:   Wed Oct 21 21:03:07 EDT 2009

Your friendly daemon,
Cyrador

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Pavan Balaji, le Wed 21 Oct 2009 17:46:28 -0500, a écrit :
> Suppose you compiled hwloc with gcc, you'll get the following in your
> new config file --
> 
> /* Note the hwloc_ prefix added by the AX_PREFIX_CONFIG_H macro */
> #define hwloc_restrict __restrict
> 
> Now, if the application is using suncc, and it tries to find what
> spelling for restrict it needs to use, it'll come up with --
> 
> #define restrict _Restrict
> 
> These do not conflict at all; hwloc will use its definition of restrict
> and the application will use its. Why is this a problem?

Because the hwloc headers are full with hwloc_restrict qualifiers,
translated into __restrict, which suncc will not understand, and thus
hwloc.h will not compile.

Samuel

Re: [hwloc-devel] MPICH2 question

[Pavan Balaji]

On 10/21/2009 04:44 PM, Samuel Thibault wrote:
> Pavan Balaji, le Wed 21 Oct 2009 16:40:42 -0500, a écrit :
>>> Please read this thread:
>>>
>>> http://www.open-mpi.org/community/lists/hwloc-devel/2009/09/0054.php
>> Thanks. Is the only issue to not use AC_C_RESTRICT that of conflicting
>> name space?
> 
> Not only, it's also that you never know in advance which compiler will
> be used while including hwloc.h, and thus what _that_ compiler will
> support, thus the dynamic discovery.

I'm not sure I understand the problem here. Let me illustrate what I
mean. Suppose you are using gcc to compile hwloc and suncc to compile
some application using hwloc. Let's say gcc uses __restrict and suncc
uses _Restrict (I didn't check what these compilers use; these are just
for illustration).

hwloc configure.in will have --

AX_PREFIX_CONFIG_H([include/hwloc_config.h], [hwloc])
AC_C_RESTRICT

Suppose you compiled hwloc with gcc, you'll get the following in your
new config file --

/* Note the hwloc_ prefix added by the AX_PREFIX_CONFIG_H macro */
#define hwloc_restrict __restrict

Now, if the application is using suncc, and it tries to find what
spelling for restrict it needs to use, it'll come up with --

#define restrict _Restrict

These do not conflict at all; hwloc will use its definition of restrict
and the application will use its. Why is this a problem?

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Pavan Balaji, le Wed 21 Oct 2009 16:40:42 -0500, a écrit :
> > Please read this thread:
> > 
> > http://www.open-mpi.org/community/lists/hwloc-devel/2009/09/0054.php
> 
> Thanks. Is the only issue to not use AC_C_RESTRICT that of conflicting
> name space?

Not only, it's also that you never know in advance which compiler will
be used while including hwloc.h, and thus what _that_ compiler will
support, thus the dynamic discovery.

Samuel

Re: [hwloc-devel] MPICH2 question

[Pavan Balaji]

> Please read this thread:
> 
> http://www.open-mpi.org/community/lists/hwloc-devel/2009/09/0054.php

Thanks. Is the only issue to not use AC_C_RESTRICT that of conflicting
name space? If yes, then there are ways to avoid it while still using
AC_C_RESTRICT (see
http://www.nongnu.org/autoconf-archive/ax_prefix_config_h.html).

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] v0.9 branch

[Samuel Thibault]

Brice Goglin, le Wed 21 Oct 2009 22:03:58 +0200, a écrit :
> We could either:
> * use unifdef at install time to keep the relevant #ifdef/#ifndef code
> path (I don't know how popular unifdef is)
> or
> * have different headers in the tarball, one for each OS, and only
> install the right one

Or just define HWLOC_LINUX_SYS in the public config.h and use that.

Samuel

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Pavan Balaji, le Wed 21 Oct 2009 13:23:39 -0500, a écrit :
> On 10/21/2009 10:38 AM, Samuel Thibault wrote:
> > Pavan Balaji, le Wed 21 Oct 2009 10:36:33 -0500, a écrit :
> >> On 10/21/2009 10:28 AM, Samuel Thibault wrote:
> >>> Pavan Balaji, le Wed 21 Oct 2009 09:55:36 -0500, a écrit :
>  1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
>  compiler to be C99 capable always?
> >>> No, we ended up using constructs which should pass c90 and the compilers
> >>> we have tested (aix, solaris, icc).
> >> So shouldn't the AC_PROG_CC_C99 be gotten rid of?
> > 
> > No because when C99 is available, we enable some optimization features,
> > like __hwloc_restrict.
> 
> It looks like __hwloc_restrict is not actually checking for C99, but
> instead doing something GNU specific:

It's doing both: if gcc is available, use __restrict, if C99 is
available, use restrict.

> #if (__GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 95))
> # define __hwloc_restrict __restrict
> #else
> # if __STDC_VERSION__ >= 199901L
> #  define __hwloc_restrict restrict
> # else
> #  define __hwloc_restrict
> # endif
> #endif
> 
> Wouldn't it be better to add a feature test for restrict, instead of this?

Please read this thread:

http://www.open-mpi.org/community/lists/hwloc-devel/2009/09/0054.php

Samuel

Re: [hwloc-devel] v0.9 branch

[Pavan Balaji]

The 0.9 branch doesn't build anymore -- missing openfabrics-verbs.h.

 -- Pavan

On 10/21/2009 01:34 PM, Jeff Squyres wrote:
> On Oct 20, 2009, at 8:33 PM, Jeff Squyres wrote:
> 
>> Brice -- do you need to move r1195 and r1196 to the v0.9 branch?
> 
> I effectively just brought these over to the v0.9 branch.
> 
> With all these changes, I'll cut an rc2 and post it shortly.
> 

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] v0.9 branch

[Brice Goglin]

Jeff Squyres wrote:
> On Oct 21, 2009, at 3:43 PM, Brice Goglin wrote:
>
>> Now that I try to implement it, I remember why an inline is convenient:
>> it doesn't require any build-time/run-time dependency unless you really
>> use it. If we make this code non-inline, we actually need libibverbs at
>> build time and runtime. Distro packages will have to depend on
>> libibverbs, and we'll get lots of complaints. Same for linux-libnuma.h
>> and maybe nvidia cuda one day :) So we'll probably end up splitting this
>> code out of libhwloc.so and make a libhwloc-openfabrics-verbs.so.
>>
>
> Ah, good reason.  Let's leave it as inline for 0.9.1, then.  But we do
> need to fix the topology parameter, sorry.  :-(

Ok, but we still have a problem with LINUX_SYS not being visible to
external users (it's only defined in private/config.h). Our test gets
the right Linux code because it includes private/config.h explicitely.
But all externall users will use the default/dumb code path.

We could either:
* use unifdef at install time to keep the relevant #ifdef/#ifndef code
path (I don't know how popular unifdef is)
or
* have different headers in the tarball, one for each OS, and only
install the right one

> Should we make a dlopen-like functionality for this kind of stuff for
> v1.0?  It's not hard to do with libltdl.

No idea :)

Brice

Re: [hwloc-devel] v0.9 branch

[Jeff Squyres]

On Oct 21, 2009, at 3:43 PM, Brice Goglin wrote:

Now that I try to implement it, I remember why an inline is  
convenient:
it doesn't require any build-time/run-time dependency unless you  
really
use it. If we make this code non-inline, we actually need libibverbs  
at

build time and runtime. Distro packages will have to depend on
libibverbs, and we'll get lots of complaints. Same for linux-libnuma.h
and maybe nvidia cuda one day :) So we'll probably end up splitting  
this

code out of libhwloc.so and make a libhwloc-openfabrics-verbs.so.



Ah, good reason.  Let's leave it as inline for 0.9.1, then.  But we do  
need to fix the topology parameter, sorry.  :-(


Should we make a dlopen-like functionality for this kind of stuff for  
v1.0?  It's not hard to do with libltdl.


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] v0.9 branch

[Brice Goglin]

Now that I try to implement it, I remember why an inline is convenient:
it doesn't require any build-time/run-time dependency unless you really
use it. If we make this code non-inline, we actually need libibverbs at
build time and runtime. Distro packages will have to depend on
libibverbs, and we'll get lots of complaints. Same for linux-libnuma.h
and maybe nvidia cuda one day :) So we'll probably end up splitting this
code out of libhwloc.so and make a libhwloc-openfabrics-verbs.so.

Brice



Jeff Squyres wrote:
> My vote: +1 for before v0.9.1 release.
>
>
> On Oct 21, 2009, at 3:20 PM, Brice Goglin wrote:
>
>> Jeff Squyres wrote:
>> > On Oct 20, 2009, at 8:33 PM, Jeff Squyres wrote:
>> >
>> >> Brice -- do you need to move r1195 and r1196 to the v0.9 branch?
>> >
>> > I effectively just brought these over to the v0.9 branch.
>> >
>> > With all these changes, I'll cut an rc2 and post it shortly.
>>
>>
>> Am I still supposed to make the verbs helper not inline anymore ?
>>
>> Brice
>>
>> ___
>> hwloc-devel mailing list
>> hwloc-de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel
>>
>
>

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Jeff Squyres]

On Oct 21, 2009, at 8:56 AM, Brice Goglin wrote:

> What should we return from hwloc_ibv_get_device_cpuset() if not  
#ifdef

> LINUX?

I'd say a full cpuset.
Or the full-system cpuset:
   return hwloc_cpuset_dup(hwloc_get_system_obj(topology)->cpuset);
If the latter we'll need to add a topology parameter to
hwloc_ibv_get_device_cpuset()

Oh crud, I missed your last statement about adding a topology  
parameter, so I didn't add it (i.e., to be fixed pre-final-release).


So how about this:

- make it non-inline
- add the topology parameter

--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] v0.9 branch

[Jeff Squyres]

My vote: +1 for before v0.9.1 release.


On Oct 21, 2009, at 3:20 PM, Brice Goglin wrote:


Jeff Squyres wrote:
> On Oct 20, 2009, at 8:33 PM, Jeff Squyres wrote:
>
>> Brice -- do you need to move r1195 and r1196 to the v0.9 branch?
>
> I effectively just brought these over to the v0.9 branch.
>
> With all these changes, I'll cut an rc2 and post it shortly.


Am I still supposed to make the verbs helper not inline anymore ?

Brice

___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] v0.9 branch

[Brice Goglin]

Jeff Squyres wrote:
> On Oct 20, 2009, at 8:33 PM, Jeff Squyres wrote:
>
>> Brice -- do you need to move r1195 and r1196 to the v0.9 branch?
>
> I effectively just brought these over to the v0.9 branch.
>
> With all these changes, I'll cut an rc2 and post it shortly.


Am I still supposed to make the verbs helper not inline anymore ?

Brice

Re: [hwloc-devel] v0.9 branch

[Jeff Squyres]

On Oct 20, 2009, at 8:33 PM, Jeff Squyres wrote:


Brice -- do you need to move r1195 and r1196 to the v0.9 branch?


I effectively just brought these over to the v0.9 branch.

With all these changes, I'll cut an rc2 and post it shortly.

--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] MPICH2 question

[Pavan Balaji]

On 10/21/2009 10:38 AM, Samuel Thibault wrote:
> Pavan Balaji, le Wed 21 Oct 2009 10:36:33 -0500, a écrit :
>> On 10/21/2009 10:28 AM, Samuel Thibault wrote:
>>> Pavan Balaji, le Wed 21 Oct 2009 09:55:36 -0500, a écrit :
 1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
 compiler to be C99 capable always?
>>> No, we ended up using constructs which should pass c90 and the compilers
>>> we have tested (aix, solaris, icc).
>> So shouldn't the AC_PROG_CC_C99 be gotten rid of?
> 
> No because when C99 is available, we enable some optimization features,
> like __hwloc_restrict.

It looks like __hwloc_restrict is not actually checking for C99, but
instead doing something GNU specific:

#if (__GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 95))
# define __hwloc_restrict __restrict
#else
# if __STDC_VERSION__ >= 199901L
#  define __hwloc_restrict restrict
# else
#  define __hwloc_restrict
# endif
#endif

Wouldn't it be better to add a feature test for restrict, instead of this?

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

[hwloc-devel] merging to v0.9

[Jeff Squyres]

BTW, I use this trivial script to help merge stuff over to v0.9 --  
maybe it's helpful to others...


#!/bin/csh -f

set r=$1
set otrunk=https://svn.open-mpi.org/svn/hwloc/trunk

while ("$r" != "")
echo merging in r$r...
set r_minus_one=`expr $r - 1`
set str="svn merge -r ${r_minus_one}:$r $otrunk ."
echo $str
eval $str
if ($status != "0") then
echo merge failed -- exiting
exit 1
endif

if ("$argv" != "") then
shift
set r=$1
else
set r=
endif
end


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] [hwloc] #16: hwloc build fails with strict compiler flags

[Jeff Squyres]

Cool; I filed #18 and #19 about these.  I'll fix the "mixed  
declarations and code" issue right now.



On Oct 21, 2009, at 1:45 PM, Pavan Balaji wrote:



On 10/21/2009 11:59 AM, Jeff Squyres wrote:
> Can you send a copy of your stderr?  I don't get the warnings about
> fgets output not getting checked.

Here you go:

libtool: compile:  gcc -DHAVE_CONFIG_H -I.
-I/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src
-I../include/private -I../include/hwloc
-I/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/ 
include
-I../include -std=c99 -Wall -Wmissing-prototypes -Wundef -Wpointer- 
arith

-Wcast-align -O2 -Wall -Wextra -Wno-missing-field-initializers
-Wstrict-prototypes -Wmissing-prototypes -DGCC_WALL
-Wno-unused-parameter -Wno-unused-label -Wshadow -Wmissing- 
declarations

-Wno-long-long -Wfloat-equal -Wdeclaration-after-statement -Wundef
-Wno-endif-labels -Wpointer-arith -Wbad-function-cast -Wcast-align
-Wwrite-strings -Wno-sign-compare -Waggregate-return
-Wold-style-definition -Wno-multichar -Wno-deprecated-declarations
-Wpacked -Wnested-externs -Winvalid-pch -Wno-pointer-sign
-Wvariadic-macros -std=c89 -Wno-format-zero-length -Wno-type-limits
-D_POSIX_C_SOURCE=199506L -MT cpuset.lo -MD -MP -MF .deps/cpuset.Tpo  
-c
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
cpuset.c

-o cpuset.o >/dev/null 2>&1
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
cpuset.c:

In function 'hwloc_snprintf':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
cpuset.c:37:

warning: implicit declaration of function 'vsnprintf'
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
cpuset.c:37:

warning: nested extern declaration of 'vsnprintf'
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:

In function 'hwloc_linux_set_thread_cpubind':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:170:

warning: ISO C90 forbids mixed declarations and code
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:

In function 'hwloc_parse_sysfs_unsigned':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:241:

warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:

In function 'hwloc_read_cpuset_mask':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:326:

warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:346:

warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:

In function 'look_cpuinfo':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:863:

warning: ignoring return value of 'fscanf', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:

In function 'hwloc__get_dmi_info':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:917:

warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/ 
topology-linux.c:931:

warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result

 -- Pavan

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

[hwloc-devel] Trac ticket mails

[Jeff Squyres]

The IU sysadmins fixed something with trac today such that we should  
now get mails for trac ticket actions (to the hwloc-bugs list).


Also, FWIW, I also follow the Trac RSS feed, which shows commits,  
ticket actions, and wiki changes (customize the URL parameters to your  
liking):


https://svn.open-mpi.org/trac/hwloc/timeline?ticket=on&changeset=on&milestone=on&wiki=on&max=50&daysback=90&format=rss

--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] MPICH2 question

[Jeff Squyres]

On Oct 21, 2009, at 1:18 PM, Samuel Thibault wrote:


> configuring/building hwloc with icc results in a *lot*
> of warnings; I didn't test functionality).

Mmm, I have already successfully tested with icc9.

The kind of warnings I've seen were not worth fixing to my mind:  
unused

parameters, mostly, are you using some particular option?




I didn't look closely at what the warnings were -- I just saw a lot of  
them go by.  I can look closer.


I didn't use any particular configure options -- just --prefix.  This  
was on RHEL4 using icc 11.1.056 (I just updated my intel compilers  
this morning to the latest latest latest).


(/me looks closer)

Wow, you're right -- there's *truckloads* of variable-never-referenced  
and variable-set-but-not-used warnings.  Those should be cleaned up  
IMHO, but they can wait until after 0.9.1, also IMHO.  I'll file a  
ticket against 1.0.


My pathscale compiler license has expired, and the license server for  
my PGI license has gone down -- so I can't check any other compilers  
at the moment.


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] [hwloc] #16: hwloc build fails with strict compiler flags

[Pavan Balaji]

On 10/21/2009 11:59 AM, Jeff Squyres wrote:
> Can you send a copy of your stderr?  I don't get the warnings about
> fgets output not getting checked.

Here you go:

libtool: compile:  gcc -DHAVE_CONFIG_H -I.
-I/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src
-I../include/private -I../include/hwloc
-I/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/include
-I../include -std=c99 -Wall -Wmissing-prototypes -Wundef -Wpointer-arith
-Wcast-align -O2 -Wall -Wextra -Wno-missing-field-initializers
-Wstrict-prototypes -Wmissing-prototypes -DGCC_WALL
-Wno-unused-parameter -Wno-unused-label -Wshadow -Wmissing-declarations
-Wno-long-long -Wfloat-equal -Wdeclaration-after-statement -Wundef
-Wno-endif-labels -Wpointer-arith -Wbad-function-cast -Wcast-align
-Wwrite-strings -Wno-sign-compare -Waggregate-return
-Wold-style-definition -Wno-multichar -Wno-deprecated-declarations
-Wpacked -Wnested-externs -Winvalid-pch -Wno-pointer-sign
-Wvariadic-macros -std=c89 -Wno-format-zero-length -Wno-type-limits
-D_POSIX_C_SOURCE=199506L -MT cpuset.lo -MD -MP -MF .deps/cpuset.Tpo -c
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/cpuset.c
-o cpuset.o >/dev/null 2>&1
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/cpuset.c:
In function 'hwloc_snprintf':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/cpuset.c:37:
warning: implicit declaration of function 'vsnprintf'
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/cpuset.c:37:
warning: nested extern declaration of 'vsnprintf'
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:
In function 'hwloc_linux_set_thread_cpubind':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:170:
warning: ISO C90 forbids mixed declarations and code
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:
In function 'hwloc_parse_sysfs_unsigned':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:241:
warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:
In function 'hwloc_read_cpuset_mask':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:326:
warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:346:
warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:
In function 'look_cpuinfo':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:863:
warning: ignoring return value of 'fscanf', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:
In function 'hwloc__get_dmi_info':
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:917:
warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result
/home/balaji/projects/mpich2/hydra/hydra/tools/bind/hwloc/hwloc/src/topology-linux.c:931:
warning: ignoring return value of 'fgets', declared with attribute
warn_unused_result

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] MPICH2 question

[Pavan Balaji]

> Pavan -- is it a problem to always compile hwloc with gcc?

Yeah, this will be a problem. For us to enable hwloc by default, it'll
need to build with all compilers and on all platforms without errors
(and hopefully without warnings either).

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Jeff Squyres, le Wed 21 Oct 2009 12:50:55 -0400, a écrit :
> configuring/building hwloc with icc results in a *lot*  
> of warnings; I didn't test functionality).

Mmm, I have already successfully tested with icc9.

The kind of warnings I've seen were not worth fixing to my mind: unused
parameters, mostly, are you using some particular option?

Samuel

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Jeff Squyres, le Wed 21 Oct 2009 12:46:38 -0400, a écrit :
> I'd be surprised if there's a system out there that doesn't have some  
> flavor of egrep that satisfies AC_PROG_EGREP (especially if Libtool  
> uses it heavily).  Do we know if this is the case, or is this a  
> hypothetical that a suitable egrep might not be found?

Well, all I can say is that I've had no problem on the tested OSes, i.e.
notably solaris, AIX, HP-UX and OSF (without the GNU tools).

Samuel

Re: [hwloc-devel] [hwloc] #16: hwloc build fails with strict compiler flags

[Jeff Squyres]

Pavan --

Can you send a copy of your stderr?  I don't get the warnings about  
fgets output not getting checked.


Thanks!


On Oct 21, 2009, at 9:39 AM, hwloc wrote:


#16: hwloc build fails with strict compiler flags
--- 
+

Reporter:  balaji  |Owner:
Type:  defect  |   Status:  closed
Priority:  normal  |Milestone:  v0.9.1
 Version:  |   Resolution:  fixed
Keywords:  |
--- 
+


Comment(by balaji):

 The vsnprintf warnings occur because snprintf and vsnprintf are  
present

 only in C99, not C89. There are a few solutions possible:

 1. Check in configure to (i) add a prototype for snprintf/vsnprintf  
where
 needed and (ii) add an alternative implementation for them for  
platforms

 that don't provide them.

 2. An alternative (simpler) solution is to include MPL
 (https://svn.mcs.anl.gov/repos/mpi/mpich2/trunk/src/mpl) into hwloc  
and

 just use MPL_snprintf and friends everywhere.

 3. Check if snprintf/vsnprintf exist in configure and abort if they  
don't.
 Other libraries relying on hwloc can see this error and not build  
hwloc in

 those cases.

 Not sure if either approach is acceptable for you guys, so I'm  
leaving

 this ticket as closed. Please reopen if appropriate.

 Btw, there are some other warnings too because the return values of  
fgets

 and fscanf are not checked. But those are relatively minor, IMHO.

--
Ticket URL: 
hwloc 
Portable hardware locality




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] MPICH2 question

[Jeff Squyres]

FWIW, I see at least some GNU-isms in the hwloc code that might be  
problematic for embedding hwloc in other code bases that don't use gcc  
to compile.  E.g., in OMPI, we'd prefer to use the same compiler suite  
to compile hwloc that was used to compile OMPI itself (e.g., intel,  
PGI, ...etc. -- configuring/building hwloc with icc results in a *lot*  
of warnings; I didn't test functionality).


I don't know if we want to target that for v0.9.1 or not -- my first  
reaction is to postpone this to v1.0 in order to get v0.9.1 out the  
door.  Fully bake in all the embedding -- including cleaning up any  
potential GNU-isms -- for v1.0.


Pavan -- is it a problem to always compile hwloc with gcc?


On Oct 21, 2009, at 11:39 AM, Pavan Balaji wrote:



On 10/21/2009 10:38 AM, Samuel Thibault wrote:
> Pavan Balaji, le Wed 21 Oct 2009 10:36:33 -0500, a écrit :
>> On 10/21/2009 10:28 AM, Samuel Thibault wrote:
>>> Pavan Balaji, le Wed 21 Oct 2009 09:55:36 -0500, a écrit :
 1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
 compiler to be C99 capable always?
>>> No, we ended up using constructs which should pass c90 and the  
compilers

>>> we have tested (aix, solaris, icc).
>> So shouldn't the AC_PROG_CC_C99 be gotten rid of?
>
> No because when C99 is available, we enable some optimization  
features,

> like __hwloc_restrict.

Gotcha! Thanks for the clarification.

 -- Pavan

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] MPICH2 question

[Jeff Squyres]

On Oct 21, 2009, at 10:20 AM, Samuel Thibault wrote:


I've checked configure, only the check for egrep may fail and does not
provide any fallback which we could have used. It's only used for the
documentation generation, Jeff, maybe we can find an alternative to
egrep for what we use it for?




We use it for running latex (i.e., looking for some warning messages  
in the latex output to know if we need to run it again).  This could  
be coded around and/or removed entirely if desired.


However, I notice that libtool makes extensive use of egrep.   
Specifically, even if I remove AC_PROG_EGREP, it's checked for later  
in configure and is used a lot by libtool.


I'd be surprised if there's a system out there that doesn't have some  
flavor of egrep that satisfies AC_PROG_EGREP (especially if Libtool  
uses it heavily).  Do we know if this is the case, or is this a  
hypothetical that a suitable egrep might not be found?


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Jeff Squyres]

On Oct 21, 2009, at 10:26 AM, Samuel Thibault wrote:

> What I meant is that *our* ibverbs.h code is Linux specific (it  
uses a

> sysfs specific nice feature of OFED/Linux).

Can't we just make it a real function and not an inline?  As it is now
it will not include the linux version in applications since LINUX_SYS
won't be defined.




Yep; I have no objection to this.  I don't see much of a need for this  
function to be inlined.


Do you want to do it?

--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] MPICH2 question

[Pavan Balaji]

On 10/21/2009 10:38 AM, Samuel Thibault wrote:
> Pavan Balaji, le Wed 21 Oct 2009 10:36:33 -0500, a écrit :
>> On 10/21/2009 10:28 AM, Samuel Thibault wrote:
>>> Pavan Balaji, le Wed 21 Oct 2009 09:55:36 -0500, a écrit :
 1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
 compiler to be C99 capable always?
>>> No, we ended up using constructs which should pass c90 and the compilers
>>> we have tested (aix, solaris, icc).
>> So shouldn't the AC_PROG_CC_C99 be gotten rid of?
> 
> No because when C99 is available, we enable some optimization features,
> like __hwloc_restrict.

Gotcha! Thanks for the clarification.

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Pavan Balaji, le Wed 21 Oct 2009 10:36:33 -0500, a écrit :
> 
> On 10/21/2009 10:28 AM, Samuel Thibault wrote:
> > Pavan Balaji, le Wed 21 Oct 2009 09:55:36 -0500, a écrit :
> >> 1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
> >> compiler to be C99 capable always?
> > 
> > No, we ended up using constructs which should pass c90 and the compilers
> > we have tested (aix, solaris, icc).
> 
> So shouldn't the AC_PROG_CC_C99 be gotten rid of?

No because when C99 is available, we enable some optimization features,
like __hwloc_restrict.

Samuel

Re: [hwloc-devel] MPICH2 question

[Pavan Balaji]

On 10/21/2009 10:28 AM, Samuel Thibault wrote:
> Pavan Balaji, le Wed 21 Oct 2009 09:55:36 -0500, a écrit :
>> 1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
>> compiler to be C99 capable always?
> 
> No, we ended up using constructs which should pass c90 and the compilers
> we have tested (aix, solaris, icc).

So shouldn't the AC_PROG_CC_C99 be gotten rid of?

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Pavan Balaji, le Wed 21 Oct 2009 09:55:36 -0500, a écrit :
> 1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
> compiler to be C99 capable always?

No, we ended up using constructs which should pass c90 and the compilers
we have tested (aix, solaris, icc).

> 2. I believe AM_CONDITIONAL is automake-1.11 specific.

I would be very surprised as it's a very basic operation. It is in my
1.9 manual at least so it should be fine.

Thanks for the extra review,
Samuel

Re: [hwloc-devel] MPICH2 question

[Pavan Balaji]

 I am not sure how hard it'd be to avoid errors during configure. Are we
 sure PKG_* macros or other external things will never use AC_MSG_ERROR ?
>>> In principle ac macros always have an "what if not found" part which
>>> allows us to fallback nicely.
>> Note that if you decide to take this approach, it is important that
>> neither the configure nor the make fail.
> 
> I've checked configure, only the check for egrep may fail and does not
> provide any fallback which we could have used. It's only used for the
> documentation generation, Jeff, maybe we can find an alternative to
> egrep for what we use it for?

I made a quick pass as well. Here are some comments:

1. I see a AC_PROG_CC_C99 in the configure.ac. Do you require the
compiler to be C99 capable always? If yes, then you might want to check
the return value $ac_cv_prog_cc_c99 and do something with it (maybe abort).

2. I believe AM_CONDITIONAL is automake-1.11 specific. Can someone
verify this? If this is correct, then your AM_INIT_AUTOMAKE should
contain 1.11 as a prereq.

> make is supposed to always succeed (unless bugs of course :) )

Great! Thanks.

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Samuel Thibault]

Brice Goglin, le Wed 21 Oct 2009 14:10:21 +0200, a écrit :
> What I meant is that *our* ibverbs.h code is Linux specific (it uses a
> sysfs specific nice feature of OFED/Linux).

Can't we just make it a real function and not an inline?  As it is now
it will not include the linux version in applications since LINUX_SYS
won't be defined.

Samuel

Re: [hwloc-devel] MPICH2 question

[Samuel Thibault]

Pavan Balaji, le Tue 20 Oct 2009 20:19:59 -0500, a écrit :
> >> I am not sure how hard it'd be to avoid errors during configure. Are we
> >> sure PKG_* macros or other external things will never use AC_MSG_ERROR ?
> > 
> > In principle ac macros always have an "what if not found" part which
> > allows us to fallback nicely.
> 
> Note that if you decide to take this approach, it is important that
> neither the configure nor the make fail.

I've checked configure, only the check for egrep may fail and does not
provide any fallback which we could have used. It's only used for the
documentation generation, Jeff, maybe we can find an alternative to
egrep for what we use it for?

make is supposed to always succeed (unless bugs of course :) )

Samuel

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Jeff Squyres]

Ok, finally done.  It was somewhat painful because I also discovered  
that doxygen man pages weren't being generated for openfabrics-verbs.h  
and linux.h properly, so I fixed that as well.


Note that I also renamed hwloc_ibverbs_get_device_cpuset(),  
hwloc_ibv_get_device_cpuset() -- might as well use the same prefix as  
the verbs API.  It'll be recognizable to verbs programmers that way.



On Oct 21, 2009, at 8:56 AM, Brice Goglin wrote:


Jeff Squyres wrote:
> On Oct 21, 2009, at 8:41 AM, Jeff Squyres (jsquyres) wrote:
>
>> > Or, we could install it only on Linux for now? (and document  
this in

>> > the
>> > doxyfile)
>>
>
>
> Actually, a colleague from Sun tells me that Sun's verbs stack is
> available.
>
> What should we return from hwloc_ibv_get_device_cpuset() if not  
#ifdef

> LINUX?


I'd say a full cpuset.
Or the full-system cpuset:
   return hwloc_cpuset_dup(hwloc_get_system_obj(topology)->cpuset);
If the latter we'll need to add a topology parameter to
hwloc_ibv_get_device_cpuset()

Brice

___
hwloc-devel mailing list
hwloc-de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel




--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Brice Goglin]

Jeff Squyres wrote:
> On Oct 21, 2009, at 8:41 AM, Jeff Squyres (jsquyres) wrote:
>
>> > Or, we could install it only on Linux for now? (and document this in
>> > the
>> > doxyfile)
>>
>
>
> Actually, a colleague from Sun tells me that Sun's verbs stack is
> available.
>
> What should we return from hwloc_ibv_get_device_cpuset() if not #ifdef
> LINUX?


I'd say a full cpuset.
Or the full-system cpuset:
   return hwloc_cpuset_dup(hwloc_get_system_obj(topology)->cpuset);
If the latter we'll need to add a topology parameter to
hwloc_ibv_get_device_cpuset()

Brice

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Jeff Squyres]

On Oct 21, 2009, at 8:41 AM, Jeff Squyres (jsquyres) wrote:


> Or, we could install it only on Linux for now? (and document this in
> the
> doxyfile)




Actually, a colleague from Sun tells me that Sun's verbs stack is  
available.


What should we return from hwloc_ibv_get_device_cpuset() if not #ifdef  
LINUX?


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Jeff Squyres]

On Oct 21, 2009, at 8:23 AM, Brice Goglin wrote:


> So I think the name would be better as of-verbs.h, or
> openfabrics-verbs.h, or ...

I'd vote for or openfabrics-verbs.h then :)



Ok.  Sorry for all the lengthy explanations -- these brands/names are  
not well differentiated to the common observer (IMHO).  :-(


I'll do the grunt work for the renaming.


> Fair enough -- I have no problems #if LINUX'ing it for this release.
> Mebbe someday if/when verbs is officially released on Solaris and
> Windows and if we get adventurous enough, we can test on those
> platforms and remove the #if LINUX protection.  Cool?

Or, we could install it only on Linux for now? (and document this in  
the

doxyfile)




Ok.  I'll add a test for that.  We're safe for the moment, because I  
don't think Sun has yet released their verbs stack publicly, nor do I  
think the Windows verbs effort is ready yet, either (but I really only  
very very loosely track these efforts, so I could be totally wrong).   
It'll help future-proof us for when these plaforms' verbs stacks are  
available -- we'll know to do proper testing at that point.


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Brice Goglin]

Jeff Squyres wrote:
> On Oct 21, 2009, at 8:10 AM, Brice Goglin wrote:
>
>> Ok :) I think I'd vote for some like ofed-verbs.h then, it'd match the
>> existing glibc-sched.h and linux-libnuma.h
>>
>
> Heh.  This is even more branding confusion that I think OpenFabrics as
> an organization has not done well to clarify...
>
> OpenFabrics Enterprise Distribution (OFED) is a) currently Linux-only,
> and b) is a combined release mechanism / software package for several
> different upper-level protocols (verbs, UDAPL, MPI, etc.) largely
> employing RDMA-based technologies.
>
> The "OFED" software is *not* the same thing as the verbs stack -- it
> *includes* the verbs stack.  For example, various Linux distros
> include the verbs stack and some other RDMA-based packages (such as
> Open MPI) but do *not* include OFED itself.  OFED is basically
> packaging and an installer/uninstaller for a whole bunch of
> OpenFabrics-sponsored/related software packages.
>
> For example, if you use the verbs stack on a RHEL distribution, it's
> not OFED.  It's the verbs that is packaged by RedHat.  Same with
> SuSE.  Same with Debian.  Same with ...etc.
>
> So I think the name would be better as of-verbs.h, or
> openfabrics-verbs.h, or ...

I'd vote for or openfabrics-verbs.h then :)

>> If we want to keep this file
>> portable, we'll need to port hwloc_ibverbs_get_device_cpuset() to
>> non-Linux OS one day, which means we need a #ifdef LINUX in this public
>> header. However, IIRC our #define LINUX_SYS is internal only so far.
>>
>
>
> Fair enough -- I have no problems #if LINUX'ing it for this release. 
> Mebbe someday if/when verbs is officially released on Solaris and
> Windows and if we get adventurous enough, we can test on those
> platforms and remove the #if LINUX protection.  Cool?

Or, we could install it only on Linux for now? (and document this in the
doxyfile)

Brice

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Jeff Squyres]

On Oct 21, 2009, at 8:10 AM, Brice Goglin wrote:


Ok :) I think I'd vote for some like ofed-verbs.h then, it'd match the
existing glibc-sched.h and linux-libnuma.h



Heh.  This is even more branding confusion that I think OpenFabrics as  
an organization has not done well to clarify...


OpenFabrics Enterprise Distribution (OFED) is a) currently Linux-only,  
and b) is a combined release mechanism / software package for several  
different upper-level protocols (verbs, UDAPL, MPI, etc.) largely  
employing RDMA-based technologies.


The "OFED" software is *not* the same thing as the verbs stack -- it  
*includes* the verbs stack.  For example, various Linux distros  
include the verbs stack and some other RDMA-based packages (such as  
Open MPI) but do *not* include OFED itself.  OFED is basically  
packaging and an installer/uninstaller for a whole bunch of  
OpenFabrics-sponsored/related software packages.


For example, if you use the verbs stack on a RHEL distribution, it's  
not OFED.  It's the verbs that is packaged by RedHat.  Same with  
SuSE.  Same with Debian.  Same with ...etc.


So I think the name would be better as of-verbs.h, or openfabrics- 
verbs.h, or ...



I thought verbs already existed on more than Linux actually. What I
meant is that *our* ibverbs.h code is Linux specific (it uses a sysfs
specific nice feature of OFED/Linux).



Ah -- I misunderstood.


If we want to keep this file
portable, we'll need to port hwloc_ibverbs_get_device_cpuset() to
non-Linux OS one day, which means we need a #ifdef LINUX in this  
public

header. However, IIRC our #define LINUX_SYS is internal only so far.




Fair enough -- I have no problems #if LINUX'ing it for this release.   
Mebbe someday if/when verbs is officially released on Solaris and  
Windows and if we get adventurous enough, we can test on those  
platforms and remove the #if LINUX protection.  Cool?


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Brice Goglin]

Jeff Squyres wrote:
> FWIW: the name has been OpenFabrics / OFED for a few years now.  Not
> that I wholly disagree -- we're still stuck with the "openib" module
> name in Open MPI because we named it several years ago when it was
> called OpenIB -- but I think the "OpenFabrics" name is pretty stable. 
> OpenIB was an informal name that existed before there was an official
> organization behind it.  OpenFabrics is the legal entity that was
> created to support all things related to this technology, so I don't
> think that name will be changing any time soon.  Indeed, there's a
> *lot* of money put into the marketing and branding with the name
> "OpenFabrics".
>
> There hasn't [yet?] been discussion of renaming 
> (or some of the other IB-centric struct/symbol names), but the whole
> package is very definitely marketed as "OpenFabrics verbs", not
> "InfiniBand verbs" (although the IB vendors certainly don't correct
> this misconception ;-) ).
>
> So I still would like to rename this file before release.

Ok :) I think I'd vote for some like ofed-verbs.h then, it'd match the
existing glibc-sched.h and linux-libnuma.h

> By the way, this file actually only works for Linux so far. Unless we
>
>
> Sun is porting the OpenFabrics verbs to Solaris (to replace their DAPL
> stack).  There is also talk of porting the verbs API to MS Windows,
> although I'm not tracking that effort at all.  If all this comes to
> fruition, it'll be 3 different platforms that expose the same verbs API.

I thought verbs already existed on more than Linux actually. What I
meant is that *our* ibverbs.h code is Linux specific (it uses a sysfs
specific nice feature of OFED/Linux). If we want to keep this file
portable, we'll need to port hwloc_ibverbs_get_device_cpuset() to
non-Linux OS one day, which means we need a #ifdef LINUX in this public
header. However, IIRC our #define LINUX_SYS is internal only so far.

Brice

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Jeff Squyres]

On Oct 21, 2009, at 1:30 AM, Brice Goglin wrote:


I named it ibverbs.h because it only works with infiniband/verbs.h
anyway. People will need the later to use it. That's why I like
ibverbs.h (or infiniband-verbs.h but it's very long). Apart from  
that, I

don't really care. At least the Infiniband name still exists, while
OpenIB/OFED/... is renamed almost every year :)



Hah!  It's actually InfiniBand(tm).  :-)

FWIW: the name has been OpenFabrics / OFED for a few years now.  Not  
that I wholly disagree -- we're still stuck with the "openib" module  
name in Open MPI because we named it several years ago when it was  
called OpenIB -- but I think the "OpenFabrics" name is pretty stable.   
OpenIB was an informal name that existed before there was an official  
organization behind it.  OpenFabrics is the legal entity that was  
created to support all things related to this technology, so I don't  
think that name will be changing any time soon.  Indeed, there's a  
*lot* of money put into the marketing and branding with the name  
"OpenFabrics".


There hasn't [yet?] been discussion of renaming   
(or some of the other IB-centric struct/symbol names), but the whole  
package is very definitely marketed as "OpenFabrics verbs", not  
"InfiniBand verbs" (although the IB vendors certainly don't correct  
this misconception ;-) ).


So I still would like to rename this file before release.


By the way, this file actually only works for Linux so far. Unless we
are sure we could make it work for non-Linux OS one day (might need  
the

#ifdef LINUX to work in public headers), we could rename it to
hwloc/linux-...verbs.h




Sun is porting the OpenFabrics verbs to Solaris (to replace their DAPL  
stack).  There is also talk of porting the verbs API to MS Windows,  
although I'm not tracking that effort at all.  If all this comes to  
fruition, it'll be 3 different platforms that expose the same verbs API.


--
Jeff Squyres
jsquy...@cisco.com

Re: [hwloc-devel] ibverbs -> not just infiniband!

[Brice Goglin]

Jeff Squyres wrote:
> I know this is somewhat pedantic, but I'd prefer to rename "ibverbs*"
> to something else.  Technically, it's OpenFabrics verbs -- not
> infiniband verbs -- which also includes at least 2 forms of ethernet
> (iWARP and RDMAoE).
>
> (working for an Ethernet-based company, I kinda have to say that :-) )
>
> Does anyone have any objections to me renaming this stuff?  I'm open
> to suggestions on names -- e.g., ofverbs.h, or verbs.h, or
> openfabrics-verbs.h, or some-other-even-longer-name-verbs.h, or ...?  
> :-)

I named it ibverbs.h because it only works with infiniband/verbs.h
anyway. People will need the later to use it. That's why I like
ibverbs.h (or infiniband-verbs.h but it's very long). Apart from that, I
don't really care. At least the Infiniband name still exists, while
OpenIB/OFED/... is renamed almost every year :)

By the way, this file actually only works for Linux so far. Unless we
are sure we could make it work for non-Linux OS one day (might need the
#ifdef LINUX to work in public headers), we could rename it to
hwloc/linux-...verbs.h

Brice