[hwloc-devel] Create success (hwloc r1.4.3a1r4608)

[MPI Team]

Creating nightly hwloc snapshot SVN tarball was a success.

Snapshot:   hwloc 1.4.3a1r4608
Start time: Mon Jul  9 21:04:38 EDT 2012
End time:   Mon Jul  9 21:07:36 EDT 2012

Your friendly daemon,
Cyrador

Re: [OMPI devel] RFC: enable the use of source in platform files

[Ralph Castain]

I keep scratching my head over this, and I just cannot figure out how this is 
going to do what you think. We "source" the platform file solely to execute any 
envar settings that are in it - i.e., to capture the CFLAGS=... and other such 
directives. We then read/parse the platform file to get all the configure 
directives - e.g., enable_debug=yes. Sourcing the platform file will set the 
envars, but won't capture the rest of the directives.

Am I missing something here? It doesn't sound like you've really even tried 
this yet - sure, chaining "source" commands will work, but do you actually get 
the desired configuration??

Hence my comment about needing to modify the parser so it ALSO follows the 
"source" directive.

On Jul 9, 2012, at 3:58 PM, Nathan Hjelm wrote:

> On Mon, Jul 09, 2012 at 03:31:33PM -0700, Ralph Castain wrote:
>> So if I understand this right, you would have multiple platform files, each 
>> "sourcing" a common one that contains the base directives? It sounds to me 
>> like you need more than the change below to make that work - you would need 
>> to interpret the platform file itself to read and execute a "source" 
>> directive inside it, wouldn't you?
> 
> That is exactly what I want to do. The change in the RFC is the only one 
> needed as platform files are sourced by ompi_load_platform.m4. This means 
> platforms can contain arbitrary m4/shell code (including the source 
> directive)! I tried my patch with a one line platform file that sourced an 
> existing platform file and it worked as expected.
> 
>> It would really help if your change (either comments or the RFC) actually 
>> explained what the heck you are doing so I wouldn't have to waste hours 
>> trying to figure out the impact of this patch :-/
>> 
> 
> The RFC does explain what the patch does but I guess I could have elaborated 
> on the implications.
> 
> Before the patch we source the platform file then cd into the platform 
> directory to find the mca parameters file. If a platform file were to have a 
> source directive then it would have to be relative to the build directory (or 
> absolute). By cding into the platform file's directory before sourcing the 
> platform file and source directives are relative to the platform file's 
> directory (or absolute). There is no impact outside of m4/shell commands 
> within the platform file that read/write/stat files.
> 
> I will add some additional comments before the commit (if there are no 
> objects of course) elaborating on the change.
> 
> -Nathan
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] RFC: enable the use of source in platform files

[Ralph Castain]

This is what was missing from your RFC - a simple explanation other than a 
one-line "How" or "Why". I know it takes five minutes to be more descriptive, 
but you've been getting awfully terse lately with these proposals.

Now that you have explained it, I have no objections.


On Jul 9, 2012, at 3:58 PM, Nathan Hjelm wrote:

> Before the patch we source the platform file then cd into the platform 
> directory to find the mca parameters file. If a platform file were to have a 
> source directive then it would have to be relative to the build directory (or 
> absolute). By cding into the platform file's directory before sourcing the 
> platform file and source directives are relative to the platform file's 
> directory (or absolute). There is no impact outside of m4/shell commands 
> within the platform file that read/write/stat files.

Re: [OMPI devel] RFC: enable the use of source in platform files

[Nathan Hjelm]

On Mon, Jul 09, 2012 at 03:31:33PM -0700, Ralph Castain wrote:
> So if I understand this right, you would have multiple platform files, each 
> "sourcing" a common one that contains the base directives? It sounds to me 
> like you need more than the change below to make that work - you would need 
> to interpret the platform file itself to read and execute a "source" 
> directive inside it, wouldn't you?

That is exactly what I want to do. The change in the RFC is the only one needed 
as platform files are sourced by ompi_load_platform.m4. This means platforms 
can contain arbitrary m4/shell code (including the source directive)! I tried 
my patch with a one line platform file that sourced an existing platform file 
and it worked as expected.

> It would really help if your change (either comments or the RFC) actually 
> explained what the heck you are doing so I wouldn't have to waste hours 
> trying to figure out the impact of this patch :-/
> 

The RFC does explain what the patch does but I guess I could have elaborated on 
the implications.

Before the patch we source the platform file then cd into the platform 
directory to find the mca parameters file. If a platform file were to have a 
source directive then it would have to be relative to the build directory (or 
absolute). By cding into the platform file's directory before sourcing the 
platform file and source directives are relative to the platform file's 
directory (or absolute). There is no impact outside of m4/shell commands within 
the platform file that read/write/stat files.

I will add some additional comments before the commit (if there are no objects 
of course) elaborating on the change.

-Nathan

Re: [OMPI devel] RFC: enable the use of source in platform files

[Ralph Castain]

So if I understand this right, you would have multiple platform files, each 
"sourcing" a common one that contains the base directives? It sounds to me like 
you need more than the change below to make that work - you would need to 
interpret the platform file itself to read and execute a "source" directive 
inside it, wouldn't you?

It would really help if your change (either comments or the RFC) actually 
explained what the heck you are doing so I wouldn't have to waste hours trying 
to figure out the impact of this patch :-/


On Jul 9, 2012, at 2:25 PM, Nathan Hjelm wrote:

> LANL's platform file's (for a particular platform) are 90% the same. By 
> allowing a source directive those settings can be kept in a common file then 
> sourced by the platform file. When something changes I wouldn't have to 
> change a dozen files only a couple.
> 
> -Nathan
> 
> On Mon, Jul 09, 2012 at 02:21:28PM -0700, Ralph Castain wrote:
>> I'm confused - how does this help maintain a platform file???
>> 
>> 
>> On Jul 9, 2012, at 2:09 PM, Nathan Hjelm wrote:
>> 
>>> When: Thurs, Jul 12, 5 PM MDT
>>> 
>>> Why: Useful feature. Enabling source may cut down on the maintenance 
>>> required to keep platform files up to date.
>>> 
>>> How: Change directories to the platform file's directory before sourcing it 
>>> (not after).
>>> 
>>> diff --git a/config/ompi_load_platform.m4 b/config/ompi_load_platform.m4
>>> index 7231f83..250e7d8 100644
>>> --- a/config/ompi_load_platform.m4
>>> +++ b/config/ompi_load_platform.m4
>>> @@ -50,7 +50,18 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [
>>># eval into environment
>>>OPAL_LOG_MSG([Loading environment file $with_platform, with contents 
>>> below])
>>>OPAL_LOG_FILE([$with_platform])
>>> -. "$with_platform"
>>> +
>>> +# setup by getting full pathname for the platform directories
>>> +platform_base="`dirname $with_platform`"
>>> +platform_file="`basename $with_platform`"
>>> +# get full pathname of where we are so we can return
>>> +platform_savedir="`pwd`"
>>> +# go to where the platform file is located
>>> +cd "$platform_base"
>>> +# get the full path to this location
>>> +platform_file_dir=`pwd`
>>> +
>>> +. "$platform_file"
>>> 
>>># see if they left us a name
>>>if test "$OMPI_PLATFORM_LOADED" != "" ; then
>>> @@ -63,14 +74,6 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [
>>> 
>>># look for default mca param file
>>> 
>>> -# setup by getting full pathname for the platform directories
>>> -platform_base="`dirname $with_platform`"
>>> -# get full pathname of where we are so we can return
>>> -platform_savedir="`pwd`"
>>> -# go to where the platform file is located
>>> -cd "$platform_base"
>>> -# get the full path to this location
>>> -platform_file_dir=`pwd`
>>># return to where we started
>>>cd "$platform_savedir"
>>> 
>>> 
>>> -Nathan
>>> ___
>>> devel mailing list
>>> de...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> 
>> 
>> ___
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] RFC: enable the use of source in platform files

[Nathan Hjelm]

LANL's platform file's (for a particular platform) are 90% the same. By 
allowing a source directive those settings can be kept in a common file then 
sourced by the platform file. When something changes I wouldn't have to change 
a dozen files only a couple.

-Nathan

On Mon, Jul 09, 2012 at 02:21:28PM -0700, Ralph Castain wrote:
> I'm confused - how does this help maintain a platform file???
> 
> 
> On Jul 9, 2012, at 2:09 PM, Nathan Hjelm wrote:
> 
> > When: Thurs, Jul 12, 5 PM MDT
> > 
> > Why: Useful feature. Enabling source may cut down on the maintenance 
> > required to keep platform files up to date.
> > 
> > How: Change directories to the platform file's directory before sourcing it 
> > (not after).
> > 
> > diff --git a/config/ompi_load_platform.m4 b/config/ompi_load_platform.m4
> > index 7231f83..250e7d8 100644
> > --- a/config/ompi_load_platform.m4
> > +++ b/config/ompi_load_platform.m4
> > @@ -50,7 +50,18 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [
> > # eval into environment
> > OPAL_LOG_MSG([Loading environment file $with_platform, with 
> > contents below])
> > OPAL_LOG_FILE([$with_platform])
> > -. "$with_platform"
> > +
> > +# setup by getting full pathname for the platform directories
> > +platform_base="`dirname $with_platform`"
> > +platform_file="`basename $with_platform`"
> > +# get full pathname of where we are so we can return
> > +platform_savedir="`pwd`"
> > +# go to where the platform file is located
> > +cd "$platform_base"
> > +# get the full path to this location
> > +platform_file_dir=`pwd`
> > +
> > +. "$platform_file"
> > 
> > # see if they left us a name
> > if test "$OMPI_PLATFORM_LOADED" != "" ; then
> > @@ -63,14 +74,6 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [
> > 
> > # look for default mca param file
> > 
> > -# setup by getting full pathname for the platform directories
> > -platform_base="`dirname $with_platform`"
> > -# get full pathname of where we are so we can return
> > -platform_savedir="`pwd`"
> > -# go to where the platform file is located
> > -cd "$platform_base"
> > -# get the full path to this location
> > -platform_file_dir=`pwd`
> > # return to where we started
> > cd "$platform_savedir"
> > 
> > 
> > -Nathan
> > ___
> > devel mailing list
> > de...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
> 
> 
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] RFC: enable the use of source in platform files

[Ralph Castain]

I'm confused - how does this help maintain a platform file???


On Jul 9, 2012, at 2:09 PM, Nathan Hjelm wrote:

> When: Thurs, Jul 12, 5 PM MDT
> 
> Why: Useful feature. Enabling source may cut down on the maintenance required 
> to keep platform files up to date.
> 
> How: Change directories to the platform file's directory before sourcing it 
> (not after).
> 
> diff --git a/config/ompi_load_platform.m4 b/config/ompi_load_platform.m4
> index 7231f83..250e7d8 100644
> --- a/config/ompi_load_platform.m4
> +++ b/config/ompi_load_platform.m4
> @@ -50,7 +50,18 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [
> # eval into environment
> OPAL_LOG_MSG([Loading environment file $with_platform, with contents 
> below])
> OPAL_LOG_FILE([$with_platform])
> -. "$with_platform"
> +
> +# setup by getting full pathname for the platform directories
> +platform_base="`dirname $with_platform`"
> +platform_file="`basename $with_platform`"
> +# get full pathname of where we are so we can return
> +platform_savedir="`pwd`"
> +# go to where the platform file is located
> +cd "$platform_base"
> +# get the full path to this location
> +platform_file_dir=`pwd`
> +
> +. "$platform_file"
> 
> # see if they left us a name
> if test "$OMPI_PLATFORM_LOADED" != "" ; then
> @@ -63,14 +74,6 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [
> 
> # look for default mca param file
> 
> -# setup by getting full pathname for the platform directories
> -platform_base="`dirname $with_platform`"
> -# get full pathname of where we are so we can return
> -platform_savedir="`pwd`"
> -# go to where the platform file is located
> -cd "$platform_base"
> -# get the full path to this location
> -platform_file_dir=`pwd`
> # return to where we started
> cd "$platform_savedir"
> 
> 
> -Nathan
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

Re: [OMPI devel] ibm/collective/bcast_f08.f90

[Jeff Squyres]

Ok, some basic non-blocking tests should be in now, including mpif.h, use mpi, 
and use mpi_f08 versions -- see r26768 in ompi-tests.

They're not great tests -- it would be wonderful if someone would write some 
better ones.  :-)  But they at least verify basic functionality.

(and they did find two bugs -- one bozo debug that I had left in the mpif.h 
ibarrier, and a problem with MPI_STATUS(ES)_IGNORE in the new ignore TKR use 
mpi implementation, which I'm still working on fixing...)


On Jul 6, 2012, at 6:35 PM, Eugene Loh wrote:

> I assume this is an orphaned file that should be removed?  (It looks like a 
> draft version of ibcast_f08.f90.)
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

[OMPI devel] RFC: enable the use of source in platform files

[Nathan Hjelm]

When: Thurs, Jul 12, 5 PM MDT

Why: Useful feature. Enabling source may cut down on the maintenance required 
to keep platform files up to date.

How: Change directories to the platform file's directory before sourcing it 
(not after).

diff --git a/config/ompi_load_platform.m4 b/config/ompi_load_platform.m4
index 7231f83..250e7d8 100644
--- a/config/ompi_load_platform.m4
+++ b/config/ompi_load_platform.m4
@@ -50,7 +50,18 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [
 # eval into environment
 OPAL_LOG_MSG([Loading environment file $with_platform, with contents 
below])
 OPAL_LOG_FILE([$with_platform])
-. "$with_platform"
+
+# setup by getting full pathname for the platform directories
+platform_base="`dirname $with_platform`"
+platform_file="`basename $with_platform`"
+# get full pathname of where we are so we can return
+platform_savedir="`pwd`"
+# go to where the platform file is located
+cd "$platform_base"
+# get the full path to this location
+platform_file_dir=`pwd`
+
+. "$platform_file"

 # see if they left us a name
 if test "$OMPI_PLATFORM_LOADED" != "" ; then
@@ -63,14 +74,6 @@ AC_DEFUN([OMPI_LOAD_PLATFORM], [

 # look for default mca param file

-# setup by getting full pathname for the platform directories
-platform_base="`dirname $with_platform`"
-# get full pathname of where we are so we can return
-platform_savedir="`pwd`"
-# go to where the platform file is located
-cd "$platform_base"
-# get the full path to this location
-platform_file_dir=`pwd`
 # return to where we started
 cd "$platform_savedir"


-Nathan

[OMPI devel] Fwd: CVE-2012-3386 Automake security fix for 'make distcheck'

[Jeff Squyres]

We had just recently bumped up the Autotools triple used to create the trunk 
and v1.7 tarballs to include Automake 1.12.1.  Due to the notice below, I have 
bumped it up to 1.12.2.  Nightly tarballs starting tonight will use this new 
version.

I have also patched the Automake that is being used to generate the v1.6 
tarballs (1.11.3) per the notice below.  Nightly tarballs starting tonight will 
use this patched version.

Since we are no longer generating tarballs for versions older than v1.6, I do 
not intend to patch any further versions of Automake, nor generate any new 
versions of older OMPI tarballs.


Begin forwarded message:

> From: Stefano Lattarini 
> Subject: CVE-2012-3386 Automake security fix for 'make distcheck'
> Date: July 9, 2012 12:26:01 PM EDT
> To: Automake List 
> Cc: , 
> 
> GNU Automake 1.12.2 as well as 1.11.6 fix a locally-exploitable
> security-related race condition that affects "make distcheck" for
> all packages that use Automake.
> 
> Before the fix, the recipe of the 'distcheck' target granted temporary
> world-write permissions on the extracted distdir.  This introduced
> a locally exploitable race condition for those who run "make distcheck"
> with a non-restrictive umask (e.g., 022) in a directory that was
> accessible by others.  A successful exploit would result in arbitrary
> code execution with the privileges of the user running "make distcheck".
> 
> It is important to stress that this vulnerability impacts not only
> the Automake package itself, but all packages with Automake-generated
> makefiles.  For an effective fix it is necessary to regenerate the
> Makefile.in files with a fixed Automake version.
> 
> For release series older than 1.11.x, no fix has been been applied to
> the the git repository, and no official new release is planned that
> fixes the vulnerability.  Users interested in having such a fix in
> older releases will have to apply it manually (the attached patch is
> what we used on the 1.11.6 and 1.12.2 release).
> 
> The issue was found and fixed by Stefano Lattarini.  Jim Meyering
> wrote a proof-of-concept script showing that the vulnerability is
> easy to exploit.
> 
From bab7065f75bb9680df8c782da06a8312e5fa95a6 Mon Sep 17 00:00:00 2001
Message-Id: 
From: Stefano Lattarini 
List-Post: devel@lists.open-mpi.org
Date: Fri, 6 Jul 2012 22:43:04 +0200
Subject: [PATCH] distcheck: never make part of $(distdir) world-writable

This fixes a locally-exploitable security vulnerability (CVE-2012-3386).

In the 'distcheck' rule, we used to make the just-extracted (from
the distribution tarball) $(distdir) directory and all its files and
subdirectories read-only; then, in order to create the '_inst' and
'_build' subdirectories in there (used by the rest of the recipe) we
made the top-level $(distdir) *world-writable* for an instant (the
time to create those two directories) before making it read-only
again.

Making that directory world-writable (albeit only briefly) introduced a
locally exploitable race condition for those who run "make distcheck" with
a non-restrictive umask (e.g., 022) in a directory that is accessible by
others.  A successful exploit would result in arbitrary code execution
with the privileges of the user running "make distcheck" -- game over.
Jim Meyering wrote a proof-of-concept script showing that such exploit is
easily implemented.

This issue is similar to the CVE-2009-4029 vulnerability:


* lib/am/distdir.am (distcheck): Don't make $(distdir) world-writable,
not even for an instant; make it user-writable instead, which is enough.

Helped-By: Jim Meyering 
Signed-off-by: Stefano Lattarini 
---
 NEWS  |9 +
 lib/am/distdir.am |2 +-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index ee16961..4975e8e 100644
--- a/NEWS
+++ b/NEWS
@@ -92,6 +92,15 @@ New in 1.12.2:
 
 Bugs fixed in 1.12.2:
 
+* SECURITY VULNERABILITIES!
+
+  - The recipe of the 'distcheck' no longer grants anymore temporary
+world-wide write permissions on the extracted distdir.  Even if such
+rights were only granted for a vanishingly small time window, the
+implied race condition proved to be enough to allow a local attacker
+to run arbitrary code with the privileges of the user running "make
+distcheck".  This is CVE-2012-3386.
+
 * Long-standing bugs:
 
   - The "recheck" targets behaves better in the face of build failures
diff --git a/lib/am/distdir.am b/lib/am/distdir.am
index e27b650..f636a1e 100644
--- a/lib/am/distdir.am
+++ b/lib/am/distdir.am
@@ -449,7 +449,7 @@ distcheck: dist
 ## Make the new source tree read-only.  Distributions ought to work in
 ## this case.  

[OMPI devel] Opal wrappers question

[Nathan Hjelm]

I am playing with the Open MPI wrappers and had a couple of questions:
 - If an argument is used to select a section (compiler_args) is that option 
supposed to be passed to the compiler (because it is)?
 - If the above is the intended behavior is there any objection to an 
additional wrapper option that prevents these arguments from being passed to 
the compiler?

-Nathan

Re: [OMPI devel] openib max_cqe

[Brad Benton]

I am running into similar issues with both Mellanox and IBM HCAs.

On a node installed with RHEL6.2 and MLNX_OFED-1.5.3-3.0.0, there is a
significant hit to locked memory when going with the device's max_cqe.
Here, for comparison's sake is the memory utilization for a simple MPI
process when using the new cq_size default, and when restricting it to 1500:

cq_size = max_cqe:
VmPeak:   348736 kB
VmSize:   348352 kB
VmLck:292096 kB
VmHWM:304896 kB
VmRSS:304896 kB
VmData:   333504 kB

cq_size = 1500
VmPeak:86720 kB
VmSize:86336 kB
VmLck: 30080 kB
VmHWM: 42880 kB
VmRSS: 42880 kB
VmData:71488 kB

For our Power systems using the IBM eHCA, the default value exhausts memory
and we can't even run.

--Brad


On Fri, Jul 6, 2012 at 5:21 AM, TERRY DONTJE wrote:

>
>
> On 7/5/2012 5:47 PM, Shamis, Pavel wrote:
>
>  I mentioned on the call that for Mellanox devices (+OFA verbs) this resource 
> is really cheap. Do you run mellanox hca + OFA verbs ?
>
>  (I'll reply because I know Terry is offline for the rest of the day)
>
> Yes, he does.
>
>  I asked because SUN used to have own verbs driver.
>
>  I noticed this on a Solaris machine, I am not sure I have the same set up
> for Linux but I'll look and see if I can reproduce the same issue on Linux.
>
> --td
>
>   The heart of the question: is it incorrect to assume that we'll consume 
> (num CQE * CQE size) registered memory for each QP opened?
>
>  QP or CQ ?  I think you don't want to assume anything there. Verbs (user and 
> kernel) do their own magic there.
> I think Mellanox should address this question.
>
> Regards,
> Pasha
> ___
> devel mailing 
> listdevel@open-mpi.orghttp://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
> --
>   Terry D. Dontje | Principal Software Engineer
> Developer Tools Engineering | +1.781.442.2631
>  Oracle * - Performance Technologies*
>  95 Network Drive, Burlington, MA 01803
> Email terry.don...@oracle.com
>
>
>
>
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>

Re: [OMPI devel] ibm/collective/bcast_f08.f90

[Jeff Squyres]

Ah, not quite.

All those non-blocking collective tests (including the Fortran) weren't quite 
finished because I wrote them because libnbc hadn't been slurped in yet.

Thanks for reminding me -- I'll go finish them now...


On Jul 6, 2012, at 6:35 PM, Eugene Loh wrote:

> I assume this is an orphaned file that should be removed?  (It looks like a 
> draft version of ibcast_f08.f90.)
> ___
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/