Re: [OMPI users] programming qsn??

2009-08-14 Thread amjad ali
Hello Mr. Eugene Loh,
THANK YOU VERY MUCH, IT WORKED. I used both ISEND and IRECV and then a
combined call to WAITALL with MPI_STATUSES_IGNORE.

with best regards,
Amjad Ali.


On Fri, Aug 14, 2009 at 6:42 AM, Eugene Loh  wrote:

> amjad ali wrote:
>
>  Please tell me that if have multiple such ISEND-RECV squrntially for
>> sending receiving data then DO we need to declare separate
>> status(MPI_STATUS_SIZE) with for example status1, status2, ; or a single
>> declaration of it will work for all??
>>
>
> First of all, it really is good form to post receives as early as possible.
>
> Anyhow, when you have multiple requests being completed at once, you need
> more than one status.  So, you declare an array of statuses.  Check the man
> page for MPI_Waitall.  E.g.,
>
> INCLUDE 'mpif.h'
>
> INTEGER REQS(M)
> INTEGER STATS(MPI_STATUS_SIZE,M)
>
> DO I = 1, M
>  CALL MPI_IRECV(BUF?, COUNT?, DATATYPE?, SOURCE?, TAG?, COMM?, REQS(I),
> IER)
> END DO
>
> DO I = 1, N
>  CALL MPI_SEND(BUF?, COUNT?, DATATYPE?, DEST?, TAG?, COMM?, IER)
> END DO
>
> CALL MPI_WAITALL(M, REQS, STATS, IER)
>
> If you don't care about the statuses, don't declare STATS and just use
>
> CALL MPI_WAITALL(M,REQS,MPI_STATUSES_IGNORE,IER)
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>


Re: [OMPI users] Bug: coll_tuned_dynamic_rules_filename and duplicate communicators

2009-08-14 Thread George Bosilca

John,

Thanks for your bug report. This issues has been fixed in commit  
r21825. I'll hope to be able to push it in our next release.


  Thanks,
george.

On Jul 7, 2009, at 13:04 , Jumper, John wrote:


I am attempting to use coll_tuned_dynamic_rules_filename to tune Open
MPI 1.3.2.  Based on my testing, it appears that the dynamic rules  
file

*only* influences the algorithm selection for MPI_COMM_WORLD.  Any
duplicate communicators will only use fixed or forced rules, which may
have much worse performance than the custom-tuned collectives in the
dynamic rules file.  The following code demonstrates the difference
between MPI_COMM_WORLD and a duplicate communicator.

test.c:
#include 

int main( int argc, char** argv ) {
 float u = 0.0, v = 0.0;
 MPI_Comm world_dup;

 MPI_Init( ,  );
 MPI_Comm_dup( MPI_COMM_WORLD, _dup );

 MPI_Allreduce( , , 1, MPI_FLOAT, MPI_SUM, world_dup );
 MPI_Barrier( MPI_COMM_WORLD );
 MPI_Allreduce( , , 1, MPI_FLOAT, MPI_SUM, MPI_COMM_WORLD );

 MPI_Finalize();
 return 0;
}

allreduce.ompi:
1
2
1
9
1
0 1 0 0

invocation:
orterun -np 9 \
   -mca btl self,sm,openib,tcp \
   -mca coll_tuned_use_dynamic_rules 1 \
   -mca coll_tuned_dynamic_rules_filename allreduce.ompi \
   -mca coll_base_verbose 1000 \
   -- test

This program is run with tracing, and the barrier is only used to
separate the allreduce calls in the trace.  The trace for one node  
is at

the end of the message, and the relevant section is the choice of
algorithms for the two allreduce calls.  The allreduce.ompi file
indicates that all size 9 communicators should use the basic linear
allreduce algorithm.  MPI_COMM_WORLD uses basic_linear, but the
world_dup communicator uses the fixed algorithm (for this message  
size,

the fixed algorithm is recursive doubling).

Thank you.

John Jumper



Trace of one process for the above program:
mca: base: components_open: opening coll components
mca: base: components_open: found loaded component basic
mca: base: components_open: component basic register function  
successful

mca: base: components_open: component basic has no open function
mca: base: components_open: found loaded component hierarch
mca: base: components_open: component hierarch has no register  
function
mca: base: components_open: component hierarch open function  
successful

mca: base: components_open: found loaded component inter
mca: base: components_open: component inter has no register function
mca: base: components_open: component inter open function successful
mca: base: components_open: found loaded component self
mca: base: components_open: component self has no register function
mca: base: components_open: component self open function successful
mca: base: components_open: found loaded component sm
mca: base: components_open: component sm has no register function
mca: base: components_open: component sm open function successful
mca: base: components_open: found loaded component sync
mca: base: components_open: component sync register function  
successful

mca: base: components_open: component sync has no open function
mca: base: components_open: found loaded component tuned
mca: base: components_open: component tuned has no register function
coll:tuned:component_open: done!
mca: base: components_open: component tuned open function successful
coll:find_available: querying coll component basic
coll:find_available: coll component basic is available
coll:find_available: querying coll component hierarch
coll:find_available: coll component hierarch is available
coll:find_available: querying coll component inter
coll:find_available: coll component inter is available
coll:find_available: querying coll component self
coll:find_available: coll component self is available
coll:find_available: querying coll component sm
coll:find_available: coll component sm is available
coll:find_available: querying coll component sync
coll:find_available: coll component sync is available
coll:find_available: querying coll component tuned
coll:find_available: coll component tuned is available
coll:base:comm_select: new communicator: MPI_COMM_WORLD (cid 0)
coll:base:comm_select: Checking all available modules
coll:base:comm_select: component available: basic, priority: 10
coll:base:comm_select: component not available: hierarch
coll:base:comm_select: component not available: inter
coll:base:comm_select: component not available: self
coll:base:comm_select: component not available: sm
coll:base:comm_select: component not available: sync
coll:tuned:module_tuned query called
coll:tuned:module_query using intra_dynamic
coll:base:comm_select: component available: tuned, priority: 30
coll:tuned:module_init called.
coll:tuned:module_init MCW & Dynamic
coll:tuned:module_init Opening [allreduce.ompi]
Reading dynamic rule for collective ID 2
Read communicator count 1 for dynamic rule for collective ID 2
Read message count 1 for dynamic rule for collective ID 2 and comm  
size

9
Done reading dynamic rule for 

Re: [OMPI users] tcp connectivity OS X and 1.3.3

2009-08-14 Thread Jody Klymak

Hi All,

Just to polish this thread off

To make openmpi work on my OS X 10.5 machine I need only:

./configure --prefix=/Network/Xgrid/openmpi
make
make install

I then edited
/Network/Xgrid/openmpi/etc/openmpi-mca-params.conf
and added

# set ports so that they are more valid than the default ones (see  
email from Ralph Castain)

btl_tcp_port_min_v4 = 36900
btl_tcp_port_range  = 32

Which may be a kludge only I need because I am being silly and running  
openMPI as an admin-privileged user.


For my environment, I don't *need* to set anything, though I find it  
useful to set:


export PATH="/Network/Xgrid/openmpi/bin/:$PATH:/usr/local/bin:/usr/ 
local/sbin"

export MANPATH="/Network/Xgrid/openmpi/man/:/usr/local/man/:$MANPATH"

Note that I do not think I need to set LD_LIBRARY_PATH (which I don't  
think OS X knows anything about anyway) nor DYLD_LIBRARY_PATH.  This  
is consistent with the FAQ that warns against setting those (though  
when I had them set there didn't seem to be any ill effects).


I'm running a long job now under torque/pbs.  I'll see if it survives  
the weekend (I had trouble with XGrid jobs dying after 10 hours or so  
due to an sshd deciding it was a security breach and killing all the  
processes).


Anyways, all seems to be working so far.  Sorry that my poor choice in  
user management caused so many mysteries.  Thanks for everyone's help.


Cheers,  Jody

--
Jody Klymak
http://web.uvic.ca/~jklymak/






Re: [OMPI users] openmpi with xgrid

2009-08-14 Thread Warner Yuen

Hi Alan,

Xgrid support for Open MPI is currently broken in the latest version  
of Open MPI. See the ticket below. However, I believe that Xgrid still  
works with one of the earlier 1.2  versions of Open MPI. I don't  
recall for sure, but I think that it's Open MPI 1.2.3.


#1777: Xgrid support is broken in the v1.3 series
- 
+--

Reporter:  jsquyres  |Owner:  brbarret
   Type:  defect|   Status:  accepted
Priority:  major |Milestone:  Open MPI 1.3.4
Version:  trunk |   Resolution:
Keywords:|
- 
+--

Changes (by bbenton):

 * milestone:  Open MPI 1.3.3 => Open MPI 1.3.4


Warner Yuen
Scientific Computing
Consulting Engineer
Apple, Inc.
email: wy...@apple.com
Tel: 408.718.2859




On Aug 14, 2009, at 6:21 AM, users-requ...@open-mpi.org wrote:



Message: 1
Date: Fri, 14 Aug 2009 14:21:30 +0100
From: Alan 
Subject: [OMPI users] openmpi with xgrid
To: us...@open-mpi.org
Message-ID:

Content-Type: text/plain; charset="utf-8"

Hi there,
I saw that http://www.open-mpi.org/community/lists/users/2007/08/3900.php 
.


I use fink, and so I changed the openmpi.info file in order to get  
openmpi

with xgrid support.

As you can see:
amadeus[2081]:~/Downloads% /sw/bin/ompi_info
Package: Open MPI root@amadeus.local Distribution
   Open MPI: 1.3.3
  Open MPI SVN revision: r21666
  Open MPI release date: Jul 14, 2009
   Open RTE: 1.3.3
  Open RTE SVN revision: r21666
  Open RTE release date: Jul 14, 2009
   OPAL: 1.3.3
  OPAL SVN revision: r21666
  OPAL release date: Jul 14, 2009
   Ident string: 1.3.3
 Prefix: /sw
Configured architecture: x86_64-apple-darwin9
 Configure host: amadeus.local
  Configured by: root
  Configured on: Fri Aug 14 12:58:12 BST 2009
 Configure host: amadeus.local
   Built by:
   Built on: Fri Aug 14 13:07:46 BST 2009
 Built host: amadeus.local
 C bindings: yes
   C++ bindings: yes
 Fortran77 bindings: yes (single underscore)
 Fortran90 bindings: yes
Fortran90 bindings size: small
 C compiler: gcc
C compiler absolute: /sw/var/lib/fink/path-prefix-10.6/gcc
   C++ compiler: g++
  C++ compiler absolute: /sw/var/lib/fink/path-prefix-10.6/g++
 Fortran77 compiler: gfortran
 Fortran77 compiler abs: /sw/bin/gfortran
 Fortran90 compiler: gfortran
 Fortran90 compiler abs: /sw/bin/gfortran
C profiling: yes
  C++ profiling: yes
Fortran77 profiling: yes
Fortran90 profiling: yes
 C++ exceptions: no
 Thread support: posix (mpi: no, progress: no)
  Sparse Groups: no
 Internal debug support: no
MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
libltdl support: yes
  Heterogeneous support: no
mpirun default --prefix: no
MPI I/O support: yes
  MPI_WTIME support: gettimeofday
Symbol visibility support: yes
  FT Checkpoint support: no  (checkpoint thread: no)
  MCA backtrace: execinfo (MCA v2.0, API v2.0, Component  
v1.3.3)

  MCA paffinity: darwin (MCA v2.0, API v2.0, Component v1.3.3)
  MCA carto: auto_detect (MCA v2.0, API v2.0, Component  
v1.3.3)

  MCA carto: file (MCA v2.0, API v2.0, Component v1.3.3)
  MCA maffinity: first_use (MCA v2.0, API v2.0, Component  
v1.3.3)

  MCA timer: darwin (MCA v2.0, API v2.0, Component v1.3.3)
MCA installdirs: env (MCA v2.0, API v2.0, Component v1.3.3)
MCA installdirs: config (MCA v2.0, API v2.0, Component v1.3.3)
MCA dpm: orte (MCA v2.0, API v2.0, Component v1.3.3)
 MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.3.3)
  MCA allocator: basic (MCA v2.0, API v2.0, Component v1.3.3)
  MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.3.3)
   MCA coll: basic (MCA v2.0, API v2.0, Component v1.3.3)
   MCA coll: hierarch (MCA v2.0, API v2.0, Component  
v1.3.3)

   MCA coll: inter (MCA v2.0, API v2.0, Component v1.3.3)
   MCA coll: self (MCA v2.0, API v2.0, Component v1.3.3)
   MCA coll: sm (MCA v2.0, API v2.0, Component v1.3.3)
   MCA coll: sync (MCA v2.0, API v2.0, Component v1.3.3)
   MCA coll: tuned (MCA v2.0, API v2.0, Component v1.3.3)
 MCA io: romio (MCA v2.0, API v2.0, Component v1.3.3)
  MCA mpool: fake (MCA v2.0, API v2.0, Component v1.3.3)
  MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.3.3)
  MCA mpool: sm (MCA v2.0, API v2.0, Component v1.3.3)
MCA pml: cm (MCA v2.0, API v2.0, Component v1.3.3)
 

Re: [OMPI users] an MPI process using about 12 file descriptors per neighbour processes - isn't it a bit too much?

2009-08-14 Thread Rolf Vandevaart

Hi Paul:
I tried the running the same way as you did and I saw the same thing.  I 
was using ClusterTools 8.2 (Open MPI 1.3.3r21324) and running on 
Solaris.  I looked at the mpirun process and it was definitely consuming 
approximately 12 file descriptors per a.out process.


 burl-ct-v440-0 59 =>limit descriptors
descriptors 1024
 burl-ct-v440-0 60 =>mpirun -np 84 a.out
Connectivity test on 84 processes PASSED.
 burl-ct-v440-0 61 =>mpirun -np 85 a.out
[burl-ct-v440-0:27083] [[38835,0],0] ORTE_ERROR_LOG: The system limit on 
number of network connections a process can open was reached in file 
oob_tcp.c at line 446

--
Error: system limit exceeded on number of network connections that can 
be open


This can be resolved by setting the mca parameter 
opal_set_max_sys_limits to 1,

increasing your limit descriptor setting (using limit or ulimit commands),
or asking the system administrator to increase the system limit.
--
 burl-ct-v440-0 62 =>

This should not be happening.  I will try and look to see what is going 
on.  The process that is complaining is the mpirun process which in this 
scenario forks/execs all the a.outs.


Rolf

On 08/14/09 08:52, Paul Kapinos wrote:

Hi OpenMPI folks,

We use Sun MPI (Cluster Tools 8.2) and also native OpenMPI 1.3.3 and we 
wonder us about the way OpenMPI devours file descriptors: on our 
computers, ulimit -n is currently set to 1024, and we found out that we 
may run maximally 84 MPI processes per box, and if we try to run 85 (or 
above) processes, we got such error message:


--
Error: system limit exceeded on number of network connections that can 
be open

.
--

Simple computing tells us, 1024/85 is about 12. This lets us believe 
that there is an single OpenMPI process, which needs 12 file descriptor 
per other MPI process.


By now, we have only one box with more than 100 CPUs on which it may be 
meaningfull to run more than 85 processes. But in the quite near future, 
many-core boxes are arising (we also ordered 128-way nehalems), so it 
may be disadvantageous to consume a lot of file descriptors per MPI 
process.



We see a possibility to awod this problem by setting the ulimit for file 
descriptor to a higher value.  This is not easy unter linux: you need 
either to recompile the kernel (which is not a choise for us), or to set 
a root process somewhere which will set the ulimit to a higher value 
(which is a security risk and not easy to implement).


We also tryed to set the opal_set_max_sys_limits to 1, as the help says 
(by adding  "-mca opal_set_max_sys_limits 1" to the command line), but 
we does not see any change of behaviour).


What is your meaning?

Best regards,
Paul Kapinos
RZ RWTH Aachen



#
 /opt/SUNWhpc/HPC8.2/intel/bin/mpiexec -mca opal_set_max_sys_limits 1 
-np 86   a.out


___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users



--

=
rolf.vandeva...@sun.com
781-442-3043
=


Re: [OMPI users] strange IMB runs

2009-08-14 Thread Michael Di Domenico
> One of the differences among MPI implementations is the default placement of
> processes within the node.  E.g., should processes by default be collocated
> on cores of the same socket or on cores of different sockets?  I don't know
> if that issue is applicable here (that is, HP MPI vs Open MPI or on
> Superdome architecture), but it's potentially an issue to look at.  With HP
> MPI, mpirun has a -cpu_bind switch for controlling placement.  With Open
> MPI, mpirun controls placement with -rankfile.
>
> E.g., what happens if you try
>
> % cat rf1
> rank 0=XX  slot=0
> rank 1=XX  slot=1
> % cat rf2
> rank 0=XX  slot=0
> rank 1=XX  slot=2
> % cat rf3
> rank 0=XX  slot=0
> rank 1=XX  slot=3
> [...etc...]
> % mpirun -np 2 --mca btl self,sm --host XX,XX -rf rf1 $PWD/IMB-MPI1 pingpong
> % mpirun -np 2 --mca btl self,sm --host XX,XX -rf rf2 $PWD/IMB-MPI1 pingpong
> % mpirun -np 2 --mca btl self,sm --host XX,XX -rf rf3 $PWD/IMB-MPI1 pingpong
> [...etc...]
>
> where XX is the name of your node and you march through all the cores on
> your Superdome node?

I tried this, but it didn't seem to make a difference either


Re: [OMPI users] strange IMB runs

2009-08-14 Thread Michael Di Domenico
On Thu, Aug 13, 2009 at 1:51 AM, Eugene Loh wrote:
> Also, I'm puzzled why you should see better results by changing
> btl_sm_eager_limit.  That shouldn't change long-message bandwidth, but only
> the message size at which one transitions from short to long messages.  If
> anything, tweaking btl_sm_max_send_size would be the variable to try.

Changing this value does tweak the 1,2,4MB messages, but all others
are the same and it's not consistent.

I've also noticed that the latency is nearly 2usec high on OpenMPI vs HPMPI


[OMPI users] openmpi with xgrid

2009-08-14 Thread Alan
Hi there,
I saw that http://www.open-mpi.org/community/lists/users/2007/08/3900.php.

I use fink, and so I changed the openmpi.info file in order to get openmpi
with xgrid support.

As you can see:
amadeus[2081]:~/Downloads% /sw/bin/ompi_info
 Package: Open MPI root@amadeus.local Distribution
Open MPI: 1.3.3
   Open MPI SVN revision: r21666
   Open MPI release date: Jul 14, 2009
Open RTE: 1.3.3
   Open RTE SVN revision: r21666
   Open RTE release date: Jul 14, 2009
OPAL: 1.3.3
   OPAL SVN revision: r21666
   OPAL release date: Jul 14, 2009
Ident string: 1.3.3
  Prefix: /sw
 Configured architecture: x86_64-apple-darwin9
  Configure host: amadeus.local
   Configured by: root
   Configured on: Fri Aug 14 12:58:12 BST 2009
  Configure host: amadeus.local
Built by:
Built on: Fri Aug 14 13:07:46 BST 2009
  Built host: amadeus.local
  C bindings: yes
C++ bindings: yes
  Fortran77 bindings: yes (single underscore)
  Fortran90 bindings: yes
 Fortran90 bindings size: small
  C compiler: gcc
 C compiler absolute: /sw/var/lib/fink/path-prefix-10.6/gcc
C++ compiler: g++
   C++ compiler absolute: /sw/var/lib/fink/path-prefix-10.6/g++
  Fortran77 compiler: gfortran
  Fortran77 compiler abs: /sw/bin/gfortran
  Fortran90 compiler: gfortran
  Fortran90 compiler abs: /sw/bin/gfortran
 C profiling: yes
   C++ profiling: yes
 Fortran77 profiling: yes
 Fortran90 profiling: yes
  C++ exceptions: no
  Thread support: posix (mpi: no, progress: no)
   Sparse Groups: no
  Internal debug support: no
 MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
 libltdl support: yes
   Heterogeneous support: no
 mpirun default --prefix: no
 MPI I/O support: yes
   MPI_WTIME support: gettimeofday
Symbol visibility support: yes
   FT Checkpoint support: no  (checkpoint thread: no)
   MCA backtrace: execinfo (MCA v2.0, API v2.0, Component v1.3.3)
   MCA paffinity: darwin (MCA v2.0, API v2.0, Component v1.3.3)
   MCA carto: auto_detect (MCA v2.0, API v2.0, Component v1.3.3)
   MCA carto: file (MCA v2.0, API v2.0, Component v1.3.3)
   MCA maffinity: first_use (MCA v2.0, API v2.0, Component v1.3.3)
   MCA timer: darwin (MCA v2.0, API v2.0, Component v1.3.3)
 MCA installdirs: env (MCA v2.0, API v2.0, Component v1.3.3)
 MCA installdirs: config (MCA v2.0, API v2.0, Component v1.3.3)
 MCA dpm: orte (MCA v2.0, API v2.0, Component v1.3.3)
  MCA pubsub: orte (MCA v2.0, API v2.0, Component v1.3.3)
   MCA allocator: basic (MCA v2.0, API v2.0, Component v1.3.3)
   MCA allocator: bucket (MCA v2.0, API v2.0, Component v1.3.3)
MCA coll: basic (MCA v2.0, API v2.0, Component v1.3.3)
MCA coll: hierarch (MCA v2.0, API v2.0, Component v1.3.3)
MCA coll: inter (MCA v2.0, API v2.0, Component v1.3.3)
MCA coll: self (MCA v2.0, API v2.0, Component v1.3.3)
MCA coll: sm (MCA v2.0, API v2.0, Component v1.3.3)
MCA coll: sync (MCA v2.0, API v2.0, Component v1.3.3)
MCA coll: tuned (MCA v2.0, API v2.0, Component v1.3.3)
  MCA io: romio (MCA v2.0, API v2.0, Component v1.3.3)
   MCA mpool: fake (MCA v2.0, API v2.0, Component v1.3.3)
   MCA mpool: rdma (MCA v2.0, API v2.0, Component v1.3.3)
   MCA mpool: sm (MCA v2.0, API v2.0, Component v1.3.3)
 MCA pml: cm (MCA v2.0, API v2.0, Component v1.3.3)
 MCA pml: csum (MCA v2.0, API v2.0, Component v1.3.3)
 MCA pml: ob1 (MCA v2.0, API v2.0, Component v1.3.3)
 MCA pml: v (MCA v2.0, API v2.0, Component v1.3.3)
 MCA bml: r2 (MCA v2.0, API v2.0, Component v1.3.3)
  MCA rcache: vma (MCA v2.0, API v2.0, Component v1.3.3)
 MCA btl: self (MCA v2.0, API v2.0, Component v1.3.3)
 MCA btl: sm (MCA v2.0, API v2.0, Component v1.3.3)
 MCA btl: tcp (MCA v2.0, API v2.0, Component v1.3.3)
MCA topo: unity (MCA v2.0, API v2.0, Component v1.3.3)
 MCA osc: pt2pt (MCA v2.0, API v2.0, Component v1.3.3)
 MCA osc: rdma (MCA v2.0, API v2.0, Component v1.3.3)
 MCA iof: hnp (MCA v2.0, API v2.0, Component v1.3.3)
 MCA iof: orted (MCA v2.0, API v2.0, Component v1.3.3)
 MCA iof: tool (MCA v2.0, API v2.0, Component v1.3.3)
 MCA oob: tcp (MCA v2.0, API v2.0, Component v1.3.3)
MCA odls: default (MCA v2.0, API v2.0, Component v1.3.3)
 MCA ras: slurm (MCA v2.0, API 

[OMPI users] an MPI process using about 12 file descriptors per neighbour processes - isn't it a bit too much?

2009-08-14 Thread Paul Kapinos

Hi OpenMPI folks,

We use Sun MPI (Cluster Tools 8.2) and also native OpenMPI 1.3.3 and we 
wonder us about the way OpenMPI devours file descriptors: on our 
computers, ulimit -n is currently set to 1024, and we found out that we 
may run maximally 84 MPI processes per box, and if we try to run 85 (or 
above) processes, we got such error message:


--
Error: system limit exceeded on number of network connections that can 
be open

.
--

Simple computing tells us, 1024/85 is about 12. This lets us believe 
that there is an single OpenMPI process, which needs 12 file descriptor 
per other MPI process.


By now, we have only one box with more than 100 CPUs on which it may be 
meaningfull to run more than 85 processes. But in the quite near future, 
many-core boxes are arising (we also ordered 128-way nehalems), so it 
may be disadvantageous to consume a lot of file descriptors per MPI 
process.



We see a possibility to awod this problem by setting the ulimit for file 
descriptor to a higher value.  This is not easy unter linux: you need 
either to recompile the kernel (which is not a choise for us), or to set 
a root process somewhere which will set the ulimit to a higher value 
(which is a security risk and not easy to implement).


We also tryed to set the opal_set_max_sys_limits to 1, as the help says 
(by adding  "-mca opal_set_max_sys_limits 1" to the command line), but 
we does not see any change of behaviour).


What is your meaning?

Best regards,
Paul Kapinos
RZ RWTH Aachen



#
 /opt/SUNWhpc/HPC8.2/intel/bin/mpiexec -mca opal_set_max_sys_limits 1 
-np 86   a.out
<>

smime.p7s
Description: S/MIME Cryptographic Signature


[OMPI users] Help: How to accomplish processors affinity

2009-08-14 Thread Lee Amy
HI,

I read some howtos at OpenMPI official site but i still have some problems here.

I build a Kerrighed Clusters with 4 nodes so they look like a big SMP
machine. every node has 1 processor with dingle core.

1) Dose MPI programs could be running on such kinds of machine? If
yes, could anyone show me some examples?

2) In this SMP machine there are 4 processors I could see. So how do I
use OpenMPI to run some programs on these CPUs? Though I read how to
make a rank file but I am still feel confused. Could anyone show me a
simple rank file example for my Clusters?

Thank you very much.

Regards,

Amy Lee