[OMPI devel] Use of orte_pointer_array in openib and udapl btls

[Tim Prins]

Hi,

The openib and udapl btls currently use the orte_pointer_array class. 
This is a problem for me as I am trying to implement the RSL. So, as far 
as I can tell, there are 3 options:


1. Move the orte_pointer_array class to opal. This would be quite simple 
to do and makes sense in that there is nothing in the orte_pointer_array 
specific to orte.


2. Change the udapl and openib btls to use a simple C array. There is 
currently a ticket filed (https://svn.open-mpi.org/trac/ompi/ticket/727) 
to do this in the openib btl.


3. Change the btls to use the ompi_pointer_array. This might not be a 
good idea since the above ticket says that the ompi_pointer array was 
intentionally not used.


Any of these options are fine with me, although if #2 is picked someone 
else would probably need to do most of the work.


Comments?

Thanks,

Tim

Re: [OMPI devel] Hybrid examples

[Edgar Gabriel]

Just to clarify, we had some conversation off-line with Jeff about that. 
You are eligible to access the ompi-tests directory, since you are a 
member of the group at UofDresden which signed the third party 
contribution, and thus is a member of Open MPI. For the sake of 
simplicity the svn accounts do not have automatically access to the 
svn-tests and svn-docs repository. For an existing svn account, it is as 
simple as sending an email to the svn repository maintainer ( Tim 
Mattox?) asking for that.


In order to issue a *new* svn account, a member has to submit a new 
Appendix A of the third party contribution -- which is a simple document 
listing all persons working on ompi and thus requiring an svn account.


Thanks
Edgar

Jeff Squyres wrote:
We've never made our ompi-tests SVN repository public mainly because  
it's mainly a collection of MPI benchmarks and tests that we've  
collected from around the net, but we've never looked into  
redistribution rights.  Hence, our SVN is not publicly readable.


As the SVN URL implies, the thread tests came from Sun, so I think  
it's up to them as to whether they want them to be public or not.




On Oct 4, 2007, at 7:22 AM, Tobias Hilbrich wrote:


Hello,

I am a developer of MARMOT (MPI Checker) and currently working on  
support for
MPI_THREAD_MULTIPLE programs. Therefore I am looking for test  
examples which
are realy rare. I heard from Matthias Müller that you have a suit  
of test

programs that use MPI_THREAD_MULTIPLE. They should be located at:
https://svn.open-mpi.org/svn/ompi-tests/trunk/sun/threads

Unfortunatly I and also the other people here in Dresden have no  
access to
these examples. So it would be nice to get a login or to get these  
examples

in some other way.

mfg Tobias Hilbrich

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel





--
Edgar Gabriel
Assistant Professor
Parallel Software Technologies Lab  http://pstl.cs.uh.edu
Department of Computer Science  University of Houston
Philip G. Hoffman Hall, Room 524Houston, TX-77204, USA
Tel: +1 (713) 743-3857  Fax: +1 (713) 743-3335

Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r16200

[George Bosilca]

As promised I patched the debugger support in Open MPI in order to  
remove the recv_context variable, and use only the shared unique_id  
field. The commit is 16492.


  Thanks,
george.

On Sep 25, 2007, at 11:20 AM, George Bosilca wrote:

He's right. By mistake I modified the mpi_interface.h file, when I  
knew that it should not be modified. This file is identical for all  
MPI implementations that provide parallel debuggers support. It  
define the interfaces between us and them ... interfaces that  
include functions as well as structures.


The unique_id field was not used in Open MPI, as it didn't seems to  
me that the tv guys are using it. Therefore, instead of using the  
unique_id field in their interface structure, I use the  
recv_context in our own version. However, as I said, the interface  
should never be changed, so Pak was right by rolling back this  
mpi_interface.h file. I'll commit another change soon, where I get  
rid of our own internal cid field in favor of the unique_id.


  Thanks,
george.

On Sep 25, 2007, at 9:15 AM, Pak Lui wrote:


Hi Tim,

Tim Mattox wrote:

Hmm, why should an external debugger dictate the members
of our communicator struct?  Does MPICH, etc. have the exact same
structure for their comunicators?  I doubt it.


Yes, only for mqs_communicator which shares between the DLL and  
TV. In
fact, the whole mpi_interface.h should not be touched. Nikolay  
Piskun of

Totalview Tech confirmed it.



If George's r16177 change is correct, mqs_taddr_t unique_id is never
used or set in OMPI.


Yes, it is not used for now. But George seems to have second thought
about using this unique_id instead of our own cid to make things
consistent. We'll stay tuned on what he has to do...



I'm sure TotalView & DLL already need to adjust what binary  
interface
they expect based on which version of OMPI they are interacting  
with...

what is one more struct difference?
If it is very important for this struct to not change, please add  
a comment

to the struct explaining why it shouldn't be changed.


I will, once I've figured out how to modify my existing comment for
r16200. Just to show you the kind of badness you'll see by taking  
things
out from the struct should show you a messed up comm name and  
comm_size,
like this on both Solaris AMD and SPARC. By putting this member  
back to

the struct, at least it can band aid the problem.

_WORLD
Comm_size5571033412557294925
Comm_rank4
Pending receives: none
Unexpected messages : no information available
Pending sends   : none

_SELF
Comm_size5571033412557294925
Comm_rank1
Pending receives: none
Unexpected messages : no information available
Pending sends   : none

_NULL
Comm_size5571033412557294925
Comm_rank0
Pending receives: none
Unexpected messages : no information available
Pending sends   : none




On 9/24/07, pak...@osl.iu.edu  wrote:

Author: paklui
Date: 2007-09-24 15:02:56 EDT (Mon, 24 Sep 2007)
New Revision: 16200
URL: https://svn.open-mpi.org/trac/ompi/changeset/16200

Log:
mqs_communicator type should not be changed as it serves as the  
interface

between Totalview and DLL.


Text files modified:
   trunk/ompi/debuggers/mpi_interface.h | 3 ++-
   1 files changed, 2 insertions(+), 1 deletions(-)

Modified: trunk/ompi/debuggers/mpi_interface.h
=== 
===

--- trunk/ompi/debuggers/mpi_interface.h(original)
+++ trunk/ompi/debuggers/mpi_interface.h2007-09-24  
15:02:56 EDT (Mon, 24 Sep 2007)

@@ -259,7 +259,8 @@
 /* A structure to represent a communicator */
 typedef struct
 {
-  mqs_tword_t local_rank;  /* The rank of this  
process Comm_rank */
+  mqs_taddr_t unique_id;   /* A unique tag  
for the communicator */
+  mqs_tword_t local_rank;  /* The rank of  
this process Comm_rank */

   mqs_tword_t size;/* Comm_size  */
   charname[64];/* the name if  
it has one */

 } mqs_communicator;
___
svn-full mailing list
svn-f...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/svn-full







--


- Pak Lui
pak@sun.com
___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel


___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




smime.p7s
Description: S/MIME cryptographic signature

Re: [OMPI devel] putting common request completion waiting code into separate inline function

[Jeff Squyres]

Gleb -

I am not overly familiar with all these portions of the pml code  
base, but it looks like not all of these places have exactly the same  
code: the inline version is much shorter than some of the original  
pml codes that it replaced.  Is the logic equivalent?


Also, a minor nit -- it would be nice if the new inline function  
conformed to our coding standards (constants on the left of ==, {}  
around all blocks, etc.).  :-)



On Oct 15, 2007, at 10:27 AM, Gleb Natapov wrote:


Hi,

   Each time a someone needs to wait for request completion he
implements the same piece of code. Why not put this code into
inline function and use it instead. Look at the included patch, it
moves the common code into ompi_request_wait_completion() function.
Does somebody have any objection against committing it to the trunk?

diff --git a/ompi/mca/crcp/coord/crcp_coord_pml.c b/ompi/mca/crcp/ 
coord/crcp_coord_pml.c

index b2392e4..eb9b9c1 100644
--- a/ompi/mca/crcp/coord/crcp_coord_pml.c
+++ b/ompi/mca/crcp/coord/crcp_coord_pml.c
@@ -3857,13 +3857,7 @@ static int coord_request_wait_all( size_t  
count,

 static int coord_request_wait( ompi_request_t * req,
ompi_status_public_t * status)
 {
-OPAL_THREAD_LOCK(_request_lock);
-ompi_request_waiting++;
-while (req->req_complete == false) {
-opal_condition_wait(_request_cond, _request_lock);
-}
-ompi_request_waiting--;
-OPAL_THREAD_UNLOCK(_request_lock);
+ompi_request_wait_completion(req);

 if( MPI_STATUS_IGNORE != status ) {
 status->MPI_TAG= req->req_status.MPI_TAG;
diff --git a/ompi/mca/pml/cm/pml_cm_recv.c b/ompi/mca/pml/cm/ 
pml_cm_recv.c

index 0e23c9a..00efffc 100644
--- a/ompi/mca/pml/cm/pml_cm_recv.c
+++ b/ompi/mca/pml/cm/pml_cm_recv.c
@@ -112,22 +112,7 @@ mca_pml_cm_recv(void *addr,
 return ret;
 }

-if (recvreq->req_base.req_ompi.req_complete == false) {
-/* give up and sleep until completion */
-if (opal_using_threads()) {
-opal_mutex_lock(_request_lock);
-ompi_request_waiting++;
-while (recvreq->req_base.req_ompi.req_complete == false)
-opal_condition_wait(_request_cond,  
_request_lock);

-ompi_request_waiting--;
-opal_mutex_unlock(_request_lock);
-} else {
-ompi_request_waiting++;
-while (recvreq->req_base.req_ompi.req_complete == false)
-opal_condition_wait(_request_cond,  
_request_lock);

-ompi_request_waiting--;
-}
-}
+ompi_request_wait_completion(>req_base.req_ompi);

 if (NULL != status) {  /* return status */
 *status = recvreq->req_base.req_ompi.req_status;
diff --git a/ompi/mca/pml/cm/pml_cm_send.c b/ompi/mca/pml/cm/ 
pml_cm_send.c

index ed9b189..f7d2e8c 100644
--- a/ompi/mca/pml/cm/pml_cm_send.c
+++ b/ompi/mca/pml/cm/pml_cm_send.c
@@ -175,23 +175,8 @@ mca_pml_cm_send(void *buf,
 MCA_PML_CM_THIN_SEND_REQUEST_RETURN(sendreq);
 return ret;
 }
-
-if (sendreq->req_send.req_base.req_ompi.req_complete  
== false) {

-/* give up and sleep until completion */
-if (opal_using_threads()) {
-opal_mutex_lock(_request_lock);
-ompi_request_waiting++;
-while (sendreq- 
>req_send.req_base.req_ompi.req_complete == false)
-opal_condition_wait(_request_cond,  
_request_lock);

-ompi_request_waiting--;
-opal_mutex_unlock(_request_lock);
-} else {
-ompi_request_waiting++;
-while (sendreq- 
>req_send.req_base.req_ompi.req_complete == false)
-opal_condition_wait(_request_cond,  
_request_lock);

-ompi_request_waiting--;
-}
-}
+
+ompi_request_wait_completion( 
>req_send.req_base.req_ompi);


 ompi_request_free( (ompi_request_t**) );
 } else {
diff --git a/ompi/mca/pml/dr/pml_dr_iprobe.c b/ompi/mca/pml/dr/ 
pml_dr_iprobe.c

index 9149174..2063c54 100644
--- a/ompi/mca/pml/dr/pml_dr_iprobe.c
+++ b/ompi/mca/pml/dr/pml_dr_iprobe.c
@@ -64,22 +64,7 @@ int mca_pml_dr_probe(int src,
 MCA_PML_DR_RECV_REQUEST_INIT(, NULL, 0,  
_mpi_char, src, tag, comm, true);

 MCA_PML_DR_RECV_REQUEST_START();

-if (recvreq.req_recv.req_base.req_ompi.req_complete == false) {
-/* give up and sleep until completion */
-if (opal_using_threads()) {
-opal_mutex_lock(_request_lock);
-ompi_request_waiting++;
-while (recvreq.req_recv.req_base.req_ompi.req_complete  
== false)
-opal_condition_wait(_request_cond,  
_request_lock);

-ompi_request_waiting--;
-opal_mutex_unlock(_request_lock);
-} else {
-ompi_request_waiting++;
-

Re: [OMPI devel] Hybrid examples

[Jeff Squyres]

We've never made our ompi-tests SVN repository public mainly because  
it's mainly a collection of MPI benchmarks and tests that we've  
collected from around the net, but we've never looked into  
redistribution rights.  Hence, our SVN is not publicly readable.


As the SVN URL implies, the thread tests came from Sun, so I think  
it's up to them as to whether they want them to be public or not.




On Oct 4, 2007, at 7:22 AM, Tobias Hilbrich wrote:


Hello,

I am a developer of MARMOT (MPI Checker) and currently working on  
support for
MPI_THREAD_MULTIPLE programs. Therefore I am looking for test  
examples which
are realy rare. I heard from Matthias Müller that you have a suit  
of test

programs that use MPI_THREAD_MULTIPLE. They should be located at:
https://svn.open-mpi.org/svn/ompi-tests/trunk/sun/threads

Unfortunatly I and also the other people here in Dresden have no  
access to
these examples. So it would be nice to get a login or to get these  
examples

in some other way.

mfg Tobias Hilbrich

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Jeff Squyres
Cisco Systems

Re: [OMPI devel] issue with --without-tm in configure?

[Jeff Squyres]

Jenny --

Can you send the information listed on the "getting help" web page?

http://www.open-mpi.org/community/help/


On Oct 17, 2007, at 3:48 PM, Jennis Pruett wrote:



Hello, All,

I'm new to this forum, but I'm told this is where I'm to ask  
questions.


I have a linux cluster, bproc, gm, and am trying to configure  
version 1.2.4.

This is my configure command line:

 ./configure
--prefix=/usr/projects/hpctools/jennyp/openmpi124flash/openmpi-1.2.4
--libdir=/usr/projects/hpctools/jennyp/openmpi124flash/ 
openmpi-1.2.4/lib64

--with-bproc --without-tm --enable-shared
--with-io_romio_flags=--with-file-system=ufs+nfs+panfs --without- 
pty_support


( I threw in the --without-pty-support just to see if it would make a
difference.)

The make and install seem to proced ok.
The test I have is an IMB-MPI1 test, and I am getting these  
warnings, no

matter what compiler I
m using.:


mpirun -n 6 ./IMB-MPI1
[n110:26208] mca: base: component_find: unable to open ras tm: file  
not

found
(ignored)
[n110:26208] mca: base: component_find: unable to open pls tm: file  
not

found
(ignored)
[n111:26215] mca: base: component_find: unable to open ras tm: file  
not

found
(ignored)
[n111:26212] mca: base: component_find: unable to open ras tm: file  
not

found
(ignored)
[n112:26216] mca: base: component_find: unable to open ras tm: file  
not

found
(ignored)
[n112:26213] mca: base: component_find: unable to open ras tm: file  
not

found
(ignored)
[n110:26211] mca: base: component_find: unable to open ras tm: file  
not

found
(ignored)
[n110:26214] mca: base: component_find: unable to open ras tm: file  
not

found
(ignored)
[n111:26212] mca: base: component_find: unable to open pls tm: file  
not

found
(ignored)
[n111:26215] mca: base: component_find: unable to open pls tm: file  
not

found
(ignored)
[n112:26213] mca: base: component_find: unable to open pls tm: file  
not

found
(ignored)
[n110:26211] mca: base: component_find: unable to open pls tm: file  
not

found
(ignored)
[n110:26214] mca: base: component_find: unable to open pls tm: file  
not

found
(ignored)
[n112:26216] mca: base: component_find: unable to open pls tm: file  
not

found
(ignored)


I was told today that the --without-tm parameter  is being ignored  
in the

configure statement.

Anyone know what is going on?

Thanks,

-- 
-

Jennis (Jenny)
Los Alamos National Laboratory
505-667-1955



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Jeff Squyres
Cisco Systems

[OMPI devel] issue with --without-tm in configure?

[Jennis Pruett]

Hello, All, 

I'm new to this forum, but I'm told this is where I'm to ask questions.

I have a linux cluster, bproc, gm, and am trying to configure version 1.2.4.
This is my configure command line:

 ./configure
--prefix=/usr/projects/hpctools/jennyp/openmpi124flash/openmpi-1.2.4
--libdir=/usr/projects/hpctools/jennyp/openmpi124flash/openmpi-1.2.4/lib64
--with-bproc --without-tm --enable-shared
--with-io_romio_flags=--with-file-system=ufs+nfs+panfs --without-pty_support

( I threw in the --without-pty-support just to see if it would make a
difference.)

The make and install seem to proced ok. 
The test I have is an IMB-MPI1 test, and I am getting these warnings, no
matter what compiler I
m using.:

 >mpirun -n 6 ./IMB-MPI1
[n110:26208] mca: base: component_find: unable to open ras tm: file not
found 
(ignored)
[n110:26208] mca: base: component_find: unable to open pls tm: file not
found 
(ignored)
[n111:26215] mca: base: component_find: unable to open ras tm: file not
found 
(ignored)
[n111:26212] mca: base: component_find: unable to open ras tm: file not
found 
(ignored)
[n112:26216] mca: base: component_find: unable to open ras tm: file not
found 
(ignored)
[n112:26213] mca: base: component_find: unable to open ras tm: file not
found 
(ignored)
[n110:26211] mca: base: component_find: unable to open ras tm: file not
found 
(ignored)
[n110:26214] mca: base: component_find: unable to open ras tm: file not
found 
(ignored)
[n111:26212] mca: base: component_find: unable to open pls tm: file not
found 
(ignored)
[n111:26215] mca: base: component_find: unable to open pls tm: file not
found 
(ignored)
[n112:26213] mca: base: component_find: unable to open pls tm: file not
found 
(ignored)
[n110:26211] mca: base: component_find: unable to open pls tm: file not
found 
(ignored)
[n110:26214] mca: base: component_find: unable to open pls tm: file not
found 
(ignored)
[n112:26216] mca: base: component_find: unable to open pls tm: file not
found 
(ignored)


I was told today that the --without-tm parameter  is being ignored in the
configure statement.

Anyone know what is going on?

Thanks,

---
Jennis (Jenny)
Los Alamos National Laboratory
505-667-1955

[OMPI devel] v1.2 branch mpi_preconnect_all

[Don Kerr]

All,

I have noticed an issue in the 1.2 branch when mpi_preconnect_all=1. The 
one way communication pattern (ranks either send or receive from each 
other) may not fully establish connection with peers. Example, if I have 
a 3 process mpi job and rank 0 does not do any mpi communication after 
MPI_Init() the other ranks attempts to connect will not be progressed (I 
have seen this with tcp and udapl).  
The preconnect pattern has changed slightly in the trunk but essentially 
it is still a one way communication, either send or receive with each 
rank. So although the issue I see in the 1.2 branch does not appear in 
the trunk I wonder if this will show up again.


An alternative to the preconnect pattern that comes to mind would be to 
perform a send and receive between all ranks to ensure that connections 
have been fully established.


Does anyone have thoughts or comments on this, or reasons not to have 
all ranks send and receive from all?


-DON

Re: [MTT devel] Handling "Interrupted system call" with MTT

[Jeff Squyres]

On Oct 17, 2007, at 9:31 AM, Ethan Mallove wrote:


Either or both of those would be fine (don't we have a timeout in
DoCommand.pm already?).


There is a timeout in DoCommand, but currently I keep
reinvoking DoCommand on each "interrupted system call" so
the timeout gets reset each time. This would not be the case
if the do-while were to go in DoCommand.


Ah -- I see what you're saying.  Good point -- I agree.


Then again, an
infinite loop is certain in the case of a command that is
*expected* to output "interrupted system call".


But only if that command *always* output "interrupted system call".   
So yes, I'm a bit paranoid about an unlikely corner case.  But we  
might as well handle it in the off-chance that it happens (and output  
a noisy error message so that you can tell if it happened, because  
that likely means that something is wrong with your cluster  
infrastructure).


And bang on your OS guys to fix the real problem while you're at  
it.  ;-)


--
Jeff Squyres
Cisco Systems

Re: [OMPI devel] RFC: delete mvapi BTL for v1.3

[Jeff Squyres]

Josh Aune confirms that it's ok for his customers to remove the mvapi  
BTL such that it will not be available from v1.3 and forward.


Since the RFC expired and only one person had any comment (which we  
resolved off-list), I have therefore svn rm'ed the mvapi BTL from the  
OMPI trunk (https://svn.open-mpi.org/trac/ompi/changeset/16483).


Woo hoo!



On Oct 11, 2007, at 2:56 PM, Jeff Squyres wrote:


Reminder -- this RFC expires tonight.

Speak now or forever hold your peace...


On Oct 5, 2007, at 7:46 AM, Jeff Squyres wrote:


WHAT: Remove the mvapi BTL for the v1.3 release.

WHY: None of the IB vendors want to maintain it anymore; our future
is OFED.  If someone still has mvapi IB drivers, they can use the
OMPI v1.2 series.

WHERE: svn rm ompi/mca/btl/mvapi

WHEN: Before the v1.3 release.

TIMEOUT: COB, Thurs, Oct 11, 2007

-

None of the IB vendors are interested in maintaining the "mvapi" BTL
anymore.  Indeed, none of us have updated it with any of the new/
interesting/better performance features that went into the openib BTL
over the past year (or more).  Additionally, some changes may be
coming in the OMPI infrastructure that would *require* some revamping
in the mvapi BTL -- and no one of Cisco, Voltaire, Mellanox is
willing to do it.

So we'd like to ditch the mvapi BTL starting with v1.3 and have the
official guidance be that if you have mvapi, you need to use the OMPI
v1.2 series (i.e., remove this from the SVN trunk in the Very Near
Future).

--
Jeff Squyres
Cisco Systems

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Jeff Squyres
Cisco Systems

___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Jeff Squyres
Cisco Systems

Re: [MTT devel] Handling "Interrupted system call" with MTT

[Ethan Mallove]

On Wed, Oct/17/2007 07:45:53AM, Jeff Squyres wrote:
> On Oct 16, 2007, at 6:36 PM, Ethan Mallove wrote:
> 
> >>> The bail is that "make" will eventually succeed or fail
> >>> with something other than "interrupted system call". Do
> >>> we need another condition?
> >>
> >> I'm just worried that Sun's NFS will somehow get in a
> >> perpetual "interrupted system call" loop such that you'll
> >> never actually break out of it.
> >
> > How about a counter? E.g., after "x" number of "interrupted
> > system call" messages, MTT moves on. Or should the "command
> > restart" go in DoCommand.pm so we can have a timeout?
>
> Either or both of those would be fine (don't we have a timeout in  
> DoCommand.pm already?).

There is a timeout in DoCommand, but currently I keep
reinvoking DoCommand on each "interrupted system call" so
the timeout gets reset each time. This would not be the case
if the do-while were to go in DoCommand. Then again, an
infinite loop is certain in the case of a command that is
*expected* to output "interrupted system call".

-Ethan

> 
> > I also noticed that our build script (which prints hundreds
> > of "interrupted system call" messages per build, but does
> > not seem to die because of them) uses "make -j 24" while MTT
> > has been using "make -j 4". I'll experiment with -j.
> 
> I know that Terry/Sun and co. spent a good amount of time trying to  
> solve the "interrupted system call" error -- they may have some more  
> information for you, such as how/why it happens...?
> 
> -- 
> Jeff Squyres
> Cisco Systems
> 
> ___
> mtt-devel mailing list
> mtt-de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel