[OMPI devel] SM BTL NUMA awareness patches

are alway welcome. -- Gleb. commit 883db5e1ce8c3b49cc1376e6acf9c2d5d0d77983 Author: Gleb Natapov <g...@voltaire.com> List-Post: devel@lists.open-mpi.org Date: Tue May 27 14:55:11 2008 +0300 Add functions to maffinity. diff --git a/opal/mca/maffinity/base/base.h

Re: [OMPI devel] Open MPI session directory location

On Tue, May 27, 2008 at 08:27:49AM -0600, Ralph H Castain wrote: > -mca orte_tmpdir_base foo Thanks! It works. But this parameter is not reported by ompi_info :( > > > > On 5/27/08 8:24 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > Hi, > >

[OMPI devel] Open MPI session directory location

Hi, Is there a way to change where Open MPI creates session directory. I can't find mca parameter that specifies this. -- Gleb.

Re: [OMPI devel] Memory hooks stuff

On Fri, May 23, 2008 at 07:19:01AM -0400, Jeff Squyres wrote: > Brian and I were chatting the other day about random OMPI stuff and > the topic of the memory hooks came up again. Brian was wondering if > we should [finally] revisit this topic -- there's a few things that > could be done to

Re: [OMPI devel] RFC: Linuxes shipping libibverbs

On Thu, May 22, 2008 at 08:30:52PM +, Dirk Eddelbuettel wrote: > > > Also, if this test depends on the Debian kernel packages, then we're > > > back to square one as some folks (like myself) run binary kernels, > > > other may just hand-compile and this test may not work as we may miss > > >

Re: [OMPI devel] RFC: Linuxes shipping libibverbs

On Thu, May 22, 2008 at 04:19:05PM -0400, Jeff Squyres wrote: > On May 22, 2008, at 4:07 PM, Dirk Eddelbuettel wrote: > > > Is there a test I could run for you? > > Can you see if /dev/infiniband exists? If it does, the OpenFabrics > kernel drivers are running. If not, they aren't. Either

Re: [OMPI devel] Threaded progress for CPCs

On Mon, May 19, 2008 at 01:38:53PM -0400, Jeff Squyres wrote: > >> 5. ...? > > What about moving posting of receive buffers into main thread. With > > SRQ it is easy: don't post anything in CPC thread. Main thread will > > prepost buffers automatically after first fragment received on the > >

Re: [OMPI devel] Threaded progress for CPCs

On Mon, May 19, 2008 at 01:52:22PM -0500, Jon Mason wrote: > On Mon, May 19, 2008 at 05:17:57PM +0300, Gleb Natapov wrote: > > On Mon, May 19, 2008 at 05:08:17PM +0300, Pavel Shamis (Pasha) wrote: > > > >> 5. ...? > > > >> > > > > What about

Re: [OMPI devel] Threaded progress for CPCs

On Mon, May 19, 2008 at 07:39:13PM +0300, Pavel Shamis (Pasha) wrote: So this solution will cost 1 buffer on each srq ... sounds acceptable for me. But I don't see too much difference compared to #1, as I understand we anyway will be need the pipe for communication with

Re: [OMPI devel] Threaded progress for CPCs

On Mon, May 19, 2008 at 05:08:17PM +0300, Pavel Shamis (Pasha) wrote: > >> 5. ...? > >> > > What about moving posting of receive buffers into main thread. With > > SRQ it is easy: don't post anything in CPC thread. Main thread will > > prepost buffers automatically after first fragment

Re: [OMPI devel] Threaded progress for CPCs

On Sun, May 18, 2008 at 11:38:36AM -0400, Jeff Squyres wrote: > ==> Remember that the goal for this work was to have a separate > progress thread *without* all the heavyweight OMPI thread locks. > Specifically: make it work in a build without --enable-progress- > threads or

Re: [OMPI devel] openib btl code review

; that Nysal found. > > Please see the most recent patch on the ticket. Looks good to me. > > > > On May 15, 2008, at 11:01 AM, Jeff Squyres wrote: > > > On May 15, 2008, at 8:46 AM, Gleb Natapov wrote: > > > >>> Any other reviewers wo

Re: [OMPI devel] openib btl code review

On Thu, May 15, 2008 at 08:14:29AM -0400, Jeff Squyres wrote: > Pasha tells me he'll be able to review the patch next week, so I'll > wait to commit until then. I added the patch to the ticket, just so > that it doesn't get lost. > > Any other reviewers would be welcome... :-) I'll look at

Re: [OMPI devel] Unbelievable situation BUG

On Sun, Apr 27, 2008 at 07:00:57PM +0300, Lenny Verkhovsky wrote: > Hi, all > > I faced the "Unbelievable situation" The situation is believable, but commit r18274, that adds this output, is not, as it doesn't take into account sequence number wrap around. > > during running IMB benchmark. >

Re: [OMPI devel] Affect of compression on modex and launch messages

On Mon, Apr 07, 2008 at 07:28:07AM -0600, Ralph H Castain wrote: > > Also can you explain how > > allgather is implemented in orte (sorry if you already explained this once > > and I missed it). > > The default method is for each proc to send its modex data to its local > daemon. The local daemon

Re: [OMPI devel] Affect of compression on modex and launch messages

On Mon, Apr 07, 2008 at 07:07:38AM -0600, Ralph H Castain wrote: > > > > On 4/7/08 7:04 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > On Fri, Apr 04, 2008 at 10:52:38AM -0600, Ralph H Castain wrote: > >> With compression "on&

Re: [OMPI devel] RFC: changes to modex

On Wed, Apr 02, 2008 at 12:08:47PM -0400, Jeff Squyres wrote: > On Apr 2, 2008, at 11:13 AM, Gleb Natapov wrote: > > On Wed, Apr 02, 2008 at 10:35:03AM -0400, Jeff Squyres wrote: > >> If we use carto to limit hcas/ports are used on a given host on a > >> per- > >

Re: [OMPI devel] RFC: changes to modex

On Wed, Apr 02, 2008 at 10:35:03AM -0400, Jeff Squyres wrote: > If we use carto to limit hcas/ports are used on a given host on a per- > proc basis, then we can include some proc_send data to say "this proc > only uses indexes X,Y,Z from the node data". The indexes can be > either uint8_ts,

Re: [OMPI devel] RFC: changes to modex

On Wed, Apr 02, 2008 at 10:21:12AM -0400, Jeff Squyres wrote: > * int ompi_modex_proc_send(...): send modex data that is specific to > this process. It is just about exactly the same as the current API > call (ompi_modex_send). > [skip] > > * int ompi_modex_node_send(...): send modex

Re: [OMPI devel] Switching away from SVN?

On Fri, Mar 21, 2008 at 08:52:03AM -0400, Jeff Squyres wrote: > Cool -- thanks Roland! > > For anyone who wants to play with the entire history of OMPI in git > (as of last night or so -- this git repository is *not* being kept in > sync with SVN), I cloned the tree that Roland created and

Re: [OMPI devel] OMPI OpenIB Credit Schema breaks Chelsio HW

On Mon, Mar 10, 2008 at 09:50:13AM -0500, Steve Wise wrote: > > I personally don't like the idea to add another layer of complexity to > > openib > > BTL code just to work around HW that doesn't follow spec. If work around > > is simple that is OK, but in this case it is not so simple and will

Re: [OMPI devel] OMPI OpenIB Credit Schema breaks Chelsio HW

On Sun, Mar 09, 2008 at 02:48:09PM -0500, Jon Mason wrote: > Issue (as described by Steve Wise): > > Currently OMPI uses qp 0 for all credit updates (by design). This breaks > when running over the chelsio rnic due to a race condition between > advertising the availability of a buffer using qp0

Re: [OMPI devel] orte can't launch process

On Thu, Mar 06, 2008 at 07:49:13AM -0500, Tim Prins wrote: > Sorry about that. I removed a field in a structure, then 'svn up' seems > to have added it back, so we were using a field that should not even > exist in a couple places. > > Should be fixed in r17757 Works again. Thanks --

[OMPI devel] orte can't launch process

Something is broken in the trunk. # mpirun -np 2 -H host1,host2 ./osu_latency -- Some of the requested hosts are not included in the current allocation. The requested hosts were specified with --host as: host1,host2 Please

Re: [OMPI devel] RDMA pipeline

t expose it to users, but set it automatically if message logging is enabled. > Thanks, > george. > > On Feb 20, 2008, at 4:29 AM, Gleb Natapov wrote: > >> On Tue, Feb 19, 2008 at 10:40:46PM -0500, George Bosilca wrote: >>> Actually, it restores the origin

Re: [OMPI devel] RDMA pipeline

On Tue, Feb 19, 2008 at 02:13:30PM -0500, George Bosilca wrote: > Few days ago during some testing I realize that the RDMA pipeline was > disabled for MX and Elan (I didn't check for the others). A quick look > into the source code, pinpointed the problem into the pml_ob1_rdma.c > file, and

Re: [OMPI devel] btl_openib_rnr_retry MCA param

now > it says RNR happened, and goes into detail into what that means -- but > that's not the real problem). > Good point. > I'll do that as well. Thanks! > > > On Feb 13, 2008, at 12:59 AM, Gleb Natapov wrote: > > > On Tue, Feb 12, 2008 at 05:41:13PM -0500, J

Re: [OMPI devel] [RFC] Remove explicit call to progress() from ob1.

The much better question is "Why they are necessary?", because if there is not good answer to this question then they should be removed, since they are harmful as they cause uncontrollable recursion calls. > > On Feb 12, 2008, at 5:27 AM, Gleb Natapov wrote: > >

Re: [OMPI devel] btl_openib_rnr_retry MCA param

On Tue, Feb 12, 2008 at 05:41:13PM -0500, Jeff Squyres wrote: > I see that in the OOB CPC for the openib BTL, when setting up the send > side of the QP, we set the rnr_retry value depending on whether the > remote receive queue is a per-peer or SRQ: > > - SRQ: btl_openib_rnr_retry MCA param

Re: [OMPI devel] Something wrong with vt?

make' ? > Which version of the autotools are you using? > > > Matthias > > On Mo, 2008-02-11 at 11:42 +0200, Gleb Natapov wrote: > > > I get the following error while "make install": > > > > make[2]: Entering directory `/home_local/glebn/build_dbg

[OMPI devel] [RFC] Remove explicit call to progress() from ob1.

Hi, I am planning to commit the following patch. Those two progress() calls are responsible for most of our deep recursion troubles. And I also think they are completely unnecessary. diff --git a/ompi/mca/pml/ob1/pml_ob1_recvreq.c b/ompi/mca/pml/ob1/pml_ob1_recvreq.c index 5899243..641176e

[OMPI devel] Something wrong with vt?

I get the following error while "make install": make[2]: Entering directory `/home_local/glebn/build_dbg/ompi/contrib/vt' Making install in vt make[3]: Entering directory `/home_local/glebn/build_dbg/ompi/contrib/vt/vt' make[3]: *** No rule to make target `install'. Stop. make[3]: Leaving

Re: [OMPI devel] 32 bit udapl warnings

On Thu, Jan 31, 2008 at 08:45:54AM -0500, Don Kerr wrote: > This was brought to my attention once before but I don't see this > message so I just plain forgot about it. :-( > uDAPL defines its pointers as uint64, "typedef DAT_UINT64 DAT_VADDR", > and pval is a "void *" which is why the message

Re: [OMPI devel] open ib btl and xrc

On Fri, Jan 18, 2008 at 11:43:03AM -0500, Jeff Squyres wrote: > I think the main savings is that mellanox hardware works better when > fewer qp's are open. I.e., it's a resource issue on the HCA, not > necessarily a savings in posting buffers to the qp. Interesting. I hear this justification

Re: [OMPI devel] [PATCH] openib btl: extensable cpcselection enablement

On Mon, Jan 14, 2008 at 08:15:23AM -0500, Jeff Squyres (jsquyres) wrote: > Any obj to bringing this stuff to the trunk? The moden string opt stuff can > be done directly on the trunk imo. Go ahead. -- Gleb.

[OMPI devel] ptmalloc and pin down cache problems again

Hi Brian, I encountered problem with ptmalloc an registration cache. I see that you (I think it was you) disabled shrinking of a heap memory allocated by sbrk by setting MORECORE_CANNOT_TRIM to 1. The comment explains that it should be done because freeing of small objects is not reentrant so if

Re: [OMPI devel] Common initialization code for IB.

On Thu, Jan 03, 2008 at 09:27:14AM -0500, Jeff Squyres wrote: > > Another > > problem is how multicast collective knows that all processes in a > > communicator are reachable via the same network, do we have a > > mechanism > > in ompi to check this? > > > Good question. > > Perhaps the

[OMPI devel] Common initialization code for IB.

Hi, In Paris we've talked about putting HCA discovery and initialization code outside of openib BTL so other components that want to use IB will be able to share common code, data and registration cache. Other components I am thinking about are ofud and multicast collectives. I started to look

Re: [OMPI devel] [ofa-general] [RFC] XRC -- make receiving XRC QP independent of any one user process

On Mon, Dec 24, 2007 at 11:49:37PM +, Tang, Changqing wrote: > > > > -Original Message- > > From: Pavel Shamis (Pasha) [mailto:pa...@dev.mellanox.co.il] > > Sent: Monday, December 24, 2007 8:03 AM > > To: Tang, Changqing > > Cc: Jack Morgenstein; Roland Dreier; > >

Re: [OMPI devel] openib xrc CPC minor nit

On Thu, Dec 20, 2007 at 05:39:36PM -0500, Jeff Squyres wrote: > Pasha -- > > I notice in the port info struct that you have a member for the lid, > but only #if HAVE_XRC. Per a comment in the code, this is supposed to > save bytes when we're using OOB (because we don't need this value in >

Re: [OMPI devel] matching code rewrite in OB1

; What do you think ? I think that coverage testing I did is enough for this code. > Rich > > > On 12/17/07 8:32 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > On Thu, Dec 13, 2007 at 08:04:21PM -0500, Richard Graham wrote: > >> > Yes,

Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r16969

On Mon, Dec 17, 2007 at 10:53:26AM -0500, Jeff Squyres wrote: > Gleb - > > Is this picture of the v1.3 long message params accurate? (see attached) Yes. -- Gleb.

Re: [OMPI devel] matching code rewrite in OB1

On Thu, Dec 13, 2007 at 08:04:21PM -0500, Richard Graham wrote: > Yes, should be a bit more clear. Need an independent way to verify that > data is matched > in the correct order ­ sending this information as payload is one way to do > this. So, > sending unique data in every message, and

Re: [OMPI devel] rb rcache component

On Sat, Dec 15, 2007 at 08:27:29AM -0500, Jeff Squyres wrote: > It doesn't look like this component is used anymore > (it's .ompi_ignore'd). > > Anyone object to svn rm'ing it on the trunk? > Not me. -- Gleb.

Re: [OMPI devel] New BTL parameter

If there is no objection I will commit this to the trunk next week. On Sun, Dec 09, 2007 at 05:34:30PM +0200, Gleb Natapov wrote: > Hi, > > Currently BTL has parameter btl_min_send_size that is no longer used. > I want to change it to be btl_rndv_eager_limit. This n

Re: [OMPI devel] matching code rewrite in OB1

also be good - actually better than hoping that one will hit out-of-order > situations. > > Rich > > > On 12/14/07 2:20 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > On Thu, Dec 13, 2007 at 06:16:49PM -0500, Richard Graham wrote: > >> The

Re: [OMPI devel] [PATCH] openib: clean-up connect to allow for new cm's

On Thu, Dec 13, 2007 at 10:49:45AM +0200, Pavel Shamis (Pasha) wrote: >> Because we want to support mixed setups and create XRC between nodes that >> support it and RC between all other nodes. >> > Ok, sounds reasonable for me. Just need make sure that the parameters name > will be user

Re: [OMPI devel] matching code rewrite in OB1

On Wed, Dec 12, 2007 at 03:10:10PM -0600, Brian W. Barrett wrote: > On Wed, 12 Dec 2007, Gleb Natapov wrote: > > > On Wed, Dec 12, 2007 at 03:46:10PM -0500, Richard Graham wrote: > >> This is better than nothing, but really not very helpful for looking at the > >>

Re: [OMPI devel] matching code rewrite in OB1

On Wed, Dec 12, 2007 at 03:52:17PM -0500, Jeff Squyres wrote: > On Dec 12, 2007, at 3:20 PM, Gleb Natapov wrote: > > >> How about making a tarball with this patch in it that can be thrown > >> at > >> everyone's MTT? (we can put the tarball on www.open-mpi.org

Re: [OMPI devel] matching code rewrite in OB1

ng it w/o some very > > strong > > reasons. Not apposed, just very cautious. > > > > Rich > > > > > > On 12/11/07 11:47 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > >> On Tue, Dec 11, 2007 at 08:36:42AM -0800, Andrew Friedley

Re: [OMPI devel] SCTP BTL exclusivity value problem

On Wed, Dec 12, 2007 at 10:31:37AM -0500, Jeff Squyres wrote: > I'd be in favor of setting the TCP exclusivity to LOW+100 and setting > SCTP exclusivity to LOW. Fine with me. > > > On Dec 12, 2007, at 10:07 AM, Gleb Natapov wrote: > > > On Wed, Dec 12, 2007 at 10:02:

Re: [OMPI devel] SCTP BTL exclusivity value problem

the SCTP BTL is being built? What kind of > environment is it? Red Hat Enterprise Linux AS release 4 (Nahant Update 5) # rpm -qa | grep sctp lksctp-tools-devel-1.0.2-6.4E.1 lksctp-tools-doc-1.0.2-6.4E.1 lksctp-tools-1.0.2-6.4E.1 > > > > On Dec 12, 2007, at 9:38 AM, Gleb Natapov

Re: [OMPI devel] [PATCH] openib: clean-up connect to allow for new cm's

On Wed, Dec 12, 2007 at 04:08:31PM +0200, Pavel Shamis (Pasha) wrote: > Gleb Natapov wrote: >> On Wed, Dec 12, 2007 at 03:37:26PM +0200, Pavel Shamis (Pasha) wrote: >> >>> Gleb Natapov wrote: >>> >>>> On Tue, Dec 11, 2007 at 08:16:07PM -0500,

Re: [OMPI devel] [PATCH] openib: clean-up connect to allow for new cm's

On Wed, Dec 12, 2007 at 03:37:26PM +0200, Pavel Shamis (Pasha) wrote: > Gleb Natapov wrote: > > On Tue, Dec 11, 2007 at 08:16:07PM -0500, Jeff Squyres wrote: > > > >> Isn't there a better way somehow? Perhaps we should have "select" > >> call *al

Re: [OMPI devel] [PATCH] openib: clean-up connect to allow for new cm's

On Tue, Dec 11, 2007 at 08:16:07PM -0500, Jeff Squyres wrote: > Isn't there a better way somehow? Perhaps we should have "select" > call *all* the functions and accept back a priority. The one with the > highest priority then wins. This is quite similar to much of the > other selection

Re: [OMPI devel] matching code rewrite in OB1

; > > > Rich > > > > > > On 12/11/07 10:54 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > >> Hi, > >> > >>I did a rewrite of matching code in OB1. I made it much simpler and 2 > >> t

[OMPI devel] matching code rewrite in OB1

Hi, I did a rewrite of matching code in OB1. I made it much simpler and 2 times smaller (which is good, less code - less bugs). I also got rid of huge macros - very helpful if you need to debug something. There is no performance degradation, actually I even see very small performance

Re: [OMPI devel] opal_condition_wait

On Tue, Dec 11, 2007 at 10:27:55AM -0500, Tim Prins wrote: > My understanding was that this behavior was not right, but upon further > inspection of the pthreads documentation this behavior seems to be > allowable. > I think that Open MPI does not implement condition variable in the strict

[OMPI devel] Changes to all BTLs.

Hi everybody, I committed changes to BTL interface. Two new parameters are now provided to descriptor allocation: endpoint and flags. I did my best to change all in tree BTLs, but I can't compile all of them, so compilation problems are possible. Can everybody test that the BTLs they care about

Re: [OMPI devel] 32-bit openib is broken on the trunk as of Nov 27th, r16799

On Wed, Dec 05, 2007 at 02:45:17PM -0500, Tim Mattox wrote: > Hello, > It appears that sometime after r16777, and by r16799, that something > was broken on the trunk's openib support for 32-bit builds. > The 64-bit tests all seem normal, as well as the 32-bit & 64-bit tests on > the 1.2 branch on

Re: [OMPI devel] opal_condition_wait

On Thu, Dec 06, 2007 at 09:46:45AM -0500, Tim Prins wrote: > Also, when we are using threads, there is a case where we do not > decrement the signaled count, in condition.h:84. Gleb put this in in > r9451, however the change does not make sense to me. I think that the > signal count should

Re: [OMPI devel] tmp XRC branches

On Fri, Nov 30, 2007 at 02:06:02PM -0500, Jeff Squyres wrote: > Are any of the XRC tmp SVN branches still relevant? Or have they now > been integrated into the trunk? > > I ask because I see 4 XRC-related branches out there under /tmp and / > tmp-public. They are not relevant any more. I'll

Re: [OMPI devel] IB/OpenFabrics pow wow

On Fri, Nov 16, 2007 at 11:36:39AM -0800, Jeff Squyres wrote: > 1. Mon, 26 Nov, 10am US East, 7am US Pacific, 5pm Israel > 2. Mon, 26 Nov, 11am US East, 8am US Pacific, 6pm Israel > 3. Thu, 29 Nov, 10am US East, 7am US Pacific, 5pm Israel > 4. Thu, 29 Nov, 11am US East, 8am US Pacific, 6pm Israel

Re: [OMPI devel] [OMPI svn] svn:open-mpi r16723

On Wed, Nov 14, 2007 at 06:44:06AM -0800, Tim Prins wrote: > Hi, > > The following files bother me about this commit: > trunk/ompi/mca/btl/sctp/sctp_writev.c > trunk/ompi/mca/btl/sctp/sctp_writev.h > > They bother me for 2 reasons: > 1. Their naming does not follow the prefix rule > 2.

Re: [OMPI devel] Multi-Rail and Open IB BTL

Sorry I missed a mail with the question. On Mon, Nov 12, 2007 at 06:03:07AM -0500, Jeff Squyres wrote: > On Nov 9, 2007, at 1:24 PM, Don Kerr wrote: > > > both, I was thinking of listing what I think are multi-rail > > requirements > > but wanted to understand what the current state of things

Re: [OMPI devel] collective problems

On Wed, Nov 07, 2007 at 11:25:43PM -0500, Patrick Geoffray wrote: > Richard Graham wrote: > > The real problem, as you and others have pointed out is the lack of > > predictable time slices for the progress engine to do its work, when relying > > on the ULP to make calls into the library... > >

Re: [OMPI devel] collective problems

On Wed, Nov 07, 2007 at 01:16:04PM -0500, George Bosilca wrote: > > On Nov 7, 2007, at 12:51 PM, Jeff Squyres wrote: > >>> The same callback is called in both cases. In the case that you >>> described, the callback is called just a little bit deeper into the >>> recursion, when in the "normal

Re: [OMPI devel] Multi-Rail and Open IB BTL

On Thu, Nov 01, 2007 at 11:15:21AM -0400, Don Kerr wrote: > How would the openib btl handle the following scenario: > Two nodes, each with two ports, all ports are on the same subnet and switch. > > Would striping occur over 4 connections or 2? Only two connections will be created. > > If 2 is

[OMPI devel] bml_btl->btl_alloc() instead of mca_bml_base_alloc() in OSC

Hi Brian, Is there a special reason why you call btl functions directly instead of using bml wrappers? What about applying this patch? diff --git a/ompi/mca/osc/rdma/osc_rdma_component.c b/ompi/mca/osc/rdma/osc_rdma_component.c index 2d0dc06..302dd9e 100644 ---

Re: [OMPI devel] RFC: Add "connect" field to openib BTL INI file

On Thu, Oct 25, 2007 at 10:55:25AM -0400, Jeff Squyres wrote: > On Oct 25, 2007, at 10:35 AM, Gleb Natapov wrote: > > > I don't think xrc should be used by default even if HW supports it. > > Only if > > special config option is set xrc should be attempted. > &

Re: [OMPI devel] RFC: Add "connect" field to openib BTL INI file

On Wed, Oct 24, 2007 at 08:01:44PM -0400, Jeff Squyres wrote: > My proposal is that the "connect" field can be added to the INI file > and take a comma-delimited list of values of acceptable CPCs for a > given device. For example, the ConnectX HCA can take the following > value: > >

Re: [OMPI devel] collective problems

on problem the fix to the problem will be a couple of lines of code. > > - Galen > > > > On 10/11/07 11:26 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > On Fri, Oct 05, 2007 at 09:43:44AM +0200, Jeff Squyres wrote: > >> David -- > &

Re: [OMPI devel] putting common request completion waiting code into separate inline function

laces :) > > > On Oct 15, 2007, at 10:27 AM, Gleb Natapov wrote: > > > Hi, > > > >Each time a someone needs to wait for request completion he > > implements the same piece of code. Why not put this code into > > inline fun

[OMPI devel] putting common request completion waiting code into separate inline function

Hi, Each time a someone needs to wait for request completion he implements the same piece of code. Why not put this code into inline function and use it instead. Look at the included patch, it moves the common code into ompi_request_wait_completion() function. Does somebody have any objection

Re: [OMPI devel] collective problems

On Fri, Oct 05, 2007 at 09:43:44AM +0200, Jeff Squyres wrote: > David -- > > Gleb and I just actively re-looked at this problem yesterday; we > think it's related to https://svn.open-mpi.org/trac/ompi/ticket/ > 1015. We previously thought this ticket was a different problem, but > our

Re: [OMPI devel] osu_bibw failing for message sizes 2097152 and larger

On Wed, Sep 19, 2007 at 10:26:15AM -0400, Dan Lacher wrote: > In doing some runs with the osu_bibw test on a single node, we have > found that it hands when using the trunk for message sizes 2097152 or > larger unless the mpool_sm_min_size is set to a number larger than the > message size. We

Re: [OMPI devel] Commit r16105

collective is executed on old communicator after setup of a new >> cid. Is this not enough to solve the problem? Some ranks may leave >> this collective call earlier than others, but none can leave it before >> all ranks enter it and at this stage new communicator is already ex

[OMPI devel] Commit r16105

George, In the comment you are saying that "a message for a not yet existing communicator can happen". Can you explain in what situation it can happen? Thanks, -- Gleb.

Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r16088

On Tue, Sep 11, 2007 at 11:30:53AM -0400, George Bosilca wrote: > > On Sep 11, 2007, at 11:05 AM, Gleb Natapov wrote: > >> On Tue, Sep 11, 2007 at 10:54:25AM -0400, George Bosilca wrote: >>> We don't want to prevent two thread from entering the code is same time. >

Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r16088

on > the same communicator, it won't work. Correct, but this is not what happens with mt_coll test. mt_coll calls commdup on the same communicator in different threads concurrently, but we handle this case inside ompi_comm_nextcid(). > > > Gleb Natapov wrote: > > On T

Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r16088

On Tue, Sep 11, 2007 at 10:14:30AM -0400, George Bosilca wrote: > Gleb, > > This patch is not correct. The code preventing the registration of the same > communicator twice is later in the code (same file in the function > ompi_comm_register_cid line 326). Once the function

Re: [OMPI devel] [devel-core] [RFC] Exit without finalize

On Thu, Sep 06, 2007 at 06:50:43AM -0600, Ralph H Castain wrote: > WHAT: Decide upon how to handle MPI applications where one or more > processes exit without calling MPI_Finalize > > WHY:Some applications can abort via an exit call instead of > calling MPI_Abort when a

[OMPI devel] opal_atomic_lifo is not really atomic.

Hi, opal_atomic_lifo implementation suffers from ABA problem. Here is the code for opal_atomic_lifo_pop: 1 do { 2 item = lifo->opal_lifo_head; 3 if( opal_atomic_cmpset_ptr( &(lifo->opal_lifo_head), 4 item, 5

Re: [OMPI devel] SM BTL hang issue

Is this trunk or 1.2? On Wed, Aug 29, 2007 at 09:40:30AM -0400, Terry D. Dontje wrote: > I have a program that does a simple bucket brigade of sends and receives > where rank 0 is the start and repeatedly sends to rank 1 until a certain > amount of time has passed and then it sends and all done

Re: [OMPI devel] openib btl header caching

On Mon, Aug 13, 2007 at 03:59:28PM -0400, Richard Graham wrote: > > > > On 8/13/07 3:52 PM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > On Mon, Aug 13, 2007 at 09:12:33AM -0600, Galen Shipman wrote: > > Here are the > > items we hav

Re: [OMPI devel] openib btl header caching

so the first macro just sets up the convertor, the second populates > all the rest of the request state in the case that we will need it > later because the fragment doesn't hit the wire. > +++ > We all agreed. >

Re: [OMPI devel] problem with system() call and openib - blocks send/recv

On Mon, Aug 06, 2007 at 09:53:20AM -0400, Bill Wichser wrote: > We have run across an issue, probably more related to openib than to > openmpi but don't know how to resolve. > > Linux kernel - 2.6.9-55.0.2.ELsmp x86_64 fork (and thus system()) is not supported by openib in this kernel. To get

Re: [OMPI devel] openib credits problem

On Thu, Jul 26, 2007 at 04:29:40PM +0300, Gleb Natapov wrote: > On Thu, Jul 26, 2007 at 09:12:26AM -0400, Jeff Squyres wrote: > > I got a problem in MTT runs last night with the openib BTL w.r.t. > > credits: > > > > [...lots of IMB output...] > > #bytes

Re: [OMPI devel] Fwd: [Open MPI] #1101: MPI_ALLOC_MEM with 0 size must be valid

On Tue, Jul 24, 2007 at 11:20:11AM -0300, Lisandro Dalcin wrote: > On 7/23/07, Jeff Squyres wrote: > > Does anyone have any opinions on this? If not, I'll go implement > > option #1. > > Sorry, Jeff... just reading this. I think your option #1 is the > better. However, I

Re: [OMPI devel] LD_LIBRARY_PATH and process launch on a head node

gt; But it *does* provide an LD_LIBRARY_PATH that is pointing to your > >>>> openmpi > >>>> installation - it says it did it right here in your debug output: > >>>> > >>>>>>> [elfit1:14752] pls:rsh: reset LD_LIB

Re: [OMPI devel] LD_LIBRARY_PATH and process launch on a head node

On Wed, Jul 18, 2007 at 09:08:38PM +0300, Gleb Natapov wrote: > On Wed, Jul 18, 2007 at 09:08:47AM -0600, Ralph H Castain wrote: > > But this will lockup: > > > > pn1180961:~/openmpi/trunk rhc$ mpirun -n 1 -host pn1180961 printenv | grep > > LD > > &g

Re: [OMPI devel] LD_LIBRARY_PATH and process launch on a head node

On Wed, Jul 18, 2007 at 09:08:47AM -0600, Ralph H Castain wrote: > But this will lockup: > > pn1180961:~/openmpi/trunk rhc$ mpirun -n 1 -host pn1180961 printenv | grep > LD > > The reason is that the hostname in this last command doesn't match the > hostname I get when I query my interfaces, so

Re: [OMPI devel] LD_LIBRARY_PATH and process launch on a head node

On Wed, Jul 18, 2007 at 07:48:17AM -0600, Ralph H Castain wrote: > I believe that was fixed in r15405 - are you at that rev level? I am on the latest revision. > > > On 7/18/07 7:27 AM, "Gleb Natapov" <gl...@voltaire.com> wrote: > > > Hi, > >

Re: [OMPI devel] Build failures of 1.2.3 on Debian hppa, mips, mipsel, s390, m68k

On Sat, Jul 14, 2007 at 01:16:42PM -0400, George Bosilca wrote: > Instead of failing at configure time, we might want to disable the > threading features and the shared memory device if we detect that we > don't have support for atomics on a specified platform. In a non > threaded build, the

Re: [OMPI devel] [devel-core] Major reduction in ORTE

On Thu, Jul 12, 2007 at 03:04:01PM -0600, Ralph H Castain wrote: > As always, any thoughts/suggestions are welcomed. > I hope Sharon's work on process affinity will be merged into the trunk before this works begins and functionality will be preserved during the work. --

Re: [OMPI devel] patch for btl_sm.c fixing segmentation fault

On Wed, Jul 11, 2007 at 01:17:02PM +0200, Christoph Niethammer wrote: > Hello, > > > Since some time I'm testing Open MPI at the HRLS. My main topic there is the > thread support of Open MPI. > > Some time ago I found a segmentation fault when running the svn-trunk > Version. > Thanks to the

Re: [OMPI devel] Ob1 segfault

On Mon, Jul 09, 2007 at 10:41:52AM -0400, Tim Prins wrote: > Gleb Natapov wrote: > > On Sun, Jul 08, 2007 at 12:41:58PM -0400, Tim Prins wrote: > >> On Sunday 08 July 2007 08:32:27 am Gleb Natapov wrote: > >>> On Fri, Jul 06, 2007 at 06:36:13PM -0400, Tim Pri

Re: [OMPI devel] Ob1 segfault

On Sun, Jul 08, 2007 at 12:41:58PM -0400, Tim Prins wrote: > On Sunday 08 July 2007 08:32:27 am Gleb Natapov wrote: > > On Fri, Jul 06, 2007 at 06:36:13PM -0400, Tim Prins wrote: > > > While looking into another problem I ran into an issue which made ob1 > > > segfault o

Re: [OMPI devel] Ob1 segfault

On Fri, Jul 06, 2007 at 06:36:13PM -0400, Tim Prins wrote: > While looking into another problem I ran into an issue which made ob1 > segfault > on me. Using gm, and running the test test_dan1 in the onesided test suite, > if I limit the gm freelist by too much, I get a segfault. That is, > >

Re: [OMPI devel] Improve OB1 performance when multiple NICs are available

; On Jun 28, 2007, at 10:06 AM, Gleb Natapov wrote: > > >Nobody except George haven't commented/complained about this patch, > >so I > >assume everybody except George are OK with it. And from George mails I > >don't understand if he is OK with me applying it to the tr

  1   2   >