We talked about this issue on the weekly OMPI engineering teleconf today.
It seems like it would be a good idea to bring over the new shared memory
revamp to the v1.5 series before it transitions to v1.6 so that it can avoid
network-mounted /tmp filesystem issues. LANL will be evaluating this;
On Nov 7, 2011, at 12:12 PM, Blosch, Edwin L wrote:
> Thanks for the valuable input. I'll change to a wait-and-watch approach.
>
> The FAQ on tuning sm says "If the session directory is located on a network
> filesystem, the shared memory BTL latency will be extremely high." And the
> title is
users] EXTERNAL: Re: How to set up state-less node /tmp for
OpenMPI usage
On 11/05/2011 09:11 AM, Blosch, Edwin L wrote:
..
>
> I know where you're coming from, and I probably didn't title the post
> correctly because I wasn't sure what to ask. But I definitely saw it,
On 11/05/2011 09:11 AM, Blosch, Edwin L wrote:
..
I know where you're coming from, and I probably didn't title the post correctly
because I wasn't sure what to ask. But I definitely saw it, and still see it,
as an OpenMPI issue. Having /tmp mounted over NFS on a stateless cluster is
not a
Thanks, Ralph,
> Having a local /tmp is typically required by Linux for proper operation as
> the OS itself needs to ensure its usage is protected, as was > previously
> stated and is reiterated in numerous books on managing Linux systems.
There is a /tmp, but it's not local. I don't know if
Original Message-
From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
Behalf Of David Turner
Sent: Thursday, November 03, 2011 6:36 PM
To: us...@open-mpi.org
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node
/tmp
for OpenMPI usage
I'm not a syste
is in the trunk for future releases), would be reasonable.
>
>
> Thanks
>
>
>
> -Original Message-----
> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
> Behalf Of David Turner
> Sent: Friday, November 04, 2011 1:38 AM
> To:
.@open-mpi.org] On Behalf
Of David Turner
Sent: Friday, November 04, 2011 1:38 AM
To: Open MPI Users
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp for
OpenMPI usage
% df /tmp
Filesystem 1K-blocks Used Available Use% Mounted on
- 1
Behalf Of Eugene Loh
Sent: Thursday, November 03, 2011 1:23 PM
To: us...@open-mpi.org
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less
node
/tmp for OpenMPI usage
Right. Actually "--mca btl ^sm". (Was missing "btl".)
On 11/3/2011 11:19 AM, Blosch, Edwin L
he application for available memory.
>>
>> On 11/3/11 8:40 PM, Ed Blosch wrote:
>>> Thanks very much, exactly what I wanted to hear. How big is /tmp?
>>>
>>> -----Original Message-----
>>> From: users-boun...@open-mpi.org [mailto:users-boun...@open
vid Turner
Sent: Thursday, November 03, 2011 6:36 PM
To: us...@open-mpi.org
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node
/tmp
for OpenMPI usage
I'm not a systems guy, but I'll pitch in anyway. On our cluster,
all the compute nodes are completely diskless. The ro
27;t quite seem to get the question.
Thanks
-Original Message-
From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
Behalf Of Ralph Castain
Sent: Thursday, November 03, 2011 5:22 PM
To: Open MPI Users
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less nod
age-
>> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
Behalf Of Eugene Loh
>> Sent: Thursday, November 03, 2011 1:23 PM
>> To: us...@open-mpi.org
>> Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node
/tmp for OpenMPI usage
>
Thanks
-Original Message-
From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf
Of Ralph Castain
Sent: Thursday, November 03, 2011 5:22 PM
To: Open MPI Users
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp for
OpenMPI usage
O
ilto:users-boun...@open-mpi.org] On Behalf
Of Ralph Castain
Sent: Thursday, November 03, 2011 5:22 PM
To: Open MPI Users
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp for
OpenMPI usage
On Nov 3, 2011, at 2:55 PM, Blosch, Edwin L wrote:
> I might be missing someth
ovember 03, 2011 1:23 PM
> To: us...@open-mpi.org
> Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp
> for OpenMPI usage
>
> Right. Actually "--mca btl ^sm". (Was missing "btl".)
>
> On 11/3/2011 11:19 AM, Blosch, Edwin L wrote:
&g
.@open-mpi.org [mailto:users-boun...@open-mpi.org] On
>> Behalf Of Eugene Loh
>> Sent: Thursday, November 03, 2011 12:54 PM
>> To: us...@open-mpi.org
>> Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less
From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf
Of Eugene Loh
Sent: Thursday, November 03, 2011 1:23 PM
To: us...@open-mpi.org
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp for
OpenMPI usage
Right. Actually "--mca btl ^sm". (W
something like --mca ^sm?
-Original Message-
From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf
Of Eugene Loh
Sent: Thursday, November 03, 2011 12:54 PM
To: us...@open-mpi.org
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp for
Ope
ehalf
Of Eugene Loh
Sent: Thursday, November 03, 2011 12:54 PM
To: us...@open-mpi.org
Subject: Re: [OMPI users] EXTERNAL: Re: How to set up state-less node /tmp for
OpenMPI usage
I've not been following closely. Why must one use shared-memory
communications? How about using other BTLs in a
On Nov 3, 2011, at 1:36 PM, Blosch, Edwin L wrote:
> Yes it sucks, so that's what led me to post my original question: If /dev/shm
> isn't the right place to put the session file, and /tmp is NFS-mounted, then
> what IS the "right" way to set up a diskless cluster? I don't think the idea
> of
I've not been following closely. Why must one use shared-memory
communications? How about using other BTLs in a "loopback" fashion?
Cross-thread response here, as this is related to the shared-memory thread:
Yes it sucks, so that's what led me to post my original question: If /dev/shm
isn't the right place to put the session file, and /tmp is NFS-mounted, then
what IS the "right" way to set up a diskless cluster? I don't th
23 matches
Mail list logo