On May 25, 2011, at 7:52 AM, Joshua Randall wrote:
> I now seem to have solved the problem by recompiling PVFS2 with:
>> /* disable shmem to allow clients and servers on the same machine */
>> setenv("MX_DISABLE_SHMEM", "1",1);
>
>
> Changed to:
>> /* disable shmem to allow clients and servers on the same machine */
>> setenv("MX_DISABLE_SHMEM", "0",1);
>
> With shared communication now enabled in Open-MX, everything appears
> to be working and I can run pvfs2-server on all three machines and use
> the filesystem from a client on any of the machines!
>
> Can you solve the mystery of why shared memory needed to be disabled
> in MX, but seems to be required in Open-MX?
>
> Josh.
Josh,
I dug back through my email and found an issue from 2007:
> For the nodes with Myri-10G cards that will be using bmi_mx, there is a known
> issue if clients and servers are run on the same node _and_ the client and
> server are run by different users. By default, MX tries to use shared memory
> for traffic between processes on the same node. This requires the processes
> to have the same owner. To avoid this issue, add MX_DISABLE_SHMEM=1 to your
> environment until we have a longer-term solution. Setting this variable will
> turn off MX shared memory and it will cause MX to send local messages to the
> switch and back.
Can you try your solution with Open-MX and have the two processes run by
different users?
Scott
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
