Scott,
I tried running the pvfs2-server under root and the clients (pvfs2-
ping, pvfs2-ls) under my user account and that appears to work just
fine using Open-MX.
Should I be expecting to be able to communicate with myself over MX by
going to the switch and back, as this suggests? If so, perhaps my
issue has something to do with my switch configuration?
I have tried the following test, which does not work:
jrandall@renton:~$ MX_DISABLE_SHMEM=1 MX_DISABLE_SELF=1 /opt/open-mx/
bin/tests/mx_pingpong
OMX: Emulating MX_DISABLE_SELF as OMX_DISABLE_SELF
OMX: Forcing self comms to disabled
OMX: Emulating MX_DISABLE_SHMEM as OMX_DISABLE_SHARED
OMX: Forcing shared comms to disabled
Starting pingpong receiver on renton:0, endpoint=1
And, in another shell on the same machine:
jrandall@renton:~$ MX_DISABLE_SHMEM=1 MX_DISABLE_SELF=1 /opt/open-mx/
bin/tests/mx_pingpong -d renton:0 -r 1
OMX: trying to open board [0,31] endpoint [0,31]
OMX: trying to open board #0 endpoint #0
OMX: successfully open board #0 endpoint #0
OMX: desc at 0x7fc9013c7000 sendq at 0x7fc9003ec000, recvq at
0x7fc8ffbec000, exp eventq at 0x7fc9013b7000, unexp at 0x7fc90139e000
OMX: Successfully attached endpoint #0 on board #0 (hostname 'renton:
0', name 'eth3', addr 00:1b:21:4d:ba:92)
OMX: created partner 0000001b214dba92 ep 0 peer index 0
OMX: created myself partner 0000001b214dba92 ep 0 peer index 0
OMX: created partner 0000001b214dba92 ep 1 peer index 0
OMX: waiting for connect reply from partner 0000001b214dba92 ep 1
Is this something that _should_ work?
Josh.
I dug back through my email and found an issue from 2007:
For the nodes with Myri-10G cards that will be using bmi_mx, there
is a known issue if clients and servers are run on the same node
_and_ the client and server are run by different users. By default,
MX tries to use shared memory for traffic between processes on the
same node. This requires the processes to have the same owner. To
avoid this issue, add MX_DISABLE_SHMEM=1 to your environment until
we have a longer-term solution. Setting this variable will turn off
MX shared memory and it will cause MX to send local messages to the
switch and back.
Can you try your solution with Open-MX and have the two processes
run by different users?
Scott
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
