Hi James,
On meth0/meth1, can you show the output of "netstat -tan |grep 51886" to
confirm that there are in fact two daemons listening on that node, and
that each one is listening on a unique ip address?
If that looks ok, you can do some diagnosis independent of PVFS from the
node that isn't able to connect. You can try a ping and traceroute of
each hostname and make sure they map to the different ip addresses that
showed up in netstat. You can also try connecting to the pvfs port via
telnet with "telnet meth0 51886" and hit ctrl-d to disconnect. This
will confirm if you can get tcp connectivity independent of any possible
pvfs configuration issues.
Assuming you get this working, I'm still not sure you'll get balanced
traffic without some extra tweaking (Emmanual Florac hinted at that in
his response too), but it is an interesting experiment. You might only
balance inbound traffic. For outbound traffic, I suspect that the
server will just send everything over whichever interface is listed
first in your routing table, regardless of where the connection was
accepted. It should be obvious from the packet counts if that happens,
though. From there you would have to either go down the bonding route
or maybe something can be done with iptables rules in your current
configuration.
-Phil
On 06/04/2010 02:47 PM, James Gao wrote:
In my understanding, the TCPBindSpecific operation would allow two
servers to operated on the same port, assuming they're on two
different ethernet devices. My first attempt did indeed use two
different ports, and this worked. I've tried two different ports with
TCPBindSpecific as well, and this did not work. The extremely
confusing thing is that the setup works fine on crystal0 and crystal1,
whereas meth0 does not work, but meth1 does.
As for the data store locations, they are in different places, defined
by the ServerOption blocks underneath.
Thanks!
-James
On Fri, Jun 4, 2010 at 11:39 AM, Becky Ligon <[email protected]
<mailto:[email protected]>> wrote:
James:
I took a quick look at your config file and you need to have different
port numbers for each server on the same physical machine and separate
storage locations for each as well. PVFS does not provide consistency
between two servers accessing the same data locations.
Becky
--
Becky Ligon
PVFS Developer
Clemson University
864-656-3865
> Interesting... My knowledge of ip routing is spotty at best. I
was under
> the
> impression that the network would view the second ethernet
adapter as a
> completely separate computer. I will try the bonding configuration.
> Thanks!
>
> -James
>
> On Fri, Jun 4, 2010 at 3:17 AM, Emmanuel Florac
> <[email protected] <mailto:[email protected]>>wrote:
>
>> Le Fri, 4 Jun 2010 00:47:35 -0700
>> James Gao <[email protected] <mailto:[email protected]>>
écrivait:
>>
>> > My ultimate goal is to be able to use the full 4 gbps of
bandwidth
>> > connected to the file servers.
>> > Any ideas? Thanks for you time!
>>
>> I suppose you set up your 2 different interfaces on each node
in the
>> same ip network (like 10.0.1.1 and 10.0.1.2). Unfortunately,
the way ip
>> routing works mean that all traffic will go through the first one.
>>
>> You should use bonding instead. Aggregate the two interfaces using
>> balance-rr or balance-alb, and you'll have a virtual 2 Gb interface
>> instead of 2 1 Gb, that should work much better.
>>
>> regards,
>> --
>>
------------------------------------------------------------------------
>> Emmanuel Florac | Direction technique
>> | Intellique
>> | <[email protected]
<mailto:[email protected]>>
>> | +33 1 78 94 84 02
>>
------------------------------------------------------------------------
>>
> _______________________________________________
> Pvfs2-users mailing list
> [email protected]
<mailto:[email protected]>
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
