Nicolas Morey Chaisemartin wrote:
Yevgeny Kliteynik a écrit :
Hi Nicolas,
Nicolas Morey Chaisemartin wrote:
Hello,
I am conducting some tests on routing non-pure fat-tree network using
the fat tree algorithm of OpenSM.
The network I am experimenting on is a 3 level fat tree, with a
pruned 3rd layer.
By providing the root_guid_file, the algorithm works great !
The problem is, we would like to add some service nodes directly on
the 3rd level switches.
I have added the cn_guid_file so the network is still recognize as a
fat tree.
OpenSM once more manage to create the routing for the network. It
provides full connectivity,
except there are no routes between non computes nodes.
I understand that the point of setting these node as not compute node
should intend they won't talk to each other, but we still need a bit
of connectivity between them to exchange few datas (pings and such).
A simple min-hop or such should be enough to generate those routes.
It will probably desequilibrate the number of routes going through
the top links, but those additional link makes virtually no traffic
at all, so in practical it shouldn't be a problem.
Fat-tree should create full connectivity as long as there is an up/down
route between ports. Do you get connectivity between these nodes with
up/down routing algorithm?
Try running it with the same root_guid_file.
-- Yevgeny
Well the route would be more down/up compared to the rest of the transfer.
(Im not sure I was clear, but when i talk of 3rd level, I mean top
level. 1st level begin the switches just above the compute nodes)
Oh, OK. I was thinking the opposite. So you connect these
non-CNs to spine switches.
I'll try this tomorrow
No need :)
Fat-tree is a variation of up/down routing. As such, down/up
routes are not allowed. You won't have connectivity between
these nodes neither in fat-tree nor in up/down routing.
Is there any reasons such a behavior wasn't implemented yet?
The idea of allowing only up/down routes is preventing credit
loops in the fabric.
Should there be one?
I guess it is possible, but these down/up routes will create
credit loops, so any traffic between these "special" nodes is
potentially bad for fabric.
Note that there's already a "connect roots" option in the
up/down routing which violates the up/down rule, but this is
only between switches, so I believe that the only traffic
that uses these routes is management traffic.
-- Yevgeny
Regards
Nicolas Morey-Chaisemartin
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-general
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
