Moni Levy wrote:
On 4/23/06, Eitan Zahavi <[EMAIL PROTECTED]> wrote:
Hi Moni,
Sorry it took me a while to get back to you (was out on vacation ...)
Moni Levy wrote:
On 4/10/06, Eitan Zahavi <[EMAIL PROTECTED]> wrote:
Hi Hal,
-----Original Message-----
From: Hal Rosenstock [mailto:[EMAIL PROTECTED]
Sent: Monday, April 10, 2006 2:00 PM
To: Eitan Zahavi
Cc: Roland Dreier; [email protected]
Subject: Re: [openib-general] IPoIB interface for unauthorized
partition
Hi Eitan,
On Mon, 2006-04-10 at 02:35, Eitan Zahavi wrote:
Hi Roland,
Roland Dreier wrote:
Eitan> I thought the intent of the IB spec when defining P_Key
Eitan> index usage (and not P_Key value) was that the P_Key
values
Eitan> would never need to be known above the driver level.
To
Eitan> avoid exposing the P_Key values we could use P_Key
index
Eitan> for creating the IPoIB interfaces.
Eitan> Does it make sense to work on a patch that would setup
Eitan> IPoIB interfaces by the P_Key index (and not by P_Key
Eitan> value)?
I don't see how this is feasible. The index that a particular
P_Key
lands at is completely undetermined -- if two nodes wanted to talk
on
partition 0x8001 say, how does one know which interface to use
without
knowing the index of that P_Key?
OK, I get it. Actually the way IPoIB defines the broadcast group
MGID exposes
P_Key anyway.
Eitan> Also I think the expected behavior for IPoIB should be
that
Eitan> IPoIB "child" interfaces should be "automatically"
Eitan> initialized by the code that brings up the interface
Eitan> (ifconfig scripts). All valid IPoIB partitions (valid =
Eitan> have corresponding broadcast groups) should be
Eitan> initialized. By doing so we provide a centralized
control
Eitan> of the partitions and their IPoIB interfaces through
the
Eitan> SM.
Not sure if this is so. I may want a partition strictly for
storage
traffic something like that, so it doesn't make sense to create an
IPoIB interface for that partition.
OpenSM provides this capability in the "partition policy":
Each partition is marked explicitly if to be used for IPoIB or not.
So through one file one could actually control the IPoIB interfaces
that will exist in the subnet.
The end node does not know the SM policy for that partition though.
My intent is to write some extension to ifup for IPoIB such that all
sub
interfaces will be automatically started (based on pre-availability
of IPoIB
broadcast MGID).
I'm not sure how ifup is related to that. From what I understand you'd
like ipoib driver to behave as follows:
1. Get an event ( or figure it out) when a new PKEY is added to the
relevant port partition table.
I prefer not to rely on new events. Instead I would like to rely on existing IB
Notices:
If we register to multicast group create/delete events (traps 66/67) IPoIB can
know about each new partition created.
I'm not sure that this is a good idea, because that way all of the
IPoIB nodes will get that event and try to join every new MC group and
partitioning by definition is good for separating a fabric. I think
that the right thing should be that only the relevant nodes try to
join the specific MCG.
A node that does not have a P_Key matching the multicast group would not
receive the Report anyway.
So there is no problem. If a node is not part of the partition it will never
know about the new group.
2. Try to join that new MC group with the MGID it created according to
the PKEY and the spec. (or maybe query for the MC group existance but
that's not atomic)
Simply join the group. We rely on these groups to be pre-created by the SM
enforcing policy dictating with partitions should
be used for IPoIB and which not.
If you let all the IPoIB nodes join every new group without checking
their PKEY tables first, they may even get joined if the SM is not
eforcing MCG to port policy.
Is that your plan ?
A compliant SM will never let them join and never report any new group to ports
not in the partition.
3. In case it fails nothing is done (no relevant MC group was
pre-created in the SM).
Exactly
4. In case it succeeds a new interface is created.
Is that what you meant ?
- Moni
If that were to be done, it would be cleanest if the child IPoIB
interface was created only if that IPoIB broadcast group for that
partition exists.
[EZ] This is exactly what I had in mind.
-- Hal
- R.
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
