I'm about to post a set of patches intended to improve mkey support
in OpenSM. These patches have been fairly rigorously tested on a small
fabric, and I believe are sufficiently stable for inclusion. The
primary intent here is threefold:
1) Fix a multitude of edge case issues with the existing
single-mkey-per-subnet support in OpenSM. For instance, the current
implementation provides no way to change an established non-zero mkey
without rebooting or manually re-keying each CA on the entire subnet.
2) Enable mkey protection across the fabric. This involves not only
setting a non-zero protection level, but also providing the SM with a
sufficient information cache to initialize the subnet on restart without
having to wait for mkey lease timeouts (provided one is set).
3) Provide a basis on which to build multiple-mkey systems for OpenSM
(be they per-host, KDF, or random) in the future.
The patches add two new cache files: a port guid-to-mkey cache, and
a neighboring link (port guid to port guid) cache.
The guid2mkey cache is used to provide a hint at the initial mkey
for a CA during initialization. It is a hint only; the SM is capable of
dealing with cases where the guid2mkey cache is incorrect, although it
may require waiting for (potentially multiple) mkey lease timeouts at
non-zero mkey protection levels. The guid2mkey cache is presented first
in the patch set, as it ends up ameliorating several corner cases in a
cleaner way than attacking them directly did.
The neighbors cache file provides an initial hint to the SM of what
port guid we may expect at the opposite end of a link that is being
initialized. This is necessary at mkey protection level 2, where we
cannot do the SubnGet necessary to determine the port guid to use in
looking up an mkey hint.
The changes to the osm_req functions to support mkeys in patch 2
now require plock to be held when called. This was generally already
the case, but there were a few spots where it was not. In most of these
cases, the plock is still not technically necessary, as they occur
during hops 0/1 when DR path traversal is trivial. I wrapped all of
these occurrences in locks in a separate patch (#3), in order to make
the changes more obvious and invite comment.
Jim
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
