Hi Todd, On 06:39 Tue 16 Dec , Todd Rimmer wrote: > > From: [email protected] [mailto:general- > > [email protected]] On Behalf Of Sasha Khapyorsky > > Sent: Monday, December 15, 2008 10:39 AM > > > > > > That's a good question - and I'm going to ask around and double check. > > > My first reaction was that you have to specify how many paths you want > > > from the query - but you're right, the spec doesn't say that. > > > > Yes, it looks like this (but I cannot understand "why" :( ). But even more > > strange (IMHO) limitation is mandatory SGID - actually it should make > > illegal such GetTable queries as all-to-all, SLID-to-all, etc.. I > > thought that it is permitted. > [Todd Rimmer] It's about scalability. An all to all query in a fabric would > return at least N^2 path records, add in an LMC, some varied SLs and PKeys > and quickly it becomes ludicrously large. For example at only 25 nodes, with > 8 SLs and 4 PKeys you could have 20,000 path records returned. With 1000 > nodes and 1 SL and 1 PKey you have 1,000,000 path records. > > The practical use of GetTable(PathRecord) is for a node to find a path to > another node, hence SGID limitation makes sense.
Of course scalability is important, but I'm not sure that discussed limitations are real solution - if some port will want to get all-to-all paths information anyway it just will send much more queries and potentially will hurt SA scalability and fabric traffic even more. > > > I'm going to do some research on my end. Are you saying that > > > IB_MAD_ATTR_PATH_RECORD should only ever return a single path? > > > > With GetTable? I think it shouldn't (for some queries it will - such as > > SLID + DLID). > [Todd Rimmer] With GetTable the NumbPath parameter is required and sets a cap > on the number of returned paths per SGID-DGID combination. This makes sense > since most applications are only prepared to use a handful of paths to each > destination. Many would only use 2 paths for a failover scheme. Those which > only need 1 path can use Get(PathRecord) in which case NumbPath is implied to > be 1. Sure. But whole discussion is about GetTable, not just Get (this case is clear for me). > For scalability, it's best to avoid queries which don't specify a destination > (eg. always provide DGID, DLID, etc). Queries without a destination can > still be huge and consume a lot of SA and memory resources to process (eg. at > 1000 nodes LMC=2, 8 SLs you get 32000 path records back). Besides its very > rare than an application really wants to know the path to EVERY other node. As I described in another email example, let look at practical SGID/DGID case when LMC=15 - this will require at least 256 records, and there is no (known for me) easy way to get this in IBA complaint way when NumbPath is limited by 7 bits. Sasha _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
