Thanks Hal, But the reason of difference in behavior of linux and windows nodes lies is probably that in windows drivers the sm_key is never set. The sm_key is defined in sa_hdr which is part of mad. If I'm not mistaken, mad buffers are zallocated.
Opensm expect OSM_DEFAULT_SM_KEY and it worked until recently because of this endianess bug. Sean, can you check if in linux code (specifically in ipoib default broadcast query/join path) the sm_key is set? Anatoly. -----Original Message----- From: Hal Rosenstock [mailto:[email protected]] Sent: Monday, November 16, 2009 10:52 PM To: Anatoly Greenblatt Cc: Sean Hefty; Fab Tillier; Tzachi Dar; Smith, Stan; [email protected] Subject: Re: [ofw] RE: installation/connectivity problems on hpc server On Mon, Nov 16, 2009 at 3:32 PM, Anatoly Greenblatt <[email protected]> wrote: > I think this bug can be noticed only when opensm runs on ppc while host > run on intel platforms or vise-versa. >From opensm_release_notes: OpenSM Compatibility -------------------- Note that OpenSM version 3.2.1 and earlier used a value of 1 in host byte order for the default SM_Key, so there is a compatibility issue with these earlier versions of OpenSM when the 3.2.2 or later version is running on a little endian machine. This affects SM handover as well as SA queries (saquery tool in infiniband-diags). > Also, I'd appreciate if anyone can point me to the place where this > sa_key filled in sa query (or how it is mapped). The --smkey option in saquery allows this to be specified. -- Hal > Thanks, > Anatoly. > > -----Original Message----- > From: Hal Rosenstock [mailto:[email protected]] > Sent: Monday, November 16, 2009 8:53 PM > To: Sean Hefty > Cc: Anatoly Greenblatt; Fab Tillier; Tzachi Dar; Smith, Stan; > [email protected] > Subject: Re: [ofw] RE: installation/connectivity problems on hpc server > > On Mon, Nov 16, 2009 at 11:46 AM, Sean Hefty <[email protected]> > wrote: >>>Stan/Sean, when you perform linux/windows dapl tests do you use > windows >>>or linux opensm and what is the value of sa_key? >> >> I always use a linux version of opensm, since I share switches with a > linux >> cluster. I don't know what value the sa_key is, but it's 99% likely > to be >> whatever the default is. >> >> Do you know what exactly the sa_key maps to? > > The default sa_key depends on which version of OpenSM is being used. > > # Note that for both values above (sm_key and sa_key) > # OpenSM version 3.2.1 and below used the default value '1' > # in a host byte order, it is fixed now but you may need to > # change the values to interoperate with old OpenSM running > # on a little endian machine. > >> Is this how the user specifies the >> SM key, or is it something else? > > There are two separate keys: sm_key and sa_key. > > -- Hal > >> - Sean >> >> _______________________________________________ >> ofw mailing list >> [email protected] >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw >> > _______________________________________________ ofw mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
