On Mar 4, 2015, at 11:56 AM, Paul Hargrove <phhargr...@lbl.gov> wrote:
> I have a system with InifniPath HCAs, where I've historically tested mtl:psm. > For some reason, that appears to have ceased working some time in the past 4 > months. > However, this report is about something else. > I am using the current master tarball: openmpi-dev-1203-g171d674.tar.bz2 > > When I ran configure, verbs support was found even though it was not my > intent to use it. > So, I am running with an explicit blt list that omits verbs and am disabling > the broken mtl:psm and mtl:ofi as well. > However, I am getting complaints from some verbs-related code: > > $ mpirun -mca btl sm,self,tcp -mca mtl ^psm,ofi -np 2 -host n15,n16 > examples/ring_c > libibverbs: Warning: couldn't open config directory '/etc/libibverbs.d'. > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 I think that warning is printed by libibverbs itself. Are you 100% sure there are no IB HCAs sitting in the head node? If there are IB HCAs but you don't want them to be used, you might want to ensure that the various verbs kernel modules don't get loaded, which is one half of the mismatch which confuses libibverbs. > -------------------------------------------------------------------------- > Fork support was requested but the library call ibv_fork_init() failed. > > Hostname: n16 > Error (22): Invalid argument > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > Fork support was requested but the library call ibv_fork_init() failed. > > Hostname: n15 > Error (22): Invalid argument > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > Fork support was requested but the library call ibv_fork_init() failed. > > Hostname: n16 > Error (22): Invalid argument > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > Fork support was requested but the library call ibv_fork_init() failed. > > Hostname: n15 > Error (22): Invalid argument > -------------------------------------------------------------------------- Hmm... I don't know enough about how show_help works, but I thought that would have at least de-duped some of this. It looks like the fork check is run once per device, so show_help may not be able to de-dupe everything. -Dave
