Roland:
Thank you very much for the info.
I hope Mellanox can tell me what to do next, Our project needs to run
2048 ranks
on a node, every rank has IB communication(most of them are sleeping, only a
few are active).
--CQ
> -----Original Message-----
> From: Roland Dreier [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, August 19, 2008 2:53 PM
> To: Tang, Changqing
> Cc: [email protected]
> Subject: Re: [ofa-general] How many processes on a node can
> open IB device ?
>
> > I have simple IBV code, which only open the device and create PD.
> > (attached below), then the code sleep there.
> >
> > When I start as many processes as I could, it fails at
> 895 copies, it fails with error:
>
> That sounds right for mlx4 with default firmware on a 4KB
> page size (ie
> x86) system. There are 1024 pages of user access registers
> available, but 128 + 1 = 129 are reserved for internal driver
> use. So that would leave 895 available for userspace use,
> exactly as you found.
>
> You should be able to build firmware that supports more
> processes, but I believe there may be some
> performance/stability tradeoffs related to that -- Mellanox
> could tell you more.
>
> - R.
>
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
