On Wed, 2012-08-08 at 22:38 +0800, Liang Zhen wrote: > LNet reserved 32 bits for network number, so you can choose a very > large network number if only have a few networks
Thanks. That was exactly what I was looking for. > but really create many networks will have some issues: > - o2iblnd will pre-allocate memory resources for each network, so it will > consume a lot of memory I am mainly looking at using network numbers to keep things better organized. For example, instead of having two different clusters each use o2ib1 for their internal networks, I could assign o2ib101 to one cluster and o2ib102 to the second (reserving o2ib[0-100] for other purposes). Any given client would probably only know about a few networks (maybe 2-3), but the lustre servers would obviously need to have more (maybe 10-20). Is there an estimate on how much memory is consumed for each network? Also, if a node has o2ib0 and o2ib5 configured, will it just allocate memory for those networks, or will it also allocate memory for o2ib[1-4] even if they are unused? (I wouldn't expect it to allocate memory, but better to find it out now than discover my mistake later.) > - Main stream LNet will have performance issue if there're many > networks, for example, hundreds, although it's not difficult to fix > this. I don't expect to have hundreds of networks, but I am curious how I would fix it if I ever did. Thanks. -- Rick Mohr HPC Systems Administrator National Institute for Computational Sciences http://www.nics.tennessee.edu/ _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
