> > Is it really returning -ENOMEM? It seems much more likely that you > > are hitting the code > > > > /* For Arbel, all MTTs must fit in the same page. */ > > if (mthca_is_memfree(dev) && > > mr->attr.max_pages * sizeof *mr->mem.arbel.mtts > PAGE_SIZE) > > return -EINVAL; > > > > I guess you could call this limit a driver design issue. > > Actually, I see this in fmr_pool.c: > > fmr = kmalloc(sizeof *fmr + params->max_pages_per_fmr * sizeof > (u64), > > GFP_KERNEL); > > Therefore, for max_pages_per_fmr = 1K, this attempts to allocate 8K > of physically contigious memory, which could explain the failure. > > One way to fix this would be to use vmalloc to allocate this buffer. > Opinions?
I don't think an order 1 allocation will fail under normal conditions. Larger allocations might fail, but I don't think vmalloc is the right solution... maybe just disable the caching in fmr_pool for larger FMRs? Anyway the issue here is defintely that mthca does not handle finding more than one page in the MTT table for memfree HCAs... _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
