A quick sanity check. When setting the cq depth in the openib btl, it checks the calculated depth against the maxmium cq depth allowed and sets the minimum of those two. However, I think it is checking the wrong variable. If I understand correctly, ib_dev_attr.max_cq represents the maximum number of cqs while ib_dev_attr.max_cqe represents the max depth allowed in each individual cq. Is this correct?
If the above is true, then I'll apply the patch below. Thanks, Jon Index: ompi/mca/btl/openib/btl_openib.c =================================================================== --- ompi/mca/btl/openib/btl_openib.c (revision 17472) +++ ompi/mca/btl/openib/btl_openib.c (working copy) @@ -140,8 +140,8 @@ if(cq_size < mca_btl_openib_component.ib_cq_size[cq]) cq_size = mca_btl_openib_component.ib_cq_size[cq]; - if(cq_size > (uint32_t)hca->ib_dev_attr.max_cq) - cq_size = hca->ib_dev_attr.max_cq; + if(cq_size > (uint32_t)hca->ib_dev_attr.max_cqe) + cq_size = hca->ib_dev_attr.max_cqe; if(NULL == hca->ib_cq[cq]) { hca->ib_cq[cq] = ibv_create_cq_compat(hca->ib_dev_context, cq_size,