Pete Wyckoff wrote:
[EMAIL PROTECTED] wrote on Fri, 29 Dec 2006 12:57 -0600:
Just a sanity check for everyone, I rebuilt release 2.5.1, and
everything works correctly.
I looked back at the 2.5.1 code, and noticed that we were just
hardcoding the max_wr value to 512, which is much less than what I'm
seeing here in 2.6.1 - (70).
I also verified that we have a 32768 max_wr value from 2.5.1's
ibv_device_query() function.
However, 2.6.1 is reporting 0?
Printed out hca_cap->max_qp_wr immediately after querying the device
ibv_query_device(), and got 32k, which is what I got w/ 2.5.1.
This is in the ib_init() function.
Then we goto:
od->nic_max_wr = hca_cap->max_qp_wr (sorry, summarising the code,
these are lines from my head)
Then in new_connection()
we do a check:
max_nr > od->nic_max_wr
at which point od->nic_max_wr is 0,
I havent searched through the flow of the code to see where its getting
overwritten yet, but will try this afternoon.
I think that is where we're getting off, havent applied the patches yet,
but, again I'll get to that this afternoon.
thx
Kyle
512 is greater that 70. We're only asking for 70 now, so fewer WRs
requested should be "easier" for the driver.
Can you step through the device setup code and make sure that all
the numbers are reasonable? In particular, print out "att" both
before and after the call to ibv_create_qp.
The main difference from the old pvfs version is that now we only
need one QP, not two. Not sure how this could cause the sort of
regression you are seeing (yet).
-- Pete
!DSPAM:4595716f197131497551287!
--
Kyle Schochenmaier
[EMAIL PROTECTED]
Research Assistant, Dr. Brett Bode
AmesLab - US Dept.Energy
Scalable Computing Laboratory
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
