On Wed, 2008-06-04 at 09:28 -0500, Steve Wise wrote: > Roland/All, > > Should the device attributes (for instance max_send_wr) be the max > supported by the HW or the max supported by the OS or something else? >
Something else. > For instance: Chelsio's HW can handle very large work queues, but since > Linux limits the size of contiguous dma coherent memory allocations, the > actual limits are much smaller. Basing the limit on an OS resource seems arbitrary and dangerous. Applications using advertised adapter resource limits will unnecessarily consume the maximum. > Which should I be using for the device > attributes? > Arbitrary knee jerk == 512. However, surveying the current app usage as well as the other manufacturers advertised limits will make it less arbitrary. > Also, the chelsio device uses a single work queue to implement the SQ > and RQ abstractions. So the max SQ depth depends on the RQ depth and > vice versa. This leads to device max attributes that aren't that useful. > So the "real" limit is the HW WQ max and therefore max SQ = HW WQ max - RQ max? Setting RQ and SQ to 512 solves this "problem." > I'm wondering what application writes should glean from these attributes... > I guess you mean "application writer"? Here's what I suggest: - Set the RQ and SQ max to some reasonable default limit (e.g. 512). - Add an escape hatch by providing module options to override the default max. Tom > Steve > > _______________________________________________ > general mailing list > general@lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general _______________________________________________ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general