Hi Doug,
This series adds completion timestamp for libibverbs.
In order to do so, we add an extensible poll cq. The problem with
extending the WC is that you could run out of the current cache
line when adding new features and degrade performance. This is solved
by introducing a custom WC.
The user creates a CQ using ibv_create_cq_ex, stating which WC fields
should be returned by this CQ. When the user calls ibv_poll_cq_ex,
this custom WC is returned. The fields orders and sizes are declared
in advanced (we avoid alignment rules by putting the fields starting
from the 64bit fields --> 8bit fields). Each WC has a wc_flags field
representing which fields are valid in this WC.
The vendor drivers could optimize those calls extensively.
Completion timestamp is added on top of these extended ibv_create_cq_ex
verb and ibv_poll_cq_ex verb. The user should call ibv_create_cq_ex
stating that this CQ should support reporting completion timestamp.
ibv_poll_cq_ex reports this raw completion timestamp value in every
packet.
In the future, a verb like the following could be added in order to
transform this time into system time:
ibv_get_timestamp(struct ibv_context *context, uint64_t raw_time,
struct timespec *ts, int flags);
The timestamp mask (number of supported bits) and the HCA's frequency
are given in ibv_query_device_ex verb.
We also give the user an ability to read the HCA's current clock.
This is done via ibv_query_values_ex. This verb could be extended
in the future for other interesting information.
Thanks,
Matan
Matan Barak (5):
Add ibv_poll_cq_ex verb
Add timestamp_mask and hca_core_clock to ibv_query_device_ex
Add support for extended ibv_create_cq
Add completion timestmap support for ibv_poll_cq_ex
Add ibv_query_values_ex
Makefile.am | 6 +-
examples/devinfo.c | 10 ++
include/infiniband/compiler.h | 89 ++++++++++++
include/infiniband/driver.h | 9 ++
include/infiniband/kern-abi.h | 26 +++-
include/infiniband/verbs.h | 318 ++++++++++++++++++++++++++++++++++++++++++
man/ibv_create_cq_ex.3 | 71 ++++++++++
man/ibv_poll_cq_ex.3 | 173 +++++++++++++++++++++++
man/ibv_query_device_ex.3 | 6 +-
src/cmd.c | 63 +++++++++
src/device.c | 44 ++++++
src/ibverbs.h | 12 ++
src/libibverbs.map | 1 +
13 files changed, 822 insertions(+), 6 deletions(-)
create mode 100644 include/infiniband/compiler.h
create mode 100644 man/ibv_create_cq_ex.3
create mode 100644 man/ibv_poll_cq_ex.3
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
