On 06/11/2015 09:35 AM, Or Gerlitz wrote:
> Hi Doug,
>
> This patchset adds the kernel control path for completion timestamping
> support by user-space verbs consumers.
>
> Timestamping is used by applications in order to know when a WQE was
> received/transmitted by the HW. The value is given is HCA hardware cycles,
> but could be easily converted as the hardware's core clock frequecny is
> available through extension of query device.
>
> Moreover, we add an ability to read the HCA's current clock. This could be
> useful on order to synchronize events to the wall clock.
>
> This functionality is achieved by adding/extending the following verbs:
>
> create_cq - create_cq is extended in order to allow passing creation flags
> to the CQ creation function. We change IB/core --> vendors API
> to be easily extendible by passing a struct which contains
> comp_vectors, cqe and the new flags parameter. In order to create
> CQ which supports timestamping, IB_CQ_FLAGS_TIMESTAMP_COMPLETION should be
> given.
>
> query_device - We extend query_device uverb further by giving the hardware's
> clock frequency and the timestamp mask (the number of timestamp
> bits which are supported). If timestamp isn't supported, 0 is returned.
>
> In order to read the timestamp in the WQE, the user needs to query the device
> for support, create an appropriate CQ (using the extanded uverb with
> IB_CQ_FLAGS_TIMESTAMP_COMPLETION) and poll the CQ with an extended poll_cq
> verb (currently,
> only implemented in user-space).
>
> In mlx4, allowing the user to read the core clock efficiently involves mapping
> this area of the hardware to user-space (being done by using a mmap command)
> and reading the clock from the correct offset of the page.
>
> This offset is returned in the vendor's specific data from mlx4's kernel
> driver
> to the mlx4's user-space driver. query_device is modified in order to support
> passing this vendor specific data. A user-space application could use a new
> verb in order to read the hardware's clock.
>
> Translating the hardware's clock into ms could be done by dividing this
> value by hca_core_clock (which is returned by the extended version of
> query_device uverb).
>
> The below V2 --> V3 changes address the review comments on the kernel bits.
>
> Jason had another comment calling for re-thinking / questioning the
> need for a dedicated uverbs_ex_cmd_mask in the IB device. This goes
> beyond the scope of this specific series.
>
> Matan and Or.
>
> Changes from V2:
> (1) Use KHZ for hca_core_clock
> (2) ib_create_cq gets const cq_attr
> (3 use {} initialization instead of memset
> (4) squashed last two mlx4 patches
>
> Changes from V1:
> (1) fixed lustre IB's code build
> (2) squashed mlx4 V1 9-11 patches into one
> (3) changed IB_CQ_FLAGS_TIMESTAMP --> IB_CQ_FLAGS_TIMESTAMP_COMPLETION
>
> Changes from V0:
> (1) Pass ib_cq_init_attr instead of cqe and comp_vector.
> (2) Fix unneeded indentation.
> (3) Change flags to u32.
> (4) Add const to create_cq's ib_cq_init_attr argument in vendor
> implementation.I've pulled this series in for 4.2. However, I forgot to add the extra reviewed-by tags before it was tagged, signed, and pushed to k.o. My apologies for that.
signature.asc
Description: OpenPGP digital signature
