> Is there a way to tell from the information returned by fi_getinfo() whether
> one should
> expect any benefit from FI_MORE with the provider one is using? I don't see
> anything,
No - this flag is merely an optimization that a provider _may_ act upon.
> and none of the provider man pages seem to mention it, at least in the v1.8.x
> branch
> I've built from. I'd prefer to avoid doing the effective equivalent of 'if
> (!usingVerbsProvider) { /* do FI_MORE stuff */ ...}', unless I absolutely
> have to.
> (I'm writing middleware, a multinode communication implementation for the
> Chapel
> language runtime, which I want to work over as wide a range of networks and
> providers
> as possible.)
My recommendation is to write the app to always use FI_MORE where it makes
sense, or just always ignore the flag. There should not be a performance
penalty if it is set but ignored by the provider.
> Sure can, although there will be a delay because right now the initiator for
> my
> fi_writemsg() calls seems to be concluding that all of them have arrived at
> the target
> before that's actually happened, so I'll have to fix that first. I don't
> know if
> that's because I'm mis-counting transactions (doesn't look like it) or have
> misunderstood the completion semantics. I'm throwing
> FI_DELIVERY_COMPLETE|FI_COMPLETION in my hints->tx_attr->op_flags, and
> believe that
> means that once the initiator sees the CQ event from a fi_writemsg() the
> processor on
> the target side must see the written bytes if it loads from the target
> memory. (In
> some cases it's seeing the old memory contents.)
Your understanding looks correct.
This could be a bug in the provider implementing DELIVERY_COMPLETE correctly.
Note that delivery complete isn't supported natively by most hardware, and
requires software based acks in order to meet that guarantee.
Are you using RDM endpoints?
- Sean
_______________________________________________
ofiwg mailing list
[email protected]
https://lists.openfabrics.org/mailman/listinfo/ofiwg
