Quoting r. Roland Dreier <[EMAIL PROTECTED]>:
> Subject: Re: ANNOUNCE: First usable version of userspace verbs
>
> Michael> Interestingly, if I rebuild libmthca with -O3 compiler
> Michael> flag, the pingpong program does not make progress.
> Michael> Building libibverbs or the test itself with -O3 has no
> Michael> such effect.
>
> I can't reproduce this on either i386 or x86-64 (Intel Nocona system).
>
> $ gcc --version
> gcc (GCC) 3.3.5 (Debian 1:3.3.5-8)
> Copyright (C) 2003 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.
>
> - R.
>
I think I have discovered the problem. It seems that with -O3 my
compiler may reorder the WQE (and possibly CQE) write with respect to
the doorbell.
This wont happen on i386 with consistent i/o ordering since the doorbell
is done in assembly, and probably not on other 32 bit architectures
since the mutex is likely to include a memory barrier.
Applying the folowing patch fixes the problem for me for x86_64.
Signed-off-by: Michael S. Tsirkin <[EMAIL PROTECTED]>
Index: src/userspace/libmthca/src/doorbell.h
===================================================================
--- src/userspace/libmthca/src/doorbell.h (revision 1972)
+++ src/userspace/libmthca/src/doorbell.h (working copy)
@@ -56,6 +56,9 @@ static inline void mthca_write64(uint32_
static inline void mthca_write64(uint32_t val[2], struct mthca_context *ctx,
int offset)
{
+ /* Sufficient for x86_64.
+ * Other architectures may need a memory barrier here. */
+ asm volatile("" ::: "memory");
*(volatile uint64_t *) (ctx->uar + offset) = *(uint64_t *) val;
}
--
MST - Michael S. Tsirkin
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
