Hi,
(1) So, I looked into the virtio-blk + cache modelling problem. To
recap, the problem can be stated as follows:
- Binary to run: "ArmVExpress-FVP-AArch64" build of current upstream
edk2
- Platform: FVP Base Model, Aarch64
- Platform setting: -C cache_state_modelled=1
When the first virtio-blk request is submitted via the virtio ring (by
writing the Queue Notify MMIO register), the Model issues the following
warning:
WARNING: Virtio available ring flags unrecognised (0xafaf). Not
fetching descriptor.
and the virtio transfer doesn't progress.
When "cache_state_modelled" is zero, this doesn't happen, and the
virtio-block device works correctly.
(2) The guest call chain triggering the warning follows:
VirtioBlkReadBlocks() [OvmfPkg/VirtioBlkDxe/VirtioBlk.c]
SynchronousRequest() [OvmfPkg/VirtioBlkDxe/VirtioBlk.c]
VirtioFlush() [OvmfPkg/Library/VirtioLib/VirtioLib.c]
VirtioMmioSetQueueNotify()
[OvmfPkg/Library/VirtioMmioDeviceLib/VirtioMmioDeviceFunctions.c]
MmioWrite32() [MdePkg/Library/BaseIoLibIntrinsic/IoLibArm.c]
I hex-dumped the ring contents (all three pages) inside the guest. It
looked normal; in particular the available ring flags were valid.
Nowhere in the ring could I find the reported 0xafaf value.
(3) The virtio guest drivers take care to perform all synchronization /
barriers that the virtio specification [1] lists. In particular please
see the "MUST perform a suitable memory barrier" references in:
3.2 Device Operation
3.2.1 Supplying Buffers to The Device
(Side note: I just noticed a typo there in step 4, namely "suitable a
memory".)
The edk2 guest drivers conform to this requirement by:
- accessing the ring area through pointers to volatile objects,
- implementing the "suitable memory barrier" by calling the
MemoryFence() BaseLib function.
The "Driver Writer's Guide for UEFI 2.3.1" [2] lists these two tools for
memory synchronization, in:
4 General Driver Design Guidelines
4.2 Maximize Platform Compatibility
4.2.6 Memory ordering
MemoryFence() is also recommended in
18 PCI Driver Design Guidelines
18.5 PCI DMA
18.5.2 Weakly ordered memory transactions
18.5.3 Bus Master Read and Write Operations
(4) On the platform in question, the MemoryFence() function is
implemented as
MemoryFence() [MdePkg/Library/BaseLib/AArch64/MemoryFence.S]
DMB SY
(5) After trying to parse parts of the ARMv8-A Architecture Reference
Manual [3], I had the idea to strengthen this primitive by replacing the
DMB with DSB. This idea didn't help at all, the FVP Base model printed
the same warning at the same spot.
(6) I grepped the ARM-specific parts of the edk2 tree for cache
invalidation primitives. I found out that I could re-qualify the virtio
ring area as uncacheable memory, by utilizing the
SetMemorySpaceAttributes() Global Coherency Domain DXE service. (See
Volume 2, Driver Execution Environment Core Interface, of the Platform
Initialization Specification [4].)
This approach is visible in the first attached patch. The call in
question resolves as:
VirtioBlkInit() [OvmfPkg/VirtioBlkDxe/VirtioBlk.c]
VirtioRingInit()
[OvmfPkg/Library/VirtioLib/VirtioLib.c]
CoreSetMemorySpaceAttributes() [MdeModulePkg/Core/Dxe/Gcd/Gcd.c]
CoreConvertSpace() [MdeModulePkg/Core/Dxe/Gcd/Gcd.c]
SetMemoryAttributes()
[ArmPkg/Library/ArmLib/AArch64/AArch64Mmu.c]
ArmCleanInvalidateDataCache()
[ArmPkg/Library/ArmLib/AArch64/AArch64Lib.c]
ArmInvalidateInstructionCache()
[ArmPkg/Library/ArmLib/AArch64/AArch64Support.S]
ArmInvalidateTlb()
[ArmPkg/Library/ArmLib/Common/AArch64/ArmLibSupport.S]
Note that in
2.3 Calling Conventions
2.3.6 AArch64 Platforms
2.3.6.1 Memory types
the UEFI 2.4 Specification [5] lists the following memory type bindings:
EFI Memory Type ARM Memory Type: ARM Memory Type:
MAIR attribute encoding Meaning
Attr<n> [7:4] [3:0]
--------------- ----------------------- ------------------------------
EFI_MEMORY_UC 0000 0000 Device-nGnRnE
(Not cacheable) (Device non-Gathering,
non-Reordering, no Early Write
Acknowledgement)
EFI_MEMORY_WC 0100 0100 Normal Memory
(Write combine) Outer non-cacheable
Inner non-cacheable
... ... ...
I tried both EFI_MEMORY_UC and EFI_MEMORY_WC.
They both worked in the sense that each got me over the immediate FVP
Base model warning, and *something* was loaded from the virtio-blk disk.
However that "something" (which should have been a grub2 binary) could
not be executed, implying that further corruption was happening.
The straightforward idea is that it's not enough to make the ring area
itself uncacheable: the scatter-gather memory areas *referenced* by the
descriptors located inside the ring have to be uncacheable too.
This would be very inconvenient, however, because the "scatter-gather
memory areas" in question are almost always local variables, of which
are many, and whose size is mostly smaller than a page.
Hence I didn't pursue this further.
(In any case, if this were deemed the "proper" solution, then I'd
suggest to extend the virtio specification [1] with a note about memory
caching attributes, somewhere near the barrier language.)
(7) The problem was ultimately solved (as a proof-of-concept) with the
second attached patch. This patch sort of proves that the current
Aarch64 implementation of MemoryFence() -- ie. DMB SY -- is not strong
enough for MemoryFence()'s advertised purpose.
Once I replaced VirtioLib's MemoryFence() primitives with
MemoryFenceX()
MemoryFence()
[MdePkg/Library/BaseLib/AArch64/MemoryFence.S]
ArmCleanInvalidateDataCache()
[ArmPkg/Library/ArmLib/AArch64/AArch64Lib.c]
MemoryFence()
[MdePkg/Library/BaseLib/AArch64/MemoryFence.S]
everything started to work.
(8) Hence I'm proposing / requesting that the MemoryFence()
implementation on Aarc64() be strengthened so that it also flushes the
data cache completely. Unfortunately I can't do this myself (ie. post a
patch) because I don't know enough about ARM.
The PoC in (7) is clearly overkill. I only squeezed the
ArmCleanInvalidateDataCache() call between two DMB SY's because I found
something frightening in the ARMv8-A Architecture Reference Manual [3]
about ordering between Data Cache manipulation and DMB instructions.
Hacking the PoC like this was easier than actually understanding the
ordering.
That said, ArmCleanInvalidateDataCache() seems to execute
dc cisw, x0 // Clean and Invalidate this line
dsb sy
isb
in some kind of a loop. (I did find DC CISW in the manual [3], but had
no idea how to say "all lines at once" in x0. Which is probably what the
loop is for.) It already includes DSB SY, which should be stronger than
the sole DMB SY that MemoryFence() currently consists of.
This suggests that reimplementing MemoryFence() as a single call to
ArmCleanInvalidateDataCache() would suffice.
Olivier, what do you think?
(9) I've included the virtio-comment list because of the typo notice in
(3), and because of the suggestion to *maybe* mention memory caching
attributes. Sorry for the noise.
Thanks,
Laszlo
[1]
http://docs.oasis-open.org/virtio/virtio/v1.0/csprd01/virtio-v1.0-csprd01.pdf
[2]
http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=UEFI_Driver_Writer%27s_Guide
[3]
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.subset.architecture.reference/index.html
[4] http://www.uefi.org/specs/download
[5] http://www.uefi.org/specs/download
From 45f2043b6f79e5384b9b61bd8cb473a44a9a9dc8 Mon Sep 17 00:00:00 2001
From: Laszlo Ersek <[email protected]>
Date: Wed, 22 Jan 2014 23:55:47 +0100
Subject: [PATCH] VirtioLib: re-qualify the virtio ring area as UC / WC
Marking the ring area as EFI_MEMORY_UC or EFI_MEMORY_WC prevents it from
being cached. It doesn't solve the whole visibility problem though because
all areas *referenced* by the descriptors would have to undergo the same
treatment. Since that's extremely impractical, this is a no-go. Instead,
MemoryFence() should just work.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Laszlo Ersek <[email protected]>
---
OvmfPkg/Library/VirtioLib/VirtioLib.inf | 1 +
OvmfPkg/Include/IndustryStandard/Virtio.h | 1 +
OvmfPkg/Library/VirtioLib/VirtioLib.c | 38 +++++++++++++++++++++++++++++--
3 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/OvmfPkg/Library/VirtioLib/VirtioLib.inf
b/OvmfPkg/Library/VirtioLib/VirtioLib.inf
index fb5897a..27d1412 100644
--- a/OvmfPkg/Library/VirtioLib/VirtioLib.inf
+++ b/OvmfPkg/Library/VirtioLib/VirtioLib.inf
@@ -32,5 +32,6 @@
BaseLib
BaseMemoryLib
DebugLib
MemoryAllocationLib
UefiBootServicesTableLib
+ DxeServicesTableLib
diff --git a/OvmfPkg/Include/IndustryStandard/Virtio.h
b/OvmfPkg/Include/IndustryStandard/Virtio.h
index fcf7b67..df001b9 100644
--- a/OvmfPkg/Include/IndustryStandard/Virtio.h
+++ b/OvmfPkg/Include/IndustryStandard/Virtio.h
@@ -148,10 +148,11 @@ typedef struct {
#pragma pack()
typedef struct {
UINTN NumPages;
VOID *Base; // deallocate only this field
+ UINT64 OrigAttr; // ... and restore these GCD attributes
volatile VRING_DESC *Desc; // QueueSize elements
VRING_AVAIL Avail;
VRING_USED Used;
UINT16 QueueSize;
} VRING;
diff --git a/OvmfPkg/Library/VirtioLib/VirtioLib.c
b/OvmfPkg/Library/VirtioLib/VirtioLib.c
index 54cf225..b4ea4ae 100644
--- a/OvmfPkg/Library/VirtioLib/VirtioLib.c
+++ b/OvmfPkg/Library/VirtioLib/VirtioLib.c
@@ -18,10 +18,12 @@
#include <Library/BaseLib.h>
#include <Library/BaseMemoryLib.h>
#include <Library/DebugLib.h>
#include <Library/MemoryAllocationLib.h>
#include <Library/UefiBootServicesTableLib.h>
+#include <Pi/PiDxeCis.h>
+#include <Library/DxeServicesTableLib.h>
#include <Library/VirtioLib.h>
/**
@@ -43,10 +45,16 @@
@retval EFI_OUT_OF_RESOURCES AllocatePages() failed to allocate contiguous
pages for the requested QueueSize. Fields of
Ring have indeterminate value.
+ @return Error codes from the SetMemorySpaceAttributes()
+ Global Coherency Domain service, if the ring
+ area could not be re-qualified as uncacheable
+ memory. Fields of Ring have indeterminate
+ value.
+
@retval EFI_SUCCESS Allocation and setup successful. Ring->Base
(and nothing else) is responsible for
deallocation.
**/
@@ -55,12 +63,14 @@ EFIAPI
VirtioRingInit (
IN UINT16 QueueSize,
OUT VRING *Ring
)
{
- UINTN RingSize;
- volatile UINT8 *RingPagesPtr;
+ UINTN RingSize;
+ EFI_STATUS Status;
+ EFI_GCD_MEMORY_SPACE_DESCRIPTOR Descriptor;
+ volatile UINT8 *RingPagesPtr;
RingSize = ALIGN_VALUE (
sizeof *Ring->Desc * QueueSize +
sizeof *Ring->Avail.Flags +
sizeof *Ring->Avail.Idx +
@@ -78,10 +88,26 @@ VirtioRingInit (
Ring->NumPages = EFI_SIZE_TO_PAGES (RingSize);
Ring->Base = AllocatePages (Ring->NumPages);
if (Ring->Base == NULL) {
return EFI_OUT_OF_RESOURCES;
}
+
+ Status = gDS->GetMemorySpaceDescriptor (
+ (EFI_PHYSICAL_ADDRESS)(UINTN) Ring->Base,
+ &Descriptor);
+ ASSERT_EFI_ERROR (Status);
+ Ring->OrigAttr = Descriptor.Attributes;
+
+ Status = gDS->SetMemorySpaceAttributes (
+ (EFI_PHYSICAL_ADDRESS)(UINTN) Ring->Base,
+ RingSize,
+ EFI_MEMORY_WC);
+ if (EFI_ERROR (Status)) {
+ FreePages (Ring->Base, Ring->NumPages);
+ return Status;
+ }
+
SetMem (Ring->Base, RingSize, 0x00);
RingPagesPtr = Ring->Base;
Ring->Desc = (volatile VOID *) RingPagesPtr;
RingPagesPtr += sizeof *Ring->Desc * QueueSize;
@@ -134,10 +160,18 @@ VOID
EFIAPI
VirtioRingUninit (
IN OUT VRING *Ring
)
{
+ EFI_STATUS Status;
+
+ Status = gDS->SetMemorySpaceAttributes (
+ (EFI_PHYSICAL_ADDRESS)(UINTN) Ring->Base,
+ Ring->NumPages * EFI_PAGE_SIZE,
+ Ring->OrigAttr);
+ ASSERT_EFI_ERROR (Status);
+
FreePages (Ring->Base, Ring->NumPages);
SetMem (Ring, sizeof *Ring, 0x00);
}
--
1.8.3.1
From 83b74495ef9853374bcc0a41591c8b80df4eba66 Mon Sep 17 00:00:00 2001
From: Laszlo Ersek <[email protected]>
Date: Thu, 23 Jan 2014 00:30:57 +0100
Subject: [PATCH] VirtioLib: flush data cache as part of MemoryFence() on
AArch64
Just a proof of concept (that actually works). Note that the VirtioNetDxe
driver contains custom ring massaging (ie. explicit MemoryFence() calls),
and that is not updated here. However, for the VirtioBlkDxe case, this
patch suffices.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Laszlo Ersek <[email protected]>
---
OvmfPkg/Library/VirtioLib/VirtioLib.inf | 2 ++
OvmfPkg/Library/VirtioLib/VirtioLib.c | 22 +++++++++++++++++-----
2 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/OvmfPkg/Library/VirtioLib/VirtioLib.inf
b/OvmfPkg/Library/VirtioLib/VirtioLib.inf
index fb5897a..6decc95 100644
--- a/OvmfPkg/Library/VirtioLib/VirtioLib.inf
+++ b/OvmfPkg/Library/VirtioLib/VirtioLib.inf
@@ -25,12 +25,14 @@
VirtioLib.c
[Packages]
MdePkg/MdePkg.dec
OvmfPkg/OvmfPkg.dec
+ ArmPkg/ArmPkg.dec
[LibraryClasses]
BaseLib
BaseMemoryLib
DebugLib
MemoryAllocationLib
UefiBootServicesTableLib
+ ArmLib
diff --git a/OvmfPkg/Library/VirtioLib/VirtioLib.c
b/OvmfPkg/Library/VirtioLib/VirtioLib.c
index 54cf225..d6b2d68 100644
--- a/OvmfPkg/Library/VirtioLib/VirtioLib.c
+++ b/OvmfPkg/Library/VirtioLib/VirtioLib.c
@@ -18,13 +18,25 @@
#include <Library/BaseLib.h>
#include <Library/BaseMemoryLib.h>
#include <Library/DebugLib.h>
#include <Library/MemoryAllocationLib.h>
#include <Library/UefiBootServicesTableLib.h>
+#include <Library/ArmLib.h>
#include <Library/VirtioLib.h>
+STATIC
+VOID
+MemoryFenceX (
+ VOID
+ )
+{
+ MemoryFence ();
+ ArmCleanInvalidateDataCache ();
+ MemoryFence ();
+}
+
/**
Configure a virtio ring.
@@ -280,18 +292,18 @@ VirtioFlush (
Indices->HeadDescIdx % Ring->QueueSize;
//
// virtio-0.9.5, 2.4.1.3 Updating the Index Field
//
- MemoryFence();
+ MemoryFenceX();
*Ring->Avail.Idx = NextAvailIdx;
//
// virtio-0.9.5, 2.4.1.4 Notifying the Device -- gratuitous notifications are
// OK.
//
- MemoryFence();
+ MemoryFenceX();
Status = VirtIo->SetQueueNotify (VirtIo, VirtQueueId);
if (EFI_ERROR (Status)) {
return Status;
}
@@ -302,18 +314,18 @@ VirtioFlush (
// synchronous, lock-step progress.
//
// Keep slowing down until we reach a poll period of slightly above 1 ms.
//
PollPeriodUsecs = 1;
- MemoryFence();
+ MemoryFenceX();
while (*Ring->Used.Idx != NextAvailIdx) {
gBS->Stall (PollPeriodUsecs); // calls AcpiTimerLib::MicroSecondDelay
if (PollPeriodUsecs < 1024) {
PollPeriodUsecs *= 2;
}
- MemoryFence();
+ MemoryFenceX();
}
- MemoryFence();
+ MemoryFenceX();
return EFI_SUCCESS;
}
--
1.8.3.1
------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
edk2-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/edk2-devel
