Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-12 Thread Laszlo Ersek 
On 04/12/18 17:02, Gao, Liming wrote:
> Laszlo:
>   On variable storage format are hex, could you submit one bugzillar for it? 
> I have some idea to simplify its description. We can reuse FV section for 
> Variable storage. Its FV section is like below. 
> 
> [FV.NVStorage]
> FileSystemGuid = gEfiSystemNvDataFvGuid
> VariableSignatureGuid = gEfiAuthenticatedVariableGuid
> VariabeRegionSize = 0x4
> FtwWorkingSpaceSize = 0x4
> FtwSpareSpaceSize = 0x4

This looks very nice; I've filed
.

Thank you!
Laszlo
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-12 Thread Gao, Liming 
Laszlo:
  On variable storage format are hex, could you submit one bugzillar for it? I 
have some idea to simplify its description. We can reuse FV section for 
Variable storage. Its FV section is like below. 

[FV.NVStorage]
FileSystemGuid = gEfiSystemNvDataFvGuid
VariableSignatureGuid = gEfiAuthenticatedVariableGuid
VariabeRegionSize = 0x4
FtwWorkingSpaceSize = 0x4
FtwSpareSpaceSize = 0x4

Thanks
Liming
> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Laszlo 
> Ersek
> Sent: Wednesday, April 11, 2018 8:26 PM
> To: Steve Capper ; edk2-devel@lists.01.org
> Cc: Ard Biesheuvel ; Leif Lindholm 
> ; Nariman Poushin
> 
> Subject: Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch 
> protocol does not appear to be registered
> 
> Hi Steve,
> 
> On 04/11/18 12:30, Steve Capper wrote:
> > Hello,
> > I am trying to debug a boot problem for a recent build of EDK2 in an
> > Arm FVP model.
> >
> > I built EDK2 from the following branches:
> > Edk2: a146c532c754106431b063fec9985a838afd82be
> > Edk2-platforms: e74f53df8b18e4aed0c9df0942ce0c30f78a68e2
> >
> > Toolchain: Debian Stretch latest (though I have also tried Linaros
> > GCC 5 latest).
> >
> > Build command:
> > GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -b
> > DEBUG -p Platform/ARM/VExpressPkg/ArmVExpress-FVP-AArch64.dsc
> >
> > The boot log can be found in the attached file. I *think* this is due
> > to the Cpu Arch protocol not actually being ready when
> > CoreConvertSpace is called by the SetMemoryAttributes from the
> > NorFlash code.
> >
> > I have found that this code path fails:
> >
> https://github.com/tianocore/edk2/blob/a146c532c754106431b063fec9985a838afd82be/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L893
> >
> > The only modification I have made is to edk2-platforms to get more
> > debug output (PcdDebugPrintErrorLevel).
> >
> > Unfortunately, I'm not sure how to fix this?
> 
> This is a bit of a problem.
> 
> The gDS->SetMemorySpaceAttributes() DXE service is allowed to fail if
> the CPU architectural protocol has not been installed yet. This is
> documented in the Platform Init spec (although riddled by a few typos --
> EFI_NOT_AVAILABLE_YET is documented under the preceding
> GetMemorySpaceDescriptor() section, and not in the correct
> SetMemorySpaceAttributes() section).
> 
> The usual solution for DXE drivers is to add an explicit dependency on
> gEfiCpuArchProtocolGuid to the DEPEX section in the INF file. You can
> see an example in commit f9a8be423cdd
> ("ArmVirtualizationPkg/PciHostBridgeDxe: MMIO aperture must not be
> uncached", 2015-02-23).
> 
> And that's where the issue is. If you check the [Depex] section in
> "ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashDxe.inf", you find
> 
> > [Depex]
> >   #
> >   # NorFlashDxe must be loaded before VariableRuntimeDxe in case empty 
> > flash needs populating with default values
> >   #
> >   BEFORE gVariableRuntimeDxeFileGuid
> 
> Why is BEFORE technically an issue here? Because, it cannot be combined
> with other depex opcodes. From the PI spec:
> 
> > If this opcode is present in a dependency expression, it must be the
> > first and only opcode in the expression. If is appears in any other
> > location in the dependency expression, then the dependency expression
> > is evaluated to FALSE.
> 
> So we can't simply combine the existent BEFORE depex with "AND
> gEfiCpuArchProtocolGuid", like you see in the commit f9a8be423cdd
> example above.
> 
> Now, more to the point, a "BEFORE" depex is *always* a hack. From the PI
> spec:
> 
> > This opcode tells the DXE Dispatcher that the DXE driver that is
> > associated with this dependency expression must be dispatched just
> > before the DXE driver with the file name specified by GUID. This means
> > that as soon as the dependency expression for the DXE driver specified
> > by GUID evaluates to TRUE, then this DXE driver must be placed in the
> > dispatch queue just before the DXE driver with the file name specified
> > by GUID.
> 
> Sometimes there's really no way around such a hack, to express an
> inter-module dependency, but even then, there is a better way to
> implement it. Namely,
> - a custom protocol GUID can be invented,
> - the pre-requisite module can install a NULL protocol interface with
>   that GUID,
> - and
>   - either the protocol GUID can be added directly to the [depex]
> section of the dependent module (if the latter module is under the
> platform's control),
>   - or the platform DSC file can hook a NULL-class library instance into
> the dependent module, such that the lib instance makes the dependent
> module *inherit* a [depex] on the protocol GUID.
> 
> In other words, if we want to order the dispatch of a
> platform-independent module (that lives in another Package), due to
> platform-specific circumstances, behind another

Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-11 Thread Steve Capper 
On 11 April 2018 at 13:25, Laszlo Ersek  wrote:
> Hi Steve,

Hi Laszlo,

[...]

> Summary:
> (1) introduce a custom GUID for "NorFlashDxe has formatted the variable
> store headers for the variable driver to read";
> (2) install the GUID in NorFlashDxe once it's done verifying and/or
> formatting the headers;
> (3) introduce a custom library instance with an empty constructor
> function, and a DEPEX on the GUID;
> (4) hook the library instance into
> "MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf"
> in all the platform DSC files where the platform build does not
> pre-format a varstore template in the FDF file;
> (5) replace the BEFORE depex with gEfiCpuArchProtocolGuid.
>

A *very* big thank you for taking the time to explain this in detail,
after a couple of reads (I mainly just build EDK2) I was able to get
something booting following the above approach :-).

Cheers,
--
Steve
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-11 Thread Laszlo Ersek 
On 04/11/18 18:10, Ard Biesheuvel wrote:
> On 11 April 2018 at 17:34, Supreeth Venkatesh
>  wrote:
>> Laszlo/Steve,
>>
>> I ran into the same problem on ArmVExpress-FVP-AArch64 while running 
>> management mode.
>> I have fixed this problem in this patch by reordering driver load order here:
>> https://lists.01.org/pipermail/edk2-devel/2018-April/023550.html
>>
>> Not sure whether Leif/Ard/Laszlo agree with this.
>>
> 
> Nope, sorry. PI has an elaborate scheme involving dependency
> expressions to manage load order of modules, and tweaking the order
> they appear in the flash volume to get things working again is not the
> way to address this.
> 
> Ironically, though, it seems my patch
> 61a7b0ec634fa3288f47929ba3ced05ff48de739 (which introduces an AFTER
> xxx depex) is the culprit here, and so I should probably take some
> responsibility to get things working again.
> 
> I will look into replacing the NorFlashDxe DEPEX with something more
> suitable to address this issue, but I am a bit swamped at the moment,
> so if anyone else beats me to it, I won't complain.

I'll try. :)

Thanks,
Laszlo
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-11 Thread Laszlo Ersek 
On 04/11/18 17:34, Supreeth Venkatesh wrote:
> Laszlo/Steve,
>
> I ran into the same problem on ArmVExpress-FVP-AArch64 while running
> management mode.
> I have fixed this problem in this patch by reordering driver load
> order here:
> https://lists.01.org/pipermail/edk2-devel/2018-April/023550.html
>
> Not sure whether Leif/Ard/Laszlo agree with this.

Unfortunately, that's not a fix, just an unreliable work-around. The
dispatch order between DXE modules is unspecified beyond the order
imposed by their dependency expressions (hence "depex"). I don't claim
that it's always feasible to fix the problem with custom NULL protocols,
but whenever it is, I suggest that we do. The protocol dependency
mechanism is clean and extensible, unlike "BEFORE" depex opcodes, or the
"naked" listing order in FDF files.

Admittedly, sometimes there's no way around specifying a precise
(partial) load order for several modules. For that, the APRIORI DXE file
should be used, if possible. And, the APRIORI DXE file should be kept as
small as possible (listing as few modules as possible).

I think we should implement the following changes.

(1) "ArmPkg/Drivers/ArmGic/ArmGicDxe.inf" produces two custom protocols
already, gHardwareInterruptProtocolGuid and
gHardwareInterrupt2ProtocolGuid. These are installed on the following
call path:

  InterruptDxeInitialize()  
[ArmPkg/Drivers/ArmGic/ArmGicDxe.c]
GicV2DxeInitialize() | GicV3DxeInitialize() 
[ArmPkg/Drivers/ArmGic/GicV2/ArmGicV2Dxe.c | 
ArmPkg/Drivers/ArmGic/GicV3/ArmGicV3Dxe.c]
  InstallAndRegisterInterruptService()  
[ArmPkg/Drivers/ArmGic/ArmGicCommonDxe.c]

The GUIDs come from "EmbeddedPkg/Include/Protocol/HardwareInterrupt.h"
and "EmbeddedPkg/Include/Protocol/HardwareInterrupt2.h", respectively.

Annotate "ArmPkg/Drivers/ArmGic/ArmGicDxe.inf" that it produces
gHardwareInterruptProtocolGuid and gHardwareInterrupt2ProtocolGuid, and
that it consumes (with a protocol notify) gEfiCpuArchProtocolGuid.

(2) Replace "AFTER gArmGicDxeFileGuid" in
"ArmPkg/Drivers/CpuDxe/CpuDxe.inf" with "gHardwareInterruptProtocolGuid
OR gHardwareInterrupt2ProtocolGuid".

(3) Replace "BEFORE gVariableRuntimeDxeFileGuid" in
"ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashDxe.inf" with
"gEfiCpuArchProtocolGuid".

(4) In all platforms that use NorFlashDxe without formatting a variable
store template at build time, make the variable driver depend on
NorFlashDxe's runtime formatting, with the trick I described earlier.


In fact, I haven't booted ArmVirtQemu in a while, and now I'm finding it
is afflicted by the same assertion failure as the one reported by Steve.
This is somewhat strange because I don't see any Arm*Pkg changes since
my commit 34711bf19833 ("ArmVirtPkg/PlatformBootManagerLib: return to
"-kernel before boot devices"", 2018-03-16). I definitely tested that
commit successfully, so I wonder what may have triggered / exposed the
issue recently. Possibly BaseTools changes? That wouldn't be surprising;
again, the dispatch order is unspecified beyond depexes.

Either way, I guess I better get to work on this.

Thanks,
Laszlo
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-11 Thread Ard Biesheuvel 
On 11 April 2018 at 17:34, Supreeth Venkatesh
 wrote:
> Laszlo/Steve,
>
> I ran into the same problem on ArmVExpress-FVP-AArch64 while running 
> management mode.
> I have fixed this problem in this patch by reordering driver load order here:
> https://lists.01.org/pipermail/edk2-devel/2018-April/023550.html
>
> Not sure whether Leif/Ard/Laszlo agree with this.
>

Nope, sorry. PI has an elaborate scheme involving dependency
expressions to manage load order of modules, and tweaking the order
they appear in the flash volume to get things working again is not the
way to address this.

Ironically, though, it seems my patch
61a7b0ec634fa3288f47929ba3ced05ff48de739 (which introduces an AFTER
xxx depex) is the culprit here, and so I should probably take some
responsibility to get things working again.

I will look into replacing the NorFlashDxe DEPEX with something more
suitable to address this issue, but I am a bit swamped at the moment,
so if anyone else beats me to it, I won't complain.

-- 
Ard.

> -Original Message-
> From: edk2-devel  On Behalf Of Laszlo Ersek
> Sent: Wednesday, April 11, 2018 7:26 AM
> To: Steve Capper ; edk2-devel@lists.01.org
> Cc: Ard Biesheuvel ; Leif Lindholm 
> ; Nariman Poushin 
> Subject: Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch 
> protocol does not appear to be registered
>
> Hi Steve,
>
> On 04/11/18 12:30, Steve Capper wrote:
>> Hello,
>> I am trying to debug a boot problem for a recent build of EDK2 in an
>> Arm FVP model.
>>
>> I built EDK2 from the following branches:
>> Edk2: a146c532c754106431b063fec9985a838afd82be
>> Edk2-platforms: e74f53df8b18e4aed0c9df0942ce0c30f78a68e2
>>
>> Toolchain: Debian Stretch latest (though I have also tried Linaro s
>> GCC 5 latest).
>>
>> Build command:
>> GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -b
>> DEBUG -p Platform/ARM/VExpressPkg/ArmVExpress-FVP-AArch64.dsc
>>
>> The boot log can be found in the attached file. I *think* this is due
>> to the Cpu Arch protocol not actually being ready when
>> CoreConvertSpace is called by the SetMemoryAttributes from the
>> NorFlash code.
>>
>> I have found that this code path fails:
>> https://github.com/tianocore/edk2/blob/a146c532c754106431b063fec9985a8
>> 38afd82be/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L893
>>
>> The only modification I have made is to edk2-platforms to get more
>> debug output (PcdDebugPrintErrorLevel).
>>
>> Unfortunately, I'm not sure how to fix this?
>
> This is a bit of a problem.
>
> The gDS->SetMemorySpaceAttributes() DXE service is allowed to fail if the CPU 
> architectural protocol has not been installed yet. This is documented in the 
> Platform Init spec (although riddled by a few typos -- EFI_NOT_AVAILABLE_YET 
> is documented under the preceding
> GetMemorySpaceDescriptor() section, and not in the correct
> SetMemorySpaceAttributes() section).
>
> The usual solution for DXE drivers is to add an explicit dependency on 
> gEfiCpuArchProtocolGuid to the DEPEX section in the INF file. You can see an 
> example in commit f9a8be423cdd
> ("ArmVirtualizationPkg/PciHostBridgeDxe: MMIO aperture must not be uncached", 
> 2015-02-23).
>
> And that's where the issue is. If you check the [Depex] section in 
> "ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashDxe.inf", you find
>
>> [Depex]
>>   #
>>   # NorFlashDxe must be loaded before VariableRuntimeDxe in case empty flash 
>> needs populating with default values
>>   #
>>   BEFORE gVariableRuntimeDxeFileGuid
>
> Why is BEFORE technically an issue here? Because, it cannot be combined with 
> other depex opcodes. From the PI spec:
>
>> If this opcode is present in a dependency expression, it must be the
>> first and only opcode in the expression. If is appears in any other
>> location in the dependency expression, then the dependency expression
>> is evaluated to FALSE.
>
> So we can't simply combine the existent BEFORE depex with "AND 
> gEfiCpuArchProtocolGuid", like you see in the commit f9a8be423cdd example 
> above.
>
> Now, more to the point, a "BEFORE" depex is *always* a hack. From the PI
> spec:
>
>> This opcode tells the DXE Dispatcher that the DXE driver that is
>> associated with this dependency expression must be dispatched just
>> before the DXE driver with the file name specified by GUID. This means
>> that as soon as the dependency expression for the DXE driver specified
>> by GUID evaluates to TRUE, then this DXE driver must be placed in the
>> dispatch queue just before the DXE driver with the file name specified
>> by GUID.
>
> Sometimes there's really no way around such a hack, to express an 
> inter-module dependency, but even then, there is a better way to implement 
> it. Namely,
> - a custom protocol GUID can be invented,
> - the pre-requisite module can install a NULL protocol interface with
>   that

Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-11 Thread Supreeth Venkatesh 
Laszlo/Steve,

I ran into the same problem on ArmVExpress-FVP-AArch64 while running management 
mode.
I have fixed this problem in this patch by reordering driver load order here:
https://lists.01.org/pipermail/edk2-devel/2018-April/023550.html

Not sure whether Leif/Ard/Laszlo agree with this.

Thanks,
Supreeth

-Original Message-
From: edk2-devel  On Behalf Of Laszlo Ersek
Sent: Wednesday, April 11, 2018 7:26 AM
To: Steve Capper ; edk2-devel@lists.01.org
Cc: Ard Biesheuvel ; Leif Lindholm 
; Nariman Poushin 
Subject: Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol 
does not appear to be registered

Hi Steve,

On 04/11/18 12:30, Steve Capper wrote:
> Hello,
> I am trying to debug a boot problem for a recent build of EDK2 in an
> Arm FVP model.
>
> I built EDK2 from the following branches:
> Edk2: a146c532c754106431b063fec9985a838afd82be
> Edk2-platforms: e74f53df8b18e4aed0c9df0942ce0c30f78a68e2
>
> Toolchain: Debian Stretch latest (though I have also tried Linaros
> GCC 5 latest).
>
> Build command:
> GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -b
> DEBUG -p Platform/ARM/VExpressPkg/ArmVExpress-FVP-AArch64.dsc
>
> The boot log can be found in the attached file. I *think* this is due
> to the Cpu Arch protocol not actually being ready when
> CoreConvertSpace is called by the SetMemoryAttributes from the
> NorFlash code.
>
> I have found that this code path fails:
> https://github.com/tianocore/edk2/blob/a146c532c754106431b063fec9985a8
> 38afd82be/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L893
>
> The only modification I have made is to edk2-platforms to get more
> debug output (PcdDebugPrintErrorLevel).
>
> Unfortunately, I'm not sure how to fix this?

This is a bit of a problem.

The gDS->SetMemorySpaceAttributes() DXE service is allowed to fail if the CPU 
architectural protocol has not been installed yet. This is documented in the 
Platform Init spec (although riddled by a few typos -- EFI_NOT_AVAILABLE_YET is 
documented under the preceding
GetMemorySpaceDescriptor() section, and not in the correct
SetMemorySpaceAttributes() section).

The usual solution for DXE drivers is to add an explicit dependency on 
gEfiCpuArchProtocolGuid to the DEPEX section in the INF file. You can see an 
example in commit f9a8be423cdd
("ArmVirtualizationPkg/PciHostBridgeDxe: MMIO aperture must not be uncached", 
2015-02-23).

And that's where the issue is. If you check the [Depex] section in 
"ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashDxe.inf", you find

> [Depex]
>   #
>   # NorFlashDxe must be loaded before VariableRuntimeDxe in case empty flash 
> needs populating with default values
>   #
>   BEFORE gVariableRuntimeDxeFileGuid

Why is BEFORE technically an issue here? Because, it cannot be combined with 
other depex opcodes. From the PI spec:

> If this opcode is present in a dependency expression, it must be the
> first and only opcode in the expression. If is appears in any other
> location in the dependency expression, then the dependency expression
> is evaluated to FALSE.

So we can't simply combine the existent BEFORE depex with "AND 
gEfiCpuArchProtocolGuid", like you see in the commit f9a8be423cdd example above.

Now, more to the point, a "BEFORE" depex is *always* a hack. From the PI
spec:

> This opcode tells the DXE Dispatcher that the DXE driver that is
> associated with this dependency expression must be dispatched just
> before the DXE driver with the file name specified by GUID. This means
> that as soon as the dependency expression for the DXE driver specified
> by GUID evaluates to TRUE, then this DXE driver must be placed in the
> dispatch queue just before the DXE driver with the file name specified
> by GUID.

Sometimes there's really no way around such a hack, to express an inter-module 
dependency, but even then, there is a better way to implement it. Namely,
- a custom protocol GUID can be invented,
- the pre-requisite module can install a NULL protocol interface with
  that GUID,
- and
  - either the protocol GUID can be added directly to the [depex]
section of the dependent module (if the latter module is under the
platform's control),
  - or the platform DSC file can hook a NULL-class library instance into
the dependent module, such that the lib instance makes the dependent
module *inherit* a [depex] on the protocol GUID.

In other words, if we want to order the dispatch of a platform-independent 
module (that lives in another Package), due to platform-specific circumstances, 
behind another module that our platform owns, then we can change our platform 
DSC to "imbue" the module with a dependency on a custom protocol GUID. We can 
then install that NULL protocol in our own module. Under this scheme the usual 
dependency resolution will order things the right way, and all the depex-es 
remain

Re: [edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-11 Thread Laszlo Ersek 
Hi Steve,

On 04/11/18 12:30, Steve Capper wrote:
> Hello,
> I am trying to debug a boot problem for a recent build of EDK2 in an
> Arm FVP model.
>
> I built EDK2 from the following branches:
> Edk2: a146c532c754106431b063fec9985a838afd82be
> Edk2-platforms: e74f53df8b18e4aed0c9df0942ce0c30f78a68e2
>
> Toolchain: Debian Stretch latest (though I have also tried Linaros
> GCC 5 latest).
>
> Build command:
> GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -b
> DEBUG -p Platform/ARM/VExpressPkg/ArmVExpress-FVP-AArch64.dsc
>
> The boot log can be found in the attached file. I *think* this is due
> to the Cpu Arch protocol not actually being ready when
> CoreConvertSpace is called by the SetMemoryAttributes from the
> NorFlash code.
>
> I have found that this code path fails:
> https://github.com/tianocore/edk2/blob/a146c532c754106431b063fec9985a838afd82be/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L893
>
> The only modification I have made is to edk2-platforms to get more
> debug output (PcdDebugPrintErrorLevel).
>
> Unfortunately, I'm not sure how to fix this?

This is a bit of a problem.

The gDS->SetMemorySpaceAttributes() DXE service is allowed to fail if
the CPU architectural protocol has not been installed yet. This is
documented in the Platform Init spec (although riddled by a few typos --
EFI_NOT_AVAILABLE_YET is documented under the preceding
GetMemorySpaceDescriptor() section, and not in the correct
SetMemorySpaceAttributes() section).

The usual solution for DXE drivers is to add an explicit dependency on
gEfiCpuArchProtocolGuid to the DEPEX section in the INF file. You can
see an example in commit f9a8be423cdd
("ArmVirtualizationPkg/PciHostBridgeDxe: MMIO aperture must not be
uncached", 2015-02-23).

And that's where the issue is. If you check the [Depex] section in
"ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashDxe.inf", you find

> [Depex]
>   #
>   # NorFlashDxe must be loaded before VariableRuntimeDxe in case empty flash 
> needs populating with default values
>   #
>   BEFORE gVariableRuntimeDxeFileGuid

Why is BEFORE technically an issue here? Because, it cannot be combined
with other depex opcodes. From the PI spec:

> If this opcode is present in a dependency expression, it must be the
> first and only opcode in the expression. If is appears in any other
> location in the dependency expression, then the dependency expression
> is evaluated to FALSE.

So we can't simply combine the existent BEFORE depex with "AND
gEfiCpuArchProtocolGuid", like you see in the commit f9a8be423cdd
example above.

Now, more to the point, a "BEFORE" depex is *always* a hack. From the PI
spec:

> This opcode tells the DXE Dispatcher that the DXE driver that is
> associated with this dependency expression must be dispatched just
> before the DXE driver with the file name specified by GUID. This means
> that as soon as the dependency expression for the DXE driver specified
> by GUID evaluates to TRUE, then this DXE driver must be placed in the
> dispatch queue just before the DXE driver with the file name specified
> by GUID.

Sometimes there's really no way around such a hack, to express an
inter-module dependency, but even then, there is a better way to
implement it. Namely,
- a custom protocol GUID can be invented,
- the pre-requisite module can install a NULL protocol interface with
  that GUID,
- and
  - either the protocol GUID can be added directly to the [depex]
section of the dependent module (if the latter module is under the
platform's control),
  - or the platform DSC file can hook a NULL-class library instance into
the dependent module, such that the lib instance makes the dependent
module *inherit* a [depex] on the protocol GUID.

In other words, if we want to order the dispatch of a
platform-independent module (that lives in another Package), due to
platform-specific circumstances, behind another module that our platform
owns, then we can change our platform DSC to "imbue" the module with a
dependency on a custom protocol GUID. We can then install that NULL
protocol in our own module. Under this scheme the usual dependency
resolution will order things the right way, and all the depex-es remain
composable with other GUIDs.

OK -- so *should* we rewrite this BEFORE depex, like described above? I
don't think so. In my opinion, the *idea* of this dependency is wrong.
The BEFORE depex was introduced in commit 6acb379fbcf8
("ArmPlatformPkg/CTA9x4: Remove Variable Storage FD file from FDF",
2011-03-31), and the idea there was to replace the build-time
pre-formatting of the flash variable store with runtime formatting. You
might ask why this replacement needed an explicit ordering hint between
drivers (i.e. why any DEPEXes had to be touched at all). Here's why:

- For *writing* non-volatile UEFI variables, a platform-independent
  protocol dependency chain already exists. At the least abstract level,
  we have the FVB (firmware volume block) protocol service, e.g.
  NorFlashDxe. At the

[edk2] Boot failure for ArmVExpress-FVP-AArch64, CpuArch protocol does not appear to be registered

2018-04-11 Thread Steve Capper 
Hello,
I am trying to debug a boot problem for a recent build of EDK2 in an
Arm FVP model.

I built EDK2 from the following branches:
Edk2: a146c532c754106431b063fec9985a838afd82be
Edk2-platforms: e74f53df8b18e4aed0c9df0942ce0c30f78a68e2

Toolchain: Debian Stretch latest (though I have also tried Linaro’s
GCC 5 latest).

Build command:
GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -b
DEBUG -p Platform/ARM/VExpressPkg/ArmVExpress-FVP-AArch64.dsc

The boot log can be found in the attached file. I *think* this is due
to the Cpu Arch protocol not actually being ready when
CoreConvertSpace is called by the SetMemoryAttributes from the
NorFlash code.

I have found that this code path fails:
https://github.com/tianocore/edk2/blob/a146c532c754106431b063fec9985a838afd82be/MdeModulePkg/Core/Dxe/Gcd/Gcd.c#L893

The only modification I have made is to edk2-platforms to get more
debug output (PcdDebugPrintErrorLevel).

Unfortunately, I'm not sure how to fix this?

Cheers,
--
Steve
NOTICE:  Booting Trusted Firmware
NOTICE:  BL1: v1.4(release):works-for-steve-328-gd1156e0b
NOTICE:  BL1: Built : 16:11:18, Apr  6 2018
NOTICE:  BL1: Booting BL2
NOTICE:  BL2: v1.4(release):works-for-steve-328-gd1156e0b
NOTICE:  BL2: Built : 16:11:21, Apr  6 2018
NOTICE:  BL1: Booting BL31
NOTICE:  BL31: v1.4(release):works-for-steve-328-gd1156e0b
NOTICE:  BL31: Built : 16:11:27, Apr  6 2018
UEFI firmware (version  built at 10:37:57 on Apr 11 2018)
add-symbol-file 
/home/steven/aarch64fw/workspace/Build/ArmVExpress-FVP-AArch64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePi/PeiUniCore/DEBUG/ArmPlatformPrePiUniCore.dll
 0x88000800
add-symbol-file 
/home/steven/aarch64fw/workspace/Build/ArmVExpress-FVP-AArch64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll
 0xFE6DD000
Loading DxeCore at 0x00FE6DC000 EntryPoint=0x00FE6DD000
Promote the memory resource
CoreInitializeMemoryServices:
  BaseAddress - 0x8C00 Length - 0x6F00 MinimalMemorySizeNeeded - 
0x5065000
AddRange: 8C00-FAFF to 7
ConvertRange: FAFFF000-FAFF to type 4
AddRange: FAFFF000-FAFF to 4
ConvertRange: FAFA-FAFE to type 6
AddRange: FAFA-FAFE to 6
ConvertRange: FAF5-FAF9 to type 5
AddRange: FAF5-FAF9 to 5
ConvertRange: FADC-FAF4 to type 3
AddRange: FADC-FAF4 to 3
ConvertRange: F5FA-FADB to type 4
AddRange: F5FA-FADB to 4
ConvertRange: F5F8C000-F5F9 to type 1
AddRange: F5F8C000-F5F9 to 1
ConvertRange: FAFA-FAFE to type 7
AddRange: FAFA-FAFE to 7
ConvertRange: FAF5-FAF9 to type 7
AddRange: FAF5-FAF9 to 7
ConvertRange: FADC-FAF4 to type 7
AddRange: FADC-FAF4 to 7
ConvertRange: F5FA-FADB to type 7
AddRange: F5FA-FADB to 7
ConvertRange: F5F8C000-F5F9 to type 7
AddRange: F5F8C000-F5F9 to 7
ConvertRange: FAFE-FAFE to type 6
AddRange: FAFE-FAFE to 6
AllocatePoolI: Type 6, Addr FAFE0018 (len 78) 160
AllocatePoolI: Type 6, Addr FAFEFF18 (len 88) 336
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B FE701250
ConvertRange: FADBF000-FADB to type 4
AddRange: FADBF000-FADB to 4
AllocatePoolI: Type 4, Addr FADBF018 (len 48) 112
AllocatePoolI: Type 4, Addr FADBFF18 (len 58) 240
AllocatePoolI: Type 4, Addr FADBFF98 (len 38) 336
ProtectUefiImageCommon - 0xFE701250
  - 0xFE6DC000 - 0x00041000
AllocatePoolI: Type 4, Addr FADBFB18 (len 50) 456
AllocatePoolI: Type 4, Addr FADBFE18 (len 38) 552
GCD:Initial GCD Memory Space Map
GCDMemType Range Capabilities Attributes  
== =  
NonExist   -000F   

FreePool: FADBFE18 (len 38) 456
AllocatePoolI: Type 4, Addr FADBFE18 (len 50) 576
AllocatePoolI: Type 4, Addr FADBFE98 (len 28) 656
GCD:Initial GCD I/O Space Map
GCDIoType  Range
== =
NonExist   - 

FreePool: FADBFE98 (len 28) 576
GCD:AddMemorySpace(Base=00088000,Length=8000)
  GcdMemoryType   = SystemMem
  Capabilities= 000E
AllocatePoolI: Type 4, Addr FADBFE98 (len 50) 696
AllocatePoolI: Type 4, Addr FADBFB98 (len 50) 816
  Status = Success
AllocatePoolI: Type 4, Addr FADBFC18 (len A8) 1,024
GCDMemType Range Capabilities Attributes  
== =  
NonExist   -00087FFF   
SystemMem  00088000-0008 800E  
NonExist   0009-000F   

FreePool: FADBFC18 (len A8) 816
GCD:AddMemorySpace(Base=8000,Length=0800)
  GcdMemoryType   = SystemMem
  Capabilities= 000E
AllocatePoolI: Type 4, Addr