date:20171110

[edk2] EDK2 build issues

2017-11-10 Thread Aaron Durbin

Hello,

Here are some observations I've encountered trying to utilize edk2 for
certain builds. Part of the problem seems to be with implicit
assumptions in how edk2 is used. I'm trying to build things using edk2
from a clean enviroment on an automated builder. i.e. there isn't a
workspace that exists on one persons computer for the lifetime of
development.

1. BaseTools can't build in parallel. The tests are racey which result
in test failures. Because of this one has to build these in serial
instead of in parallel.

==
FAIL: testHelp (TianoCompress.Tests)
--
Traceback (most recent call last):
File
"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-/work/fsp-cnl-/Edk2/BaseTools/Tests/TianoCompress.py",
line 34, in testHelp
self.assertTrue(result == 0)
AssertionError: False is not true

==
FAIL: testRandomDataCycles (TianoCompress.Tests)
--
Traceback (most recent call last):
File
"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-/work/fsp-cnl-/Edk2/BaseTools/Tests/TianoCompress.py",
line 65, in testRandomDataCycles
self.compressionTestCycle(data)
File
"/build/zoombini/tmp/portage/sys-boot/fsp-cnl-/work/fsp-cnl-/Edk2/BaseTools/Tests/TianoCompress.py",
line 44, in compressionTestCycle
self.assertTrue(result == 0)
AssertionError: False is not true

In addition, it seems compilation even breaks trying to build a parser:

VfrSyntax.cpp:53:1: error: expected class-name before '{' token
{^M
^
VfrSyntax.cpp: In constructor 'CVfrDLGLexer::CVfrDLGLexer(DLGFileInput*)':
VfrSyntax.cpp:55:36: error: class 'CVfrDLGLexer' does not have any
field named 'VfrLexer'
CVfrDLGLexer (DLGFileInput *F) : VfrLexer (F) {};^M
^

This just slows down builds needing to things serially.

2. It appears the BaseTools uses the ARCH environment variable. I'm
not sure of the origins, but ARCH seems like a complete misnomer for
the *host* you are trying to build tools on -- not the target. Trying
to incorporate edk2 builds into a portage environment effectively
breaks because of this as ARCH refers to target architecture -- not
host builder's ARCH.

3. This more of an observation, but the tools definition seems to make
quite the leap on how consistent compilers are of a certain version.
e.g. GCC 4.9 can be built with all kinds of default options that edk2
implicitly assumes are set based on some distribution's default flags?
And in order to extend toolchain support one needs to create a series
of entries associated with a certain family. Barrier to entry is
pretty high in teasing out where to put things in the ~8000 line
tools_def.template.

Thanks for the consideration.

-Aaron
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] stdlib broken strncasecmp.c

2017-11-10 Thread David F.

I would submit a patch but over the years of fixes and enhancements,
they don't go anywhere. Then a couple years later someone does a
half-baked fix and I have to run merge resolution on svn.

Anyway, the problem with that is the s1 compare to 0 should be before
the s1++ (otherwise it matches when the s1 string is shorter than the
s2 string).
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 2/4] OvmfPkg/Sec/Ia32: seed the temporary RAM with PcdInitValueInTempStack

2017-11-10 Thread Laszlo Ersek

On 11/10/17 19:11, Laszlo Ersek wrote:
> On 11/10/17 16:56, Ard Biesheuvel wrote:
>> On 10 November 2017 at 15:49, Laszlo Ersek  wrote:
>>> This allows the PEI core to report the maximum temporary SEC/PEI stack
>>> usage on the DEBUG_INFO level, in the PeiCheckAndSwitchStack() function
>>> [MdeModulePkg/Core/Pei/Dispatcher/Dispatcher.c]:
>>>
>>> * Normal boot:
>>>
 Temp Stack : BaseAddress=0x814000 Length=0x4000
 Temp Heap  : BaseAddress=0x81 Length=0x4000
 Total temporary memory:32768 bytes.
   temporary memory stack ever used:   3664 bytes. <
   temporary memory heap used for HobList: 5904 bytes.
   temporary memory heap occupied by memory pages: 0 bytes.
>>>
>>> * S3 resume (with PEI decompression / SMM):
>>>
 Temp Stack : BaseAddress=0x814000 Length=0x4000
 Temp Heap  : BaseAddress=0x81 Length=0x4000
 Total temporary memory:32768 bytes.
   temporary memory stack ever used:   3428 bytes. <
   temporary memory heap used for HobList: 4816 bytes.
   temporary memory heap occupied by memory pages: 0 bytes.
>>>
>>> I unit-tested this change by transitorily adding an infinite loop right
>>> after the "rep stosd", and dumping the guest's temp SEC/PEI RAM (32KB
>>> currently) while the guest was stuck in the loop. The dump includes one
>>> dword from before and after the temp SEC/PEI RAM:
>>>
 $ virsh qemu-monitor-command GUEST_NAME --hmp 'xp /8194wx 0x80FFFC'

 0080fffc: 0x 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5
 0081000c: 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5
 ...
 00817fec: 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5
 00817ffc: 0x5aa55aa5 0x
>>>
>>> Cc: Ard Biesheuvel 
>>> Cc: Jordan Justen 
>>> Cc: Ruiyu Ni 
>>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=747
>>> Contributed-under: TianoCore Contribution Agreement 1.1
>>> Signed-off-by: Laszlo Ersek 
>>> ---
>>>  OvmfPkg/Sec/SecMain.inf|  1 +
>>>  OvmfPkg/Sec/Ia32/SecEntry.nasm | 13 +
>>>  2 files changed, 14 insertions(+)
>>>
>>> diff --git a/OvmfPkg/Sec/SecMain.inf b/OvmfPkg/Sec/SecMain.inf
>>> index 711b59530907..6051cb3c6c4c 100644
>>> --- a/OvmfPkg/Sec/SecMain.inf
>>> +++ b/OvmfPkg/Sec/SecMain.inf
>>> @@ -71,6 +71,7 @@ [Pcd]
>>>gEfiMdePkgTokenSpaceGuid.PcdGuidedExtractHandlerTableAddress
>>>gUefiOvmfPkgTokenSpaceGuid.PcdGuidedExtractHandlerTableSize
>>>gUefiOvmfPkgTokenSpaceGuid.PcdOvmfDecompressionScratchEnd
>>> +  gEfiMdeModulePkgTokenSpaceGuid.PcdInitValueInTempStack
>>>
>>>  [FeaturePcd]
>>>gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire
>>> diff --git a/OvmfPkg/Sec/Ia32/SecEntry.nasm b/OvmfPkg/Sec/Ia32/SecEntry.nasm
>>> index 54d074e621f6..1d426fafa888 100644
>>> --- a/OvmfPkg/Sec/Ia32/SecEntry.nasm
>>> +++ b/OvmfPkg/Sec/Ia32/SecEntry.nasm
>>> @@ -29,6 +29,7 @@ extern ASM_PFX(SecCoreStartupWithStack)
>>>  ; @param[in]  EAX   Initial value of the EAX register (BIST: Built-in Self 
>>> Test)
>>>  ; @param[in]  DI'BP': boot-strap processor, or 'AP': application 
>>> processor
>>>  ; @param[in]  EBP   Pointer to the start of the Boot Firmware Volume
>>> +; @param[in]  ESSet to LINEAR_SEL in TransitionFromReal16To32BitFlat
>>
>> What does this mean? Does it belong in this patch? (Knowing you, and
>> noticing that the next patch adds it to the x86 version of this code
>> as well, I am sure it probably does, but I just need you to explain it
>> to me :-))
> 
> See the STOSD instruction in the Intel SDM, or just on the web:
> 
> http://x86.renejeschke.de/html/file_module_x86_id_306.html
> 
>> Stores a byte, word, or doubleword from the AL, AX, or EAX register,
>> respectively, into the destination operand. The destination operand is
>> a memory location, the address of which is read from either the ES:EDI
>> or the ES:DI registers (depending on the address-size attribute of the
>> instruction, 32 or 16, respectively). The ES segment cannot be
>> overridden with a segment override prefix.
> 
> It means that whatever we put in EDI, it will be relative to the segment
> "identified by" ES. (See the code comment near "mov edi": "base address,
> relative to ES".)
> 
> Above I put "identified by" in quotes, because the definition of
> "identified by" depends on the mode the processor is in.
> 
> (Instead of a botched attempt at writing up x86 segmentation (plus,
> optionally, paging) generally :) , I'll just leave these references
> here:
> 
> * Intel SDM: 2.2 MODES OF OPERATION; Figure 2-3. Transitions Among the
>   Processor's Operating Modes
> 
> * https://en.wikipedia.org/wiki/Unreal_mode
> 
> * https://en.wikipedia.org/wiki/X86_memory_segmentation#80286_protected_mode
> 
> * https://en.wikipedia.org/wiki/X86_memory_segmentation#80386_protected_mode
> )
> 
> So using STOSD and ES:EDI as an example, here's a hugely inaccurate list
> of possible

Re: [edk2] [PATCH 2/4] OvmfPkg/Sec/Ia32: seed the temporary RAM with PcdInitValueInTempStack

2017-11-10 Thread Laszlo Ersek

On 11/10/17 16:56, Ard Biesheuvel wrote:
> On 10 November 2017 at 15:49, Laszlo Ersek  wrote:
>> This allows the PEI core to report the maximum temporary SEC/PEI stack
>> usage on the DEBUG_INFO level, in the PeiCheckAndSwitchStack() function
>> [MdeModulePkg/Core/Pei/Dispatcher/Dispatcher.c]:
>>
>> * Normal boot:
>>
>>> Temp Stack : BaseAddress=0x814000 Length=0x4000
>>> Temp Heap  : BaseAddress=0x81 Length=0x4000
>>> Total temporary memory:32768 bytes.
>>>   temporary memory stack ever used:   3664 bytes. <
>>>   temporary memory heap used for HobList: 5904 bytes.
>>>   temporary memory heap occupied by memory pages: 0 bytes.
>>
>> * S3 resume (with PEI decompression / SMM):
>>
>>> Temp Stack : BaseAddress=0x814000 Length=0x4000
>>> Temp Heap  : BaseAddress=0x81 Length=0x4000
>>> Total temporary memory:32768 bytes.
>>>   temporary memory stack ever used:   3428 bytes. <
>>>   temporary memory heap used for HobList: 4816 bytes.
>>>   temporary memory heap occupied by memory pages: 0 bytes.
>>
>> I unit-tested this change by transitorily adding an infinite loop right
>> after the "rep stosd", and dumping the guest's temp SEC/PEI RAM (32KB
>> currently) while the guest was stuck in the loop. The dump includes one
>> dword from before and after the temp SEC/PEI RAM:
>>
>>> $ virsh qemu-monitor-command GUEST_NAME --hmp 'xp /8194wx 0x80FFFC'
>>>
>>> 0080fffc: 0x 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5
>>> 0081000c: 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5
>>> ...
>>> 00817fec: 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5 0x5aa55aa5
>>> 00817ffc: 0x5aa55aa5 0x
>>
>> Cc: Ard Biesheuvel 
>> Cc: Jordan Justen 
>> Cc: Ruiyu Ni 
>> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=747
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Laszlo Ersek 
>> ---
>>  OvmfPkg/Sec/SecMain.inf|  1 +
>>  OvmfPkg/Sec/Ia32/SecEntry.nasm | 13 +
>>  2 files changed, 14 insertions(+)
>>
>> diff --git a/OvmfPkg/Sec/SecMain.inf b/OvmfPkg/Sec/SecMain.inf
>> index 711b59530907..6051cb3c6c4c 100644
>> --- a/OvmfPkg/Sec/SecMain.inf
>> +++ b/OvmfPkg/Sec/SecMain.inf
>> @@ -71,6 +71,7 @@ [Pcd]
>>gEfiMdePkgTokenSpaceGuid.PcdGuidedExtractHandlerTableAddress
>>gUefiOvmfPkgTokenSpaceGuid.PcdGuidedExtractHandlerTableSize
>>gUefiOvmfPkgTokenSpaceGuid.PcdOvmfDecompressionScratchEnd
>> +  gEfiMdeModulePkgTokenSpaceGuid.PcdInitValueInTempStack
>>
>>  [FeaturePcd]
>>gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire
>> diff --git a/OvmfPkg/Sec/Ia32/SecEntry.nasm b/OvmfPkg/Sec/Ia32/SecEntry.nasm
>> index 54d074e621f6..1d426fafa888 100644
>> --- a/OvmfPkg/Sec/Ia32/SecEntry.nasm
>> +++ b/OvmfPkg/Sec/Ia32/SecEntry.nasm
>> @@ -29,6 +29,7 @@ extern ASM_PFX(SecCoreStartupWithStack)
>>  ; @param[in]  EAX   Initial value of the EAX register (BIST: Built-in Self 
>> Test)
>>  ; @param[in]  DI'BP': boot-strap processor, or 'AP': application 
>> processor
>>  ; @param[in]  EBP   Pointer to the start of the Boot Firmware Volume
>> +; @param[in]  ESSet to LINEAR_SEL in TransitionFromReal16To32BitFlat
>
> What does this mean? Does it belong in this patch? (Knowing you, and
> noticing that the next patch adds it to the x86 version of this code
> as well, I am sure it probably does, but I just need you to explain it
> to me :-))

See the STOSD instruction in the Intel SDM, or just on the web:

http://x86.renejeschke.de/html/file_module_x86_id_306.html

> Stores a byte, word, or doubleword from the AL, AX, or EAX register,
> respectively, into the destination operand. The destination operand is
> a memory location, the address of which is read from either the ES:EDI
> or the ES:DI registers (depending on the address-size attribute of the
> instruction, 32 or 16, respectively). The ES segment cannot be
> overridden with a segment override prefix.

It means that whatever we put in EDI, it will be relative to the segment
"identified by" ES. (See the code comment near "mov edi": "base address,
relative to ES".)

Above I put "identified by" in quotes, because the definition of
"identified by" depends on the mode the processor is in.

(Instead of a botched attempt at writing up x86 segmentation (plus,
optionally, paging) generally :) , I'll just leave these references
here:

* Intel SDM: 2.2 MODES OF OPERATION; Figure 2-3. Transitions Among the
  Processor's Operating Modes

* https://en.wikipedia.org/wiki/Unreal_mode

* https://en.wikipedia.org/wiki/X86_memory_segmentation#80286_protected_mode

* https://en.wikipedia.org/wiki/X86_memory_segmentation#80386_protected_mode
)

So using STOSD and ES:EDI as an example, here's a hugely inaccurate list
of possible address calculations, dependent on processor mode:

* real mode:

  physical_address = ES * 0x10 + DI

  E.g., A000:1234 means physical address A1234.

* big real mode (segmentation

51 matches

Mail list logo