Re: [edk2] [PATCH 0/4] MdeModulePkg: add support for dispatching foreign arch PE/COFF images

[Shi, Steven]

Hi Ard,
Just my curious, are you supporting the below idea of QEMU in UEFI?

QEMU in UEFI - Alexander Graf
https://www.youtube.com/watch?v=uxvAH1Q4Mx0
http://events17.linuxfoundation.org/sites/events/files/slides/QEMU%20in%20UEFI.pdf



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Ard Biesheuvel
> Sent: Wednesday, September 12, 2018 9:22 PM
> To: edk2-devel@lists.01.org
> Cc: Ni, Ruiyu ; Zimmer, Vincent
> ; Dong, Eric ; Andrew
> Fish ; ag...@suse.de; Richardson, Brian
> ; Kinney, Michael D
> ; Laszlo Ersek ; Zeng,
> Star 
> Subject: [edk2] [PATCH 0/4] MdeModulePkg: add support for dispatching
> foreign arch PE/COFF images
> 
> Add the basic plumbing to DXE core, the PCI bus driver and the boot manager
> to allow PE/COFF images to be dispatched that target an architecture that is
> not native for the platform, but which is supported by an emulator.
> 
> One implementation of such an emulator can be found here:
> https://github.com/ardbiesheuvel/X86EmulatorPkg
> 
> Cc: Zimmer Vincent 
> Cc: Brian Richardson 
> Cc: Michael D Kinney 
> Cc: Andrew Fish 
> Cc: Laszlo Ersek 
> Cc: Leif Lindholm 
> Cc: Star Zeng 
> Cc: Eric Dong 
> Cc: Ruiyu Ni 
> 
> Ard Biesheuvel (4):
>   MdeModulePkg: introduce PE/COFF image emulator protocol
>   MdeModulePkg/DxeCore: invoke the emulator protocol for foreign images
>   MdeModulePkg/PciBusDxe: invoke PE/COFF emulator for foreign option
> ROMs
>   MdeModulePkg/UefiBootManagerLib: allow foreign Driver images
> 
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h   |  1 +
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf  |  1 +
>  .../Bus/Pci/PciBusDxe/PciOptionRomSupport.c   | 16 +-
>  MdeModulePkg/Core/Dxe/DxeMain.h   |  3 ++
>  MdeModulePkg/Core/Dxe/DxeMain.inf |  1 +
>  MdeModulePkg/Core/Dxe/Image/Image.c   | 39 +++---
>  .../Include/Protocol/PeCoffImageEmulator.h| 51 +++
>  .../Library/UefiBootManagerLib/BmLoadOption.c | 26 +-
>  .../Library/UefiBootManagerLib/InternalBm.h   |  1 +
>  .../UefiBootManagerLib/UefiBootManagerLib.inf |  1 +
>  MdeModulePkg/MdeModulePkg.dec |  4 ++
>  11 files changed, 136 insertions(+), 8 deletions(-)
>  create mode 100644
> MdeModulePkg/Include/Protocol/PeCoffImageEmulator.h
> 
> --
> 2.17.1
> 
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] MdeModulePkg/EhciDxe: fix host controller reset condition in BindingStart

[Shi, Steven]

> Steven, can you confirm the patch fixes the issue you saw?
Yes. The patch fixes my issue in 
https://bugzilla.tianocore.org/show_bug.cgi?id=1129 .

Thanks
Steven Shi

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2] BaseTools/GenFw: Add X64 GOTPCREL Support to GenFw

[Shi, Steven]

Hi Zenith,
I like your patch. Sorry for the late response. I see your patch  has no impact 
to current supported relocation types, and I also hope we can add more 
relocation types in our GenFw tool. 
Below are some code style issues in your patch:

 
> +STATIC
> +VOID
> +FindElfGOTSectionFromGOTEntryElfRva (
> +  Elf64_Addr GOTEntryElfRva
> +  )
> +{
> +  UINT32 i;
> +  if (mGOTShdr != NULL) {
> +if (GOTEntryElfRva >= mGOTShdr->sh_addr &&
> +GOTEntryElfRva <  mGOTShdr->sh_addr + mGOTShdr->sh_size)
> +  return;

We usually use the {} to brace the  if branch statements, even it has only one 
statement,  like this:
if (GOTEntryElfRva >= mGOTShdr->sh_addr &&
GOTEntryElfRva <  mGOTShdr->sh_addr + mGOTShdr->sh_size){
return;
}


> +STATIC
> +BOOLEAN
> +AccumulateCoffGOTEntries (
> +  UINT32 GOTCoffEntry
> +  )
> +{
> +  UINT32 i;
> +  if (mGOTCoffEntries != NULL) {
> +for (i = 0; i < mGOTNumCoffEntries; i++)
> +  if (mGOTCoffEntries[i] == GOTCoffEntry)
> +return FALSE;
Same issue as above, we usually always use {} to include if true branch 
statement like this:
  if (mGOTCoffEntries[i] == GOTCoffEntry){
return FALSE;
  }

> +STATIC
> +int
> +__comparator (
> +  const void* lhs,
> +  const void* rhs
> +  )
> +{
> +  if (*(const UINT32*)lhs < *(const UINT32*)rhs)
> +return -1;
> +  return *(const UINT32*)lhs > *(const UINT32*)rhs;
> +}
We usually don't use '__' as prefix for a function, could you enhance the 
comparator() name and add {} to include if branch statement?


> +STATIC
> +VOID
> +EmitGOTRelocations (
> +  VOID
> +  )
> +{
> +  UINT32 i;
> +  if (mGOTCoffEntries == NULL)
> +return;
Please add {} to include if branch statement


> +if (mEhdr->e_machine == EM_X86_64 && RelShdr->sh_info ==
> mGOTShindex) {
> +  //
> +  // Tack relocations for GOT entries after other relocations for
> +  //   the section the GOT is in, as it's usually found at the end
> +  //   of the section.
> +  //
> +  EmitGOTRelocations();
> +}
>}
>  }
>}
> 
> +  if (mEhdr->e_machine == EM_X86_64) {
> +//
> +// Just in case GOT is in a section with no other relocations
> +//
> +EmitGOTRelocations();
> +  }
Could we combine the two EmitGOTRelocations() invokes and only keep the second 
one?

Thanks
Steven
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [RFC PATCH 00/11] GCC/X64: use hidden visibility for LTO PIE code

[Shi, Steven]

Hi Ard, Zenith,
Thank you both explained the complete knowledge about ELF GOT, LTO, PIC/PIE, 
machine code mode and GCC visibility #pragma. It is pretty good to read them 
all in one picture. And I believe copying these explain to a edk2 wiki page in 
GitHub could be very useful for other edk2 developers. 

>From code change impact view, I see to use the hidden visibility for LTO, 
>which is to remove the !defined(USING_LTO) in X64/ProcessorBind.h actually, 
>need to change other 20+ files overall the edk2. The cost looks not small. We 
>might need more justification to accept such change.  Does the hidden 
>visibility in LTO can improve the LTO build code size? Is there any other 
>benefit?

Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Tuesday, June 12, 2018 11:23 PM
> To: edk2-devel@lists.01.org
> Cc: Ard Biesheuvel ; Kinney, Michael D
> ; Gao, Liming ; Ni,
> Ruiyu ; Wu, Hao A ; Leif
> Lindholm ; Justen, Jordan L
> ; Andrew Fish ; Zeng, Star
> ; Dong, Eric ; Laszlo Ersek
> ; Zenith432 ; Shi,
> Steven 
> Subject: [RFC PATCH 00/11] GCC/X64: use hidden visibility for LTO PIE code
> 
> The GCC toolchain uses PIE mode when building code for X64, because it
> is the most efficient in size: it uses relative references where
> possible, but still uses 64-bit quantities for absolute symbol
> references, which is optimal for executables that need to be converted
> to PE/COFF using GenFw.
> 
> Enabling PIE mode has a couple of side effects though, primarily caused
> by the fact that the primary application area of GCC is to build programs
> for userland. GCC will assume that ELF symbols should be preemptible (which
> makes sense for PIC but not for PIE, but this simply seems to be the result
> of code being shared between the two modes), and it will attempt to keep
> absolute references close to each other so that dynamic relocations that
> trigger CoW for text pages have the smallest possible footprint.
> 
> These side effects can be mititgated by overriding the visibility of all
> symbol definitions *and* symbol references, using a special #pragma. This
> will inform the compiler that symbol preemption and dynamic relocations
> are not a concern, and that all symbol references can be emitted as direct
> relative references rather than relative references to a GOT entry containing
> the absolute address. Unsurprisingly, this leads to better and smaller code.
> 
> Unfortunately, we have not been able to set this override when LTO is in
> effect, because the LTO code generator infers from the hidden visibility
> of all symbols that none of the code is reachable, and discards it all,
> leading to corrupt, empty binaries.
> 
> We can work around this by overriding the visibility for symbols that are
> module entry points. So implement this for all occcurrences of the symbol
> '_ModuleEntryPoint', and enable 'hidden' visibility in LTO builds as well.
> 
> Note that all the changes in this series resolve to no-ops if USING_LTO
> is not #defined.
> 
> Code can be found here:
> https://github.com/ardbiesheuvel/edk2/tree/x64-lto-visibility
> 
> Cc: Michael D Kinney 
> Cc: Liming Gao 
> Cc: Ruiyu Ni 
> Cc: Hao Wu 
> Cc: Leif Lindholm 
> Cc: Jordan Justen 
> Cc: Andrew Fish 
> Cc: Star Zeng 
> Cc: Eric Dong 
> Cc: Laszlo Ersek 
> Cc: Zenith432 
> Cc: "Shi, Steven" 
> 
> Ard Biesheuvel (11):
>   MdePkg/ProcessorBind.h: define macro to decorate module entry points
>   DuetPkg: annotate module entry points with EFI_ENTRYPOINT
>   EdkCompatibilityPkg: annotate module entry points with EFI_ENTRYPOINT
>   EmbeddedPkg: annotate module entry points with EFI_ENTRYPOINT
>   EmulatorPkg: annotate module entry points with EFI_ENTRYPOINT
>   IntelFrameWorkPkg: annotate module entry points with EFI_ENTRYPOINT
>   MdeModulePkg: annotate module entry points with EFI_ENTRYPOINT
>   MdePkg: annotate module entry points with EFI_ENTRYPOINT
>   Nt32Pkg: annotate module entry points with EFI_ENTRYPOINT
>   UefiCpuPkg: annotate module entry points with EFI_ENTRYPOINT
>   MdePkg/ProcessorBind.h X64: drop non-LTO limitation on visiblity
> override
> 
>  DuetPkg/DxeIpl/DxeInit.c |  1 +
>  DuetPkg/EfiLdr/EfiLoader.c   |  1 +
>  .../EntryPoints/EdkIIGlueDxeDriverEntryPoint.c   |  1 +
>  .../EntryPoints/EdkIIGluePeimEntryPoint.c|  1 +
>  .../EntryPoints/EdkIIGlueSmmDriverEntryPoint.c   |  1 +
>  .../Library/EdkIIGlueDxeSmmDriverEntryPoint.h|  1 +
>  .../Include/Library/EdkIIGluePeimEntryPoint.h|  1 +
>  .../Library/EdkIIGlueUefiDriverEntryPoint.h  |  1 +
>  EmbeddedPkg/TemplateS

Re: [edk2] [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to GenFw

[Shi, Steven]

Next week is OK, take your time. I appreciate the test cases with build steps, 
thanks!

BTW, I'm not sure where is the right place in edk2 to submit a patch's test 
cases, but I do welcome the test case contribution.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Zenith432 [mailto:zenith...@users.sourceforge.net]
> Sent: Thursday, June 7, 2018 2:09 PM
> To: Shi, Steven 
> Cc: Gao, Liming ; edk2-devel@lists.01.org
> Subject: RE: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> GenFw
> 
> As I said in other mailing - I only tested on cases in builld.  If you want
> standalone test cases I only have time next week to make some.
> 
> ------------
> On Thu, 6/7/18, Shi, Steven  wrote:
> 
>  Subject: RE: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> GenFw
>  To: "Gao, Liming" , "Zenith432"
> , "edk2-devel@lists.01.org"  de...@lists.01.org>
>  Date: Thursday, June 7, 2018, 5:24 AM
> 
>  Hi Zenith,
>  BTW,
>  besides the build pass, did you try to run a Uefi binary,
>  e.g. a simple shell application, which contain the GOTPCREL
>  relocations? If yes. Please send out your test case code as
>  well. I appreciate if you could contribute your test cases
>  as the patch together.
> 
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to GenFw

[Shi, Steven]

Hi Zenith,
BTW, besides the build pass, did you try to run a Uefi binary, e.g. a simple 
shell application, which contain the GOTPCREL relocations? If yes. Please send 
out your test case code as well. I appreciate if you could contribute your test 
cases as the patch together. 


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Shi, Steven
> Sent: Thursday, June 7, 2018 10:18 AM
> To: Shi, Steven ; Gao, Liming
> ; Zenith432 ;
> edk2-devel@lists.01.org
> Subject: RE: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> GenFw
> 
> Please see more details in
> https://bugzilla.tianocore.org/show_bug.cgi?id=970
> 
> 
> Steven Shi
> Intel\SSG\STO\UEFI Firmware
> 
> Tel: +86 021-61166522
> iNet: 821-6522
> 
> 
> > -Original Message-
> > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Shi,
> > Steven
> > Sent: Thursday, June 7, 2018 10:16 AM
> > To: Gao, Liming ; Zenith432
> > ; edk2-devel@lists.01.org
> > Subject: Re: [edk2] [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support
> > to GenFw
> >
> > Yes. If we disable the '#pragma GCC visibility push (hidden)' in
> > ProcessorBind.h, the GOTPCREL support is mandatory. 3rd party module
> > build might need it. It's more complete to add GOTPCREL support.
> >
> > The hidden #pragma can hide all ELF symbols' visibility, including 'extern'
> > functions, which is to let Linux compilers aware that we don’t need loading
> > dynamic link library in Uefi runing, please not emit the GOTPCREL based
> > relocations in the obj file, and we will ensure the linker can get all 
> > extern
> > functions statically solved. The reason of not emitting GOTPCREL based
> > relocations is because it is hard to convert the GOTPCREL based relocations
> > in ELF binary to PE/COFF related relocations, and our current GenFw has
> not
> > supported them. I'm glade if we can add the GOTPCREL support in GenFW
> to
> > totally solve this problem.
> >
> > Steven Shi
> > Intel\SSG\STO\UEFI Firmware
> >
> > Tel: +86 021-61166522
> > iNet: 821-6522
> >
> > > -Original Message-
> > > From: Gao, Liming
> > > Sent: Thursday, June 7, 2018 9:32 AM
> > > To: Zenith432 ; edk2-
> > de...@lists.01.org
> > > Cc: Shi, Steven ; Zhu, Yonghong
> > > 
> > > Subject: RE: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> > > GenFw
> > >
> > > What's purpose to support GOTPCREL in GenFw? Could you introduce
> your
> > > usage model?
> > >
> > > > -Original Message-
> > > > From: Zenith432 [mailto:zenith...@users.sourceforge.net]
> > > > Sent: Thursday, June 7, 2018 2:01 AM
> > > > To: edk2-devel@lists.01.org
> > > > Cc: Shi, Steven ; Zhu, Yonghong
> > > ; Gao, Liming 
> > > > Subject: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> > GenFw
> > > >
> > > > Adds support for the following X64 ELF relocations to GenFw
> > > >   R_X86_64_GOTPCREL
> > > >   R_X86_64_GOTPCRELX
> > > >   R_X86_64_REX_GOTPCRELX
> > > >
> > > > CC: Shi Steven 
> > > > CC: Yonghong Zhu 
> > > > CC: Liming Gao 
> > > > Contributed-under: TianoCore Contribution Agreement 1.1
> > > > Signed-off-by: Zenith432 
> > > > ---
> > > >  BaseTools/Source/C/GenFw/Elf64Convert.c | 166
> > > +++-
> > > >  BaseTools/Source/C/GenFw/elf_common.h   |  23 
> > > >  2 files changed, 188 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c
> > > b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > > > index c39bdff0..d2f9bb46 100644
> > > > --- a/BaseTools/Source/C/GenFw/Elf64Convert.c
> > > > +++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > > > @@ -94,6 +94,15 @@ STATIC Elf_Ehdr *mEhdr;
> > > >  STATIC Elf_Shdr *mShdrBase;
> > > >  STATIC Elf_Phdr *mPhdrBase;
> > > >
> > > > +//
> > > > +// GOT information
> > > > +//
> > > > +STATIC Elf_Shdr *mGOTShdr = NULL;
> > > > +STATIC UINT32   mGOTShindex = 0;
> > > > +STATIC UINT32   *mGOTCoffEntries = NULL;
> > > > +STATIC UINT32   mGOTMaxCoffEntries = 0;
> > > > +STATIC UINT32   mGOTNumCoffEntries = 0;
> > > > +
> > > >  //
> > > >  // Coff information
> &

Re: [edk2] [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to GenFw

[Shi, Steven]

Please see more details in https://bugzilla.tianocore.org/show_bug.cgi?id=970 


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Shi,
> Steven
> Sent: Thursday, June 7, 2018 10:16 AM
> To: Gao, Liming ; Zenith432
> ; edk2-devel@lists.01.org
> Subject: Re: [edk2] [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support
> to GenFw
> 
> Yes. If we disable the '#pragma GCC visibility push (hidden)' in
> ProcessorBind.h, the GOTPCREL support is mandatory. 3rd party module
> build might need it. It's more complete to add GOTPCREL support.
> 
> The hidden #pragma can hide all ELF symbols' visibility, including 'extern'
> functions, which is to let Linux compilers aware that we don’t need loading
> dynamic link library in Uefi runing, please not emit the GOTPCREL based
> relocations in the obj file, and we will ensure the linker can get all extern
> functions statically solved. The reason of not emitting GOTPCREL based
> relocations is because it is hard to convert the GOTPCREL based relocations
> in ELF binary to PE/COFF related relocations, and our current GenFw has not
> supported them. I'm glade if we can add the GOTPCREL support in GenFW to
> totally solve this problem.
> 
> Steven Shi
> Intel\SSG\STO\UEFI Firmware
> 
> Tel: +86 021-61166522
> iNet: 821-6522
> 
> > -Original Message-
> > From: Gao, Liming
> > Sent: Thursday, June 7, 2018 9:32 AM
> > To: Zenith432 ; edk2-
> de...@lists.01.org
> > Cc: Shi, Steven ; Zhu, Yonghong
> > 
> > Subject: RE: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> > GenFw
> >
> > What's purpose to support GOTPCREL in GenFw? Could you introduce your
> > usage model?
> >
> > > -Original Message-
> > > From: Zenith432 [mailto:zenith...@users.sourceforge.net]
> > > Sent: Thursday, June 7, 2018 2:01 AM
> > > To: edk2-devel@lists.01.org
> > > Cc: Shi, Steven ; Zhu, Yonghong
> > ; Gao, Liming 
> > > Subject: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> GenFw
> > >
> > > Adds support for the following X64 ELF relocations to GenFw
> > >   R_X86_64_GOTPCREL
> > >   R_X86_64_GOTPCRELX
> > >   R_X86_64_REX_GOTPCRELX
> > >
> > > CC: Shi Steven 
> > > CC: Yonghong Zhu 
> > > CC: Liming Gao 
> > > Contributed-under: TianoCore Contribution Agreement 1.1
> > > Signed-off-by: Zenith432 
> > > ---
> > >  BaseTools/Source/C/GenFw/Elf64Convert.c | 166
> > +++-
> > >  BaseTools/Source/C/GenFw/elf_common.h   |  23 
> > >  2 files changed, 188 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c
> > b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > > index c39bdff0..d2f9bb46 100644
> > > --- a/BaseTools/Source/C/GenFw/Elf64Convert.c
> > > +++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > > @@ -94,6 +94,15 @@ STATIC Elf_Ehdr *mEhdr;
> > >  STATIC Elf_Shdr *mShdrBase;
> > >  STATIC Elf_Phdr *mPhdrBase;
> > >
> > > +//
> > > +// GOT information
> > > +//
> > > +STATIC Elf_Shdr *mGOTShdr = NULL;
> > > +STATIC UINT32   mGOTShindex = 0;
> > > +STATIC UINT32   *mGOTCoffEntries = NULL;
> > > +STATIC UINT32   mGOTMaxCoffEntries = 0;
> > > +STATIC UINT32   mGOTNumCoffEntries = 0;
> > > +
> > >  //
> > >  // Coff information
> > >  //
> > > @@ -322,6 +331,117 @@ GetSymName (
> > >return StrtabContents + Sym->st_name;
> > >  }
> > >
> > > +//
> > > +// Find the ELF section hosting the GOT from an ELF Rva
> > > +//   of a single GOT entry.  Normally, GOT is placed in
> > > +//   ELF .text section, so assume once we find in which
> > > +//   section the GOT is, all GOT entries are there, and
> > > +//   just verify this.
> > > +//
> > > +STATIC
> > > +VOID
> > > +FindElfGOTSectionFromGOTEntryElfRva (
> > > +  Elf64_Addr GOTEntryElfRva
> > > +  )
> > > +{
> > > +  UINT32 i;
> > > +  if (mGOTShdr != NULL) {
> > > +if (GOTEntryElfRva >= mGOTShdr->sh_addr &&
> > > +GOTEntryElfRva <  mGOTShdr->sh_addr + mGOTShdr->sh_size)
> > > +  return;
> > > +Error (NULL, 0, 3000, "Unsupported",
> > "FindElfGOTSectionFromGOTEntryElfRva: GOT entries found in multiple
> > sections.")

Re: [edk2] [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to GenFw

[Shi, Steven]

Yes. If we disable the '#pragma GCC visibility push (hidden)' in 
ProcessorBind.h, the GOTPCREL support is mandatory. 3rd party module build 
might need it. It's more complete to add GOTPCREL support.

The hidden #pragma can hide all ELF symbols' visibility, including 'extern' 
functions, which is to let Linux compilers aware that we don’t need loading 
dynamic link library in Uefi runing, please not emit the GOTPCREL based 
relocations in the obj file, and we will ensure the linker can get all extern 
functions statically solved. The reason of not emitting GOTPCREL based 
relocations is because it is hard to convert the GOTPCREL based relocations in 
ELF binary to PE/COFF related relocations, and our current GenFw has not 
supported them. I'm glade if we can add the GOTPCREL support in GenFW to 
totally solve this problem.

Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Gao, Liming
> Sent: Thursday, June 7, 2018 9:32 AM
> To: Zenith432 ; edk2-devel@lists.01.org
> Cc: Shi, Steven ; Zhu, Yonghong
> 
> Subject: RE: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to
> GenFw
> 
> What's purpose to support GOTPCREL in GenFw? Could you introduce your
> usage model?
> 
> > -Original Message-
> > From: Zenith432 [mailto:zenith...@users.sourceforge.net]
> > Sent: Thursday, June 7, 2018 2:01 AM
> > To: edk2-devel@lists.01.org
> > Cc: Shi, Steven ; Zhu, Yonghong
> ; Gao, Liming 
> > Subject: [PATCH] BaseTools/GenFw: Add X64 GOTPCREL Support to GenFw
> >
> > Adds support for the following X64 ELF relocations to GenFw
> >   R_X86_64_GOTPCREL
> >   R_X86_64_GOTPCRELX
> >   R_X86_64_REX_GOTPCRELX
> >
> > CC: Shi Steven 
> > CC: Yonghong Zhu 
> > CC: Liming Gao 
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Zenith432 
> > ---
> >  BaseTools/Source/C/GenFw/Elf64Convert.c | 166
> +++-
> >  BaseTools/Source/C/GenFw/elf_common.h   |  23 
> >  2 files changed, 188 insertions(+), 1 deletion(-)
> >
> > diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c
> b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > index c39bdff0..d2f9bb46 100644
> > --- a/BaseTools/Source/C/GenFw/Elf64Convert.c
> > +++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > @@ -94,6 +94,15 @@ STATIC Elf_Ehdr *mEhdr;
> >  STATIC Elf_Shdr *mShdrBase;
> >  STATIC Elf_Phdr *mPhdrBase;
> >
> > +//
> > +// GOT information
> > +//
> > +STATIC Elf_Shdr *mGOTShdr = NULL;
> > +STATIC UINT32   mGOTShindex = 0;
> > +STATIC UINT32   *mGOTCoffEntries = NULL;
> > +STATIC UINT32   mGOTMaxCoffEntries = 0;
> > +STATIC UINT32   mGOTNumCoffEntries = 0;
> > +
> >  //
> >  // Coff information
> >  //
> > @@ -322,6 +331,117 @@ GetSymName (
> >return StrtabContents + Sym->st_name;
> >  }
> >
> > +//
> > +// Find the ELF section hosting the GOT from an ELF Rva
> > +//   of a single GOT entry.  Normally, GOT is placed in
> > +//   ELF .text section, so assume once we find in which
> > +//   section the GOT is, all GOT entries are there, and
> > +//   just verify this.
> > +//
> > +STATIC
> > +VOID
> > +FindElfGOTSectionFromGOTEntryElfRva (
> > +  Elf64_Addr GOTEntryElfRva
> > +  )
> > +{
> > +  UINT32 i;
> > +  if (mGOTShdr != NULL) {
> > +if (GOTEntryElfRva >= mGOTShdr->sh_addr &&
> > +GOTEntryElfRva <  mGOTShdr->sh_addr + mGOTShdr->sh_size)
> > +  return;
> > +Error (NULL, 0, 3000, "Unsupported",
> "FindElfGOTSectionFromGOTEntryElfRva: GOT entries found in multiple
> sections.");
> > +exit(EXIT_FAILURE);
> > +  }
> > +  for (i = 0; i < mEhdr->e_shnum; i++) {
> > +Elf_Shdr *shdr = GetShdrByIndex(i);
> > +if (GOTEntryElfRva >= shdr->sh_addr &&
> > +GOTEntryElfRva <  shdr->sh_addr + shdr->sh_size) {
> > +  mGOTShdr = shdr;
> > +  mGOTShindex = i;
> > +  return;
> > +}
> > +  }
> > +  Error (NULL, 0, 3000, "Invalid", "FindElfGOTSectionFromGOTEntryElfRva:
> ElfRva 0x%016LX for GOT entry not found in any section.",
> > GOTEntryElfRva);
> > +  exit(EXIT_FAILURE);
> > +}
> > +
> > +//
> > +// Stores locations of GOT entries in COFF image.
> > +//   Returns TRUE if GOT entry is new.
> > +//   Simple implementation as number of GOT
> > +//   entries is expected to be low.
> > +//
> > +
> > +STATIC
> > +BOOLEAN
> > +AccumulateC

Re: [edk2] [PATCH v2 0/8] RFC: ovmf: preliminary TPM2 support

[Shi, Steven]

It works in my side after specify the full path to swtpm tpmemu.sock in qemu 
command options. Thanks!


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Stefan Berger [mailto:stef...@linux.vnet.ibm.com]
> Sent: Friday, March 9, 2018 9:54 PM
> To: Shi, Steven <steven@intel.com>; Marc-André Lureau
> <marcandre.lur...@gmail.com>
> Cc: edk2-devel@lists.01.org; ler...@redhat.com; pjo...@redhat.com; Yao,
> Jiewen <jiewen@intel.com>; qemu-de...@nongnu.org;
> javi...@redhat.com
> Subject: Re: [edk2] [PATCH v2 0/8] RFC: ovmf: preliminary TPM2 support
> 
> On 03/08/2018 10:03 PM, Shi, Steven wrote:
> > Hi Marcandre,
> > Thanks for your command steps and I tried them, but my qemu failed to
> connect the socket tpmemu.sock. When I added the control channel to the
> TPM, the swtpm socket command stuck there and never exit. Not sure
> whether it was successful.
> > Below are the command steps running output in my side
> >
> >> Then you can run:
> >> mkdir tpmstatedir
> >> swtpm_setup.sh --tpm2 --tpm-state tpmstatedir
> > $ swtpm_setup.sh --tpm2 --tpm-state tpmstatedir
> > Starting vTPM manufacturing as jshi19:jshi19 @ 2018年03月09日 星期
> 五 10时28分39秒
> > TPM is listening on TCP port 47364.
> > Successfully authored TPM state.
> > Ending vTPM manufacturing @ 2018年03月09日 星期五 10时28分39
> 秒
> >
> >> Run the emulator:
> >> swtpm socket --tpmstate dir=tpmstatedir --ctrl
> type=unixio,path=tpmemu.sock  --tpm2
> > $ swtpm socket --tpmstate dir=tpmstatedir --ctrl
> type=unixio,path=tpmemu.sock --tpm2
> > (the swtpm socket command stuck there and never exit)
> >
> >> Run qemu (from git) with ovmf (with this series):
> >> qemu ... -chardev socket,id=chrtpm,path=tpmemu.sock -tpmdev
> >> emulator,id=tpm0,chardev=chrtpm  -device tpm-crb,tpmdev=tpm0
> >> -drive if=pflash,format=raw,file=OVMF_CODE.fd,readonly -drive
> >> if=pflash,format=raw,file=OVMF_VARS.fd ..
> > $ qemu-system-x86_64  -serial file:serial.log -m 5120 -hda fat:. -monitor
> stdio --enable-kvm -smp 4 -bios ../Ovmf3264/NOOPT_GCC5/FV/OVMF.fd -
> chardev socket,id=chrtpm,path=tpmemu.sock -tpmdev
> emulator,id=tpm0,chardev=chrtpm  -device tpm-crb,tpmdev=tpm0
> > qemu-system-x86_64: -chardev socket,id=chrtpm,path=tpmemu.sock:
> Failed to connect socket tpmemu.sock: No such file or directory
> 
> Try giving it both, swtpm and qemu, the full path to the socket.
> 
> 
> >
> > I use the latest version qemu as below:
> > $ qemu-system-x86_64 --version
> > QEMU emulator version 2.11.50 (v2.10.0-4184-g930b01138b-dirty)
> > Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers
> >
> > Thanks
> > Steven Shi
> >

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 0/8] RFC: ovmf: preliminary TPM2 support

[Shi, Steven]

Hi Marcandre,
Thanks for your command steps and I tried them, but my qemu failed to connect 
the socket tpmemu.sock. When I added the control channel to the TPM, the swtpm 
socket command stuck there and never exit. Not sure whether it was successful. 
Below are the command steps running output in my side

> Then you can run:
> mkdir tpmstatedir
> swtpm_setup.sh --tpm2 --tpm-state tpmstatedir
$ swtpm_setup.sh --tpm2 --tpm-state tpmstatedir
Starting vTPM manufacturing as jshi19:jshi19 @ 2018年03月09日 星期五 10时28分39秒
TPM is listening on TCP port 47364.
Successfully authored TPM state.
Ending vTPM manufacturing @ 2018年03月09日 星期五 10时28分39秒

> Run the emulator:
> swtpm socket --tpmstate dir=tpmstatedir --ctrl type=unixio,path=tpmemu.sock  
> --tpm2
$ swtpm socket --tpmstate dir=tpmstatedir --ctrl type=unixio,path=tpmemu.sock 
--tpm2
(the swtpm socket command stuck there and never exit)

> Run qemu (from git) with ovmf (with this series):
> qemu ... -chardev socket,id=chrtpm,path=tpmemu.sock -tpmdev
> emulator,id=tpm0,chardev=chrtpm  -device tpm-crb,tpmdev=tpm0
> -drive if=pflash,format=raw,file=OVMF_CODE.fd,readonly -drive
> if=pflash,format=raw,file=OVMF_VARS.fd ..
$ qemu-system-x86_64  -serial file:serial.log -m 5120 -hda fat:. -monitor stdio 
--enable-kvm -smp 4 -bios ../Ovmf3264/NOOPT_GCC5/FV/OVMF.fd -chardev 
socket,id=chrtpm,path=tpmemu.sock -tpmdev emulator,id=tpm0,chardev=chrtpm  
-device tpm-crb,tpmdev=tpm0
qemu-system-x86_64: -chardev socket,id=chrtpm,path=tpmemu.sock: Failed to 
connect socket tpmemu.sock: No such file or directory

I use the latest version qemu as below:
$ qemu-system-x86_64 --version
QEMU emulator version 2.11.50 (v2.10.0-4184-g930b01138b-dirty)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

Thanks
Steven Shi

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 0/8] RFC: ovmf: preliminary TPM2 support

[Shi, Steven]

Hi Marcandre,
> I test with qemu & swtpm/libtpms (tpm2 branches, swtpm_setup.sh --tpm2 
> --tpm-state tpmstatedir)
> $ swtpm socket --tpmstate tpmstatedir --ctrl type=unixio,path=tpmsock  --tpm2 
> &

Where is the swtpm_setup.sh? And could you tell how to build & install the 
swtpm?

Thanks
Steven Shi
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] MdeModulePkg/SmmCore: Fix hang due to already-freed memory deference

[Shi, Steven]

Hi Laszlo,
Check the DispatchHandle valid in internal handle set before using it to 
reference its Signature data is majorly to avoid use-after-free problem here, 
it also can defense if an input handle is invalid but has a valid signature 
occasionally or deliberately. 

> Generally, if client code violates an interface contract, then the called 
> function is not responsible for catching the error and preventing undefined 
> behavior. 
> For "quality of service", we can go to certain lengths nonetheless, but it 
> should hopefully not hurt valid client code.
If the called function is an interface function, I think it is necessary to 
validate the inputs before use them to reference other internal data. This way 
can make the service code more secure.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Laszlo Ersek
> Sent: Friday, February 2, 2018 12:12 AM
> To: Ni, Ruiyu ; edk2-devel@lists.01.org
> Cc: Yao, Jiewen ; Zeng, Star 
> Subject: Re: [edk2] [PATCH] MdeModulePkg/SmmCore: Fix hang due to
> already-freed memory deference
> 
> Hello Ray,
> 
> On 02/01/18 11:15, Ruiyu Ni wrote:
> > SmiHandlerUnRegister() validates the DispatchHandle by checking
> > whether the first 32bit matches to a certain signature
> > (SMI_HANDLER_SIGNATURE).
> > But if a caller calls *UnRegister() twice and the memory freed by
> > first call still contains the signature, the second hang may hang.
> >
> > The patch fixes this issue by locating the DispatchHandle
> > in all SMI handlers, instead of checking the signature.
> >
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Ruiyu Ni 
> > Cc: Jiewen Yao 
> > Cc: Star Zeng 
> > ---
> >  MdeModulePkg/Core/PiSmmCore/Smi.c | 37
> -
> >  1 file changed, 32 insertions(+), 5 deletions(-)
> 
> I'm mildly curious: can we just zero out the signature when the
> de-registration / freeing happens? Otherwise, the nested loop added
> below will penalize (performance-wise) correctly written client code as
> well.
> 
> > diff --git a/MdeModulePkg/Core/PiSmmCore/Smi.c
> > b/MdeModulePkg/Core/PiSmmCore/Smi.c
> > index ad483a1877ce..6596ea9560d1 100644
> > --- a/MdeModulePkg/Core/PiSmmCore/Smi.c
> > +++ b/MdeModulePkg/Core/PiSmmCore/Smi.c
> > @@ -290,6 +290,7 @@ SmiHandlerUnRegister (
> >SmiEntry = SmiHandler->SmiEntry;
> >
> >RemoveEntryList (>Link);
> > +  SmiHandler->Signature = 0;
> >FreePool (SmiHandler);
> >
> >if (SmiEntry == NULL) {
> 
> Generally, if client code violates an interface contract, then the
> called function is not responsible for catching the error and preventing
> undefined behavior. For "quality of service", we can go to certain
> lengths nonetheless, but it should hopefully not hurt valid client code.
> 
> For example, I seem to remember that the list data structure
> implementation checks the internal consistency (which can be costly)
> only if a PCD is set to a certain value. Is that right? Is it an option
> here? (If the above zeroing is not good for some reason.)
> 
> Anyway, I'm asking mainly for my own education.
> 
> Thanks!
> Laszlo
> 
> > diff --git a/MdeModulePkg/Core/PiSmmCore/Smi.c
> b/MdeModulePkg/Core/PiSmmCore/Smi.c
> > index ad483a1877..0c09e7fa10 100644
> > --- a/MdeModulePkg/Core/PiSmmCore/Smi.c
> > +++ b/MdeModulePkg/Core/PiSmmCore/Smi.c
> > @@ -1,7 +1,7 @@
> >  /** @file
> >SMI management.
> >
> > -  Copyright (c) 2009 - 2017, Intel Corporation. All rights reserved.
> > +  Copyright (c) 2009 - 2018, Intel Corporation. All rights reserved.
> >This program and the accompanying materials are licensed and made
> available
> >under the terms and conditions of the BSD License which accompanies
> this
> >distribution.  The full text of the license may be found at
> > @@ -276,14 +276,41 @@ SmiHandlerUnRegister (
> >  {
> >SMI_HANDLER  *SmiHandler;
> >SMI_ENTRY*SmiEntry;
> > +  LIST_ENTRY   *EntryLink;
> > +  LIST_ENTRY   *HandlerLink;
> >
> > -  SmiHandler = (SMI_HANDLER *) DispatchHandle;
> > -
> > -  if (SmiHandler == NULL) {
> > +  if (DispatchHandle == NULL) {
> >  return EFI_INVALID_PARAMETER;
> >}
> >
> > -  if (SmiHandler->Signature != SMI_HANDLER_SIGNATURE) {
> > +  //
> > +  // Look for it in root SMI handlers
> > +  //
> > +  SmiHandler = NULL;
> > +  for ( HandlerLink = GetFirstNode ()
> > +  ; !IsNull (, HandlerLink) &&
> (SmiHandler != DispatchHandle)
> > +  ; HandlerLink = GetNextNode (,
> HandlerLink)
> > +  ) {
> > +SmiHandler = CR (HandlerLink, SMI_HANDLER, Link,
> SMI_HANDLER_SIGNATURE);
> > +  }
> > +
> > +  //
> > +  // Look for it in non-root SMI handlers
> > +  //
> > +  for ( EntryLink = GetFirstNode ()
> > +  ; !IsNull (, EntryLink) && (SmiHandler !=

Re: [edk2] [PATCH v2 2/3] BaseTools/tools_def CLANG38: add -Wno-unused-const-variable

[Shi, Steven]

Thank you!

Reviewed-by: Shi Steven <steven@intel.com>


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Thursday, December 7, 2017 5:29 PM
> To: edk2-devel@lists.01.org
> Cc: leif.lindh...@linaro.org; Gao, Liming <liming@intel.com>; Zhu,
> Yonghong <yonghong@intel.com>; Shi, Steven <steven@intel.com>;
> evan.ll...@arm.com; ler...@redhat.com; Ard Biesheuvel
> <ard.biesheu...@linaro.org>
> Subject: [PATCH v2 2/3] BaseTools/tools_def CLANG38: add -Wno-unused-
> const-variable
> 
> Commit 8b6366f87584 ("BaseTools/GCC: set -Wno-unused-const-variable
> on RELEASE builds") suppresses warnings about unused constant
> variables in RELEASE builds when building with GCC, given that they
> break the build under our warnings-as-errors policy.
> 
> Do the same for CLANG38.
> 
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=790
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Ard Biesheuvel <ard.biesheu...@linaro.org>
> ---
> v2: new patch
> 
>  BaseTools/Conf/tools_def.template | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/BaseTools/Conf/tools_def.template
> b/BaseTools/Conf/tools_def.template
> index c0189d4c8d30..703884fc49a7 100755
> --- a/BaseTools/Conf/tools_def.template
> +++ b/BaseTools/Conf/tools_def.template
> @@ -5664,7 +5664,7 @@ DEFINE CLANG38_X64_PREFIX   =
> ENV(CLANG38_BIN)
>  DEFINE CLANG38_IA32_TARGET  = -target i686-pc-linux-gnu
>  DEFINE CLANG38_X64_TARGET   = -target x86_64-pc-linux-gnu
> 
> -DEFINE CLANG38_WARNING_OVERRIDES= -Wno-parentheses-equality -
> Wno-tautological-compare -Wno-tautological-constant-out-of-range-
> compare -Wno-empty-body  -Wno-varargs
> +DEFINE CLANG38_WARNING_OVERRIDES= -Wno-parentheses-equality -
> Wno-tautological-compare -Wno-tautological-constant-out-of-range-
> compare -Wno-empty-body -Wno-unused-const-variable -Wno-varargs
>  DEFINE CLANG38_ALL_CC_FLAGS = DEF(GCC44_ALL_CC_FLAGS)
> DEF(CLANG38_WARNING_OVERRIDES) -fno-stack-protector -mms-bitfields -
> Wno-address -Wno-shift-negative-value -Wno-unknown-pragmas -Wno-
> incompatible-library-redeclaration -fno-asynchronous-unwind-tables -mno-
> sse -mno-mmx -msoft-float -mno-implicit-float  -ftrap-
> function=undefined_behavior_has_been_optimized_away_by_clang -
> funsigned-char -fno-ms-extensions -Wno-null-dereference -Wno-unknown-
> warning-option
> 
>  ###
> --
> 2.11.0

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] BaseTools/tools_def: add CLANG38 LTO versions for AARCH64 and ARM

[Shi, Steven]

Hi Ard,
I'm OK for the patch.
BTW, would you mind to help add the "-Wno-unused-const-variable" in the 
CLANG38_WARNING_OVERRIDES? I happen to need add it in CLANG38 
(https://bugzilla.tianocore.org/show_bug.cgi?id=790) and I see it seems a 
shared flag for ARM/AARCH64 as well.

Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Wednesday, December 6, 2017 7:39 PM
> To: edk2-devel@lists.01.org
> Cc: leif.lindh...@linaro.org; Gao, Liming <liming@intel.com>; Zhu,
> Yonghong <yonghong@intel.com>; Shi, Steven <steven@intel.com>;
> evan.ll...@arm.com; ler...@redhat.com; Ard Biesheuvel
> <ard.biesheu...@linaro.org>
> Subject: [PATCH] BaseTools/tools_def: add CLANG38 LTO versions for
> AARCH64 and ARM
> 
> Extend the CLANG38 toolchain definition so it can be used for
> ARM and AARCH64 as well. Note that this requires llvm-ar and
> the LLVMgold.so linker plugin.
> 
> In preparation of doing the same for GCC5, this toolchain version
> also departs from the custom of using -O0 for DEBUG builds, which
> makes them needlessly slow. Instead, let's add a NOOPT flavor as
> well, and enable optimization for DEBUG like the other architectures
> do. (Note that this will require some trivial changes to the platform
> description files)
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Ard Biesheuvel <ard.biesheu...@linaro.org>
> ---
>  BaseTools/Conf/tools_def.template | 98 +++-
>  1 file changed, 95 insertions(+), 3 deletions(-)
> 
> diff --git a/BaseTools/Conf/tools_def.template
> b/BaseTools/Conf/tools_def.template
> index 91b135c2e569..6ee720d7586e 100755
> --- a/BaseTools/Conf/tools_def.template
> +++ b/BaseTools/Conf/tools_def.template
> @@ -399,8 +399,8 @@ DEFINE DTC_BIN = ENV(DTC_PREFIX)dtc
>  #   Intel(r) ACPI Compiler from
>  #   https://acpica.org/downloads
>  #   CLANG38  -Linux-  Requires:
> -# Clang v3.8, LLVMgold plugin and GNU binutils 
> 2.26
> targeting x86_64-linux-gnu
> -# Clang v3.9 or later, LLVMgold plugin and GNU 
> binutils 2.28
> targeting x86_64-linux-gnu
> +# Clang v3.8, LLVMgold plugin and GNU binutils 
> 2.26
> targeting x86_64-linux-gnu, aarch64-linux-gnu or arm-linux-gnueabi
> +# Clang v3.9 or later, LLVMgold plugin and GNU 
> binutils 2.28
> targeting x86_64-linux-gnu, aarch64-linux-gnu or arm-linux-gnueabi
>  #Optional:
>  # Required to build platforms or ACPI tables:
>  #   Intel(r) ACPI Compiler from
> @@ -5652,6 +5652,7 @@ RELEASE_CLANG35_AARCH64_CC_FLAGS =
> DEF(CLANG35_AARCH64_CC_FLAGS) $(ARCHCC_FLAGS)
>  *_CLANG38_*_MAKE_PATH   = make
>  *_CLANG38_*_*_DLL   = ENV(CLANG38_DLL)
>  *_CLANG38_*_ASL_PATH= DEF(UNIX_IASL_BIN)
> +*_CLANG38_*_DTC_PATH= DEF(DTC_BIN)
> 
>  *_CLANG38_*_APP_FLAGS   =
>  *_CLANG38_*_ASL_FLAGS   = DEF(IASL_FLAGS)
> @@ -5663,7 +5664,8 @@ DEFINE CLANG38_X64_PREFIX   =
> ENV(CLANG38_BIN)
>  DEFINE CLANG38_IA32_TARGET  = -target i686-pc-linux-gnu
>  DEFINE CLANG38_X64_TARGET   = -target x86_64-pc-linux-gnu
> 
> -DEFINE CLANG38_ALL_CC_FLAGS = DEF(GCC44_ALL_CC_FLAGS) -Wno-
> empty-body -fno-stack-protector -mms-bitfields -Wno-address -Wno-shift-
> negative-value -Wno-parentheses-equality -Wno-unknown-pragmas -Wno-
> tautological-constant-out-of-range-compare -Wno-incompatible-library-
> redeclaration -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -
> msoft-float -mno-implicit-float  -ftrap-
> function=undefined_behavior_has_been_optimized_away_by_clang -
> funsigned-char -fno-ms-extensions -Wno-null-dereference -Wno-
> tautological-compare -Wno-unknown-warning-option -Wno-varargs
> +DEFINE CLANG38_WARNING_OVERRIDES= -Wno-parentheses-equality -
> Wno-tautological-compare -Wno-tautological-constant-out-of-range-
> compare -Wno-empty-body  -Wno-varargs
> +DEFINE CLANG38_ALL_CC_FLAGS = DEF(GCC44_ALL_CC_FLAGS)
> DEF(CLANG38_WARNING_OVERRIDES) -fno-stack-protector -mms-bitfields -
> Wno-address -Wno-shift-negative-value -Wno-unknown-pragmas -Wno-
> incompatible-library-redeclaration -fno-asynchronous-unwind-tables -mno-
> sse -mno-mmx -msoft-float -mno-implicit-float  -ftrap-
> function=undefined_behavior_has_been_optimized_away_by_clang -
> funsigned-char -fno-ms-extensions -Wno-null-dereference -Wno-unknown-
> warning-o

Re: [edk2] [PATCH] PcAtChipsetPkg/IsaAcpiDxe: Restore PCI attributes correctly

[Shi, Steven]

This patch can fix the Bug 405 issue 
(https://bugzilla.tianocore.org/show_bug.cgi?id=405). Thank you!



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ni, Ruiyu
> Sent: Tuesday, November 7, 2017 1:36 PM
> To: Laszlo Ersek <ler...@redhat.com>; Shi, Steven <steven@intel.com>
> Cc: Zeng, Star <star.z...@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [PATCH] PcAtChipsetPkg/IsaAcpiDxe: Restore PCI attributes
> correctly
> 
> Laszlo,
> Sure I will add the Bugzilla url in the commit message.
> 
> Steven,
> Could you please check whether this patch can fix your "reconnect -r" hang?
> 
> Thanks/Ray
> 
> > -Original Message-
> > From: Laszlo Ersek [mailto:ler...@redhat.com]
> > Sent: Tuesday, November 7, 2017 7:23 AM
> > To: Ni, Ruiyu <ruiyu...@intel.com>; edk2-devel@lists.01.org
> > Cc: Zeng, Star <star.z...@intel.com>; Shi, Steven <steven@intel.com>
> > Subject: Re: [PATCH] PcAtChipsetPkg/IsaAcpiDxe: Restore PCI attributes
> > correctly
> >
> > Hi Ray,
> >
> > On 11/03/17 09:28, Ruiyu Ni wrote:
> > > The original code enables some BITs in PCI attributes in Start(), but
> > > wrongly to disable these BITs in Stop().
> > >
> > > The correct behavior is to save the original PCI attributes before
> > > enables some BITs in Start(), and restore to original value in Stop().
> > >
> > > Contributed-under: TianoCore Contribution Agreement 1.1
> > > Signed-off-by: Ruiyu Ni <ruiyu...@intel.com>
> > > Cc: Star Zeng <star.z...@intel.com>
> > > Cc: Laszlo Ersek <ler...@redhat.com>
> > > ---
> > >  PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c | 44
> > > +
> > > PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.h |  3 ++-
> > >  2 files changed, 25 insertions(+), 22 deletions(-)
> >
> > Is this for <https://bugzilla.tianocore.org/show_bug.cgi?id=405>?
> >
> > If so, can you please add the reference to the commit message?
> >
> > Also, I think we should ask Steven to test the patch. (CC'd.)
> >
> > I'll try to comment more later.
> >
> > Thanks
> > Laszlo
> >
> >
> > >
> > > diff --git a/PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
> > > b/PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
> > > index 32381b112d..60d2fb5a5b 100644
> > > --- a/PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
> > > +++ b/PcAtChipsetPkg/IsaAcpiDxe/PcatIsaAcpi.c
> > > @@ -172,6 +172,7 @@ PcatIsaAcpiDriverBindingStart (
> > >EFI_PCI_IO_PROTOCOL  *PciIo;
> > >PCAT_ISA_ACPI_DEV*PcatIsaAcpiDev;
> > >UINT64   Supports;
> > > +  UINT64   OriginalAttributes;
> > >BOOLEAN  Enabled;
> > >
> > >Enabled = FALSE;
> > > @@ -210,9 +211,18 @@ PcatIsaAcpiDriverBindingStart (
> > >if (Supports == 0 || Supports == (EFI_PCI_IO_ATTRIBUTE_ISA_IO |
> > EFI_PCI_IO_ATTRIBUTE_ISA_IO_16)) {
> > >  Status = EFI_UNSUPPORTED;
> > >  goto Done;
> > > -  }
> > > +  }
> > > +
> > > +  Status = PciIo->Attributes (
> > > +PciIo,
> > > +EfiPciIoAttributeOperationGet,
> > > +0,
> > > +
> > > +);
> > > +  if (EFI_ERROR (Status)) {
> > > +goto Done;
> > > +  }
> > >
> > > -  Enabled = TRUE;
> > >Status = PciIo->Attributes (
> > >  PciIo,
> > >  EfiPciIoAttributeOperationEnable, @@ -222,7
> > > +232,8 @@ PcatIsaAcpiDriverBindingStart (
> > >if (EFI_ERROR (Status)) {
> > >  goto Done;
> > >}
> > > -
> > > +
> > > +  Enabled = TRUE;
> > >//
> > >// Allocate memory for the PCAT ISA ACPI Device structure
> > >//
> > > @@ -239,9 +250,10 @@ PcatIsaAcpiDriverBindingStart (
> > >//
> > >// Initialize the PCAT ISA ACPI Device structure
> > >//
> > > -  PcatIsaAcpiDev->Signature = PCAT_ISA_ACPI_DEV_SIGNATURE;
> > > -  PcatIsaAcpiDev->Handle= Controller;
> > > -  PcatIsaAcpiDev->PciIo = PciIo;
> > > +  PcatIsaAcpiDev->Signature = PCAT_ISA_ACPI_DEV_SIGNATURE;
> > > +  PcatIsaAcpiDev->Handle= Controller;
> > > +  PcatIsaAcpiDev-&

Re: [edk2] [PATCH] Maintainers.txt: add Laszlo Ersek as UefiCpuPkg reviewer

[Shi, Steven]

Hi Laszlo,
I happen hope to know how to well test the IOMMU/VT-D feature on OVMF. Can you 
give me some hints?


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Laszlo Ersek
> Sent: Tuesday, October 24, 2017 6:53 PM
> To: edk2-devel-01 
> Cc: Ni, Ruiyu ; Dong, Eric ;
> Andrew Fish ; Leif Lindholm ;
> Kinney, Michael D 
> Subject: [edk2] [PATCH] Maintainers.txt: add Laszlo Ersek as UefiCpuPkg
> reviewer
> 
> UefiCpuPkg hosts a number of modules (SMM infrastructure,
> MP-initialization) that may behave differently enough between
> virtualization guests and physical boards that they merit regression
> testing and review specifically from a virtualization perspective.
> 
> Add Laszlo as a UefiCpuPkg reviewer (not maintainer) so that he be CC'd on
> all UefiCpuPkg patches.
> 
> > R: Package Reviewer: Cc address for patches and questions. Reviewers
> >help maintainers review code, but don't have push access.
> 
> Cc: Andrew Fish 
> Cc: Eric Dong 
> Cc: Jeff Fan 
> Cc: Leif Lindholm 
> Cc: Michael D Kinney 
> Cc: Ruiyu Ni 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Laszlo Ersek 
> ---
> 
> Notes:
> Recently there have been two UefiCpuPkg patches on the list that have
> each spent hardly more than 24 hours in review:
> 
> * d76c2da8d50d ("MdeModulePkg/PiSmmCore: Install Protocol when S3
> resume
>   finished.", 2017-10-11),
> 
> * 0594ec417c89 ("UefiCpuPkg/MpInitLib: Enhance waiting for AP
>   initialization logic.", 2017-10-23).
> 
> Both times, I was out of office, and didn't get a chance to comment. The
> first commit broke OVMF:
> 
>   http://mid.mail-archive.com/8d678707-fb07-0326-acfd-
> 878bb7e8c...@redhat.com
>   https://github.com/tianocore/edk2/commit/152e8d7670d5
> 
> while the second commit has a good chance to break OVMF on at least
> some
> virt hosts:
> 
>   https://lists.01.org/pipermail/edk2-devel/2017-October/016309.html
> 
> I was not CC'd either time. Had I been CC'd, the automated out-of-office
> reply that I always set up would have been received on the submitter's
> side, and I could have tested and/or commented on the patches in
> question, before they were pushed.
> 
> My intent as a UefiCpuPkg Reviewer is to be inserted in the patch review
> process for UefiCpuPkg. I should be explicitly CC'd; and my feedback
> should please be waited upon. I think I've proved myself reasonably
> responsive; I intend to follow up in a few days every time. My review
> should definitely not replace Eric's review, but I'd like to get an
> *official* chance to raise questions before a patch is pushed. Whenever
> I have nothing to add, I'll send an Acked-by. Thanks.
> 
>  Maintainers.txt | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/Maintainers.txt b/Maintainers.txt
> index c1a5723af9db..297a9bfd2b56 100644
> --- a/Maintainers.txt
> +++ b/Maintainers.txt
> @@ -245,6 +245,7 @@ M: Jaben Carsey 
>  UefiCpuPkg
>  W: https://github.com/tianocore/tianocore.github.io/wiki/UefiCpuPkg
>  M: Eric Dong 
> +R: Laszlo Ersek 
> 
>  UnixPkg
>  W: https://github.com/tianocore/tianocore.github.io/wiki/UnixPkg
> --
> 2.14.1.3.gb7cf6e02401b
> 
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] UefiCpuPkg/MtrrLib: Fix MtrrDebugPrintAllMtrrsWorker to avoid hang

[Shi, Steven]

Reviewed-by: Steven Shi <steven@intel.com>


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ni, Ruiyu
> Sent: Tuesday, October 17, 2017 9:47 AM
> To: edk2-devel@lists.01.org
> Cc: Shi, Steven <steven@intel.com>; Laszlo Ersek <ler...@redhat.com>
> Subject: [PATCH] UefiCpuPkg/MtrrLib: Fix MtrrDebugPrintAllMtrrsWorker to
> avoid hang
> 
> ARRAY_SIZE(Mtrrs->Variables.Mtrr) was used in
> MtrrDebugPrintAllMtrrsWorker() to parse the MTRR registers.
> Instead, the actual variable MTRR count should be used.
> Otherwise, the uninitialized random data in MtrrSetting may cause
> MtrrLibSetMemoryType() hang.
> 
> Steven Shi found this bug in QEMU when using Q35 chip.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Ruiyu Ni <ruiyu...@intel.com>
> Cc: Steven Shi <steven@intel.com>
> Cc: Laszlo Ersek <ler...@redhat.com>
> ---
>  UefiCpuPkg/Library/MtrrLib/MtrrLib.c | 9 ++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/UefiCpuPkg/Library/MtrrLib/MtrrLib.c
> b/UefiCpuPkg/Library/MtrrLib/MtrrLib.c
> index 2fd1d0153e..cb22558103 100644
> --- a/UefiCpuPkg/Library/MtrrLib/MtrrLib.c
> +++ b/UefiCpuPkg/Library/MtrrLib/MtrrLib.c
> @@ -2776,6 +2776,7 @@ MtrrDebugPrintAllMtrrsWorker (
>  UINTN RangeCount;
>  UINT64MtrrValidBitsMask;
>  UINT64MtrrValidAddressMask;
> +UINT32VariableMtrrCount;
>  MTRR_MEMORY_RANGE Ranges[
>ARRAY_SIZE (mMtrrLibFixedMtrrTable) * sizeof (UINT64) + 2 *
> ARRAY_SIZE (Mtrrs->Variables.Mtrr) + 1
>];
> @@ -2785,6 +2786,8 @@ MtrrDebugPrintAllMtrrsWorker (
>return;
>  }
> 
> +VariableMtrrCount = GetVariableMtrrCountWorker ();
> +
>  if (MtrrSetting != NULL) {
>Mtrrs = MtrrSetting;
>  } else {
> @@ -2802,7 +2805,7 @@ MtrrDebugPrintAllMtrrsWorker (
>DEBUG((DEBUG_CACHE, "Fixed MTRR[%02d]   : %016lx\n", Index, Mtrrs-
> >Fixed.Mtrr[Index]));
>  }
> 
> -for (Index = 0; Index < ARRAY_SIZE (Mtrrs->Variables.Mtrr); Index++) {
> +for (Index = 0; Index < VariableMtrrCount; Index++) {
>if (((MSR_IA32_MTRR_PHYSMASK_REGISTER *)
> >Variables.Mtrr[Index].Mask)->Bits.V == 0) {
>  //
>  // If mask is not valid, then do not display range
> @@ -2829,11 +2832,11 @@ MtrrDebugPrintAllMtrrsWorker (
>  RangeCount = 1;
> 
>  MtrrLibGetRawVariableRanges (
> -  >Variables, ARRAY_SIZE (Mtrrs->Variables.Mtrr),
> +  >Variables, VariableMtrrCount,
>MtrrValidBitsMask, MtrrValidAddressMask, RawVariableRanges
>);
>  MtrrLibApplyVariableMtrrs (
> -  RawVariableRanges, ARRAY_SIZE (RawVariableRanges),
> +  RawVariableRanges, VariableMtrrCount,
>Ranges, ARRAY_SIZE (Ranges), 
>);
> 
> --
> 2.12.2.windows.2

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] BaseTools/Conf: Support LLVM39 and LLVM40 in CLANG38 toolchain

[Shi, Steven]

Besides the GetBestLanguage() , there are several other lib functions have the 
same problem which are listed in attachment 
https://bugzilla.tianocore.org/attachment.cgi?id=97 of the Bug410. These 
functions can also cause the build fails with CLANG38 in my side with clang 
version 4.0 or clang 5.0.

I'm OK to enable the -Wvarargs again if we think these lib ABI updates are 
acceptable.

Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Marvin H?user
> Sent: Monday, September 25, 2017 8:30 PM
> To: edk2-devel@lists.01.org
> Cc: Laszlo Ersek <ler...@redhat.com>; Gao, Liming <liming@intel.com>
> Subject: Re: [edk2] [PATCH] BaseTools/Conf: Support LLVM39 and LLVM40 in
> CLANG38 toolchain
> 
> Hey Liming and Laszlo,
> 
> Thanks for your answers!
> 
> I rather thought of this (I didn't check whether Language1 should be CONST,
> but doesn't matter for now):
> 
> CHAR8 *
> EFIAPI
> GetBestLanguage (
>   IN CONST CHAR8  *SupportedLanguages,
>   IN BOOLEAN  Iso639Language,
>   IN CHAR8   *Language1,
>   ...
>   );
> 
> This would be compatible with all existing code because the VA list solely
> consists of CHAR8 pointers.
> The only difference would be that one cannot pass just NULL after
> Iso639Language, but this scenario makes no sense anyway.
> One would just use Langauge1 on the first iteration and, as part of a do-
> while-loop, assign the VA args via VA_ARG().
> 
> Best regards,
> Marvin.
> 
> > -Original Message-
> > From: Gao, Liming [mailto:liming@intel.com]
> > Sent: Monday, September 25, 2017 11:58 AM
> > To: Laszlo Ersek <ler...@redhat.com>; Marvin H?user
> > <marvin.haeu...@outlook.com>; edk2-devel@lists.01.org
> > Subject: RE: [edk2] [PATCH] BaseTools/Conf: Support LLVM39 and LLVM40
> in
> > CLANG38 toolchain
> >
> > Laszlo:
> >   This is a better way. The bug
> > https://bugzilla.tianocore.org/show_bug.cgi?id=410 only lists
> > GetBestLanguage() API. I am not sure whether they are similar cases in
> edk2.
> > If have, we had better fix them together.
> >
> > Thanks
> > Liming
> > >-Original Message-
> > >From: Laszlo Ersek [mailto:ler...@redhat.com]
> > >Sent: Monday, September 25, 2017 4:13 PM
> > >To: Gao, Liming <liming@intel.com>; Marvin Häuser
> > ><marvin.haeu...@outlook.com>; edk2-devel@lists.01.org
> > >Subject: Re: [edk2] [PATCH] BaseTools/Conf: Support LLVM39 and
> LLVM40
> > >in
> > >CLANG38 toolchain
> > >
> > >On 09/25/17 06:59, Gao, Liming wrote:
> > >> Marvin:
> > >>   My concern is that the fix is an incompatible change. It requires
> > >> to modify
> > >library API. In fact, this is an undefined behavior. But, current
> > >compiler (VS, GCC and Clang) makes it work.  So, I prefer to keep the
> > >code as-is, and disable this warning first. If you find any real issue,
> > >we can return back and figure out the solution.
> > >
> > >I think we can draw a parallel here to GetVariable() and GetVariable2().
> > >GetVariable() is now unavailable if
> > DISABLE_NEW_DEPRECATED_INTERFACES
> > >is defined.
> > >
> > >I think the right solution would be to
> > >- introduce GetBestLanguage2(),
> > >- migrate all the current call sites to GetBestLanguage2() -- I counted
> > >  18, and the updates should be easy --,
> > >- and then make GetVariable() conditional on
> > >  not-DISABLE_NEW_DEPRECATED_INTERFACES.
> > >
> > >
> > >CHAR8 *
> > >EFIAPI
> > >GetBestLanguage2 (
> > >  IN CONST CHAR8  *SupportedLanguages,
> > >  IN INTN Iso639Language,
> > >  ...
> > >  );
> > >
> > >I don't feel strongly about this question, I just think technically
> > >this would be best. CLANG warnings are valuable.
> > >
> > >Thanks
> > >Laszlo
> > >
> > >>> -Original Message-
> > >>> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf
> > >>> Of Marvin H?user
> > >>> Sent: Friday, September 22, 2017 11:53 PM
> > >>> To: edk2-devel@lists.01.org
> > >>> Cc: Gao, Liming <liming@intel.com>
> > >>> Subject: Re: [edk2] [PATCH] BaseTools/Conf: Support LLVM39 and
> > >>> LLVM40
> > >in
> > >>> CLANG38 toolchain
> 

Re: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups

[Shi, Steven]

Hi Paulo,
Your fix works for me. The "Null Pointer Use" issue disappears after apply your 
new assignment patch. Thanks!


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Paulo Alcantara [mailto:pca...@zytor.com]
> Sent: Monday, September 11, 2017 9:52 PM
> To: Shi, Steven <steven@intel.com>; Laszlo Ersek <ler...@redhat.com>;
> edk2-devel-01 <edk2-devel@lists.01.org>
> Cc: Ni, Ruiyu <ruiyu...@intel.com>; Dong, Eric <eric.d...@intel.com>; Zeng,
> Star <star.z...@intel.com>; Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: Re: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups
> 
> Hi Steven,
> 
> On 11/09/2017 10:07, Shi, Steven wrote:
> > Hi Laszlo,
> > Your code is good. But there is a "Null Pointer Use" issue, which is a
> undefined behavior in C spec, in
> edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c, line:696, column: 7 and
> line:707, column: 14.
> >
> > Line695:
> >FileIdentifierDesc = PrivFileData->File.FileIdentifierDesc;
> >if (IS_FID_DIRECTORY_FILE (FileIdentifierDesc)) {
> > ...
> >
> > The above FileIdentifierDesc can be NULL if you run a .efi in a Udf 
> > partition.
> And the later IS_FID_DIRECTORY_FILE (FileIdentifierDesc) is unsafe if
> FileIdentifierDesc == NULL. You can test it with below extra code which add
> debug output when FileIdentifierDesc == NULL, and check the log after load
> and run something in a Udf partition.
> >
> > +++ b/MdeModulePkg/Universal/Disk/UdfDxe/File.c
> > @@ -693,6 +693,11 @@ UdfSetPosition (
> > PrivFileData = PRIVATE_UDF_FILE_DATA_FROM_THIS (This);
> >
> > FileIdentifierDesc = PrivFileData->File.FileIdentifierDesc;
> > +  if (FileIdentifierDesc == NULL) {
> > +DEBUG ((EFI_D_ERROR, "FileIdentifierDesc == NULL!\n"));
> > +return Status;
> > +  }
> > +
> > if (IS_FID_DIRECTORY_FILE (FileIdentifierDesc)) {
> >   //
> >   // If the file handle is a directory, the _only_ position that may be 
> > set is
> >
> > I followed the Paulo's suggestion, and found this issue when using the Udf
> partition of Windows 10 Enterprise ISO image on qemu. Below is my qemu
> command:
> > /usr/local/bin/qemu-system-x86_64 -m 5120 -enable-kvm  -machine pc-
> q35-2.9 -bios OVMF.fd -serial file:serial.log  -cdrom /media/jshi19/My\
> Passport/OSImage/ISO/winserver/en_windows_server_2016_x64_dvd_9327
> 751.iso
> >
> > I've submitted a bug for this issue:
> https://bugzilla.tianocore.org/show_bug.cgi?id=704.
> >
> > I found this issue by edk2 llvm undefined behavior sanitizer, and below is
> the sanitizer output. FYI.
> >
> /home/jshi19/wksp_efi/edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c,
> line:0x02B8, column:0x0007 ErrorType = NullPointerUse: member access
> within null pointer of type 'UDF_FILE_IDENTIFIER_DESCRIPTOR'
> > ASAN MEMORY ACCESS check fail! __ubsan_handle_type_mismatch_v1 is
> called:
> > UdfSetPosition
> >
> /home/jshi19/wksp_efi/edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c:
> 696:7
> > EfiShellFindFilesInDir
> >
> /home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:213
> 6:18
> > ShellSearchHandle
> >
> /home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:241
> 3:14
> > EfiShellFindFiles
> >
> /home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:258
> 6:18
> > EfiShellOpenFileList
> >
> /home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:266
> 6:12
> > ShellOpenFileMetaArg
> >
> /home/jshi19/wksp_efi/edk2/ShellPkg/Library/UefiShellLib/UefiShellLib.c:15
> 70:14
> >
> >
> /home/jshi19/wksp_efi/edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c,
> line:0x02C3, column:0x000E ErrorType = NullPointerUse: member access
> within null pointer of type 'UDF_FILE_IDENTIFIER_DESCRIPTOR'
> > ASAN MEMORY ACCESS check fail! __ubsan_handle_type_mismatch_v1 is
> called:
> > ...
> 
> Thank you for catching this bug up. I think this issue is related to
> accessing a File Identifier Descriptor from a root directory file --
> e.g., this should be "FileIdentifierDesc =
> PrivFileData->Root->FileIdentifierDesc;"
> 
> Could you please replace the broken assignment with:
> 
> FileIdentifierDesc = _FILE(PrivFileData)->FileIdentifierDesc;
> ASSERT (FileIdentifierDesc != NULL);
> ...
> 
> Tell me if that worked for you. If so, I could send a formal patch
> fixing this issue later.
> 
> (BTW, I do apologize the late replies but I'm only 

Re: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups

[Shi, Steven]

Hi Laszlo,
Your code is good. But there is a "Null Pointer Use" issue, which is a 
undefined behavior in C spec, in 
edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c, line:696, column: 7 and 
line:707, column: 14.

Line695:
  FileIdentifierDesc = PrivFileData->File.FileIdentifierDesc;
  if (IS_FID_DIRECTORY_FILE (FileIdentifierDesc)) {
   ...

The above FileIdentifierDesc can be NULL if you run a .efi in a Udf partition. 
And the later IS_FID_DIRECTORY_FILE (FileIdentifierDesc) is unsafe if 
FileIdentifierDesc == NULL. You can test it with below extra code which add 
debug output when FileIdentifierDesc == NULL, and check the log after load and 
run something in a Udf partition.

+++ b/MdeModulePkg/Universal/Disk/UdfDxe/File.c
@@ -693,6 +693,11 @@ UdfSetPosition (
   PrivFileData = PRIVATE_UDF_FILE_DATA_FROM_THIS (This);

   FileIdentifierDesc = PrivFileData->File.FileIdentifierDesc;
+  if (FileIdentifierDesc == NULL) {
+DEBUG ((EFI_D_ERROR, "FileIdentifierDesc == NULL!\n"));
+return Status;
+  }
+
   if (IS_FID_DIRECTORY_FILE (FileIdentifierDesc)) {
 //
 // If the file handle is a directory, the _only_ position that may be set 
is

I followed the Paulo's suggestion, and found this issue when using the Udf 
partition of Windows 10 Enterprise ISO image on qemu. Below is my qemu command:
/usr/local/bin/qemu-system-x86_64 -m 5120 -enable-kvm  -machine pc-q35-2.9 
-bios OVMF.fd -serial file:serial.log  -cdrom /media/jshi19/My\ 
Passport/OSImage/ISO/winserver/en_windows_server_2016_x64_dvd_9327751.iso

I've submitted a bug for this issue: 
https://bugzilla.tianocore.org/show_bug.cgi?id=704. 

I found this issue by edk2 llvm undefined behavior sanitizer, and below is the 
sanitizer output. FYI.
/home/jshi19/wksp_efi/edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c, 
line:0x02B8, column:0x0007 ErrorType = NullPointerUse: member access within 
null pointer of type 'UDF_FILE_IDENTIFIER_DESCRIPTOR'
ASAN MEMORY ACCESS check fail! __ubsan_handle_type_mismatch_v1 is called: 
UdfSetPosition
/home/jshi19/wksp_efi/edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c:696:7
EfiShellFindFilesInDir
/home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:2136:18
ShellSearchHandle
/home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:2413:14
EfiShellFindFiles
/home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:2586:18
EfiShellOpenFileList
/home/jshi19/wksp_efi/edk2/ShellPkg/Application/Shell/ShellProtocol.c:2666:12
ShellOpenFileMetaArg
/home/jshi19/wksp_efi/edk2/ShellPkg/Library/UefiShellLib/UefiShellLib.c:1570:14

/home/jshi19/wksp_efi/edk2/MdeModulePkg/Universal/Disk/UdfDxe/File.c, 
line:0x02C3, column:0x000E ErrorType = NullPointerUse: member access within 
null pointer of type 'UDF_FILE_IDENTIFIER_DESCRIPTOR'
ASAN MEMORY ACCESS check fail! __ubsan_handle_type_mismatch_v1 is called:
...

Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Paulo Alcantara [mailto:pca...@zytor.com]
> Sent: Sunday, September 10, 2017 11:52 PM
> To: Shi, Steven <steven@intel.com>; Laszlo Ersek <ler...@redhat.com>;
> edk2-devel-01 <edk2-devel@lists.01.org>
> Cc: Ni, Ruiyu <ruiyu...@intel.com>; Dong, Eric <eric.d...@intel.com>; Zeng,
> Star <star.z...@intel.com>; Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: Re: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups
> 
> 
> 
> On 10/09/2017 11:27, Shi, Steven wrote:
> > OK. Does the UDF image you created correctly show up as CD-ROM
> content in Linux, e.g Fedora?
> 
> The Fedora image I have (Fedora-Workstation-Live-x86_64-26-1.5.iso) does
> not contain a valid UDF file system, so you cannot use it for testing.
> 
> To make sure it really doesn't, I used a "Philips UDF Conformance Tool"
> that you can grab in [1], and the tool didn't find any valid UDF file
> system in it.
> 
> I've been using an unmodified Windows 10 Enterprise image that does
> contain a valid UDF bridge disk image (ISO9660+ElTorito+UDF) to perform
> my tests.
> 
> Additionally, I use an USB stick that I format it with 'sudo mkudffs -b
> 512 --media-type=hd /dev/sdX' and copy some files to it for testing
> 
> BTW, when testing with OVMF AARCH64, I was using my USB stick that
> showed up correctly as 'fsX' in UEFI shell, but with the Windows 10
> Enterprise ISO image, it didn't. I looked at the logs to see what was
> going on and I found out that the emulated CD-ROM drive is reporting a
> block size of 512 instead of 2048 -- which seems wrong to me. With
> 'qemu-system-x86_64 -cdrom', it reports a block size of 2048.
> 
> The Partition driver is still able to find a valid ElTorito partition
> because, regardless the device block size, it always use a logical block
> size of 2048 starting at 32K. In UDF

Re: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups

[Shi, Steven]

OK. Does the UDF image you created correctly show up as CD-ROM content in 
Linux, e.g Fedora?


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Laszlo Ersek [mailto:ler...@redhat.com]
> Sent: Sunday, September 10, 2017 9:52 PM
> To: Shi, Steven <steven@intel.com>; edk2-devel-01  de...@lists.01.org>
> Cc: Ni, Ruiyu <ruiyu...@intel.com>; Dong, Eric <eric.d...@intel.com>; Zeng,
> Star <star.z...@intel.com>; Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: Re: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups
> 
> Hi Steven,
> 
> On 09/10/17 10:38, Laszlo Ersek wrote:
> > On 09/10/17 06:24, Shi, Steven wrote:
> >> Hi Laszlo,
> >> How could we configure the Qemu and test the UDF driver on OVMF?
> >
> > I guess you would format e.g. a DVD image with UDF, and attach it to
> > QEMU like any other CD-ROM.
> 
> I tried to look into this -- I tried several things, but nothing
> produced an UDF image file that, when attached to the VM, would show up
> in the UEFI shell as FSn:
> 
> Google returned a bunch of pages, but all I found was:
> - tips that didn't work (see above),
> - confused users (like me) looking for solutions.
> 
> So, at the moment, I have no idea how authoring UDF DVD images is
> possible on Linux, so that they'd be recognized in edk2.
> 
> (I'm interested in the edk2 UDF driver not because I want to "author"
> UDF DVD images (ISO9660+ElTorito works just fine), but because some
> optical media images that were given to me are formatted UDF-only. They
> can be translated into ISO9660+ElTorito off-line, but that's a chore.)
> 
> Thanks,
> Laszlo
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups

[Shi, Steven]

Hi Laszlo,
How could we configure the Qemu and test the UDF driver on OVMF?  
BTW, how could we configure the Qemu to create a full feature scope machine to 
include all possible devices in it, e.g. USB, ISA, SD/MMC, network etc. 


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Laszlo Ersek
> Sent: Sunday, September 10, 2017 8:13 AM
> To: edk2-devel-01 
> Cc: Ni, Ruiyu ; Dong, Eric ; Zeng,
> Star ; Ard Biesheuvel 
> Subject: [edk2] [PATCH 0/5] MdeModulePkg: UDF fixes and cleanups
> 
> Repo:   https://github.com/lersek/edk2.git
> Branch: udf_fixes_cleanups
> 
> Patches #2, #3 and #4 are needed (and enough) for me to build OVMF for
> IA32 and X64 with clang-3.8, after the UDF introduction.
> 
> Patches #1 and #5 are cleanups that I felt fit before patch #2 and after
> patch #4, respectively.
> 
> Cc: Ard Biesheuvel 
> Cc: Eric Dong 
> Cc: Paulo Alcantara 
> Cc: Ruiyu Ni 
> Cc: Star Zeng 
> 
> Thanks
> Laszlo
> 
> Laszlo Ersek (5):
>   MdeModulePkg/UdfDxe: ASSERT() valid ReadFileInfo Flags for INLINE_DATA
> req
>   MdeModulePkg/UdfDxe: don't return unset Status if INLINE_DATA req
> succeeds
>   MdeModulePkg/UdfDxe: replace zero-init of local variables with
> ZeroMem()
>   MdeModulePkg/PartitionDxe: don't divide 64-bit values with C operators
>   MdeModulePkg/PartitionDxe: remove always false comparison
> 
>  MdeModulePkg/Universal/Disk/PartitionDxe/Udf.c| 9 +++--
>  MdeModulePkg/Universal/Disk/UdfDxe/File.c | 6 --
>  MdeModulePkg/Universal/Disk/UdfDxe/FileSystemOperations.c | 5 +
>  3 files changed, 16 insertions(+), 4 deletions(-)
> 
> --
> 2.14.1.3.gb7cf6e02401b
> 
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] "practical" memory allocation limit?

[Shi, Steven]

OK, got it. Thanks.



For the why the 64bits DXE usually prefer allocations below 4GB, there is a 
good white paper elaborate the memory service initialization flows and can 
explain the reason. Please see the page 23 in below white paper (I have mark 
the answer in red).



https://github.com/tianocore-docs/Docs/blob/master/White_Papers/A_Tour_Beyond_BIOS_Securiy_Enhancement_to_Mitigate_Buffer_Overflow_in_UEFI.pdf

Heap Management in EDK II
In UEFI, the DxeCore maintains the heap usage. The UEFI driver or application 
may call
AllocatePages/FreePages/AllocatePool/FreePool to allocate or free the resource, 
or call
GetMemoryMap() to review all of the memory usage.
[Heap Initialization]
When DxeIpl transfers control to the DxeCore, all of the resource information 
is reported in a
Hand-off-Block (HOB) [PI] list. The DxeCore constructs the heap based upon the 
HOB
information. See figure 4-2 Heap Initialization.
1) The DxeCore needs to find one region to serve as the initial memory in
CoreInitializeMemoryServices()
(https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Dxe/Gcd/G
cd.c). The function is responsible for priming the memory map so that memory 
allocations
and resource allocations can be made. If the memory region described by the 
PHIT HOB is
big enough to hold BIN and minimum initial memory, this memory region is used as
highest priority. It can make the memory BIN allocation to be at the same 
memory region
with PHIT that has better compatibility to avoid memory fragmentation. Usually 
the BIN
size is already considered by platform PEIM when the platform PEIM calls
InstallPeiMemory() to PEI core.








Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522





> -Original Message-

> From: Laszlo Ersek [mailto:ler...@redhat.com]

> Sent: Monday, August 28, 2017 10:40 PM

> To: Shi, Steven <steven@intel.com>; edk2-devel-01  de...@lists.01.org>

> Cc: Kinney, Michael D <michael.d.kin...@intel.com>; Gao, Liming

> <liming@intel.com>; Ard Biesheuvel <ard.biesheu...@linaro.org>

> Subject: Re: [edk2] "practical" memory allocation limit?

>

> On 08/28/17 16:25, Shi, Steven wrote:

> > Hi Laszlo,

> >

> > I happen to have a question about how to correctly get the system memory

> size on Qemu.

> >

> > In the QemuInitializeRam() of OvmfPkg\PlatformPei\MemDetect.c, I add

> debug info as below to trace the GetSystemMemorySizeBelow4gb() and

> GetSystemMemorySizeAbove4gb() output. But the output results seems not

> right if my input memory size > 4GB. See my commands and trace outputs in

> below.

> >

> >

> >

> > MemDetect.c  Line 602:

> >

> >   //

> >

> >   // Determine total memory size available

> >

> >   //

> >

> >   LowerMemorySize = GetSystemMemorySizeBelow4gb ();

> >

> >   UpperMemorySize = GetSystemMemorySizeAbove4gb ();

> >

> >   DEBUG ((EFI_D_INFO, "LowerMemorySize= 0x%x\n", LowerMemorySize));

> >

> >   DEBUG ((EFI_D_INFO, "UpperMemorySize= 0x%x\n", UpperMemorySize));

> >

> >

> >

> > My test commands and trace outputs:

> >

> > $ /opt/qemu/bin/qemu-system-x86_64 -m 5120 -enable-kvm -hda

> /home/jshi19/workspace/simics5-project/images/luv-

> v2.1_diskboot_gpt_x86_64_.img -machine pc-q35-2.9 -bios OVMF.fd -serial

> file:serial.log

> >

> > LowerMemorySize= 0x8000 //2GB, but should not it be 4GB?

> >

> > UpperMemorySize= 0xC000 //3GB, but should not it be 1GB?

>

> No, this is correct; the values returned are system memory *amounts*.

> (On the QEMU command line, the "-m" switch says how much system

> memory

> you have, not the highest address in system memory.)

>

> In the 4GB address space, you don't have *just* system memory, you have

> a whole bunch of MMIO too. (Most of that is used for 32-bit MMIO PCI BAR

> allocation, some is used for the pflash chip range, LAPICs, etc.) On Q35

> you also have 256MB assigned to the MMCONFIG / ECAM area (which

> provides

> direct MMIO access to PCI config space for PCI 256 buses).

>

> The physical memory map also depends on the board model; on i440fx, the

> 32-bit system memory can go up to 3GB, while on Q35, it only goes up to

> 2GB.

>

> So, if you pass 5GB to a Q35 machine, 2GB of that are placed in the

> address space at [0, 2G), and the other 3GB are placed in the address

> space at [4G, 7G).

>

> > $ /opt/qemu/bin/qemu-system-x86_64 -m 6144 -enable-kvm -hda

> /home/jshi19/workspace/simics5-project/images/luv-

> v2.1_diskboot_gpt_x86_64_.img -machine pc-q35-2.9 -bios OVMF.fd -serial

> file:serial.log

> >

> > LowerMemorySize= 0x8000

Re: [edk2] "practical" memory allocation limit?

[Shi, Steven]

Hi Laszlo,

I happen to have a question about how to correctly get the system memory size 
on Qemu.

In the QemuInitializeRam() of OvmfPkg\PlatformPei\MemDetect.c, I add debug info 
as below to trace the GetSystemMemorySizeBelow4gb() and 
GetSystemMemorySizeAbove4gb() output. But the output results seems not right if 
my input memory size > 4GB. See my commands and trace outputs in below.



MemDetect.c  Line 602:

  //

  // Determine total memory size available

  //

  LowerMemorySize = GetSystemMemorySizeBelow4gb ();

  UpperMemorySize = GetSystemMemorySizeAbove4gb ();

  DEBUG ((EFI_D_INFO, "LowerMemorySize= 0x%x\n", LowerMemorySize));

  DEBUG ((EFI_D_INFO, "UpperMemorySize= 0x%x\n", UpperMemorySize));



My test commands and trace outputs:

$ /opt/qemu/bin/qemu-system-x86_64 -m 5120 -enable-kvm -hda 
/home/jshi19/workspace/simics5-project/images/luv-v2.1_diskboot_gpt_x86_64_.img 
-machine pc-q35-2.9 -bios OVMF.fd -serial file:serial.log

LowerMemorySize= 0x8000 //2GB, but should not it be 4GB?

UpperMemorySize= 0xC000 //3GB, but should not it be 1GB?





$ /opt/qemu/bin/qemu-system-x86_64 -m 6144 -enable-kvm -hda 
/home/jshi19/workspace/simics5-project/images/luv-v2.1_diskboot_gpt_x86_64_.img 
-machine pc-q35-2.9 -bios OVMF.fd -serial file:serial.log

LowerMemorySize= 0x8000 //2GB, but should not it be 4GB?

UpperMemorySize= 0x0 // 0GB, but should not it be 2GB?









Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522



> -Original Message-

> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of

> Laszlo Ersek

> Sent: Monday, August 28, 2017 4:43 PM

> To: edk2-devel-01 

> Cc: Kinney, Michael D ; Gao, Liming

> ; Ard Biesheuvel 

> Subject: [edk2] "practical" memory allocation limit?

>

> Hi,

>

> I've been curious about it for a long time, so I guess I might as well

> ask the question:

>

> Even if OVMF's DXE phase is built for X64, and even if I give the VM

> memory above 4GB -- confirmed by the MEMMAP command in the UEFI shell

> --, in the OVMF log I see practically all allocations coming from the

> 32-bit address space.

>

> OVMF produces all its memory resource descriptor HOBs in PEI, all

> "tested". (No memory is added in DXE.) So why aren't DXE allocations

> served from 64-bit space?

>

> Something seems to be artificially limiting run-of-the-mill pool and

> page allocations in X64 OVMF; what is it? (It doesn't work like this in

> ArmVirtQemu.)

>

> Thanks!

> Laszlo

> ___

> edk2-devel mailing list

> edk2-devel@lists.01.org

> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH] BaseTools/Conf: Support LLVM39 and LLVM40 in CLANG38 toolchain

[Shi Steven]

From: "Shi, Steven" <steven@intel.com>

Add LLVM39 and LLVM40 support in CLANG38 toolchain

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/tools_def.template | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
index 1fa3ca3..2f83341 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -380,7 +380,8 @@ DEFINE SOURCERY_CYGWIN_TOOLS = /cygdrive/c/Program 
Files/CodeSourcery/Sourcery G
 #   Intel(r) ACPI Compiler from
 #   https://acpica.org/downloads
 #   CLANG38  -Linux-  Requires:
-# Clang v3.8 or later, LLVMgold plugin and GNU 
binutils 2.26 targeting x86_64-linux-gnu
+# Clang v3.8, LLVMgold plugin and GNU binutils 
2.26 targeting x86_64-linux-gnu
+# Clang v3.9 or later, LLVMgold plugin and GNU 
binutils 2.28 targeting x86_64-linux-gnu
 #Optional:
 # Required to build platforms or ACPI tables:
 #   Intel(r) ACPI Compiler from
@@ -5512,7 +5513,7 @@ DEFINE CLANG38_X64_PREFIX   = ENV(CLANG38_BIN)
 DEFINE CLANG38_IA32_TARGET  = -target i686-pc-linux-gnu
 DEFINE CLANG38_X64_TARGET   = -target x86_64-pc-linux-gnu
 
-DEFINE CLANG38_ALL_CC_FLAGS = DEF(GCC44_ALL_CC_FLAGS) -Wno-empty-body 
-fno-stack-protector -mms-bitfields -Wno-address -Wno-shift-negative-value 
-Wno-parentheses-equality -Wno-unknown-pragmas 
-Wno-tautological-constant-out-of-range-compare 
-Wno-incompatible-library-redeclaration -fno-asynchronous-unwind-tables 
-mno-sse -mno-mmx -msoft-float -mno-implicit-float  
-ftrap-function=undefined_behavior_has_been_optimized_away_by_clang 
-funsigned-char -fno-ms-extensions -Wno-null-dereference 
-Wno-tautological-compare -Wno-unknown-warning-option
+DEFINE CLANG38_ALL_CC_FLAGS = DEF(GCC44_ALL_CC_FLAGS) -Wno-empty-body 
-fno-stack-protector -mms-bitfields -Wno-address -Wno-shift-negative-value 
-Wno-parentheses-equality -Wno-unknown-pragmas 
-Wno-tautological-constant-out-of-range-compare 
-Wno-incompatible-library-redeclaration -fno-asynchronous-unwind-tables 
-mno-sse -mno-mmx -msoft-float -mno-implicit-float  
-ftrap-function=undefined_behavior_has_been_optimized_away_by_clang 
-funsigned-char -fno-ms-extensions -Wno-null-dereference 
-Wno-tautological-compare -Wno-unknown-warning-option -Wno-varargs
 
 ###
 # CLANG38 IA32 definitions
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 1/1] BaseTools/tools_def.template: revert to large code model for X64/GCC5/LTO

[Shi, Steven]

It is a link flag misuse in our GCC build toolchains, not compiler/linker’s 
problem. Below patch can fix the wrong assembly function relocation type in PIE 
binary. With below patch, all the GCC5, GCC49 and GCC48 can build correctly 
images of OvmfPkgIa32X64 and OvmfPkgX64 platforms without my previous simple 
work around in my side. Please try it in your side.

Since we are using GCC as linker command, we MUST pass -pie to ld with 
“-Wl,-pie”, not just “--pie”  or “-fpie”.

So, this means we never correctly build small+PIE 64bits code with GCC 
toolchains before (CLANG38 is correct). If you failed to enable PIE/LTO build 
before, it is worthy to revisit those failures with “-Wl,-pie”. FYI.






diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
index 1fa3ca3..9e46d65 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -4375,7 +4375,7 @@ DEFINE GCC44_IA32_X64_DLINK_COMMON   = -nostdlib 
-Wl,-n,-q,--gc-sections -z comm
DEFINE GCC44_IA32_X64_ASLDLINK_FLAGS = DEF(GCC44_IA32_X64_DLINK_COMMON) 
-Wl,--entry,ReferenceAcpiTable -u ReferenceAcpiTable
DEFINE GCC44_IA32_X64_DLINK_FLAGS= DEF(GCC44_IA32_X64_DLINK_COMMON) 
-Wl,--entry,$(IMAGE_ENTRY_POINT) -u $(IMAGE_ENTRY_POINT) 
-Wl,-Map,$(DEST_DIR_DEBUG)/$(BASE_NAME).map
DEFINE GCC44_IA32_DLINK2_FLAGS   = -Wl,--defsym=PECOFF_HEADER_SIZE=0x220 
DEF(GCC_DLINK2_FLAGS_COMMON)
-DEFINE GCC44_X64_DLINK_FLAGS = DEF(GCC44_IA32_X64_DLINK_FLAGS) 
-Wl,-melf_x86_64,--oformat=elf64-x86-64
+DEFINE GCC44_X64_DLINK_FLAGS = DEF(GCC44_IA32_X64_DLINK_FLAGS) 
-Wl,-melf_x86_64,--oformat=elf64-x86-64 -Wl,-pie
DEFINE GCC44_X64_DLINK2_FLAGS= -Wl,--defsym=PECOFF_HEADER_SIZE=0x228 
DEF(GCC_DLINK2_FLAGS_COMMON)
DEFINE GCC44_ASM_FLAGS   = DEF(GCC_ASM_FLAGS)

@@ -4455,7 +4455,7 @@ DEFINE GCC49_IA32_X64_DLINK_COMMON   = -nostdlib 
-Wl,-n,-q,--gc-sections -z comm
DEFINE GCC49_IA32_X64_ASLDLINK_FLAGS = DEF(GCC49_IA32_X64_DLINK_COMMON) 
-Wl,--entry,ReferenceAcpiTable -u ReferenceAcpiTable
DEFINE GCC49_IA32_X64_DLINK_FLAGS= DEF(GCC49_IA32_X64_DLINK_COMMON) 
-Wl,--entry,$(IMAGE_ENTRY_POINT) -u $(IMAGE_ENTRY_POINT) 
-Wl,-Map,$(DEST_DIR_DEBUG)/$(BASE_NAME).map
DEFINE GCC49_IA32_DLINK2_FLAGS   = DEF(GCC48_IA32_DLINK2_FLAGS)
-DEFINE GCC49_X64_DLINK_FLAGS = DEF(GCC49_IA32_X64_DLINK_FLAGS) 
-Wl,-melf_x86_64,--oformat=elf64-x86-64
+DEFINE GCC49_X64_DLINK_FLAGS = DEF(GCC49_IA32_X64_DLINK_FLAGS) 
-Wl,-melf_x86_64,--oformat=elf64-x86-64 -Wl,-pie
DEFINE GCC49_X64_DLINK2_FLAGS= DEF(GCC48_X64_DLINK2_FLAGS)
DEFINE GCC49_ASM_FLAGS   = DEF(GCC48_ASM_FLAGS)
DEFINE GCC49_ARM_ASM_FLAGS   = DEF(GCC48_ARM_ASM_FLAGS)





Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522



> -Original Message-

> From: Laszlo Ersek [mailto:ler...@redhat.com]

> Sent: Tuesday, August 15, 2017 11:46 PM

> To: Shi, Steven <steven@intel.com>

> Cc: edk2-devel-01 <edk2-devel@lists.01.org>; Alex Williamson

> <alex.william...@redhat.com>; Ard Biesheuvel

> <ard.biesheu...@linaro.org>; Justen, Jordan L <jordan.l.jus...@intel.com>;

> Gao, Liming <liming@intel.com>; Kinney, Michael D

> <michael.d.kin...@intel.com>

> Subject: Re: [edk2] [PATCH 1/1] BaseTools/tools_def.template: revert to

> large code model for X64/GCC5/LTO

>

> On 08/12/17 05:05, Shi, Steven wrote:

> > OK. I can reproduce the failure with -smp 4 and -m 5120 in my side.

> >

> > It looks a linker bug about assemble function support in PIC/PIE

> > code.  You know, if we only have C code, the compiler/linker will

> > generate all the machine code and guarantee all the address reference

> > are position independent under PIC/PIE build. But if we mix manually

> > written assemble code in the C code,  the linker cannot really control

> > the address reference in the assemble code, and  might got confused.

>

> This is an incorrect description of the situation.

>

> The address reference is *not* in assembly code. (It used to be in

> assembly code, but Mike changed that earlier, for XCODE5 compatibility.)

>

> At this moment the address reference is in *C code*. The C code takes

> the address of an external function, and assigns it to a field in a data

> structure. The assembly code calls the function through this field. The

> assembly code makes no reference to the called function by name. See

> Mike's commit 3b2928b46987:

>

> -movrax, ASM_PFX(InitializeFloatingPointUnits)

> +movrax, qword [esi + InitializeFloatingPointUnitsAddress]

>  subrsp, 20h

>  call   rax   ; Call assembly function to initialize FPU 
> per UEFI spec

>

> And, indeed, it is *not* the assembly code that's being miscompiled. It

> is the C-

Re: [edk2] [PATCH 1/1] BaseTools/tools_def.template: revert to large code model for X64/GCC5/LTO

[Shi, Steven]

OK. I can reproduce the failure with -smp 4 and -m 5120 in my side. 
It looks a linker bug about assemble function support in PIC/PIE code.  You 
know, if we only have C code, the compiler/linker will generate all the machine 
code and guarantee all the address reference are position independent under 
PIC/PIE build. But if we mix manually written assemble code in the C code,  the 
linker cannot really control the address reference in the assemble code, and  
might got confused. So, it is not seldom we could see the compiler/linker 
generate wrong code for mixed code,  especially with very high level 
optimization, e.g. LTO.

Globally change memory model from small to large will bring not trivial impact 
(+15%) to code size, espcial for the uncomperssed option rom dirver. Below is 
some data of OvmfPkgX64.dsc platform.
Dxecore.efi CpuDxe.efi  CpuMpPeiPeiCore.efi
Small+PIE:  139520  47360   30144   46720
Large:  165696  55360   34496   53504

A simpler workaround could be to add a C function wrapper around the assemble 
lib function as below. This simple workaround works in my side.  But it is 
necessary to find this issue's root cause and fix it in the compiler/linker. I 
will try to raise this issue to compiler/linker guys.

diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c 
b/UefiCpuPkg/Library/MpInitLib/MpLib.c
old mode 100644
new mode 100755
index a3eea29..7afe434
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
@@ -738,6 +738,15 @@ WaitApWakeup (
   }
 }

+VOID
+EFIAPI
+InitializeFloatingPointUnitsWrapper (
+  VOID
+  )
+{
+  InitializeFloatingPointUnits();
+}
+
 /**
   This function will fill the exchange info structure.

@@ -771,7 +780,7 @@ FillExchangeInfoData (

   ExchangeInfo->EnableExecuteDisable = IsBspExecuteDisableEnabled ();

-  ExchangeInfo->InitializeFloatingPointUnitsAddress = 
(UINTN)InitializeFloatingPointUnits;
+  ExchangeInfo->InitializeFloatingPointUnitsAddress = 
(UINTN)InitializeFloatingPointUnitsWrapper;

   //
   // Get the BSP's data of GDT and IDT


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Laszlo Ersek [mailto:ler...@redhat.com]
> Sent: Friday, August 11, 2017 7:18 PM
> To: Shi, Steven <steven@intel.com>; edk2-devel-01  de...@lists.01.org>
> Cc: Ard Biesheuvel <ard.biesheu...@linaro.org>; Justen, Jordan L
> <jordan.l.jus...@intel.com>; Gao, Liming <liming@intel.com>; Kinney,
> Michael D <michael.d.kin...@intel.com>
> Subject: Re: [edk2] [PATCH 1/1] BaseTools/tools_def.template: revert to large
> code model for X64/GCC5/LTO
> 
> Hi Steven,
> 
> On 08/11/17 07:28, Shi, Steven wrote:
> > Hi Laszlo,
> >
> > I'm trying to reproduce your boot failure with OVMF in my Ubuntu
> > system, but not succeed.  My GCC was built from GCC main trunk code
> > in 20170601, and my ld linker is version 2.28. Could you try the ld
> > 2.28 with your gcc-7.1 and check whether it works in your side?  Or,
> > do you know where can I download the gcc-7.1 pre-built binaries?
> This was reproduced on a stock Fedora 26 installation:
> 
> https://getfedora.org/
> 
> > jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ build -p
> OvmfPkg/OvmfPkgX64.dsc -t GCC5 -DSECURE_BOOT_ENABLE=TRUE -a X64 -b
> DEBUG  -DDEBUG_ON_SERIAL_PORT -n 5
> >
> > jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ build -p
> OvmfPkg/OvmfPkgIa32X64.dsc -t GCC5 -DSECURE_BOOT_ENABLE=TRUE -a IA32
> -a X64 -b DEBUG  -DDEBUG_ON_SERIAL_PORT -n 5
> 
> These build commands look good.
> 
> > jshi19@jshi19-
> desktop:~/wksp_efi/Laszlo/edk2/Build/OvmfX64/DEBUG_GCC5/FV$ qemu-
> system-x86_64  -bios OVMF.fd -serial file:serial.log -m 512 -hda fat:. 
> -monitor
> stdio --enable-kvm
> >
> > jshi19@jshi19-
> desktop:~/wksp_efi/Laszlo/edk2/Build/Ovmf3264/DEBUG_GCC5/FV$ qemu-
> system-x86_64  -bios OVMF.fd -serial file:serial.log -m 512 -hda fat:. 
> -monitor
> stdio --enable-kvm
> 
> The QEMU command lines are missing two options:
> 
> * First, you have to make sure that the VM has enough memory under 4GB
> so that the PEI RAM can grow above the 2GB limit. You are using the
> i440fx board type, so that's OK (its 32-bit memory can go up to 3GB),
> but the "-m" switch is too small. Instead, I recommend:
> 
>   -m 5120
> 
> This will give the VM 5GB of RAM, 3GB of which will be placed under the
> 4GB mark, and 2GB will be placed above.
> 
> * Second, there must be at least one AP (application processor) for
> CpuMpPei to boot up. So please add something like:
> 
>   -smp 4
> 
> Thanks
> Laszlo
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 1/1] BaseTools/tools_def.template: revert to large code model for X64/GCC5/LTO

[Shi, Steven]

Hi Laszlo,

I'm trying to reproduce your boot failure with OVMF in my Ubuntu system, but 
not succeed.  My GCC was built from GCC main trunk code in 20170601, and my ld 
linker is version 2.28. Could you try the ld 2.28 with your gcc-7.1 and check 
whether it works in your side?  Or, do you know where can I download the 
gcc-7.1 pre-built binaries?





jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ gcc --version

gcc (GCC) 8.0.0 20170601 (experimental)



jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ ld --version

GNU ld (GNU Binutils) 2.28



Below are my trying steps, and the Qemu can boot well into shell:

jshi19@jshi19-desktop:~/wksp_efi$ git clone https://github.com/lersek/edk2.git

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ git checkout gcc7_lto

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ git checkout 
7ef0dae092afcfb6fab7e8372c78097672168c4a

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ rm Build -r

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ rm Conf/tools_def.txt

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ rm Conf/target.txt

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ rm Conf/build_rule.txt

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ source edksetup.sh

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ make -C BaseTools/Source/C/ clean

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ make -C BaseTools/Source/C/
jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ build -p OvmfPkg/OvmfPkgX64.dsc 
-t GCC5 -DSECURE_BOOT_ENABLE=TRUE -a X64 -b DEBUG  -DDEBUG_ON_SERIAL_PORT -n 5

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2$ build -p 
OvmfPkg/OvmfPkgIa32X64.dsc -t GCC5 -DSECURE_BOOT_ENABLE=TRUE -a IA32 -a X64 -b 
DEBUG  -DDEBUG_ON_SERIAL_PORT -n 5

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2/Build/OvmfX64/DEBUG_GCC5/FV$ 
qemu-system-x86_64  -bios OVMF.fd -serial file:serial.log -m 512 -hda fat:. 
-monitor stdio --enable-kvm

jshi19@jshi19-desktop:~/wksp_efi/Laszlo/edk2/Build/Ovmf3264/DEBUG_GCC5/FV$ 
qemu-system-x86_64  -bios OVMF.fd -serial file:serial.log -m 512 -hda fat:. 
-monitor stdio --enable-kvm







Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522





> -Original Message-

> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of

> Laszlo Ersek

> Sent: Friday, August 11, 2017 8:34 AM

> To: edk2-devel-01 

> Cc: Ard Biesheuvel ; Justen, Jordan L

> ; Gao, Liming ; Kinney,

> Michael D 

> Subject: [edk2] [PATCH 1/1] BaseTools/tools_def.template: revert to large

> code model for X64/GCC5/LTO

>

> Several users have recently reported boot failures with OVMF. The symptoms

> are: blank screen and all VCPUs pegged at 100%. Alex Williamson has found

> the commit (via bisection) that exposes the issue. In this patch I'll

> attempt to analyze the symptoms and fix the root problem.

>

> (1) "UefiCpuPkg/Library/MpInitLib" is an IA32/X64 library instance that

> provides multiprocessing services. It is built into the CpuMpPei

> module (for the PEI phase) and the CpuDxe module (for the DXE and BDS

> phases). When either of these modules starts up during boot, it

> briefly boots up all of the CPUs, perfoms some counting /

> synchronization, and then all the APs (application processors) are

> quiesced until further work arrives.

>

> The APs boot in real mode, and need assembly language init code (a

> "reset vector") that gradually takes them into 64-bit mode, before

> they can call back into normal C code. The reset vector code is

> assembled at build time, but it is copied as data under 1MB (for real

> mode execution by the APs) during boot, whenever CpuMpPei or CpuDxe

> launches.

>

> Part of the reset vector code is FPU initialization. The reset vector

> in "UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm" calls the assembly

> language function InitializeFloatingPointUnits(), which is however

> provided by an external library. Normally, when the object files were

> linked together, this function call would result in an absolute

> reference (and matching relocation, performed at module loading), and

> the instance of the reset vector that was copied under 1MB during boot

> would refer to the same, unchanged, absolute address of

> InitializeFloatingPointUnits(), residing in CpuMpPei or CpuDxe.

>

> This didn't work with X64 XCODE5 builds however. XCODE5 prefers

> RIP-relative addressing for X64 (i.e., the assembly language function

> call depended on the distance between the call site and the callee).

> When the call site was copied under 1MB but

> InitializeFloatingPointUnits() would not move, the call broke.

>

> This was tracked in TianoCore BZ#565 and fixed in commit 3b2928b46987

> ("UefiCpuPkg/MpInitLib: Fix X64 XCODE5/NASM compatibility issues",

> 2017-05-17). The 

Re: [edk2] [PATCH] ShellPkg: Avoid buffer out-of-bound access

[Shi, Steven]

Reviewed-by: Steven Shi <steven@intel.com>

Thanks
Steven Shi


> -Original Message-
> From: Ni, Ruiyu
> Sent: Wednesday, July 26, 2017 4:22 PM
> To: edk2-devel@lists.01.org
> Cc: Shi, Steven <steven@intel.com>
> Subject: [PATCH] ShellPkg: Avoid buffer out-of-bound access
> 
> PathSize is the number of bytes in PathForReturn buffer so
> PathForReturn[PathSize - 1] incorrectly accesses the last
> character in the buffer,
> PathForReturn[PathSize / sizeof (CHAR16) - 1] should be used.
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Ruiyu Ni <ruiyu...@intel.com>
> Cc: Steven Shi <steven@intel.com>
> ---
>  ShellPkg/Application/Shell/ShellProtocol.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ShellPkg/Application/Shell/ShellProtocol.c
> b/ShellPkg/Application/Shell/ShellProtocol.c
> index b3b8acc0d0..991fb58ca7 100644
> --- a/ShellPkg/Application/Shell/ShellProtocol.c
> +++ b/ShellPkg/Application/Shell/ShellProtocol.c
> @@ -477,7 +477,7 @@ EfiShellGetFilePathFromDevicePath(
>  //  UEFI Shell spec section 3.7)
>  if ((PathSize != 0)&&
>  (PathForReturn != NULL)&&
> -(PathForReturn[PathSize - 1] != L'\\') &&
> +(PathForReturn[PathSize / sizeof (CHAR16) - 1] != L'\\') &&
>  (AlignedNode->PathName[0]!= L'\\')) {
>PathForReturn = StrnCatGrow (, , L"\\", 1);
>  }
> --
> 2.12.2.windows.2

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] MdeModulePkg: Fix use-after-free error in InstallConfigurationTable()

[Shi, Steven]

Reviewed-by: Steven Shi <steven@intel.com>


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Zeng, Star
> Sent: Monday, June 19, 2017 6:24 PM
> To: edk2-devel@lists.01.org
> Cc: Shi, Steven <steven@intel.com>; Yao, Jiewen <jiewen@intel.com>;
> Gao, Liming <liming@intel.com>; Zeng, Star <star.z...@intel.com>
> Subject: [PATCH] MdeModulePkg: Fix use-after-free error in
> InstallConfigurationTable()
> 
> From: "Shi, Steven" <steven@intel.com>
> 
> When installing configuration table and the original
> gDxeCoreST->ConfigurationTable[] buffer happen to be not big enough to
> add a new table, the CoreInstallConfigurationTable() enter the branch
> of line 113 in InstallConfigurationTable.c to free the old
> gDxeCoreST->ConfigurationTable[] buffer and allocate a new bigger one.
> The problem happens at line 139 CoreFreePool(), which is to free the
> old gDxeCoreST->ConfigurationTable[] buffer. The CoreFreePool()'s
> behavior is to free the buffer firstly, then call the
> InstallMemoryAttributesTableOnMemoryAllocation (PoolType) to update
> the EfiRuntimeServices type memory info, the
> CoreInstallConfigurationTable() will be re-entered by CoreFreePool()
> in its calling stack, then use-after-free read error will happen at
> line 59 of InstallConfigurationTable.c and use-after-free write error
> will happen at line 151 and 152 of InstallConfigurationTable.c.
> 
> The patch is to update System table to the new table pointer before
> calling CoreFreePool() to free the old table.
> 
> The case above is in DxeCore, but not in PiSmmCore.
> The change in PiSmmCore is to be consistent with DxeCore.
> 
> Cc: Jiewen Yao <jiewen@intel.com>
> Cc: Liming Gao <liming@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Steven Shi <steven@intel.com>
> Signed-off-by: Star Zeng <star.z...@intel.com>
> ---
>  .../Core/Dxe/Misc/InstallConfigurationTable.c  | 34 
> --
>  .../Core/PiSmmCore/InstallConfigurationTable.c | 34 
> --
>  2 files changed, 50 insertions(+), 18 deletions(-)
>  mode change 100644 => 100755
> MdeModulePkg/Core/Dxe/Misc/InstallConfigurationTable.c
> 
> diff --git a/MdeModulePkg/Core/Dxe/Misc/InstallConfigurationTable.c
> b/MdeModulePkg/Core/Dxe/Misc/InstallConfigurationTable.c
> old mode 100644
> new mode 100755
> index e4735db7ba45..dcdeb7f45803
> --- a/MdeModulePkg/Core/Dxe/Misc/InstallConfigurationTable.c
> +++ b/MdeModulePkg/Core/Dxe/Misc/InstallConfigurationTable.c
> @@ -1,7 +1,7 @@
>  /** @file
>UEFI Miscellaneous boot Services InstallConfigurationTable service
> 
> -Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved.
> +Copyright (c) 2006 - 2017, Intel Corporation. All rights reserved.
>  This program and the accompanying materials
>  are licensed and made available under the terms and conditions of the BSD
> License
>  which accompanies this distribution.  The full text of the license may be
> found at
> @@ -42,6 +42,7 @@ CoreInstallConfigurationTable (
>  {
>UINTN   Index;
>EFI_CONFIGURATION_TABLE *EfiConfigurationTable;
> +  EFI_CONFIGURATION_TABLE *OldTable;
> 
>//
>// If Guid is NULL, then this operation cannot be performed
> @@ -68,7 +69,7 @@ CoreInstallConfigurationTable (
>  if (Table != NULL) {
>//
>// If Table is not NULL, then this is a modify operation.
> -  // Modify the table enty and return.
> +  // Modify the table entry and return.
>//
>gDxeCoreST->ConfigurationTable[Index].VendorTable = Table;
> 
> @@ -134,15 +135,30 @@ CoreInstallConfigurationTable (
>);
> 
>  //
> -// Free Old Table
> +// Record the old table pointer.
>  //
> -CoreFreePool (gDxeCoreST->ConfigurationTable);
> -  }
> +OldTable = gDxeCoreST->ConfigurationTable;
> 
> -  //
> -  // Update System Table
> -  //
> -  gDxeCoreST->ConfigurationTable = EfiConfigurationTable;
> +//
> +// As the CoreInstallConfigurationTable() may be re-entered by
> CoreFreePool()
> +// in its calling stack, updating System table to the new table 
> pointer
> must
> +// be done before calling CoreFreePool() to free the old table.
> +// It can make sure the gDxeCoreST->ConfigurationTable point to the
> new table
> +// and avoid the errors of use-after-free to the old table by the 
> reenter
> of
> +// CoreInstallConfigurationTa

Re: [edk2] Testing SMM with QEMU, KVM and libvirt

[Shi, Steven]

Hi Laszlo,
I want to see the serial debug output and hope to save it to a local file. How 
could I update the ovmf.fedora.q35.template to define it?


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Laszlo Ersek [mailto:ler...@redhat.com]
> Sent: Tuesday, February 21, 2017 6:18 PM
> To: Shi, Steven <steven@intel.com>; edk2-devel-01  de...@ml01.01.org>
> Cc: Tian, Feng <feng.t...@intel.com>; Justen, Jordan L
> <jordan.l.jus...@intel.com>; Yao, Jiewen <jiewen@intel.com>; Kinney,
> Michael D <michael.d.kin...@intel.com>; Fan, Jeff <jeff@intel.com>; Zeng,
> Star <star.z...@intel.com>
> Subject: Re: [edk2] Testing SMM with QEMU, KVM and libvirt
> 
> On 02/21/17 10:22, Shi, Steven wrote:
> > Hi Laszlo,
> > I wonder if you could offer a Ubuntu version wiki for the
> > Testing-SMM-with-QEMU,-KVM-and-libvirt?
> 
> Sorry, I can't do that.
> 
> To give you a retrospective on the current article (which targets
> Fedora), writing and testing that article took ~14 hours, on an
> operating system that I'm both familiar with and can readily install in
> our internal server farm (called "Beaker").
> 
> My method was to start with a wiped-clean physical machine, install a
> fresh, clean Fedora system, and build it all up from there. I wanted to
> make sure that all missing dependencies would be thrown in my face, so I
> could explicitly document them for readers.
> 
> This approach paid off very well (it caught a whole lot of dependencies
> that I "thought" would be available by default, but weren't!), but it
> was also the *primary* time sink while writing the article.
> 
> "Porting" the article to another Linux distribution would mean an almost
> complete rewrite. The package names are different, the package contents
> are different, the package inter-dependencies are different, the
> virtualization tools may have different versions available, the generic
> tools may be different, and so on.
> 
> > I'm trying to port your
> > steps to my Ubuntu 16.04, but meet lots of troubles. The Ubuntu
> > apt-get virsh version is too old to support smm feature in your
> > ovmf.fedora.q35.template,
> 
> That's *exactly* my point.
> 
> > and I have to build the new version libvirt
> > by myself. I meet lots of failures when configure the new version
> > virsh, and wonder if you could help.
> 
> Sorry, I don't have time for that. I don't know Ubuntu at all, have no
> contact to Ubuntu developers, and cannot even auto-install Ubuntu easily
> on a headless server in our internal server pool.
> 
> Frankly, one goal of using Fedora 25 for the host operating system was
> *exactly* that the user could avoid this kind of struggle with the
> virtualization toolstack, and they could focus on rebuilding *only* what
> was unavoidable.
> 
> (It is bad enough that at the moment I must have instructions in there
> for building QEMU from source -- once QEMU 2.9 is released and Fedora 25
> picks it up, I think I will go ahead and replace that section of the
> article, with a simple package installation command. I'll also update
> references elsewhere, such as in the domain templates.)
> 
> So, unfortunately, what you are asking for is a complete rewrite of the
> article, for Ubuntu, which I don't know and have no access to, in the
> isolated server environment that is necessary for writing and testing
> such an article.
> 
> I'm not trying to "push" Fedora with this -- a fresh Ubuntu release
> should be entirely suitable for this I *guess*, but the devil is in the
> details, and you'll need an Ubuntu person, with a corporate(-like)
> Ubuntu environment, to write that article for you.
> 
> I do confirm that I intend to support the Fedora setup with high
> priority, so if you have questions about that, I'll do my best to answer.
> 
> Thanks,
> Laszlo
> 
> 
> >
> >
> >
> > Steven Shi
> > Intel\SSG\STO\UEFI Firmware
> >
> > Tel: +86 021-61166522
> > iNet: 821-6522
> >
> >
> >> -Original Message-
> >> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> >> Laszlo Ersek
> >> Sent: Tuesday, February 7, 2017 9:07 PM
> >> To: edk2-devel-01 <edk2-de...@ml01.01.org>
> >> Cc: Tian, Feng <feng.t...@intel.com>; Justen, Jordan L
> >> <jordan.l.jus...@intel.com>; Yao, Jiewen <jiewen@intel.com>; Kinney,
> >> Michael D <michael.d.kin...@intel.com>; Fan, Jeff <jeff@intel.com>;
> Zeng,
> >> Star <star.z...@intel.com>
> >&

Re: [edk2] Testing SMM with QEMU, KVM and libvirt

[Shi, Steven]

Hi Laszlo,
I wonder if you could offer a Ubuntu version wiki for the 
Testing-SMM-with-QEMU,-KVM-and-libvirt?  I'm trying to port your steps to my 
Ubuntu 16.04, but meet lots of troubles. The Ubuntu apt-get virsh version is 
too old to support smm feature in your ovmf.fedora.q35.template, and I have to 
build the new version libvirt by myself. I meet lots of failures when configure 
the new version virsh, and wonder if you could help. 



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Laszlo Ersek
> Sent: Tuesday, February 7, 2017 9:07 PM
> To: edk2-devel-01 
> Cc: Tian, Feng ; Justen, Jordan L
> ; Yao, Jiewen ; Kinney,
> Michael D ; Fan, Jeff ; Zeng,
> Star 
> Subject: [edk2] Testing SMM with QEMU, KVM and libvirt
> 
> Hi,
> 
> I've added the following article to the TianoCore wiki:
> 
> https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-
> QEMU,-KVM-and-libvirt
> 
> It should help both Windows and Linux desktop users build a KVM test
> machine / environment that closely resembles mine. Such an environment
> is useful for testing and regression-testing new MP and SMM features and
> bugfixes.
> 
> The initial setup is not short, but once you got it up and running, it's
> very simple to rebuild OVMF with the edk2 changes, install the firmware
> binary in the right place (see the article) and then click the Play
> button on the Fedora 25 and Windows 10 guests, to see the changes in action.
> 
> If you have smaller updates or structural reorgs for the document,
> there's no need to ask me, just go ahead and do them.
> 
> If some significant information is missing that you'd like me to add, I
> think I'd prefer new TianoCore BZs at this time (Product: Tianocore
> Feature Requests, Component: Web Content, Assignee: yours truly). I
> don't know when I'll have time again to dig into this.
> 
> Sorry if I forgot someone off the CC list.
> 
> Thanks!
> Laszlo
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] OVMF not booting when built with GCC5 toolset?

[Shi, Steven]

Hi Cran,
This issue is just fixed in GCC main trunk. I tried below latest GCC trunk 
version and it can pass build the edk2 with GCC5 toolset. FYI.

$ gcc --version
gcc (GCC) 7.0.0 20160828 (experimental)


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

From: Shi, Steven
Sent: Tuesday, August 23, 2016 6:49 PM
To: 'Bruce Cran' <br...@cran.org.uk>; 'Michael Zimmermann' 
<sigmaepsilo...@gmail.com>
Cc: 'edk2-devel-01' <edk2-devel@lists.01.org>; Lu, Hongjiu 
<hongjiu...@intel.com>
Subject: RE: [edk2] OVMF not booting when built with GCC5 toolset?

It's confirmed it is a GCC6 regression bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70955


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

From: Shi, Steven
Sent: Tuesday, August 23, 2016 4:10 PM
To: 'Bruce Cran' <br...@cran.org.uk<mailto:br...@cran.org.uk>>; 'Michael 
Zimmermann' <sigmaepsilo...@gmail.com<mailto:sigmaepsilo...@gmail.com>>
Cc: 'edk2-devel-01' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Subject: RE: [edk2] OVMF not booting when built with GCC5 toolset?


Hi Cran,

This issue might be a GCC6 LTO regression bug, and I've filed a bug to GCC 
community as below. Please don't use the GCC6 with edk2 GCC5 toolchain for now, 
the GCC5 (gcc-5) should works.



https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77341

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50818





GCC6.10 lto fails to support below MS ABI buildin of va list, but GCC5 works. 
This is a regression bug for GCC6. I suspect current GCC trunk also has this 
regression bug.

  __builtin_ms_va_list ap;

  __builtin_ms_va_start (ap, n);

  __builtin_ms_va_end (ap);



Below is my testcase:

$cat test2.c



#include 



int

__attribute__((ms_abi))

foo (int n, ...)

{

  __builtin_ms_va_list ap;

  int sum = 0;



  __builtin_ms_va_start (ap, n);



  while (n--) {

sum += __builtin_va_arg (ap, int);

printf("sum = %d\n", sum);

  }

  __builtin_ms_va_end (ap);



  return sum;

}



int main ()

{

  int res = foo (10, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10);



  if (res != 55)

__builtin_abort ();



  return 0;

}

$ gcc --version

gcc (GCC) 6.1.0



$ gcc -flto -Os test2.c

$ ./a.out

Segmentation fault (core dumped)



$ gcc-5 -flto -Os test2.c

$ ./a.out

sum = 1

sum = 3

sum = 6

sum = 10

sum = 15

sum = 21

sum = 28

sum = 36

sum = 45

sum = 55







Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522




___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] OVMF not booting when built with GCC5 toolset?

[Shi, Steven]

It's confirmed it is a GCC6 regression bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70955


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

From: Shi, Steven
Sent: Tuesday, August 23, 2016 4:10 PM
To: 'Bruce Cran' <br...@cran.org.uk>; 'Michael Zimmermann' 
<sigmaepsilo...@gmail.com>
Cc: 'edk2-devel-01' <edk2-devel@lists.01.org>
Subject: RE: [edk2] OVMF not booting when built with GCC5 toolset?


Hi Cran,

This issue might be a GCC6 LTO regression bug, and I've filed a bug to GCC 
community as below. Please don't use the GCC6 with edk2 GCC5 toolchain for now, 
the GCC5 (gcc-5) should works.



https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77341

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50818





GCC6.10 lto fails to support below MS ABI buildin of va list, but GCC5 works. 
This is a regression bug for GCC6. I suspect current GCC trunk also has this 
regression bug.

  __builtin_ms_va_list ap;

  __builtin_ms_va_start (ap, n);

  __builtin_ms_va_end (ap);



Below is my testcase:

$cat test2.c



#include 



int

__attribute__((ms_abi))

foo (int n, ...)

{

  __builtin_ms_va_list ap;

  int sum = 0;



  __builtin_ms_va_start (ap, n);



  while (n--) {

sum += __builtin_va_arg (ap, int);

printf("sum = %d\n", sum);

  }

  __builtin_ms_va_end (ap);



  return sum;

}



int main ()

{

  int res = foo (10, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10);



  if (res != 55)

__builtin_abort ();



  return 0;

}

$ gcc --version

gcc (GCC) 6.1.0



$ gcc -flto -Os test2.c

$ ./a.out

Segmentation fault (core dumped)



$ gcc-5 -flto -Os test2.c

$ ./a.out

sum = 1

sum = 3

sum = 6

sum = 10

sum = 15

sum = 21

sum = 28

sum = 36

sum = 45

sum = 55







Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522




___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] OVMF not booting when built with GCC5 toolset?

[Shi, Steven]

Hi Cran,

This issue might be a GCC6 LTO regression bug, and I've filed a bug to GCC 
community as below. Please don't use the GCC6 with edk2 GCC5 toolchain for now, 
the GCC5 (gcc-5) should works.



https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77341

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50818





GCC6.10 lto fails to support below MS ABI buildin of va list, but GCC5 works. 
This is a regression bug for GCC6. I suspect current GCC trunk also has this 
regression bug.

  __builtin_ms_va_list ap;

  __builtin_ms_va_start (ap, n);

  __builtin_ms_va_end (ap);



Below is my testcase:

$cat test2.c



#include 



int

__attribute__((ms_abi))

foo (int n, ...)

{

  __builtin_ms_va_list ap;

  int sum = 0;



  __builtin_ms_va_start (ap, n);



  while (n--) {

sum += __builtin_va_arg (ap, int);

printf("sum = %d\n", sum);

  }

  __builtin_ms_va_end (ap);



  return sum;

}



int main ()

{

  int res = foo (10, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10);



  if (res != 55)

__builtin_abort ();



  return 0;

}

$ gcc --version

gcc (GCC) 6.1.0



$ gcc -flto -Os test2.c

$ ./a.out

Segmentation fault (core dumped)



$ gcc-5 -flto -Os test2.c

$ ./a.out

sum = 1

sum = 3

sum = 6

sum = 10

sum = 15

sum = 21

sum = 28

sum = 36

sum = 45

sum = 55







Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522




___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] BaseTools/GenFw: ignore dynamic RELA sections

[Shi, Steven]

Hi Ard,
I like you path, and it makes the code look more clear. But in fact, old code 
should already ignore the dynamic RELA sections implicitly. 

Like below code, when the GenFw use .rela section info value, which is 0, to 
get .rela target applying section address in SecShdr, it will point to NULL 
section [0]. And the next section type checking, which is (*Filter)(SecShdr), 
will fails and then stop the .rela section relocation process.

Line 685 of BaseTools\Source\C\GenFw\Elf64Convert.c 
//
// Relocation section found.  Now extract section information that the 
relocations
// apply to in the ELF data and the new COFF data.
//
SecShdr = GetShdrByIndex(RelShdr->sh_info);
SecOffset = mCoffSectionsOffset[RelShdr->sh_info];

//
// Only process relocations for the current filter type.
//
if (RelShdr->sh_type == SHT_RELA && (*Filter)(SecShdr)) {
...  ...
}


But anyway, I still support you add this patch.



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Monday, August 22, 2016 6:14 PM
> To: edk2-devel@lists.01.org; Gao, Liming <liming@intel.com>; Shi,
> Steven <steven@intel.com>; Zhu, Yonghong <yonghong@intel.com>
> Cc: Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: [PATCH] BaseTools/GenFw: ignore dynamic RELA sections
> 
> When building PIE (ET_DYN) executables, an additional RELA section is
> emitted (in addition to the per-section .rela.text and .rela.data sections)
> that is intended to be resolved at runtime by a ET_DYN compatible loader.
> 
> At the moment, due to the fact that we don't support GOT based relocations,
> this dynamic RELA section only contains relocations that are redundant,
> i.e., each R_xxx_RELATIVE relocation it contains duplicates a R_xxx_ABS64
> relocation appear in .rela.text or .rela.data, and so we can simply ignore
> this section (and we already ignore it in practice due to the fact that it
> points to the NULL section, which has the SHF_ALLOC bit cleared)
> 
> For example,
> 
>   Section Headers:
> [Nr] Name  Type Address   Offset
>  Size  EntSize  Flags  Link  Info  Align
> [ 0]   NULL   
>       0 0 0
> [ 1] .text PROGBITS 0240  00c0
>  427c  0008  AX   0 0 64
> [ 2] .rela.textRELA   9310
>  1bf0  0018   I   7 1 8
> [ 3] .data PROGBITS 44c0  4340
>  46d0    WA   0 0 64
> [ 4] .rela.dataRELA   af00
>  0600  0018   I   7 3 8
> [ 5] .rela RELA 8bc0  8a10
>  0600  0018   0 0 8
> [ 6] .shstrtab STRTAB     b500
>  0037     0 0 1
> [ 7] .symtab   SYMTAB     9010
>  0210  0018   817 8
> [ 8] .strtab   STRTAB     9220
>  00eb     0 0 1
> 
>   Relocation section '.rela.data' at offset 0xaf00 contains 64 entries:
> Offset  Info   Type   Sym. ValueSym. Name + 
> Addend
>   4800  00010001 R_X86_64_64   0240 .text +
> 3f5b
>   4808  00010001 R_X86_64_64   0240 .text +
> 3f63
>   4810  00010001 R_X86_64_64   0240 .text +
> 3f79
>   4818  00010001 R_X86_64_64   0240 .text +
> 3f90
>   4820  00010001 R_X86_64_64   0240 .text +
> 3fa6
>   ...
> 
>   Relocation section '.rela' at offset 0x8a10 contains 64 entries:
> Offset  Info   Type   Sym. ValueSym. Name + 
> Addend
>   4800  0008 R_X86_64_RELATIVE419b
>   4808  0008 R_X86_64_RELATIVE41a3
>   4810  0008 R_X86_64_RELATIVE41b9
>   4818  0008 R_X86_64_RELATIVE41d0
>   4820  0008 R_X86_64_RELATIVE41e6
>   4828  0008 R_X86_64_RELATIVE  

Re: [edk2] [PATCH] BaseTools/GccBase.lds: don't copy RELA section to PE/COFF

[Shi, Steven]

Ard,
Your patch rock! Thank you let us know and contribute this optimization. 


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Gao, Liming
> Sent: Monday, August 22, 2016 4:47 PM
> To: Ard Biesheuvel <ard.biesheu...@linaro.org>
> Cc: edk2-devel-01 <edk2-devel@lists.01.org>
> Subject: Re: [edk2] [PATCH] BaseTools/GccBase.lds: don't copy RELA section
> to PE/COFF
> 
> Ard:
>Thanks for the detail information. Your patch makes sense.  Reviewed-by:
> Liming Gao <liming@intel.com>
> 
> > -Original Message-
> > From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> > Ard Biesheuvel
> > Sent: Monday, August 22, 2016 3:26 PM
> > To: Gao, Liming <liming@intel.com>
> > Cc: edk2-devel-01 <edk2-devel@lists.01.org>
> > Subject: Re: [edk2] [PATCH] BaseTools/GccBase.lds: don't copy RELA
> section
> > to PE/COFF
> >
> > On 22 August 2016 at 09:22, Gao, Liming <liming@intel.com> wrote:
> > > Ard:
> > >Sorry. I miss this patch. GccBase.lds is used by GCC44~GCC5.
> > GCC44~GCC49 doesn't enable LTO, GCC5 enables LTO. That means RELA
> > section is not used with LTO or without LTO. Right?
> >
> > The allocatable RELA section is only emitted when linking a PIE binary
> > (i.e., the -pie linker switch, not the -fpie compiler switch). It
> > contains ELF relocations that the PE/COFF loader does not understand.
> > CLANG38LTO is the first toolchain to use the -pie linker switch, GCC5
> > does use PIE objects but does not create a PIE binary at link time.
> >
> > In some cases, (i.e., when we add support for GOT based relocations),
> > the contents of this RELA section must be taken into account by GenFw,
> > but currently, it can only contain relocations that are duplicates of
> > the relocations we encounter in the non-allocatable RELA sections
> > (.rela.text, .rela.data etc etc)
> >
> > So it never makes sense to copy it into the PE/COFF binary, regardless
> > of whether we inspect its contents or not.
> >
> > --
> > Ard.
> >
> > >
> > > Thanks
> > > Liming
> > >> -Original Message-
> > >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> > >> Sent: Monday, August 22, 2016 3:11 PM
> > >> To: Shi, Steven <steven@intel.com>; Zhu, Yonghong
> > >> <yonghong@intel.com>; Gao, Liming <liming@intel.com>;
> edk2-
> > >> devel-01 <edk2-devel@lists.01.org>
> > >> Cc: Ard Biesheuvel <ard.biesheu...@linaro.org>
> > >> Subject: Re: [PATCH] BaseTools/GccBase.lds: don't copy RELA section to
> > >> PE/COFF
> > >>
> > >> On 10 August 2016 at 10:55, Ard Biesheuvel <ard.biesheu...@linaro.org>
> > >> wrote:
> > >> > The CLANG38 toolchain creates a PIE binary at link time. This is
> > >> > necessary since the LTO code generation may otherwise result in
> > >> > code that cannot execute correctly when loaded above 2 GB.
> > >> >
> > >> > PIE executables contain a RELA section consisting of dynamic
> > >> > relocation entries that are intended for consumption by the loader
> > >> > at runtime. For this reason, it has the SHF_ALLOC attribute set by
> > >> > default, and will be identified by GenFw as a section that needs to
> > >> > be copied into the PE/COFF binary, resulting in waste of space since
> > >> > the PE/COFF loader does not use this data at all.
> > >> >
> > >> > So mark the RELA section as informational: this will prevent the
> > >> > linker from setting the SHF_ALLOC attribute, causing GenFw to
> > >> > ignore it.
> > >> >
> > >> > DxeCore.efi before:
> > >> >
> > >> > Detected 'X64' type PE/COFF image consisting of 3 sections
> > >> > Section alignment:  0x40
> > >> > File alignment: 0x40
> > >> > Section '.text' @ 0x0240
> > >> > File offset:0x240
> > >> > Virtual size:   0x21000
> > >> > Raw size:   0x21000
> > >> > Section '.data' @ 0x00021240
> > >> > File offset:0x21240
> > >> > Virtual size:   0x3640
> > >> >  

Re: [edk2] OVMF not booting when built with GCC5 toolset?

[Shi, Steven]

Hi Cran,
OK. I can reproduce your build failure with GCC 6.1.0. We will take a look at 
it. Thank you report it.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Bruce Cran [mailto:br...@cran.org.uk]
> Sent: Friday, August 19, 2016 10:47 PM
> To: Shi, Steven <steven@intel.com>; Michael Zimmermann
> <sigmaepsilo...@gmail.com>
> Cc: edk2-devel-01 <edk2-devel@lists.01.org>
> Subject: Re: [edk2] OVMF not booting when built with GCC5 toolset?
> 
> On 8/19/2016 8:35 AM, Bruce Cran wrote:
> > When I built and ran it using the same commands (with the exception of
> > adding "-vnc :0" to the qemu cmdline) I got the same problem as
> > before, and serial.log contained just:
> >
> >
> SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x30202C7
> 8SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x30202C
> 78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x30202
> C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x3020
> 2C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x302
> 02C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x30
> 202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x3
> 0202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0x
> 30202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(0
> x30202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack(
> 0x30202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStack
> (0x30202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithStac
> k(0x30202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithSta
> ck(0x30202C78SecCoreStartupWithStack(0x30202C78SecCoreStartupWithSt
> ack(0x30202C78SecCoreStartupWithStack(0x30202C78
> >
> 
> Sorry, I'd added -O0 to the debug build flags to let me better debug
> code. With that removed, I now get the following in serial.log:
> 
> SecCoreStartupWithStack(0x0, 0x0)
> Register PPI Notify: 
> Install PPI: 
> Install PPI: 
> The 8519680th FV start address is 0x0810248, size is 0xFFFD13C4,
> handle is 0x827F01
> Register PPI Notify: 
> Register PPI Notify: 
> Install PPI: 
> Install PPI: 
> Loading PEIM at 0x000 EntryPoint=0x000 
> SecCoreStartupWithStack(0x, 0xAE569B3A)
> Register PPI Notify: 
> SecCoreStartupWithStack(0x, 0xAE569B3A)
> Register PPI Notify: 
> SecCoreStartupWithStack(0x, 0xAE569B3A)
> Register PPI Notify: 
> SecCoreStartupWithStack(0x, 0xAE569B3A)
> Register PPI Notify: 
> SecCoreStartupWithStack(0x, 0xAE569B3A)
> Register PPI Notify: 
> 
> Those last 2 lines repeat about 100 times.
> 
> --
> Bruce

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v2] BaseTools-CLANG38: Add -O3 in DLINK2 flag

[Shi, Steven]

CLANG38 build fail after CC_FLAG is added in the link rule.
This failure is because the CLANG38 enable the LTO through LLVMgold.so
linker plugin, but the LLVMgold.so plugin cannot accept the clang -Oz
CC flag as build option. After CC_FLAG is added in the link rule,
the LLVMgold.so plugin reports linking error. LLVMgold.so only accept
-O0 ~ -O3, and you can see it in the LLVM gold plugin source code
in below:

http://llvm.org/svn/llvm-project/llvm/tags/RELEASE_380/final/tools/gold/
gold-plugin.cpp line173:

if (opt[1] < '0' || opt[1] > '3')
   message(LDPL_FATAL, "Optimization level must be between 0 and 3");

Add -O3 in the *_CLANG38_*_DLINK2_FLAGS to override the -Oz flag in
*_CLANG38_*_CC_FLAGS to pass LLVMgold.so linking.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/tools_def.template | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
index 974656c..42a3ca0 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -5489,7 +5489,7 @@ DEFINE CLANG38_ALL_CC_FLAGS = 
DEF(GCC44_ALL_CC_FLAGS) -Wno-empty-body -f
 DEBUG_CLANG38_IA32_CC_FLAGS = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto 
-march=i586 DEF(CLANG38_IA32_TARGET) -g
 RELEASE_CLANG38_IA32_CC_FLAGS   = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto 
-march=i586 DEF(CLANG38_IA32_TARGET)
 *_CLANG38_IA32_DLINK_FLAGS  = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz 
-Wl,-melf_i386 -Wl,--oformat=elf32-i386
-*_CLANG38_IA32_DLINK2_FLAGS = DEF(GCC5_IA32_DLINK2_FLAGS)
+*_CLANG38_IA32_DLINK2_FLAGS = DEF(GCC5_IA32_DLINK2_FLAGS) -O3
 *_CLANG38_IA32_RC_FLAGS = DEF(GCC_IA32_RC_FLAGS)
 *_CLANG38_IA32_OBJCOPY_FLAGS=
 *_CLANG38_IA32_NASM_FLAGS   = -f elf32
@@ -5518,7 +5518,7 @@ RELEASE_CLANG38_IA32_CC_FLAGS   = 
DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -
 DEBUG_CLANG38_X64_CC_FLAGS = DEF(CLANG38_ALL_CC_FLAGS) -m64 
"-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto 
DEF(CLANG38_X64_TARGET) -g
 RELEASE_CLANG38_X64_CC_FLAGS   = DEF(CLANG38_ALL_CC_FLAGS) -m64 
"-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto 
DEF(CLANG38_X64_TARGET)
 *_CLANG38_X64_DLINK_FLAGS  = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz 
-Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie -mcmodel=small
-*_CLANG38_X64_DLINK2_FLAGS = DEF(GCC5_X64_DLINK2_FLAGS)
+*_CLANG38_X64_DLINK2_FLAGS = DEF(GCC5_X64_DLINK2_FLAGS) -O3
 *_CLANG38_X64_RC_FLAGS = DEF(GCC_X64_RC_FLAGS)
 *_CLANG38_X64_OBJCOPY_FLAGS=
 *_CLANG38_X64_NASM_FLAGS   = -f elf64
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] OVMF not booting when built with GCC5 toolset?

[Shi, Steven]

Hi Cran,

I build OVMF GCC X64 image, can boot Qemu into shell with below commands. I can 
enter shell, reconnect -r, exit to setup, and the vga display looks good to me. 
What build command did you use? 
 
build -t GCC5 -a X64  -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT
qemu-system-x86_64 -bios OVMF.fd -serial file:serial.log -m 4096 -hda fat:. 
-monitor stdio -s


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Bruce Cran
> Sent: Friday, August 19, 2016 12:59 AM
> To: Michael Zimmermann 
> Cc: edk2-devel-01 
> Subject: Re: [edk2] OVMF not booting when built with GCC5 toolset?
> 
> On 8/18/2016 10:57 AM, Michael Zimmermann wrote:
> > IA32 or X64? I noticed that EmulatorPkg-X64 just produces a segfault
> > when booting. IA32 works just fine. I didn't try to use GCC49 yet.
> 
> X64 - both DEBUG and RELEASE.
> 
> Bruce
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH] BaseTools-CLANG38: Add O3 in Dlink2 flag

[Shi, Steven]

CLANG38 build fail after CC_FLAG is added in the link rule.
This failure is because the CLANG38 enable the LTO through LLVMgold.so
linker plugin, but the LLVMgold.so plugin cannot accept the clang -Oz
CC flag as build option. After CC_FLAG is added in the link rule,
the LLVMgold.so plugin reports linking error. LLVMgold.so only accept
-O0 ~ -O3, and you can see it in the LLVM gold plugin source code
in below:

http://llvm.org/svn/llvm-project/llvm/tags/RELEASE_380/final/tools/gold/
gold-plugin.cpp line173:

if (opt[1] < '0' || opt[1] > '3')
   message(LDPL_FATAL, "Optimization level must be between 0 and 3");

Add -O3 in the *_CLANG38_X64_DLINK2_FLAGS to override the -Oz flag in
*_CLANG38_X64_CC_FLAGS to pass LLVMgold.so linking.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/tools_def.template | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
index 974656c..c8e80c9 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -5518,7 +5518,7 @@ RELEASE_CLANG38_IA32_CC_FLAGS   = 
DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -
 DEBUG_CLANG38_X64_CC_FLAGS = DEF(CLANG38_ALL_CC_FLAGS) -m64 
"-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto 
DEF(CLANG38_X64_TARGET) -g
 RELEASE_CLANG38_X64_CC_FLAGS   = DEF(CLANG38_ALL_CC_FLAGS) -m64 
"-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto 
DEF(CLANG38_X64_TARGET)
 *_CLANG38_X64_DLINK_FLAGS  = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz 
-Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie -mcmodel=small
-*_CLANG38_X64_DLINK2_FLAGS = DEF(GCC5_X64_DLINK2_FLAGS)
+*_CLANG38_X64_DLINK2_FLAGS = DEF(GCC5_X64_DLINK2_FLAGS) -O3
 *_CLANG38_X64_RC_FLAGS = DEF(GCC_X64_RC_FLAGS)
 *_CLANG38_X64_OBJCOPY_FLAGS=
 *_CLANG38_X64_NASM_FLAGS   = -f elf64
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 3/3] BaseTools GCC: add the compiler flags to the linker command line

[Shi, Steven]

>

> Could you try putting -O2 in the DLINK2_FLAGS for Clang 38? (i.e., so

> that the last -O option the linker sees is not -Oz)

>

Ard,

Add -O2 or -O3 in DLINK2_FLAGS works as below, thank your suggestion!



"/home/jshi19/clang38/bin/clang" -o 
/home/jshi19/edk2-fork/Build/OvmfX64/RELEASE_CLANG38/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll
 -nostdlib -Wl,-n,-q,--gc-sections -z common-page-size=0x40 
-Wl,--entry,_ModuleEntryPoint -u _ModuleEntryPoint 
-Wl,-Map,/home/jshi19/edk2-fork/Build/OvmfX64/RELEASE_CLANG38/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.map
 -flto -Wl,-Oz -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie 
-mcmodel=small 
-Wl,--start-group,@/home/jshi19/edk2-fork/Build/OvmfX64/RELEASE_CLANG38/X64/MdeModulePkg/Core/Dxe/DxeMain/OUTPUT/static_library_files.lst,--end-group
 -g -fshort-wchar -fno-strict-aliasing -Wall -Werror -Wno-array-bounds 
-ffunction-sections -fdata-sections -include AutoGen.h -fno-common 
-DSTRING_ARRAY_NAME=DxeCoreStrings -Wno-empty-body -fno-stack-protector 
-fno-builtin -mms-bitfields -Wno-address -Wno-shift-negative-value 
-Wno-parentheses-equality -Wno-unknown-pragmas 
-Wno-tautological-constant-out-of-range-compare -Wno-incompatible-library-rede
 claration -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -msoft-float 
-mno-implicit-float 
-ftrap-function=undefined_behavior_has_been_optimized_away_by_clang 
-funsigned-char -fno-ms-extensions -Wno-null-dereference 
-Wno-tautological-compare -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone 
-mcmodel=small -fpie -Oz -flto -target x86_64-pc-linux-gnu -DMDEPKG_NDEBUG 
-mno-mmx -mno-sse -Wl,--defsym=PECOFF_HEADER_SIZE=0x228 
-Wl,--script=/home/jshi19/edk2-fork/BaseTools/Scripts/GccBase.lds -Wno-error 
-O3 -v

clang version 3.8.0 (tags/RELEASE_380/final)

Target: x86_64-pc-linux-gnu

Thread model: posix

InstalledDir: /home/jshi19/clang38/bin

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7.4

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.3

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0

Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0

Candidate multilib: .;@m64

Candidate multilib: 32;@m32

Candidate multilib: x32;@mx32

Selected multilib: .;@m64

"/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 
-dynamic-linker /lib64/ld-linux-x86-64.so.2 -o 
/home/jshi19/edk2-fork/Build/OvmfX64/RELEASE_CLANG38/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll
 -u _ModuleEntryPoint -L/usr/lib/gcc/x86_64-linux-gnu/5.4.0 
-L/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu 
-L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu 
-L/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../.. 
-L/home/jshi19/clang38/bin/../lib -L/lib -L/usr/lib -plugin 
/home/jshi19/clang38/bin/../lib/LLVMgold.so -plugin-opt=mcpu=x86-64 
-plugin-opt=O3 -n -q --gc-sections -z common-page-size=0x40 --entry 
_ModuleEntryPoint -Map 
/home/jshi19/edk2-fork/Build/OvmfX64/RELEASE_CLANG38/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.map
 -Oz -melf_x86_64 --oformat=elf64-x86-64 -pie --start-group 
@/home/jshi19/edk2-fork/Build/OvmfX64/RELEASE_CLANG38/X64/MdeModulePkg/Core/Dxe/DxeMain/OUTPUT/static_library_files.lst
 --end-group --defsym=PECO
 FF_HEADER_SIZE=0x228 
--script=/home/jshi19/edk2-fork/BaseTools/Scripts/GccBase.lds



Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522


___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 3/3] BaseTools GCC: add the compiler flags to the linker command line

[Shi, Steven]

Hi Ard,

CLANG38 has a build failure as below for this patch. This failure is because 
the CLANG38 enable the LTO through LLVMgold.so linker plugin, but the 
LLVMgold.so plugin cannot accept the clang -Oz cc flag as build option. After 
CC_FLAG is added in the link rule, the LLVMgold.so plugin reports link error. 
LLVMgold.so only accept -O0~-O3, and you can see it in the LLVM gold plugin 
source code in below:



http://llvm.org/svn/llvm-project/llvm/tags/RELEASE_380/final/tools/gold/gold-plugin.cpp



  if (opt[1] < '0' || opt[1] > '3')

line173：message(LDPL_FATAL, "Optimization level must be between 0 and 
3");



I hope to know is it mandatory to add compiler CC_FLAG in the linker rule for 
you? If it is mandatory, I have to introduce a new link rule for CLANG38 to 
remove the compiler CC_FLAG. What do you think?





"/home/jshi19/clang38/bin/clang" -o 
/home/jshi19/edk2-fork/Build/OvmfX64/DEBUG_CLANG38/X64/OvmfPkg/Sec/SecMain/DEBUG/SecMain.dll
 -nostdlib -Wl,-n,-q,--gc-sections -z common-page-size=0x40 
-Wl,--entry,_ModuleEntryPoint -u _ModuleEntryPoint 
-Wl,-Map,/home/jshi19/edk2-fork/Build/OvmfX64/DEBUG_CLANG38/X64/OvmfPkg/Sec/SecMain/DEBUG/SecMain.map
 -flto -Wl,-Oz -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie 
-mcmodel=small 
-Wl,--start-group,@/home/jshi19/edk2-fork/Build/OvmfX64/DEBUG_CLANG38/X64/OvmfPkg/Sec/SecMain/OUTPUT/static_library_files.lst,--end-group
 -g -fshort-wchar -fno-strict-aliasing -Wall -Werror -Wno-array-bounds 
-ffunction-sections -fdata-sections -include AutoGen.h -fno-common 
-DSTRING_ARRAY_NAME=SecMainStrings -Wno-empty-body -fno-stack-protector 
-fno-builtin -mms-bitfields -Wno-address -Wno-shift-negative-value 
-Wno-parentheses-equality -Wno-unknown-pragmas 
-Wno-tautological-constant-out-of-range-compare 
-Wno-incompatible-library-redeclaration -fno-asynchronous-unwind-tables 
-mno-sse -mno-mmx -msoft-float -mno-implicit-float 
-ftrap-function=undefined_behavior_has_been_optimized_away_by_clang 
-funsigned-char -fno-ms-extensions -Wno-null-dereference 
-Wno-tautological-compare -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone 
-mcmodel=small -fpie -Oz -flto -target x86_64-pc-linux-gnu -g -mno-mmx -mno-sse 
-Wl,--defsym=PECOFF_HEADER_SIZE=0x228 
-Wl,--script=/home/jshi19/edk2-fork/BaseTools/Scripts/GccBase.lds -Wno-error -v

clang version 3.8.0 (tags/RELEASE_380/final)

Target: x86_64-pc-linux-gnu

Thread model: posix

InstalledDir: /home/jshi19/clang38/bin

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.7.4

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.3

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0

Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0

Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0

Candidate multilib: .;@m64

Candidate multilib: 32;@m32

Candidate multilib: x32;@mx32

Selected multilib: .;@m64

"/usr/bin/ld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 
-dynamic-linker /lib64/ld-linux-x86-64.so.2 -o 
/home/jshi19/edk2-fork/Build/OvmfX64/DEBUG_CLANG38/X64/OvmfPkg/Sec/SecMain/DEBUG/SecMain.dll
 -u _ModuleEntryPoint -L/usr/lib/gcc/x86_64-linux-gnu/5.4.0 
-L/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../../x86_64-linux-gnu 
-L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu 
-L/usr/lib/gcc/x86_64-linux-gnu/5.4.0/../../.. 
-L/home/jshi19/clang38/bin/../lib -L/lib -L/usr/lib -plugin 
/home/jshi19/clang38/bin/../lib/LLVMgold.so -plugin-opt=mcpu=x86-64 
-plugin-opt=Oz -n -q --gc-sections -z common-page-size=0x40 --entry 
_ModuleEntryPoint -Map 
/home/jshi19/edk2-fork/Build/OvmfX64/DEBUG_CLANG38/X64/OvmfPkg/Sec/SecMain/DEBUG/SecMain.map
 -Oz -melf_x86_64 --oformat=elf64-x86-64 -pie --start-group 
@/home/jshi19/edk2-fork/Build/OvmfX64/DEBUG_CLANG38/X64/OvmfPkg/Sec/SecMain/OUTPUT/static_library_files.lst
 --end-group --defsym=PECOFF_HEADER_SIZE=0x228 
--script=/home/jshi19/edk2-fork/BaseTools/Scripts/GccBase.lds

/usr/bin/ld: error: Optimization level must be between 0 and 3

clang-3.8: error: linker command failed with exit code 1 (use -v to see 
invocation)







Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522





> -Original Message-

> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of

> Ard Biesheuvel

> Sent: Tuesday, August 02, 2016 10:40 PM

> To: Zhu, Yonghong ; Gao, Liming

> ; Justen, Jordan L ;

> edk2-devel@lists.01.org; leif.lindh...@linaro.org

> Cc: sigmaepsilo...@gmail.com; Ard Biesheuvel 

> Subject: [edk2] [PATCH 3/3] BaseTools GCC: add the compiler flags to the

> linker command line

>

> Now that we invoke GCC as the linker for the GCC toolchain family,

> we can pass the 

Re: [edk2] [PATCH] BaseTools X64: fold PLT relocations into simple relative references

[Shi, Steven]

OK, it is. But it is a bit not very clear.





Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522





> -Original Message-

> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]

> Sent: Thursday, August 04, 2016 4:55 PM

> To: Shi, Steven <steven@intel.com>

> Cc: Zhu, Yonghong <yonghong@intel.com>; Gao, Liming

> <liming@intel.com>; Justen, Jordan L <jordan.l.jus...@intel.com>;

> edk2-devel@lists.01.org; misch...@offblast.org

> Subject: Re: [PATCH] BaseTools X64: fold PLT relocations into simple relative

> references

>

> On 4 August 2016 at 10:54, Shi, Steven 
> <steven@intel.com<mailto:steven@intel.com>> wrote:

> > Hi Ard,

> > I don't see you add below code for case R_X86_64_PLT32. Is it right?

> >

> > *(UINT32 *)Targ = (UINT32) (*(UINT32 *)Targ

> >   + (mCoffSectionsOffset[Sym->st_shndx] - SymShdr->sh_addr)

> >   - (SecOffset - SecShdr->sh_addr));

> >

>

> Isn't it identical to the code for R_X86_64_PC32?
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH] BaseTools X64: fold PLT relocations into simple relative references

[Shi, Steven]

Hi Ard,
I don't see you add below code for case R_X86_64_PLT32. Is it right?

*(UINT32 *)Targ = (UINT32) (*(UINT32 *)Targ
  + (mCoffSectionsOffset[Sym->st_shndx] - SymShdr->sh_addr)
  - (SecOffset - SecShdr->sh_addr));


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Thursday, August 04, 2016 4:46 PM
> To: Shi, Steven <steven@intel.com>; Zhu, Yonghong
> <yonghong@intel.com>; Gao, Liming <liming@intel.com>; Justen,
> Jordan L <jordan.l.jus...@intel.com>; edk2-devel@lists.01.org
> Cc: misch...@offblast.org; Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: [PATCH] BaseTools X64: fold PLT relocations into simple relative
> references
> 
> For X64/GCC, we use position independent code with hidden visibility
> to inform the compiler that symbols references are never resolved at
> runtime, which removes the need for PLTs and GOTs. However, in some
> cases GCC has been reported to still emit PLT based relocations, which
> we need to handle in the ELF to PE/COFF perform by GenFw.
> 
> Unlike GOT based relocations, which are non-trivial to handle since the
> indirections in the code can not be fixed up easily (although relocation
> types exist for X64 that annotate relocation targets as suitable for
> relaxation), PLT relocations simply point to jump targets, and we can
> relax such relocations by resolving them using the symbol directly rather
> than via a PLT entry that does nothing more than tail call the function
> we already know it is going to call (since all symbol references are
> resolved in the same module).
> 
> So handle R_X86_64_PLT32 as a R_X86_64_PC32 relocation.
> 
> Suggested-by: Steven Shi <steven@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Ard Biesheuvel <ard.biesheu...@linaro.org>
> ---
>  BaseTools/Source/C/GenFw/Elf64Convert.c | 11 +++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c
> b/BaseTools/Source/C/GenFw/Elf64Convert.c
> index 944c94b8f8b4..7cbff0df0996 100644
> --- a/BaseTools/Source/C/GenFw/Elf64Convert.c
> +++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
> @@ -785,6 +785,17 @@ WriteSections64 (
>  *(INT32 *)Targ = (INT32)((INT64)(*(INT32 *)Targ) - 
> SymShdr->sh_addr
> + mCoffSectionsOffset[Sym->st_shndx]);
>  VerboseMsg ("Relocation:  0x%08X", *(UINT32*)Targ);
>  break;
> +
> +  case R_X86_64_PLT32:
> +//
> +// Treat R_X86_64_PLT32 relocations as R_X86_64_PC32: this is
> +// possible since we know all code symbol references resolve to
> +// definitions in the same module (UEFI has no shared libraries),
> +// and so there is never a reason to jump via a PLT entry,
> +// allowing us to resolve the reference using the symbol 
> directly.
> +//
> +VerboseMsg ("Treating R_X86_64_PLT32 as R_X86_64_PC32 ...");
> +/* fall through */
>case R_X86_64_PC32:
>  //
>  // Relative relocation: Symbol - Ip + Addend
> --
> 2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

Ard,
Pls go ahead to add R_X86_64_PLT32 firstly. Below is my original path. And for 
the GOTPCREL type ones, I could finish enhance them after I back from vacation 
next week. Thanks.
https://github.com/shijunjing/edk2/commit/dd8b1c4625cbffc7e149571266e0ae0581bd207d



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Thursday, August 04, 2016 4:55 AM
> To: Justen, Jordan L <jordan.l.jus...@intel.com>
> Cc: Shi, Steven <steven@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>; edk2-devel-01 <edk2-devel@lists.01.org>;
> af...@apple.com; Gao, Liming <liming@intel.com>;
> misch...@offblast.org
> Subject: Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf
> relocation types for PIC/PIE code
> 
> 
> > On 3 aug. 2016, at 22:53, Jordan Justen <jordan.l.jus...@intel.com> wrote:
> >
> >> On 2016-08-03 13:47:09, Ard Biesheuvel wrote:
> >>
> >>> On 3 aug. 2016, at 22:13, Jordan Justen <jordan.l.jus...@intel.com>
> wrote:
> >>>
> >>> Where does this patch stand? I think Ard was asking for it to be
> >>> split, but also wondering if it was really required...
> >>>
> >>> Some devs are still reporting unsupported relocation errors during the
> >>> build. (mischief on irc, for example.)
> >>
> >> The patch is not correct in its current form. PLT relocations (0x4)
> >> can be handled ok, so we can split that off and merge it. The GOT
> >> based relocation handling needs non-trivial rework, and may
> >> currently create corrupt binaries. Which unhandled reloc types are
> >> being reported?
> >
> > "unsupported ELF EM_X86_64 relocation 0x4"
> >
> > You might join irc if you want to get some more info from mischief
> > about it.
> >
> 
> That's the plt one, which we can fix easily. I'll whip up a patch in the 
> morning
> (11 pm here now)
> 
> 
> >
> >>
> >>>
> >>>> On 2016-08-02 05:00:43, Ard Biesheuvel wrote:
> >>>> On 2 August 2016 at 13:40, Shi, Steven <steven@intel.com> wrote:
> >>>>>>
> >>>>>> CoffAddFixup() must be used for absolute symbol references only.
> These
> >>>>>> instructions contain relative symbol references, which are
> >>>>>> recalculated in WriteSections64().
> >>>>>>
> >>>>>> The only absolute symbol reference is the GOT entry for 'n', and your
> >>>>>> code (in WriteRelocations64()) calculates the address of the GOT
> entry
> >>>>>> (which is always in .text BTW) and adds a fixup for it, i.e.,
> >>>>>>
> >>>>>> +  CoffAddFixup(
> >>>>>> +(UINT32)(UINTN)((UINT64)
> >>>>>> mCoffSectionsOffset[RelShdr->sh_info] + GoTPcRelPtrOffset),
> >>>>>> +EFI_IMAGE_REL_BASED_DIR64);
> >>>>>>
> >>>>>> This code adds a fixup to the PE/COFF .reloc section for the GOT
> entry
> >>>>>> containing the address of 'n', and the instructions perform a IP
> >>>>>> relative load of the contents of the GOT entry to retrieve the address
> >>>>>> of 'n'.
> >>>>>>
> >>>>>> By adding two fixups, the PE/COFF loader will apply the load offset
> >>>>>> twice, resulting in an incorrect value.
> >>>>> OK, I get your point now. Yes, the current patch could generate
> multiple fixups for the same GOT relocation entry. How about we introduce a
> simple IsDuplicatedCoffFixup() to check whether a converting fixup offset is
> duplicated before we use CoffAddFixup() to really add it? If it is new, we add
> it, otherwise just skip it.
> >>>>
> >>>> That could work, but you have to be aware that fixups are best emitted
> >>>> in the order they need to be applied in the binary, or it will become
> >>>> very inefficient. (Please refer to the PE/COFF spec section that
> >>>> explains the layout of the .reloc section)
> >>>>
> >>>> What it comes down to is that relocations are grouped by target page,
> >>>> and for every place in the page that requires a relocation to be
> >>>> applied, a 4 bit type is emitted followed by a 12-bit offset, which is
> >>>> the offset into the current page. If you emit fixups for the current
> >>>> instruction, followed by one for the GOT, it will basically take two
> >>>> 'page switches' every time.
> >>>>
> >>>> So it would be better to simply emit the relocations, but introduce a
> >>>> sorting pass that merges all duplicates as well.
> >>>>
> >>>> Thanks,
> >>>> Ard.
> >>>> ___
> >>>> edk2-devel mailing list
> >>>> edk2-devel@lists.01.org
> >>>> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v5 1/4] BaseTools-Conf:Remove short dash in ar flag for LLVM

[Shi, Steven]

Both binutils ar and LLVM ar support "cr", but LLVM ar doens't
support add "-" in the flags, and llvm-ar cannot accept "-cr".
So remove the short dash "-" to make llvm archives work.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
Reviewed-by: Ard Biesheuvel <ard.biesheu...@linaro.org>
---
 BaseTools/Conf/build_rule.template | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template

diff --git a/BaseTools/Conf/build_rule.template 
b/BaseTools/Conf/build_rule.template
old mode 100644
new mode 100755
index 9adf391..6e7b7d4
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -266,7 +266,7 @@
 "$(SLINK)" $(SLINK_FLAGS) /OUT:${dst} @$(OBJECT_FILES_LIST)
 
 <Command.GCC, Command.GCCLD>
-"$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
+"$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
 
 
 "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v5 4/4] ShellPkg-UefiShellCommandLib: Add EFIAPI in VA_List library function

[Shi, Steven]

Add EFIAPI in CatPrint library function. Every function which uses
variable list need explicit use EFIAPI to force use MS ABI. This change
is needed to pass CLANG38 build.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c | 1 +
 1 file changed, 1 insertion(+)
 mode change 100644 => 100755 
ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c

diff --git a/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c 
b/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
old mode 100644
new mode 100755
index d157ebb..979693a
--- a/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
+++ b/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
@@ -85,6 +85,7 @@ typedef struct {
 
 **/
 EFI_STATUS
+EFIAPI
 CatPrint (
   IN OUT POOL_PRINT   *Str,
   IN CHAR16   *Fmt,
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v5 0/4] Introduce CLANG38 toolchain in edk2

[Shi, Steven]

Please review my new commits in public branch:
https://github.com/shijunjing/edk2/commits/llvm_v5

The difference from V4(https://github.com/shijunjing/edk2/commits/llvm_v4):
1. Remove the GCC5 path prefix for binutils objcopy, because a clang user may 
not set GCC5_BIN.
2. Follow GCC5 to disable the LTO for ASLCC flags.
3. Add CLANG38 in the "Supported Tool Chains" part.

The difference from V2(https://github.com/shijunjing/edk2/commits/llvm_v2):
1. Seperate the CLANG38 from the other two toolchains (GCC5, CLANGSCAN38), and 
only focus on CLANG38 in this serial.
2. Base on current GCC5 in edk2 to enhance the CLANG38.
3. Seperate the GenFW new relocation types support patch from this serial, and 
I will send it later in new serial.
4. Add EFIAPI for UefiShellCommandLib function.

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
OvmfPkgIa32X64.dsc).
Test compiler and linker version: LLVM 3.8, GNU ld 2.26.

Example steps to use the CLANG38 tool chain to build OVMF platform:
1. Download and extract the llvm 3.8.0 Pre-Built Binaries from 
http://www.llvm.org/releases/ (e.g. 
http://www.llvm.org/releases/3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz
 and extract it as ~/clang38).
2. Copy LLVMgold.so from 
https://github.com/shijunjing/edk2/blob/llvm/BaseTools/Bin/LLVMgold.so to above 
clang lib folder (e.g.~/clang38/lib/LLVMgold.so)
3. Install new version linker with plugin support (e.g. ld 2.26 in GNU Binutils 
2.26 or Ubuntu16.04)
$ cd edk2
$ export CLANG38_BIN=path/to/your/clang38/ (e.g. export 
CLANG38_BIN=~/clang38/bin/)
$ source edksetup.sh
$ make -C BaseTools/Source/C
$ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT
$ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
$ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096 -hda 
fat:.


Shi, Steven (4):
  BaseTools-Conf:Remove short dash in ar flag for LLVM
  BaseTools-Conf:Introduce CLANG38 new toolchain for x86
  ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin
  ShellPkg-UefiShellCommandLib: Add EFIAPI in VA_List library function

 BaseTools/Conf/build_rule.template |  2 +-
 BaseTools/Conf/tools_def.template  | 87 ++
 .../Library/UefiShellCommandLib/ConsistMapping.c   |  1 +
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c|  2 +-
 4 files changed, 90 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template
 mode change 100644 => 100755 
ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v5 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Shi, Steven]

This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
CLANG38 enable LLVM Link Time Optimization (LTO) and code size
optimization flag (-Oz) by default for aggressive code size
improvement. CLANG38 X64 code is small code model + PIE.

CLANG LTO needs PIE in link flags to generate PIE code correctly,
otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
hang in 64bits SEC at high address because of small model code
displacement overflow).

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
OvmfPkgIa32X64.dsc).
Test compiler and linker version: LLVM 3.8, GNU ld 2.26.

Example steps to use the CLANG38 tool chain to build OVMF platform:
1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
extract it as ~/clang38).
2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so)
3. Install new version linker with plugin support (e.g. ld 2.26 in
GNU Binutils 2.26 or Ubuntu16.04)
$ cd edk2
$ git checkout llvm
$ export CLANG38_BIN=path/to/your/clang38/
(e.g. export CLANG38_BIN=~/clang38/bin/)
$ source edksetup.sh
$ make -C BaseTools/Source/C
$ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
-DDEBUG_ON_SERIAL_PORT
$ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
$ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
 -hda fat:.

If you want, you can build and install GNU Binutils 2.26 as below steps
in Ubuntu:
Download binutils-2.26 source code from http://ftp.gnu.org/gnu/binutils/
 and extract it to ~/binutils-2.26
$sudo apt-get install bison
$sudo apt-get install flex
Install other necessary binutils build tools if missing
$ mkdir build
$ cd build
$ ../binutils-2.26/configure --enable-gold --enable-plugins
--disable-werror --prefix=/usr
$ make -j 5
$ sudo make install

If you want, you can build LLVMgold.so as below steps
Download llvm-3.8.0 source code from http://www.llvm.org/releases/
3.8.0/llvm-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src
Download clang3.8.0 source code from http://www.llvm.org/releases/
3.8.0/cfe-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src/tools/clang
Refer http://clang.llvm.org/get_started.html to Install other necessary
clang build tools if missing
$ mkdir llvm38build
$ cd llvm38build
If your GNU Binutils 2.26 is in /home/jshi19/binutils-2.26,
$ cmake ../llvm-3.8.0.src -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release"
-DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_C_COMPILER="/usr/bin/gcc"
-DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
$ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/tools_def.template | 87 +++
 1 file changed, 87 insertions(+)
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template

diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
old mode 100644
new mode 100755
index fd9eccb..6489039
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -381,6 +381,12 @@ DEFINE SOURCERY_CYGWIN_TOOLS = /cygdrive/c/Program 
Files/CodeSourcery/Sourcery G
 # Required to build platforms or ACPI tables:
 #   Intel(r) ACPI Compiler from
 #   https://acpica.org/downloads
+#   CLANG38  -Linux-  Requires:
+# Clang v3.8 or later, LLVMgold plugin and GNU 
binutils 2.26 targeting x86_64-linux-gnu
+#Optional:
+# Required to build platforms or ACPI tables:
+#   Intel(r) ACPI Compiler from
+#   https://acpica.org/downloads
 #   ELFGCC  -Linux-  Requires:
 # GCC(this tool chain uses whatever version of gcc 
and binutils that is installed in /usr/bin)
 #Optional:
@@ -5428,6 +5434,87 @@ RELEASE_CLANG35_AARCH64_CC_FLAGS = 
DEF(CLANG35_AARCH64_CC_FLAGS) $(ARCHCC_FLAGS)
 
 

 #
+# Clang 3.8 - This configuration is used to compile under Linux to produce
+#  PE/COFF binaries using LLVM/Clang 3.8 with Link Time Optimization enabled
+#
+
+*_CLANG38_*_*_FAMILY= GCC
+*_CLANG38_*_MAKE_PATH   = make
+*_CLANG38_*_*_DLL   = ENV(CLANG38_DLL)
+*_CLANG38_*_ASL_PATH= DEF(UNIX_IASL_BIN)
+
+*_CLANG38_*_APP_FLAGS   =
+*_CLANG38_*_ASL_FLAG

[edk2] [PATCH v5 3/4] ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

[Shi, Steven]

Use explicit CopyMem to replace compiler builtin to do the structure
values assignment. This change is needed to pass CLANG38 build.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
Reviewed-by: Jaben Carsey <jaben.car...@intel.com>
---
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

diff --git a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c 
b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
old mode 100644
new mode 100755
index 666ee9d..5c50797
--- a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
+++ b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
@@ -342,7 +342,7 @@ ShellCommandRunTftp (
 goto Error;
   }
 
-  Mtftp4ConfigData = DefaultMtftp4ConfigData;
+  CopyMem (, , sizeof 
(EFI_MTFTP4_CONFIG_DATA));
 
   //
   // Check the host IPv4 address
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Shi, Steven]

Hello Ard,

> 
> Hello Steven,
> 
> On 3 August 2016 at 08:48, Shi, Steven <steven@intel.com> wrote:
> > This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
> > CLANG38 enable LLVM Link Time Optimization (LTO) and code size
> > optimization flag (-Oz) by default for aggressive code size
> > improvement. CLANG38 X64 code is small code model + PIE.
> >
> > CLANG LTO needs PIE in link flags to generate PIE code correctly,
> > otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
> > hang in 64bits SEC at high address because of small model code
> > displacement overflow).
> >
> 
> This is probably caused by the same issue I am addressing with the
> series I sent out yesterday, to pass the CC flags to the DLINK
> command.
> 
> The reason is that code is generated by the link pass, so it needs to
> see the same -fpie -mcmodel=small options that we passed to the
> compiler as wel.
> 
> Could you check whether replacing '-Wl,-pie' with -fpie does the trick
> as well? As I mentioned before, creating a PIE executable at link time
> is not the same as generatic position independent code at compile time
> (whether it is via $(CC) or via $(DLINK)). The PIE executable will
> contain a .rela section that partially overlaps with other absolute
> relocations, so it is best to avoid it.

[Steven]: I just tried it. No, replacing '-Wl,-pie' with -fpie cannot works for 
clang38. With -fpie in link flags, the OvmfPkgX64 still hang in 64bits SEC at 
high address.


> 
> Some more comments below
> 
> > +*_CLANG38_IA32_OBJCOPY_PATH = DEF(GCC5_IA32_PREFIX)objcopy
> 
> Why are you using the GCC5 prefix here? A clang user may not set GCC5_BIN
> 
[Steven]: OK, I will remove the GCC5 prefix.


> > +*_CLANG38_IA32_ASLCC_FLAGS  = -x c -Os -m32
> DEF(CLANG38_IA32_TARGET) -flto
> 
> Does LTO make any sense for ACPI tables?
> 
[Steven]: OK, I will follow GCC5 to disable the LTO for ASLCC flags.


Thanks
Steven
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Shi, Steven]

This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
CLANG38 enable LLVM Link Time Optimization (LTO) and code size
optimization flag (-Oz) by default for aggressive code size
improvement. CLANG38 X64 code is small code model + PIE.

CLANG LTO needs PIE in link flags to generate PIE code correctly,
otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
hang in 64bits SEC at high address because of small model code
displacement overflow).

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
OvmfPkgIa32X64.dsc).
Test compiler and linker version: LLVM 3.8, GNU ld 2.26.

Example steps to use the CLANG38 tool chain to build OVMF platform:
1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
extract it as ~/clang38).
2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so)
3. Install new version linker with plugin support (e.g. ld 2.26 in
GNU Binutils 2.26 or Ubuntu16.04)
$ cd edk2
$ git checkout llvm
$ export CLANG38_BIN=path/to/your/clang38/
(e.g. export CLANG38_BIN=~/clang38/bin/)
$ source edksetup.sh
$ make -C BaseTools/Source/C
$ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
-DDEBUG_ON_SERIAL_PORT
$ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
$ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
 -hda fat:.

If you want, you can build and install GNU Binutils 2.26 as below steps
in Ubuntu:
Download binutils-2.26 source code from http://ftp.gnu.org/gnu/binutils/
 and extract it to ~/binutils-2.26
$sudo apt-get install bison
$sudo apt-get install flex
Install other necessary binutils build tools if missing
$ mkdir build
$ cd build
$ ../binutils-2.26/configure --enable-gold --enable-plugins
--disable-werror --prefix=/usr
$ make -j 5
$ sudo make install

If you want, you can build LLVMgold.so as below steps
Download llvm-3.8.0 source code from http://www.llvm.org/releases/
3.8.0/llvm-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src
Download clang3.8.0 source code from http://www.llvm.org/releases/
3.8.0/cfe-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src/tools/clang
Refer http://clang.llvm.org/get_started.html to Install other necessary
clang build tools if missing
$ mkdir llvm38build
$ cd llvm38build
If your GNU Binutils 2.26 is in /home/jshi19/binutils-2.26,
$ cmake ../llvm-3.8.0.src -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release"
-DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_C_COMPILER="/usr/bin/gcc"
-DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
$ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/tools_def.template | 81 +++
 1 file changed, 81 insertions(+)
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template

diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
old mode 100644
new mode 100755
index fd9eccb..6d964f1
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -5428,6 +5428,87 @@ RELEASE_CLANG35_AARCH64_CC_FLAGS = 
DEF(CLANG35_AARCH64_CC_FLAGS) $(ARCHCC_FLAGS)
 
 

 #
+# Clang 3.8 - This configuration is used to compile under Linux to produce
+#  PE/COFF binaries using LLVM/Clang 3.8 with Link Time Optimization enabled
+#
+
+*_CLANG38_*_*_FAMILY= GCC
+*_CLANG38_*_MAKE_PATH   = DEF(GCC5_IA32_PREFIX)make
+*_CLANG38_*_*_DLL   = ENV(CLANG38_DLL)
+*_CLANG38_*_ASL_PATH= DEF(UNIX_IASL_BIN)
+
+*_CLANG38_*_APP_FLAGS   =
+*_CLANG38_*_ASL_FLAGS   = DEF(IASL_FLAGS)
+*_CLANG38_*_ASL_OUTFLAGS= DEF(IASL_OUTFLAGS)
+
+DEFINE CLANG38_IA32_PREFIX  = ENV(CLANG38_BIN)
+DEFINE CLANG38_X64_PREFIX   = ENV(CLANG38_BIN)
+
+DEFINE CLANG38_IA32_TARGET  = -target i686-pc-linux-gnu
+DEFINE CLANG38_X64_TARGET   = -target x86_64-pc-linux-gnu
+
+DEFINE CLANG38_ALL_CC_FLAGS = DEF(GCC44_ALL_CC_FLAGS) -Wno-empty-body 
-fno-stack-protector -fno-builtin -mms-bitfields -Wno-address 
-Wno-shift-negative-value -Wno-parentheses-equality -Wno-unknown-pragmas 
-Wno-tautological-constant-out-of-range-compare 
-Wno-incompatible-library-redeclaration -fno-asynchronous-unwind-tables 
-mno-sse -mno-mmx -msoft-float -mno-implicit-float  
-ftrap-function=undefined_behavior_has_been_optimized_away_by_clang 
-funsigned-char -fno-ms-extensions -Wno-null-dereference 
-Wno-tautolog

[edk2] [PATCH 0/4] Introduce CLANG38 toolchain in edk2

[Shi, Steven]

Please review my new commits in public branch:
https://github.com/shijunjing/edk2/commits/llvm_v4

The difference from last V2(https://github.com/shijunjing/edk2/commits/llvm_v2):
1. Seperate the CLANG38 from the other two toolchains (GCC5, CLANGSCAN38), and 
only focus on CLANG38 in this serial.
2. Base on current GCC5 in edk2 to enhance the CLANG38.
3. Seperate the GenFW new relocation types support patch from this serial, and 
I will send it later in new serial.
4. Add EFIAPI for UefiShellCommandLib function.

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
OvmfPkgIa32X64.dsc).
Test compiler and linker version: LLVM 3.8, GNU ld 2.26.

Example steps to use the CLANG38 tool chain to build OVMF platform:
1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
extract it as ~/clang38).
2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so)
3. Install new version linker with plugin support (e.g. ld 2.26 in
GNU Binutils 2.26 or Ubuntu16.04)
$ cd edk2
$ git checkout llvm
$ export CLANG38_BIN=path/to/your/clang38/
(e.g. export CLANG38_BIN=~/clang38/bin/)
$ source edksetup.sh
$ make -C BaseTools/Source/C
$ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
-DDEBUG_ON_SERIAL_PORT
$ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
$ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
 -hda fat:.


Shi, Steven (4):
  BaseTools-Conf:Remove short dash in ar flag for LLVM
  BaseTools-Conf:Introduce CLANG38 new toolchain for x86
  ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin
  ShellPkg-UefiShellCommandLib: Add EFIAPI in VA_List library function

 BaseTools/Conf/build_rule.template |  2 +-
 BaseTools/Conf/tools_def.template  | 81 ++
 .../Library/UefiShellCommandLib/ConsistMapping.c   |  1 +
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c|  2 +-
 4 files changed, 84 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template
 mode change 100644 => 100755 
ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH 1/4] BaseTools-Conf:Remove short dash in ar flag for LLVM

[Shi, Steven]

Both binutils ar and LLVM ar support "cr", but LLVM ar doens't
support add "-" in the flags, and llvm-ar cannot accept "-cr".
So remove the short dash "-" to make llvm archives work.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/build_rule.template | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template

diff --git a/BaseTools/Conf/build_rule.template 
b/BaseTools/Conf/build_rule.template
old mode 100644
new mode 100755
index 9adf391..6e7b7d4
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -266,7 +266,7 @@
 "$(SLINK)" $(SLINK_FLAGS) /OUT:${dst} @$(OBJECT_FILES_LIST)
 
 <Command.GCC, Command.GCCLD>
-"$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
+"$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
 
 
 "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH 4/4] ShellPkg-UefiShellCommandLib: Add EFIAPI in VA_List library function

[Shi, Steven]

Add EFIAPI in CatPrint library function. Every function which uses
variable list need explicit use EFIAPI to force use MS ABI. This change
is needed to pass CLANG38 build.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c | 1 +
 1 file changed, 1 insertion(+)
 mode change 100644 => 100755 
ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c

diff --git a/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c 
b/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
old mode 100644
new mode 100755
index d157ebb..979693a
--- a/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
+++ b/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
@@ -85,6 +85,7 @@ typedef struct {
 
 **/
 EFI_STATUS
+EFIAPI
 CatPrint (
   IN OUT POOL_PRINT   *Str,
   IN CHAR16   *Fmt,
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH 3/4] ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

[Shi, Steven]

Use explicit CopyMem to replace compiler builtin to do the structure
values assignment. This change is needed to pass CLANG38 build.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
Reviewed-by: Jaben Carsey <jaben.car...@intel.com>
---
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

diff --git a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c 
b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
old mode 100644
new mode 100755
index 666ee9d..5c50797
--- a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
+++ b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
@@ -342,7 +342,7 @@ ShellCommandRunTftp (
 goto Error;
   }
 
-  Mtftp4ConfigData = DefaultMtftp4ConfigData;
+  CopyMem (, , sizeof 
(EFI_MTFTP4_CONFIG_DATA));
 
   //
   // Check the host IPv4 address
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Build traceback with new CLANG35 toolset

[Shi, Steven]

Cran,
We will send CLANG38 patch to support X86, and the new CLANG38 patch will base 
on current GCC5 implementation. FYI.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of
> Bruce Cran
> Sent: Wednesday, August 03, 2016 2:56 AM
> To: Ard Biesheuvel 
> Cc: edk2-devel-01 
> Subject: Re: [edk2] Build traceback with new CLANG35 toolset
> 
> On 8/2/2016 12:53 PM, Ard Biesheuvel wrote:
> 
> > CLANG35 is not new, and currently only supports ARM and AARCH64
> 
> Oops - thanks. I was thinking of CLANG38 that hasn't been committed yet.
> 
> --
> Bruce
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v5 0/8] BaseTools: add support for GCC5 in LTO mode

[Shi, Steven]

Ard,
Thank you to check in GCC5! 


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Tuesday, August 02, 2016 5:03 PM
> To: Shi, Steven <steven@intel.com>; Zhu, Yonghong
> <yonghong@intel.com>; Gao, Liming <liming@intel.com>; Justen,
> Jordan L <jordan.l.jus...@intel.com>; edk2-devel-01  de...@lists.01.org>
> Cc: Leif Lindholm <leif.lindh...@linaro.org>; Laszlo Ersek
> <ler...@redhat.com>; Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: Re: [PATCH v5 0/8] BaseTools: add support for GCC5 in LTO mode
> 
> On 1 August 2016 at 10:01, Ard Biesheuvel <ard.biesheu...@linaro.org>
> wrote:
> > This v5 to introduce GCC5 is now a 8 piece series, including some
> > preparatory cleanup patches that allow all GCC4x and CLANG35 toolchains
> > to switch to using 'gcc' as the linker. This allows us to get rid of
> > the wrapper script to marshall ld arguments in order to make them
> > understandable by gcc, which is fragile and likely to cause problems in
> > the future.
> >
> > Since there appears to be a natural split between the 'legacy' GCC
> > toolchains UNIXGCC, ELFGCC, and CYGGCC[xASL], both in term of
> supported
> > architectures [IA32, X64, IPF] vs [IA32, X64, ARM, AARCH64], and in
> > terms of maintenance, these toolchains are not moved to using 'gcc' as
> > the linker, and instead, a new BUILDRULEFAMILY is introduced called
> GCCLD
> > that will retain the old behavior.
> >
> > The result is that GCC5 can align much more closely with its predecessors,
> > making the expected maintenance burden of supporting GCC back to v4.4
> > much lower.
> >
> > Changes since v4:
> > - added patch to use 'protected' visibility only for the libraries that
> >   define the module entry points (_ModuleEntryPoint), to prevent them
> from
> >   being optimized away by the LTO routines
> > - added Jordan's ack/RBs
> > - add some extra comments to tools_def.template (#8)
> >
> 
> Thanks all. Committed as
> 
> 1c63516075b3 BaseTools CLANG35: drop problematic use-movt and save-
> temps options
> ff54bcdf2e4e ArmVirtPkg/ArmVirtPrePiUniCoreRelocatable: ignore .hash
> and .note sections
> befb3ba51502 BaseTools UNIXGCC ELFGCC CYGGCC: clone GCC build rule
> family into GCCLD
> a1b8baccc30b BaseTools GCC: use 'gcc' as the linker command for GCC44
> and later
> e1458aaded8e ArmPkg: add prebuilt glue binaries for GCC5 LTO support
> 7fd5d619806d BaseTools GCC: drop GNU notes section from EFI image
> 4a8466d4baba BaseTools GCC: introduce GCC5 toolchain to support GCC
> v5.x in LTO mode
> 
> with Leif and Liming's R-b. I dropped patch #7, and instead made the
> visibility pragma conditional on whether LTO is disabled.
> 
> --
> Ard.
> 
> 
> > Changes since v3:
> > - like Steven does in his GCC5LTO patch, add -fno-builtin to IA32 and X64
> >   CC_FLAGS; this addresses a build issue reported by Liming
> > - add -Os the the linker flags as well, for AARCH64 this does not seem to
> make
> >   a difference, but it is arguably correct since the LTO processing at link
> >   time involves code generation as well
> > - add Laszlo's ack to #2
> > - new patch #6 to omit the autogenerated build-id from the PE/COFF binary
> >
> > Changes since v2:
> > - add license headers to LTO glue files for ARM and AARCH64 (#5)
> > - get rid of lto-ld-wrapper script
> >
> > Ard Biesheuvel (8):
> >   BaseTools CLANG35: drop problematic use-movt and save-temps options
> >   ArmVirtPkg/ArmVirtPrePiUniCoreRelocatable: ignore .hash and .note
> > sections
> >   BaseTools UNIXGCC ELFGCC CYGGCC: clone GCC build rule family into
> > GCCLD
> >   BaseTools GCC: use 'gcc' as the linker command for GCC44 and later
> >   ArmPkg: add prebuilt glue binaries for GCC5 LTO support
> >   BaseTools GCC: drop GNU notes section from EFI image
> >   MdePkg GCC/X64: avoid 'hidden' visibility for module entry points
> >   BaseTools GCC: introduce GCC5 toolchain to support GCC v5.x in LTO
> > mode
> >
> >  ArmPkg/GccLto/liblto-aarch64.a | 
> > Bin 0 -> 1016
> bytes
> >  ArmPkg/GccLto/liblto-aarch64.s |  
> > 27 ++
> >  ArmPkg/GccLto/liblto-arm.a | 
> > Bin 0 -> 2096 bytes
> >  ArmPkg/GccLto/liblto-arm.s |  
> > 61 
> >  ArmVirtPkg/PrePi/ArmVirtPre

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

> 
> CoffAddFixup() must be used for absolute symbol references only. These
> instructions contain relative symbol references, which are
> recalculated in WriteSections64().
> 
> The only absolute symbol reference is the GOT entry for 'n', and your
> code (in WriteRelocations64()) calculates the address of the GOT entry
> (which is always in .text BTW) and adds a fixup for it, i.e.,
> 
> +  CoffAddFixup(
> +(UINT32)(UINTN)((UINT64)
> mCoffSectionsOffset[RelShdr->sh_info] + GoTPcRelPtrOffset),
> +EFI_IMAGE_REL_BASED_DIR64);
> 
> This code adds a fixup to the PE/COFF .reloc section for the GOT entry
> containing the address of 'n', and the instructions perform a IP
> relative load of the contents of the GOT entry to retrieve the address
> of 'n'.
> 
> By adding two fixups, the PE/COFF loader will apply the load offset
> twice, resulting in an incorrect value.
> 
OK, I get your point now. Yes, the current patch could generate multiple fixups 
for the same GOT relocation entry. How about we introduce a simple 
IsDuplicatedCoffFixup() to check whether a converting fixup offset is 
duplicated before we use CoffAddFixup() to really add it? If it is new, we add 
it, otherwise just skip it.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v5 0/8] BaseTools: add support for GCC5 in LTO mode

[Shi, Steven]

Ard,
Where can I check out your v5 patch?


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Monday, August 01, 2016 4:01 PM
> To: Shi, Steven <steven@intel.com>; Zhu, Yonghong
> <yonghong@intel.com>; Gao, Liming <liming@intel.com>; Justen,
> Jordan L <jordan.l.jus...@intel.com>; edk2-devel@lists.01.org
> Cc: leif.lindh...@linaro.org; ler...@redhat.com; Ard Biesheuvel
> <ard.biesheu...@linaro.org>
> Subject: [PATCH v5 0/8] BaseTools: add support for GCC5 in LTO mode
> 
> This v5 to introduce GCC5 is now a 8 piece series, including some
> preparatory cleanup patches that allow all GCC4x and CLANG35 toolchains
> to switch to using 'gcc' as the linker. This allows us to get rid of
> the wrapper script to marshall ld arguments in order to make them
> understandable by gcc, which is fragile and likely to cause problems in
> the future.
> 
> Since there appears to be a natural split between the 'legacy' GCC
> toolchains UNIXGCC, ELFGCC, and CYGGCC[xASL], both in term of supported
> architectures [IA32, X64, IPF] vs [IA32, X64, ARM, AARCH64], and in
> terms of maintenance, these toolchains are not moved to using 'gcc' as
> the linker, and instead, a new BUILDRULEFAMILY is introduced called GCCLD
> that will retain the old behavior.
> 
> The result is that GCC5 can align much more closely with its predecessors,
> making the expected maintenance burden of supporting GCC back to v4.4
> much lower.
> 
> Changes since v4:
> - added patch to use 'protected' visibility only for the libraries that
>   define the module entry points (_ModuleEntryPoint), to prevent them from
>   being optimized away by the LTO routines
> - added Jordan's ack/RBs
> - add some extra comments to tools_def.template (#8)
> 
> Changes since v3:
> - like Steven does in his GCC5LTO patch, add -fno-builtin to IA32 and X64
>   CC_FLAGS; this addresses a build issue reported by Liming
> - add -Os the the linker flags as well, for AARCH64 this does not seem to
> make
>   a difference, but it is arguably correct since the LTO processing at link
>   time involves code generation as well
> - add Laszlo's ack to #2
> - new patch #6 to omit the autogenerated build-id from the PE/COFF binary
> 
> Changes since v2:
> - add license headers to LTO glue files for ARM and AARCH64 (#5)
> - get rid of lto-ld-wrapper script
> 
> Ard Biesheuvel (8):
>   BaseTools CLANG35: drop problematic use-movt and save-temps options
>   ArmVirtPkg/ArmVirtPrePiUniCoreRelocatable: ignore .hash and .note
> sections
>   BaseTools UNIXGCC ELFGCC CYGGCC: clone GCC build rule family into
> GCCLD
>   BaseTools GCC: use 'gcc' as the linker command for GCC44 and later
>   ArmPkg: add prebuilt glue binaries for GCC5 LTO support
>   BaseTools GCC: drop GNU notes section from EFI image
>   MdePkg GCC/X64: avoid 'hidden' visibility for module entry points
>   BaseTools GCC: introduce GCC5 toolchain to support GCC v5.x in LTO
> mode
> 
>  ArmPkg/GccLto/liblto-aarch64.a | Bin 
> 0 -> 1016 bytes
>  ArmPkg/GccLto/liblto-aarch64.s |  27 
> ++
>  ArmPkg/GccLto/liblto-arm.a | Bin 
> 0 -> 2096 bytes
>  ArmPkg/GccLto/liblto-arm.s |  61 
> 
>  ArmVirtPkg/PrePi/ArmVirtPrePiUniCoreRelocatable.inf|   2 
> +-
>  ArmVirtPkg/PrePi/Scripts/PrePi-PIE.lds |   3 
> +
>  BaseTools/Conf/build_rule.template |  31 
> +-
>  BaseTools/Conf/tools_def.template  | 350
> +++-
>  BaseTools/Scripts/GccBase.lds  |   6 
> +
>  EmulatorPkg/Unix/Host/Host.inf |   6 
> +-
>  MdePkg/Include/X64/ProcessorBind.h |   9 
> +-
>  MdePkg/Library/DxeCoreEntryPoint/DxeCoreEntryPoint.inf |   2 
> +
>  MdePkg/Library/PeiCoreEntryPoint/PeiCoreEntryPoint.inf |   2 
> +
>  MdePkg/Library/PeimEntryPoint/PeimEntryPoint.inf   |   2 
> +
>  MdePkg/Library/UefiApplicationEntryPoint/UefiApplicationEntryPoint.inf |
> 2 +
>  MdePkg/Library/UefiDriverEntryPoint/UefiDriverEntryPoint.inf   |   2 
> +
>  16 files changed, 396 insertions(+), 109 deletions(-)
>  create mode 100644 ArmPkg/GccLto/liblto-aarch64.a
>  create mode 100644 ArmPkg/GccLto/liblto-aarch64.s
>  create mode 100644 ArmPkg/GccLto/liblto-arm.a
>  create mode 100644 ArmPkg/GccLto/liblto-arm.s
> 
> --
> 2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

> On 1 August 2016 at 09:54, Shi, Steven <steven@intel.com> wrote:
> >> On 1 August 2016 at 09:19, Shi, Steven
> <steven@intel.com<mailto:steven@intel.com>> wrote:
> >> >> >>
> >> >> >> The fact that it works does not make it safe. Having multiple fixups
> >> >> >> for the same symbol in the .reloc section is a problem, and so is
> >> >> >> reapplying GOTPCRELX to places where the original instruction has
> >> been
> >> >> >> replaced by the linker.
> >> >> >>
> >> >> > [Steven]: I still don't understand why there will be multiple fixups 
> >> >> > for
> the
> >> >> same symbol in the .reloc section?
> >> >> >
> >> >>
> >> >> Remember this example
> >> >>
> >> >> >> > int n;
> >> >> >> > int f () { return n; }
> >> >> >> > int g () { return n; }
> >> >> >> > int h () { return n; }
> >> >>
> >> >> If every 'return n' results in a GOTPCREL relocation, how are you
> >> >> going to make sure that the GOT entry for 'n' is only fixed up a
> >> >> single time?
> >> >
> >> > [Steven]: the 'return n' will not result in relocation, but the 'int n' 
> >> > will
> result
> >> in the relocation in GOT. The three 'return n' will point to the same 'int 
> >> n'
> >> relocation item. So, we need only fixup 'int n' once, all three 'return n' 
> >> will
> >> use the correct global 'n' value.
> >>
> >> Every 'return n' will result in a GOTPCREL relocation against n. And
> >> your code emits a relocation for the GOT entry every time.
> >>
> > [Steven]: I don't think so. please give a real case and offer its source 
> > code
> to prove " Every 'return n' will result in a GOTPCREL relocation against n ".
> >
> 
> Compiling the code above using
> 
> gcc -c -O -fpic /tmp/pie.c -o pie.o
> 
> and dumping it using
> 
> readelf -r pie.o
> 
> gives me
> 
> Relocation section '.rela.text' at offset 0x250 contains 3 entries:
>   Offset  Info   Type   Sym. ValueSym. Name + 
> Addend
> 0003  000a002a R_X86_64_REX_GOTP 0004 n -
> 4
> 000d  000a002a R_X86_64_REX_GOTP 0004 n -
> 4
> 0017  000a002a R_X86_64_REX_GOTP 0004 n -
> 4
> ...
[Steven]: In this example, the pie.o is just the object file which is not 
linked. And if you link this pie.o file, the linker will solve all these three 
R_X86_64_REX_GOTP symbol with same 'int n' symbol address, and will create only 
one relocation item for 'int n' in the linked executable relocation section. 
So, we only need fixup once for the 'int n' in the linked executable.

Thanks
Steven
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

> On 1 August 2016 at 09:19, Shi, Steven 
> <steven@intel.com<mailto:steven@intel.com>> wrote:

> >> >>

> >> >> The fact that it works does not make it safe. Having multiple fixups

> >> >> for the same symbol in the .reloc section is a problem, and so is

> >> >> reapplying GOTPCRELX to places where the original instruction has

> been

> >> >> replaced by the linker.

> >> >>

> >> > [Steven]: I still don't understand why there will be multiple fixups for 
> >> > the

> >> same symbol in the .reloc section?

> >> >

> >>

> >> Remember this example

> >>

> >> >> > int n;

> >> >> > int f () { return n; }

> >> >> > int g () { return n; }

> >> >> > int h () { return n; }

> >>

> >> If every 'return n' results in a GOTPCREL relocation, how are you

> >> going to make sure that the GOT entry for 'n' is only fixed up a

> >> single time?

> >

> > [Steven]: the 'return n' will not result in relocation, but the 'int n' 
> > will result

> in the relocation in GOT. The three 'return n' will point to the same 'int n'

> relocation item. So, we need only fixup 'int n' once, all three 'return n' 
> will

> use the correct global 'n' value.

>

> Every 'return n' will result in a GOTPCREL relocation against n. And

> your code emits a relocation for the GOT entry every time.

>

[Steven]: I don't think so. please give a real case and offer its source code 
to prove " Every 'return n' will result in a GOTPCREL relocation against n ".



>  BTW, the 'int n' relocation type in your code on X64 should be

> R_X86_64_GLOB_DAT

> >

>

> R_X86_64_GLOB_DAT is a dynamic relocation type. These are only emitted

> when linking a shared object or a PIE executable, which I would like

> to avoid as well.

>

> The problem with PIE executables is that the .rela.xxx sections

> emitted for each section, and the single .rela section containing the

> dynamic relocations overlap with each other, i.e., places containing

> absolute symbol addresses in the binary will appear in both, and

> emitting reloc fixups for all relocations will result in duplicates.

>

[Steven]: the current GenFw will ignore the single .rela section, because the 
.rela section info is 0, which target invalid section at all. See below 
sections example. Current GenFw will only handle the .rela.text and .rela.data 
relocation sections.



Section Headers:

  [Nr] Name  Type Address   Offset

   Size  EntSize  Flags  Link  Info  Align

  [ 0]   NULL   

        0 0 0

  [ 1] .note.gnu.build-i NOTE   0100

   0024     A   0 0 4

  [ 2] .text PROGBITS 0240  0340

   59f8  0008  AX   0 0 64

  [ 3] .rela.textRELA   8728

   44d0  0018   I  10 2 8

  [ 4] .data PROGBITS 5c40  5d40

   0a68    WA   0 0 64

  [ 5] .rela.dataRELA   cbf8

   04b0  0018   I  10 4 8

  [ 6] .rela RELA 66c0  67c0

   04b0  0018   A   0 0 8



> > You can see the 'int myglob' in Eli's example in

> http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-<http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/>

> shared-libraries/<http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/>.
>  The 'int myglob' is same as your 'int n' example.

> >

> > int myglob = 42;

> > int ml_func(int a, int b)

> > {

> > return myglob + a + b;

> > }

> >

>

> Yes, and every reference to 'myglob' will result in a GOTPCREL

> relocation. We must not emit a fixup for myglob every time.

[Steven]: Please give a real case and offer its source code to prove "every 
reference to 'myglob' will result in a GOTPCREL relocation".


___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

> >>
> >> The fact that it works does not make it safe. Having multiple fixups
> >> for the same symbol in the .reloc section is a problem, and so is
> >> reapplying GOTPCRELX to places where the original instruction has been
> >> replaced by the linker.
> >>
> > [Steven]: I still don't understand why there will be multiple fixups for the
> same symbol in the .reloc section?
> >
> 
> Remember this example
> 
> >> > int n;
> >> > int f () { return n; }
> >> > int g () { return n; }
> >> > int h () { return n; }
> 
> If every 'return n' results in a GOTPCREL relocation, how are you
> going to make sure that the GOT entry for 'n' is only fixed up a
> single time?

[Steven]: the 'return n' will not result in relocation, but the 'int n' will 
result in the relocation in GOT. The three 'return n' will point to the same 
'int n' relocation item. So, we need only fixup 'int n' once, all three 'return 
n' will use the correct global 'n' value. BTW, the 'int n' relocation type in 
your code on X64 should be R_X86_64_GLOB_DAT

You can see the 'int myglob' in Eli's example in 
http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/.
 The 'int myglob' is same as your 'int n' example.

int myglob = 42;
int ml_func(int a, int b)
{
return myglob + a + b;
}



___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

> >>
> >> I am also concerned about the GOTPCRELX/REX_GOTPCRELX relocations.
> >> Reading the x86_64 ABI docs, it appears that these may refer to
> >> instructions that have been modified by the linker. In that case, how
> >> do we deal with the relocation? Also, according to the doc, mov
> >> instructions may be emitted by the linker in some cases that are only
> >> valid in the lowest 2 GB of the address space.
> >>
> > [Steven]: Frankly to say, the x86_64 ABI docs is only good for compiler
> domain developer and not very good for other domain developers to
> understand it.
> > My overall understanding for these different relocation type is like this:
> compiler generate PIC code with different "level of indirection to all global
> data and function references in the code." And these different level of
> indirection is implemented through GOT and PLT structure with different
> addressing calculation pattern. The different calculation patterns are the
> different relocation types which are defined  by  x86_64 ABI Table 4.9. We
> don't need worry about how compiler correctly generate code to work with
> these relocation types, we just need correctly understand their addressing
> calculation pattern.
> >
> > The GOTPCRELX/REX_GOTPCRELX has the same calculation definition in
> x86_64 ABI Table 4.9 as "G + GOT + A - P". So, I assume their difference is 
> not
> in the relocation calculation pattern, but how to co-work with specific
> instructions to finish these calculation in a hardware optimized way.
> >
> 
> No, that is not what these are for. The special types mark
> instructions that can be converted by the linker into simpler
> sequences if the symbol turns out to be in the same module. From the
> doc:
> 
> mov foo@GOTPCREL(%rip), %reg
> 
> could be converted by the linker into
> 
> lea foo(%rip), %reg
> 
> if the reference to 'foo' is satisfied by a non-preemptible local
> definition. This is a useful optimization, since it eliminates a
> memory load. The problem is that we cannot recalculate such
> relocations in GenFw without checking whether the linker has applied
> this optimization or not.
> 
[Steven]: Do you mean the linker will apply above optimization but not remove 
the original GOTPCREL item? It sounds like a severe linker bug.

> 
> The fact that it works does not make it safe. Having multiple fixups
> for the same symbol in the .reloc section is a problem, and so is
> reapplying GOTPCRELX to places where the original instruction has been
> replaced by the linker.
> 
[Steven]: I still don't understand why there will be multiple fixups for the 
same symbol in the .reloc section?



___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

> >
> > That was not my point. With your code, how many
> > EFI_IMAGE_REL_BASED_DIR64 fixups are added to the .reloc section for
> > the GOT entry of 'n'?
> >
> > int n;
> > int f () { return n; }
> > int g () { return n; }
> > int h () { return n; }

[Steven]: If the above global " n " need GOTPCREL type relocation. It should 
need only once EFI_IMAGE_REL_BASED_DIR64 fixups in my code.

> 
> I am also concerned about the GOTPCRELX/REX_GOTPCRELX relocations.
> Reading the x86_64 ABI docs, it appears that these may refer to
> instructions that have been modified by the linker. In that case, how
> do we deal with the relocation? Also, according to the doc, mov
> instructions may be emitted by the linker in some cases that are only
> valid in the lowest 2 GB of the address space.
> 
[Steven]: Frankly to say, the x86_64 ABI docs is only good for compiler domain 
developer and not very good for other domain developers to understand it. 
My overall understanding for these different relocation type is like this: 
compiler generate PIC code with different "level of indirection to all global 
data and function references in the code." And these different level of 
indirection is implemented through GOT and PLT structure with different 
addressing calculation pattern. The different calculation patterns are the 
different relocation types which are defined  by  x86_64 ABI Table 4.9. We 
don't need worry about how compiler correctly generate code to work with these 
relocation types, we just need correctly understand their addressing 
calculation pattern.

The GOTPCRELX/REX_GOTPCRELX has the same calculation definition in x86_64 ABI 
Table 4.9 as "G + GOT + A - P". So, I assume their difference is not in the 
relocation calculation pattern, but how to co-work with specific instructions 
to finish these calculation in a hardware optimized way.

One important thing worthy to mention that our gcc link script (GccBase.lds) 
has forced the GOT related sections , e.g. '.got', '.got.plt' , into .text 
section. So, all the GOT relocation fixup target .text section in fact.

I find below article help me a lot to understand some PIC simple relocation 
types. Hope it also can help you. I wish Eli, the author of below articles, 
could be invited as one author of x86_64 ABI spec, which will surely make the 
spec be more readable to other domain developers. :) 
Position Independent Code (PIC) in shared libraries 
http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/
Position Independent Code (PIC) in shared libraries on x64 
http://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64

>
> All in all, I think supporting GOT based relocations is a can of
> worms, and I would prefer to get rid of them completely if we can
> (i.e., using hidden visibility even for LTO, I have a fix for that I
> will sent out separately)
>
[Steven]: I agree we should try to avoid generating complex relocation type 
code for Uefi. But to make Uefi code build more portable to various version 
compilers and linker, I think it is still necessary to support these GOT based 
relocations. 
I've tested the GenFw new relocation types support with both GCC5/GCC49 (with 
default visibility) and CLANG38 in my branch in before. It can works. I suggest 
we could just keep these code there for reference.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

> OK. Could we do that in a separate patch?
[Steven]: Yes. We could separate it in a new patch.

> >> Is this necessary? I would expect the GOT entry itself to be already
> >> covered by a R_X86_64_64 relocation, so I don't think there is a need
> >> to emit a EFI_IMAGE_REL_BASED_DIR64 PE/COFF reloc here.
> > [Steven]: Do you ask whether it is still necessary to support the
> GOTPCREL/GOTPCRELX/REX_GOTPCRELX new relocation types? I think they
> are nice to have now, since new type support has no impact to old ones, we
> could just leave the code there for reference.
> 
> No. The question is whether it is necessary to emit the
> EFI_IMAGE_REL_BASED_DIR64 fixup reloc here. The relocation place is
> the instruction, not the GOT entry, and I would expect the GOT entry
> to be covered by a R_X86_64_64 relocation already
[Steven]: Yes, it is necessary. Even the R_X86_64_64 relocation need to emit 
the EFI_IMAGE_REL_BASED_DIR64 fixup reloc like below in the original GenFw 
code. You can test it like this, remove the below CoffAddFixup() code and build 
the OVMF with GCC5, you will see the Ovmf boot failure with cpu exception. 

case R_X86_64_64:
  VerboseMsg ("EFI_IMAGE_REL_BASED_DIR64 Offset: 0x%08X", 
mCoffSectionsOffset[RelShdr->sh_info] + (Rel->r_offset - 
SecShdr->sh_addr));
  CoffAddFixup(
(UINT32) ((UINT64) mCoffSectionsOffset[RelShdr->sh_info]
+ (Rel->r_offset - SecShdr->sh_addr)),
EFI_IMAGE_REL_BASED_DIR64);

Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [RFC PATCH] BaseTools GCC: add support for GCC/X64 and GCC/AARCH64 in LTO mode

[Shi, Steven]

> " Again, this is not an attempt to interfere with the great work Steven is 
> doing."

Hi Ard,
Go ahead, don't worry about me, I'm happy to see you cover the GCC5 work in 
your serial patch. I'm OK to base on your serial patches, not mine, to improve 
the GCC5 new toolchain in the future. We are all contributing the edk2 
community, and in fact, I should thank you to include IA32 and X64 in your 
continued work.



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Monday, July 18, 2016 1:26 PM
> To: edk2-devel@lists.01.org; ler...@redhat.com; af...@apple.com; Gao,
> Liming <liming@intel.com>; Shi, Steven <steven@intel.com>; Zhu,
> Yonghong <yonghong@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>; Justen, Jordan L <jordan.l.jus...@intel.com>
> Cc: Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: [RFC PATCH] BaseTools GCC: add support for GCC/X64 and
> GCC/AARCH64 in LTO mode
> 
> This introduces support for a new toolchain GCC5 which is identical to
> GCC49, except for the fact that it enables LTO for X64 and AARCH64.
> 
> In order to allow the same FAMILY and BUILDRULEFAMILY to be reused, a
> shell script is used that translates LD arguments to GCC arguments on
> the fly. I.e., most -xxx arguments are translated to their -Wl,xxx
> pass throught equivalent, and some other arguments are interpreted
> by the script directly.
> 
> Using the same FAMILY and BUILDRULEFAMILY is important, since
> many .DSC
> and .INF files contain [BuildOptions] sections that set DLINK flags for,
> e.g., all GCC/X64 targets, and so using GCC as the linker instead of LD
> (which LTO requires) would either require a complete new toolchain FAMILY
> (which would make the GCC5 unavailable without changes to many packages,
> including out of tree ones), or a new BUILDRULEFAMILY, which has similar
> problems.
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Ard Biesheuvel <ard.biesheu...@linaro.org>
> ---
> 
> Again, this is not an attempt to interfere with the great work Steven is
> doing. I am simply concerned that adding a new FAMILY or
> BUILDRULEFAMILY
> for GCC in LTO mode further fragments the support for GCC, which is
> already difficult to maintain. For example, packages that set BuildOptions
> only for the GCC/LTO FAMILY could not be built with GCC4x without changes.
> 
> The use of a shell script is a hack, and to support this approach for GCC
> on Windows, we should probably change it to a shell script. We may even
> add BaseTools support for converting a LD command line string to a GCC
> one directly. The point of this RFC is to illustrate is that such translation
> is feasible, and avoids the pitfall of having to introduce a new FAMILY or
> BUILDRULEFAMILY
> 
> I have included LTO support for AARCH64 as well to illustrate that the
> same approach works there too. Note that the resulting build is BROKEN,
> i.e., LTO for AARCH64 does not actually work at the moment, but this is
> under investigation.
> 
>  BaseTools/Conf/tools_def.template   | 156 
>  BaseTools/Scripts/lto-ld-wrapper.sh |  26 
>  2 files changed, 182 insertions(+)
> 
> diff --git a/BaseTools/Conf/tools_def.template
> b/BaseTools/Conf/tools_def.template
> index f9b26fad44de..647450c1c232 100644
> --- a/BaseTools/Conf/tools_def.template
> +++ b/BaseTools/Conf/tools_def.template
> @@ -197,6 +197,9 @@ DEFINE GCC48_X64_PREFIX= ENV(GCC48_BIN)
>  DEFINE GCC49_IA32_PREFIX   = ENV(GCC49_BIN)
>  DEFINE GCC49_X64_PREFIX= ENV(GCC49_BIN)
> 
> +DEFINE GCC5_IA32_PREFIX= ENV(GCC5_BIN)
> +DEFINE GCC5_X64_PREFIX = ENV(GCC5_BIN)
> +
>  DEFINE UNIX_IASL_BIN   = ENV(IASL_PREFIX)iasl
>  DEFINE WIN_ASL_BIN_DIR = C:\ASL
>  DEFINE WIN_IASL_BIN= DEF(WIN_ASL_BIN_DIR)\iasl.exe
> @@ -4450,6 +4453,27 @@ DEFINE GCC49_AARCH64_DLINK2_FLAGS=
> DEF(GCC48_AARCH64_DLINK2_FLAGS)
>  DEFINE GCC49_ARM_ASLDLINK_FLAGS  =
> DEF(GCC48_ARM_ASLDLINK_FLAGS)
>  DEFINE GCC49_AARCH64_ASLDLINK_FLAGS  =
> DEF(GCC48_AARCH64_ASLDLINK_FLAGS)
> 
> +DEFINE GCC5_IA32_CC_FLAGS   = DEF(GCC49_IA32_CC_FLAGS)
> +DEFINE GCC5_X64_CC_FLAGS= DEF(GCC49_X64_CC_FLAGS)
> +DEFINE GCC5_IA32_X64_DLINK_COMMON   =
> DEF(GCC49_IA32_X64_DLINK_COMMON)
> +DEFINE GCC5_IA32_X64_ASLDLINK_FLAGS =
> DEF(GCC49_IA32_X64_ASLDLINK_FLAGS)
> +DEFINE GCC5_IA32_X64_DLINK_FLAGS=
> DEF(GCC49_IA32_X64_DLINK_FLAGS)
> +DEFINE GCC5_IA32_DLINK2_FLAGS   = DEF(GCC49_IA32_DLINK2_FLAGS)
> +DEFINE GCC5_X64_DLINK_FLAGS = D

Re: [edk2] [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building with -fpic

[Shi, Steven]

You clone and can try the GCC LTO build on my below branch, and welcome 
directly debug on it.
https://github.com/shijunjing/edk2/tree/llvm_v2



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Saturday, July 16, 2016 10:48 PM
> To: Shi, Steven <steven@intel.com>
> Cc: Zhu, Yonghong <yonghong@intel.com>; Gao, Liming
> <liming@intel.com>; Kinney, Michael D <michael.d.kin...@intel.com>;
> Justen, Jordan L <jordan.l.jus...@intel.com>; ler...@redhat.com;
> af...@apple.com; edk2-devel@lists.01.org
> Subject: Re: [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building
> with -fpic
> 
> On 16 July 2016 at 15:54, Shi, Steven <steven@intel.com> wrote:
> > Hi Ard,
> > Sorry for my late response. Setting all symbols visibility as hidden will 
> > block
> LTO on GCC. GCC LTO need symbols to be public to let linker freely do the
> link time optimization across local to external. Below is the GCC5 build
> failure in my side with your patch.
> >
> 
> Do you have any references that support this? Hidden visibility has
> nothing to do with linkage. It only controls the symbols that are
> exported from a shared library. Knowing whether a symbol is ever
> exported from a shared library allows the compiler to generate better
> code.
> 
> 
> > "gcc" -o /home/jshi19/edk2-
> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain
> /DEBUG/PeiCore.dll -Os -flto -nostdlib -Wl,-n -Wl,-q -Wl,--gc-sections -z
> common-page-size=0x40 -Wl,--entry,_ModuleEntryPoint -Wl,-
> u,_ModuleEntryPoint -Wl,-Map,/home/jshi19/edk2-
> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain
> /DEBUG/PeiCore.map -mcmodel=small -Wl,-melf_x86_64 -Wl,--
> oformat=elf64-x86-64 -Wl,--start-group,,@/home/jshi19/edk2-
> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain
> /OUTPUT/static_library_files.lst -Wl,--end-group -Wl,--
> defsym=PECOFF_HEADER_SIZE=0x228 -Wl,--script=/home/jshi19/edk2-
> fork3/BaseTools/Scripts/GccBase.lds
> > "objcopy"  /home/jshi19/edk2-
> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain
> /DEBUG/PeiCore.dll
> > objcopy: error: the input file '/home/jshi19/edk2-
> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain
> /DEBUG/PeiCore.dll' has no sections
> 
> This looks like an issue with --gc-sections. Could you check the map
> file why everything gets discarded?
> 
> > make: *** [/home/jshi19/edk2-
> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain
> /DEBUG/PeiCore.dll] Error 1
> > GNUmakefile:478: recipe for target '/home/jshi19/edk2-
> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain
> /DEBUG/PeiCore.dll' failed
> >
> >
> > Steven Shi
> > Intel\SSG\STO\UEFI Firmware
> >
> > Tel: +86 021-61166522
> > iNet: 821-6522
> >
> >> -Original Message-
> >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> >> Sent: Thursday, July 14, 2016 9:16 PM
> >> To: Shi, Steven <steven@intel.com>; Zhu, Yonghong
> >> <yonghong@intel.com>; Gao, Liming <liming@intel.com>;
> Kinney,
> >> Michael D <michael.d.kin...@intel.com>; Justen, Jordan L
> >> <jordan.l.jus...@intel.com>; ler...@redhat.com; af...@apple.com;
> edk2-
> >> de...@lists.01.org
> >> Cc: Ard Biesheuvel <ard.biesheu...@linaro.org>
> >> Subject: [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building
> with
> >> -fpic
> >>
> >> When building position independent (PIC) ELF objects, the GCC compiler
> >> assumes that each symbol with external linkage may potentially end up
> >> being exported from a shared library, which means that each of those
> >> symbols may be subject to symbol preemption, i.e., the executable
> >> linking to the shared library at runtime may override symbols exported
> >> by the shared library, and every internal reference held by the shared
> >> library itself *must* be made to point to the overridden version instead.
> >>
> >> For this reason, PIC code symbol references always go via the Global
> >> Offset Table (GOT), even if the code in question references symbols that
> >> are defined in the same compilation unit. The GOT refers to each symbol
> >> by absolute address, and so each entry is subject to runtime relocation.
> >>
> >> Since not every symbol with external linkage is ultimately exported from
> >> a shared libr

Re: [edk2] [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section type

[Shi, Steven]

If PIE can generate smaller code than PIC, why not use PIE? Why “We should only 
enable it if we must” ?


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
Sent: Saturday, July 16, 2016 10:45 PM
To: Shi, Steven <steven@intel.com>
Cc: Andrew Fish <af...@apple.com>; edk2-devel-01 <edk2-devel@lists.01.org>; 
Gao, Liming <liming@intel.com>; Justen, Jordan L 
<jordan.l.jus...@intel.com>; Kinney, Michael D <michael.d.kin...@intel.com>
Subject: Re: [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section 
type

On 16 July 2016 at 15:54, Shi, Steven 
<steven@intel.com<mailto:steven@intel.com>> wrote:

Hi Ard,

Sorry for my late response. For the PIC and PIE selection, my personal opinion 
is to prefer the PIE rather than PIC, because PIE usually has even smaller code 
size than PIC, although their difference is small (especially if enable LTO). 
Below are some example data in my side. Also, like Andrew mentioned, PIE is 
more secure than PIC, and more and more users use PIE rather than PIC to 
generate the code.

A PIE executable is built from PIC (*) objects, and simply ​contains a .rela 
section and a _DYNAMIC entry point that allows a runtime linker to load it at 
arbitrary offsets in memory. We already have this 'security' in EDK2 because we 
use relocatable PE/COFF executables, so PIE does not bring us anything in terms 
of features.

(*) AARCH64 allows PIE executables to be built from non-PIC objects when using 
the small model

From a code generation point of view, they are identical, and only differ in 
the linking step.







I use your gcc-x64-opt and my llvm_v2 branches to collect the GCC49 and GCC5 
data

Code size,

Byte,

the smaller the  better


GCC49 with hidden visibility

(PIE = PIC)


GCC49 with public visibility + my patch for new PIC/PIE relocation types 
support in GenFw

(PIC > PIE)


GCC5 LTO

(PIC > PIE)


Shell X64 DEBUG PIC


1,138,048


5,148,104


899,136


Shell X64 DEBUG PIE


1,138,048


5,130,464


899,072


CpuDxe X64 DEBUG PIC


67,456


68,608


36,032


CpuDxe X64 DEBUG PIE


67,456


67,456


35,968


UsbKbDxe X64 DEBUG PIC


46,784


47,296


26,752


UsbKbDxe X64 DEBUG PIE


46,784


46,784


26,688

​

​You are changing several parameters at the same time, comparing non-PIE/hidden 
with PIE/default visibility.

PIE is not an optimization. PIE is an executable format that can be understood 
by an ELF runtime linker, ​and so enabling PIE is not a goal in itself. We 
should only enable it if we must.

--
Ard.


___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building with -fpic

[Shi, Steven]

Hi Ard,
Sorry for my late response. Setting all symbols visibility as hidden will block 
LTO on GCC. GCC LTO need symbols to be public to let linker freely do the link 
time optimization across local to external. Below is the GCC5 build failure in 
my side with your patch. 

"gcc" -o 
/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll
 -Os -flto -nostdlib -Wl,-n -Wl,-q -Wl,--gc-sections -z common-page-size=0x40 
-Wl,--entry,_ModuleEntryPoint -Wl,-u,_ModuleEntryPoint 
-Wl,-Map,/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.map
 -mcmodel=small -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 
-Wl,--start-group,,@/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/OUTPUT/static_library_files.lst
 -Wl,--end-group -Wl,--defsym=PECOFF_HEADER_SIZE=0x228 
-Wl,--script=/home/jshi19/edk2-fork3/BaseTools/Scripts/GccBase.lds
"objcopy"  
/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll
objcopy: error: the input file 
'/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll'
 has no sections
make: *** 
[/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll]
 Error 1
GNUmakefile:478: recipe for target 
'/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll'
 failed


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Thursday, July 14, 2016 9:16 PM
> To: Shi, Steven <steven@intel.com>; Zhu, Yonghong
> <yonghong@intel.com>; Gao, Liming <liming@intel.com>; Kinney,
> Michael D <michael.d.kin...@intel.com>; Justen, Jordan L
> <jordan.l.jus...@intel.com>; ler...@redhat.com; af...@apple.com; edk2-
> de...@lists.01.org
> Cc: Ard Biesheuvel <ard.biesheu...@linaro.org>
> Subject: [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building with
> -fpic
> 
> When building position independent (PIC) ELF objects, the GCC compiler
> assumes that each symbol with external linkage may potentially end up
> being exported from a shared library, which means that each of those
> symbols may be subject to symbol preemption, i.e., the executable
> linking to the shared library at runtime may override symbols exported
> by the shared library, and every internal reference held by the shared
> library itself *must* be made to point to the overridden version instead.
> 
> For this reason, PIC code symbol references always go via the Global
> Offset Table (GOT), even if the code in question references symbols that
> are defined in the same compilation unit. The GOT refers to each symbol
> by absolute address, and so each entry is subject to runtime relocation.
> 
> Since not every symbol with external linkage is ultimately exported from
> a shared library, the GCC compiler allows control over symbol visibility
> using attributes, command line arguments and pragmas, where 'hidden'
> means that the symbol is only referenced by the shared library itself.
> Due to the poor hygiene in EDK2 regarding the use of the 'static'
> modifier, many symbols that are local to their compilation unit end up
> being referenced indirectly via the GOT when building PIC code.
> 
> In UEFI, there are no shared libraries and so there is no need to deal
> with symbol preemption, and we can mark every symbol reference 'hidden'.
> The only method that applies to all symbol definitions as well as
> declarations is the #pragma. So set the visibility 'hidden' pragma when
> building PIC code for X64 using GCC.
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Ard Biesheuvel <ard.biesheu...@linaro.org>
> ---
>  MdePkg/Include/X64/ProcessorBind.h | 10 ++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/MdePkg/Include/X64/ProcessorBind.h
> b/MdePkg/Include/X64/ProcessorBind.h
> index 705104af062a..96df78fca07d 100644
> --- a/MdePkg/Include/X64/ProcessorBind.h
> +++ b/MdePkg/Include/X64/ProcessorBind.h
> @@ -27,6 +27,16 @@
>  #pragma pack()
>  #endif
> 
> +#if defined(__GNUC__) && defined(__pic__)
> +//
> +// Mark all symbol declarations and references as hidden, meaning they will
> not
> +// be exported from a shared library, and thus will not be subject to symbol
> +// preemption. This allows the compiler to refer to symbols directly using
> +// relative references rather than via the GOT, which contains absolute
> symbol
> +// addresses that are subject to runtime relocation.
> +//
> +#pragma GCC visibility push (hidden)
> +#endif
> 
>  #if defined(__INTEL_COMPILER)
>  //
> --
> 2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section type

[Shi, Steven]

Hi Ard,

Sorry for my late response. For the PIC and PIE selection, my personal opinion 
is to prefer the PIE rather than PIC, because PIE usually has even smaller code 
size than PIC, although their difference is small (especially if enable LTO). 
Below are some example data in my side. Also, like Andrew mentioned, PIE is 
more secure than PIC, and more and more users use PIE rather than PIC to 
generate the code.





I use your gcc-x64-opt and my llvm_v2 branches to collect the GCC49 and GCC5 
data

Code size,

Byte,

the smaller the  better


GCC49 with hidden visibility

(PIE = PIC)


GCC49 with public visibility + my patch for new PIC/PIE relocation types 
support in GenFw

(PIC > PIE)


GCC5 LTO

(PIC > PIE)


Shell X64 DEBUG PIC


1,138,048


5,148,104


899,136


Shell X64 DEBUG PIE


1,138,048


5,130,464


899,072


CpuDxe X64 DEBUG PIC


67,456


68,608


36,032


CpuDxe X64 DEBUG PIE


67,456


67,456


35,968


UsbKbDxe X64 DEBUG PIC


46,784


47,296


26,752


UsbKbDxe X64 DEBUG PIE


46,784


46,784


26,688




The PIE is based on PIC and can further optimize the local global variables and 
functions . Below is an example for IA32 PIE and PIC difference.

http://stackoverflow.com/questions/2463150/fpie-position-independent-executable-option-gcc-ld

http://www.openbsd.org/papers/nycbsdcon08-pie/mgp4.html









Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522





> -Original Message-

> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]

> Sent: Wednesday, July 13, 2016 11:06 PM

> To: Andrew Fish <af...@apple.com>

> Cc: Shi, Steven <steven@intel.com>; edk2-devel-01  de...@lists.01.org>; Gao, Liming <liming@intel.com>; Justen, Jordan L

> <jordan.l.jus...@intel.com>; Kinney, Michael D

> <michael.d.kin...@intel.com>

> Subject: Re: [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf

> section type

>

> On 12 July 2016 at 19:33, Andrew Fish <af...@apple.com> wrote:

> >

> >> On Jul 12, 2016, at 9:55 AM, Ard Biesheuvel <ard.biesheu...@linaro.org>

> wrote:

> [...]

> >> OK, so in summary, the AMD64 small model not only limits the relative

> >> range but also the absolute placement of the code? This is different

> >> from AArch64, where the tiny and small models only limit the range of

> >> symbol references, but the code can still execute anywhere (and

> >> relative jumps have a range that is +/- 128 MB, regardless of the code

> >> model)

> >>

> >> So I understand why you need the build the objects using -fpic. But

> >> why are you linking a PIE executable? Is that required? The only thing

> >> that it buys you is the redundant RELA section with the SHF_ALLOC

> >> attribute set, and its contents are redundant in the presence of

> >> -q/--emit-relocs.

> >>

> >> A quick test build of DxeCore.efi in RELEASE mode with GCC48 shows

> >> that linking small model PIC objects without PIE already results in a

> >> significant size reduction (182 KB -> 160 KB) I don't have any newer

> >> X64 GCC versions installed, so I did not try any of those. (Note that

> >> this requires your patch to add support for PIC-specific ELF

> >> relocations to GenFw)

> >>

> >

> > Ard,

> >

> > I'm not sure if this is related, but the Xcode/clang linker for x86_64 only

> supports PIE. This is only an issue when making .S (.nasm) files common as

> some times they will fail to link with Xcode and we have to change code to

> use RIP relative addressing.

> >

> > I remember Steven was having a code generation issue with clang and

> turning on PIE fixed it. We also need the PIE support for the macOS native

> tools, but they generate Mach-O and do not use GenFw.

> >

> > So at this point I think PIE is required for clang. I guess Steven could

> investigate using other models, but I'm guessing that is not a common usage

> case with clang as PIE is required for macOS and also from ASLR. Thus it

> might be broken, or it might not be obvious how to make clang and the linker

> happy.

> >

>

> Hello Andrew,

>

> As I responded in reply to another patch in this series, I think it

> makes sense to separate the various changes, and assess each on merit

> and/or necessity in isolation. The MS ABI va_list change appears to

> allow -O2 optimization to be used all the way back to GCC 4.7, and

> this does not require any other changes, so this should be a separate

> patch. Using small model PIC code could be enabled on top of that, and

> if the numbers look good, we could also enable that all the way back

> to GCC 4.7. If we need PIE linking, but only for LLVM, I propose we

> only enable it for Clang/llvm.

>

> Regards,

> Ard.
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 0/5] MdePkg BaseTools: GCC optimization for X64

[Shi, Steven]

Hi Ard,
Sorry for my late response. Thank you help to sync the improvement to other GCC 
X64 toolchains, especial for the MS VA intrinsics one. I once mentioned I could 
sync them as below, and I'm glad you already done it. :)

http://thread.gmane.org/gmane.comp.bios.edk2.devel/13751/focus=14015 


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Saturday, July 16, 2016 6:16 AM
> To: edk2-devel@lists.01.org; ler...@redhat.com; af...@apple.com; Gao,
> Liming <liming....@intel.com>; Shi, Steven <steven@intel.com>; Zhu,
> Yonghong <yonghong@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>; Justen, Jordan L <jordan.l.jus...@intel.com>
> Cc: br...@cran.org.uk; pbonz...@redhat.com; Ard Biesheuvel
> <ard.biesheu...@linaro.org>
> Subject: [PATCH v2 0/5] MdePkg BaseTools: GCC optimization for X64
> 
> This is v2 of the series to enable compiler optimization under GCC for builds
> targetting X64. It includes a patch from Steven Shi, taken from his series to
> introduce support for GCC 5.x and Clang 3.8/X64 to EDK2.
> 
> The first patch fixes the issue that __builtin_unreachable() is not
> implemented by GCC 4.4 or earlier.
> 
> Patch #2 is Steven's patch to switch to the flavor of VA_LIST that is
> explicitly modeled after the MS implementation. This by itself is an
> improvement, since the open coded implementation that performs
> arithmetic
> on the address of explicit arguments to obtain the variadic arguments is
> fragile and difficult to maintain, and should be best avoided.
> 
> Patch #3 enables -Os optimization for X64/RELEASE and X86/DEBUG all the
> way
> back to GCC44. This aligns X64 with IA32, which is already built using -Os
> for both DEBUG and RELEASE.
> 
> Note that the reported breakage regarding the use of -Os [0] is likely caused
> by the poor man's __builtin_ms_va_list (NO_BUILTIN_VA_FUNCS) that we
> used for
> GCC/X64 before Steven's patch above switches us to a properly defined
> builtin
> type. (The poor man's implementation uses pointer arithmetic involving the
> address of the last named argument to calculate pointers to the variadic
> arguments and dereferences them to obtain the argument values. I'm quite
> surprised it works as well as it does at -O0, to be honest)
> 
> I tested this change with both Ovmf and EmulatorPkg built in various ways
> and with various versions, with the caveat that I did not always use a
> matching
> binutils (i.e., of the same era). Since the issues this series deal with are
> all code generation issues, I think this is reasonable, but more testing would
> be appreciated. (v2: this still applies, I have tested DEBUG and RELEASE 
> builds
> using GCC44, GCC46, GCC47 and GCC48, but all on the same Ubuntu 14.04
> system,
> which means they all use the same ld version 2.24.90)
> 
> [0] http://thread.gmane.org/gmane.comp.bios.tianocore.devel/10963
> 
> Patch #4 applies the visibility 'hidden' GCC pragma globally. Please refer to
> the commit log for the motivation.
> 
> Patch #5 switches GCC/X64 to the PIC small code model, which results in
> smaller
> code.
> 
> Changes in v2:
> - Switch to -Os optimization, both for DEBUG and RELEASE builds. This turns
> out
> to work fine (as far as I could establish) and gives the best results in terms
> of code size. The following numbers were produced with the following
> invocation
> 
> build -a X64 -t GCC44 -t GCC46 -t GCC47 -t GCC48 -p
> OvmfPkg/OvmfPkgX64.dsc \
>   -D SECURE_BOOT_ENABLE  -b DEBUG -b RELEASE
> 
> DEBUG_GCC44
> 
> SECFV [14%Full] 212992 total, 31632 used, 181360 free
> FVMAIN_COMPACT [74%Full] 1753088 total, 1300904 used, 452184 free
> DXEFV [50%Full] 10485760 total, 5272464 used, 5213296 free
> PEIFV [25%Full] 917504 total, 231496 used, 686008 free
> 
> DEBUG_GCC46
> 
> SECFV [14%Full] 212992 total, 31728 used, 181264 free
> FVMAIN_COMPACT [73%Full] 1753088 total, 1291672 used, 461416 free
> DXEFV [50%Full] 10485760 total, 5263888 used, 5221872 free
> PEIFV [25%Full] 917504 total, 231656 used, 685848 free
> 
> DEBUG_GCC47
> 
> SECFV [14%Full] 212992 total, 31440 used, 181552 free
> FVMAIN_COMPACT [73%Full] 1753088 total, 1290584 used, 462504 free
> DXEFV [49%Full] 10485760 total, 5234224 used, 5251536 free
> PEIFV [25%Full] 917504 total, 230152 used, 687352 free
> 
> DEBUG_GCC48
> 
> SECFV [14%Full] 212992 total, 31280 used, 181712 free
> FVMAIN_COMPACT [73%Full] 1753088 total, 1296208 used, 456880 free
> DXEFV [49%Full] 10485760 total, 5229424 used, 5256336 free
> PEIFV [25%Full] 917504 total, 229512 used, 687992 free
> 
&g

Re: [edk2] [PATCH 1/7] BaseTools: Enable three new toolchains

[Shi, Steven]

Hi Chevalier,
Sorry, your email was blocked to me and I just see it now. Yes, I should don't 
the x86_64-pc-linux-gnu target for IA32. Thank you for the patch to correct it.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ronny Chevalier [mailto:chevalier.ro...@gmail.com]
> Sent: Thursday, July 07, 2016 6:15 PM
> To: Shi, Steven <steven@intel.com>
> Cc: edk2-devel@lists.01.org; Gao, Liming <liming@intel.com>; Kinney,
> Michael D <michael.d.kin...@intel.com>; af...@apple.com
> Subject: Re: [edk2] [PATCH 1/7] BaseTools: Enable three new toolchains
> 
> On Tue, Jun 28, 2016 at 4:18 PM, Shi, Steven <steven@intel.com> wrote:
> > +DEFINE CLANG38_ALL_CC_FLAGS = -c -fshort-wchar -fno-strict-aliasing
> -Wall -Werror -Wno-array-bounds -Wno-empty-body -ffunction-sections -
> fdata-sections -include AutoGen.h -
> DSTRING_ARRAY_NAME=$(BASE_NAME)Strings -fno-stack-protector -fno-
> builtin -mms-bitfields -Wno-address -Wno-shift-negative-value -Wno-
> parentheses-equality -Wno-unknown-pragmas -Wno-tautological-constant-
> out-of-range-compare -Wno-incompatible-library-redeclaration -target
> x86_64-pc-linux-gnu -fno-asynchronous-unwind-tables -mno-sse -mno-mmx
> -msoft-float -mno-implicit-float  -ftrap-
> function=undefined_behavior_has_been_optimized_away_by_clang -
> funsigned-char -fno-ms-extensions -fno-common -Wno-null-dereference -
> Wno-tautological-compare
> > +
> 
> By default you add the x86_64-pc-linux-gnu target for all
> architectures, which is wrong.
> 
> > +DEFINE CLANG38_ALL_DLINK_FLAGS  = -flto -nostdlib -Wl,-n -Wl,-q -Wl,--
> gc-sections -z common-page-size=0x40 -Wl,--entry,$(IMAGE_ENTRY_POINT)
> -Wl,-u,$(IMAGE_ENTRY_POINT) -Wl,-
> Map,$(DEST_DIR_DEBUG)/$(BASE_NAME).map
> > +DEFINE CLANG38_IA32_DLINK2_FLAGS = -Wl,--
> defsym=PECOFF_HEADER_SIZE=0x220 -Wl,--
> script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
> > +DEFINE CLANG38_X64_DLINK2_FLAGS  = -Wl,--
> defsym=PECOFF_HEADER_SIZE=0x228 -Wl,--
> script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
> > +
> > +###
> > +# CLANG38 IA32 definitions
> > +###
> > +*_CLANG38_IA32_OBJCOPY_PATH =
> DEF(GCC49_IA32_PREFIX)objcopy
> > +*_CLANG38_IA32_CC_PATH  = DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_SLINK_PATH   = DEF(CLANG38_IA32_PREFIX)llvm-
> ar
> > +*_CLANG38_IA32_DLINK_PATH   = DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_ASLDLINK_PATH=
> DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_ASM_PATH = DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_PP_PATH  = DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_VFRPP_PATH   = DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_ASLCC_PATH   = DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_ASLPP_PATH   = DEF(CLANG38_IA32_PREFIX)clang
> > +*_CLANG38_IA32_RC_PATH  = DEF(GCC49_IA32_PREFIX)objcopy
> > +
> > +*_CLANG38_IA32_ASLCC_FLAGS  = -x c -Os -m32 -target x86_64-pc-
> linux-gnu -flto
> 
> And you used the wrong target again here.
> 
> Maybe you could use a definition, like for the CLANG35 target, and add
> DEF(CLANG38_IA32_TARGET) where appropriate.
> 
> I attached a patch (on top of your current patch) that does this to
> better explain what I am saying.
> 
> What do you think?
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section type

[Shi, Steven]

If the cost of the large model is due to its use of 64-bit quantities, how is 
this any better with PIC/PIE binaries? The primary difference is that you now 
have a Global Offset Table (GOT) entry for all symbol references, which also 
contains 64-bit values. Do you have any numbers comparing the large model to 
PIC/PIE? You should also be aware that GCC's default visibility setting results 
in all symbol references to go via the GOT, which may not be what you want. 
(This is required for ELF symbol preemption, which does not make sense in a PIE 
executable, but GCC does not know about that when it creates the .o from the 
.c). Also, do you have any comparisons for the size of the .reloc section?

[Steven] As I mentioned in previous email, it is the small code size model, not 
PIC/PIE, to make the code size smaller.
“It is worthy to mention that PIE/PIC can increase code size in fact, because 
more code need to make the addressing position independent. But PIE/PIC can 
guarantee the small code model is safe in 64bits anywhere, and the small code 
can decrease the 64bits code size sharply. So the PIE+Small code overall can 
generate smaller code size than large code. A tricky scenario is that if you 
can make sure your 64bits code only run under 4GB address, you can disable PIE 
but only enable small code model, then you will get even smaller 64bits binary.”

Current GCC49 is large code model. In my last patch, I have some code size data 
to compare them as blow:
GCC49: large code model
CLANG38 and GCC 5: small code model + PIE


Module (KB)  VS2015x86 |   GCC49   | CLANG38 | GCC5

---

RELEASE X64 DxeCore.efi  95.3KB|   183KB| 101KB   | 91.2KB

DEBUG X64 DxeCore.efi 138KB |   273KB| 138KB   | 137KB




Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
Sent: Monday, July 11, 2016 10:54 PM
To: Shi, Steven <steven@intel.com>
Cc: af...@apple.com; edk2-devel-01 <edk2-devel@lists.01.org>; Gao, Liming 
<liming@intel.com>; Justen, Jordan L <jordan.l.jus...@intel.com>; Kinney, 
Michael D <michael.d.kin...@intel.com>
Subject: Re: [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section 
type

On 11 July 2016 at 16:01, Shi, Steven 
<steven@intel.com<mailto:steven@intel.com>> wrote:
Hi Ard,
>
> What I am still missing here is an explanation of the
> benefit/necessity of using PIE. What does it give us that we don't
> already have? How is it different from simply linking a shared ELF
> object and running GenFw on that?
>
[Steven]:
Using PIE is necessary, at least for x86_64, because it can guarantee the 
64bits small code model (-mcmodel=small) code can really safe to run at 
anywhere in 64bits address range. If no PIE, the small code binary will have 
trouble when run above 4GB address. Uefi firmware (especially in server 
platform) have the cases to run code above 4GB address, so I have to enable PIE 
with small code model.
Let me give you a real example, at the beginning of GCC5 and CLANG38 enabling, 
I didn't enable the PIE in fact, then I meet a weird issue on OVMF X64 firmware 
(Qemu), which is the OVMF 64bits Sec can jmp to wrong address. I attach my 
debugging email, and in that email, you can see it is because the OVMF 64bits 
Sec (small code) run at 0xfffc which is very high address above 
4GB. If you like, you can also see how I discuss this issue in llvm community 
in the thread: How to debug if LTO generate wrong 
code?(http://lists.llvm.org/pipermail/llvm-dev/2016-May/100298.html)

For the PIE/PIC and code model relation, below article can help you understand 
it:
http://eli.thegreenplace.net/2012/01/03/understanding-the-x64-code-models

​If the cost of the large model is due to its use of 64-bit quantities, how is 
this any better with PIC/PIE binaries? The primary difference is that you now 
have a Global Offset Table (GOT) entry for all symbol references, which also 
contains 64-bit values. Do you have any numbers comparing the large model to 
PIC/PIE? You should also be aware that GCC's default visibility setting results 
in all symbol references to go via the GOT, which may not be what you want. 
(This is required for ELF symbol preemption, which does not make sense in a PIE 
executable, but GCC does not know about that when it creates the .o from the 
.c). Also, do you have any comparisons for the size of the .reloc section?
​

OK, then someone might continue to ask why we need small code model and not to 
use large code model to guarantee the 64bits address safe? I see both 
GCC49_AARCH64 and CLANG35_AARCH64 already enable the small code, so I believe 
you know the answer: the large code model is VERY expensive for 64bits 
addressing in code size view, and every 64bits code and data 

Re: [edk2] [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section type

[Shi, Steven]

Hi Ard,

>

> What I am still missing here is an explanation of the

> benefit/necessity of using PIE. What does it give us that we don't

> already have? How is it different from simply linking a shared ELF

> object and running GenFw on that?

>

[Steven]:

Using PIE is necessary, at least for x86_64, because it can guarantee the 
64bits small code model (-mcmodel=small) code can really safe to run at 
anywhere in 64bits address range. If no PIE, the small code binary will have 
trouble when run above 4GB address. Uefi firmware (especially in server 
platform) have the cases to run code above 4GB address, so I have to enable PIE 
with small code model.

Let me give you a real example, at the beginning of GCC5 and CLANG38 enabling, 
I didn't enable the PIE in fact, then I meet a weird issue on OVMF X64 firmware 
(Qemu), which is the OVMF 64bits Sec can jmp to wrong address. I attach my 
debugging email, and in that email, you can see it is because the OVMF 64bits 
Sec (small code) run at 0xfffc which is very high address above 
4GB. If you like, you can also see how I discuss this issue in llvm community 
in the thread: How to debug if LTO generate wrong 
code?(http://lists.llvm.org/pipermail/llvm-dev/2016-May/100298.html)



For the PIE/PIC and code model relation, below article can help you understand 
it:

http://eli.thegreenplace.net/2012/01/03/understanding-the-x64-code-models



OK, then someone might continue to ask why we need small code model and not to 
use large code model to guarantee the 64bits address safe? I see both 
GCC49_AARCH64 and CLANG35_AARCH64 already enable the small code, so I believe 
you know the answer: the large code model is VERY expensive for 64bits 
addressing in code size view, and every 64bits code and data address cost full 
8 bytes in large code model. So we prefer to use small code model for firmware 
binary.



It is worthy to mention that PIE/PIC can increase code size in fact, because 
more code need to make the addressing position independent. But PIE/PIC can 
guarantee the small code model is safe in 64bits anywhere, and the small code 
can decrease the 64bits code size sharply. So the PIE+Small code overall can 
generate smaller code size than large code. A tricky scenario is that if you 
can make sure your 64bits code only run under 4GB address, you can disable PIE 
but only enable small code model, then you will get even smaller 64bits binary.



> OK, so how about we put the following line in GccBase.lds

>

>   .rela : { *(.rel .rela)

>

> and check for SHT_PROGBITS sections explicitly in both IsTextShdr and

> IsDataShdr (or at least filter out SHT_RELA sections)?

>

[Steven]: it sounds good to me. Current GccBase.lds already has the .rela 
define as below. Will use the section Type in IsTextShdr and IsDataShdr to 
check SHT_RELA.



  .rela ALIGN(CONSTANT(COMMONPAGESIZE)) : {

*(.rela .rela.*)

  }



>

> BTW I wonder if you still need --emit-relocs in the PIE case, since

> all relocations you need will be in the SHF_ALLOC .rela section, and

> the per-section .rela sections (which GenFw normally uses) are

> redundant.

>

[Steven]: Yes. I use -q (--emit-relocs) in current GCC5 and CLANG38 link flags. 
My understanding is the --emit-relocs is to let the whole binary fully linked, 
and it is mandatory for Uefi binary.





>

> I don't think we need to add anything, and checking for SHT_RELA (as I

> suggested above) should be sufficient. However, could you please keep

> the ELF32 and ELF64 in sync?

>

[Steven]: OK. Will sync the ELF32 with ELF64.

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section type

[Shi, Steven]

Hi Ard,

This patch can improve 1~40+ KB code size for x86_64 uefi module binary mostly 
on GCC5 and CLANG38 toolchains, the bigger binary the more improvement. Below 
is some code size data.






With this patch (Byte)


Without this patch (Byte)


GCC5 RELEASE X64 Shell.efi


826,752


872,704


GCC5 RELEASE X64 DxeCore.efi


95,104


101,952


GCC5 RELEASE X64 ConSplitterDxe.efi


19,456


21,632


CLANG38 RELEASE X64 Shell.efi


910,464


956,672


CLANG38 RELEASE X64 DxeCore.efi


103,744


109,760


CLANG38 RELEASE X64 ConSplitterDxe.efi


21,376


23,552






This patch is necessary because I've enabled the PIE (position independent 
code) in GCC5 and CLANG38 . You know PIE code will generate bigger .rela 
section which flags is SHF_ALLOC only (you can see the UsbKbDxe.dll example in 
my previous email ) .  After convert Elf to PE/Coff, we don't need these .rela 
sections in PE/Coff image any longer. But current  IsTextShdr function as below 
will see it as Text section and still keep it in final PE/Coff image. So, I 
correct it to add SHF_EXECINSTR attribute flag in Text section check logic to 
filter .rela sections.

IsTextShdr (

 Elf_Shdr *Shdr

 )

{

 return (BOOLEAN) ((Shdr->sh_flags & (SHF_WRITE | SHF_ALLOC)) == SHF_ALLOC);

}



BTW, if we hope enable PIC/PIE for GCCx_AARCH64, CLANGx_AARCH64 or RVCT in the 
future, I guess we will see the same problem.



For RVCT .rodate section support, I believe it is more reasonable to use 
SHF_EXECINSTR attribute to check Text section, rather than to design a work 
around in  IsTextShdr() to implicit include .rodata section, so

1.   Since GCCx_AARCH64, CLANGx_AARCH64 already depends on GccBase.lds, how 
about we sync RVCT Rvct-Align32.sct script with GccBase.lds section layout as 
well? This will make everything simple and easy.

2.   If you really believe it is necessary to keep RVCT script different 
from GCC and CLANG, how about we add a IsRoDataShdr() function in GenFw to 
explicit check .rodata section?







Hi Andrew,

I see the Xcode toolchain also enalbe the -PIE as default. Could you help to 
check whether this patch can improve the Xcode code size as well in your side? 
Just build the release X64 Shell.efi, DxeCore.efi modules are enough, Thank you!

https://github.com/shijunjing/edk2/commit/6884c05884d3ce75029646ca4b1cf93a06214643



And please also enhance the GccBase.lds as below, I forgot add it into above 
commit. (Sometimes, ld could use .rdata besides .rodata as the read only data 
section name. )

diff --git a/BaseTools/Scripts/GccBase.lds b/BaseTools/Scripts/GccBase.lds

old mode 100644

new mode 100755

index 32310bc..6ab45a2

--- a/BaseTools/Scripts/GccBase.lds

+++ b/BaseTools/Scripts/GccBase.lds

@@ -27,7 +27,7 @@ SECTIONS {



   .text : ALIGN(CONSTANT(COMMONPAGESIZE)) {

 *(.text .text.* .stub .gnu.linkonce.t.*)

-*(.rodata .rodata.* .gnu.linkonce.r.*)

+*(.rodata .rodata.* .rdata .gnu.linkonce.r.*)

 *(.got .got.*)



 /*





Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522



> -Original Message-

> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]

> Sent: Friday, July 08, 2016 9:52 PM

> To: Shi, Steven <steven@intel.com>

> Cc: edk2-devel-01 <edk2-devel@lists.01.org>; Gao, Liming

> <liming@intel.com>; af...@apple.com; Justen, Jordan L

> <jordan.l.jus...@intel.com>; Kinney, Michael D

> <michael.d.kin...@intel.com>

> Subject: Re: [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf

> section type

>

> On 8 July 2016 at 15:48, Shi, Steven 
> <steven@intel.com<mailto:steven@intel.com>> wrote:

> > Hi Ard,

> > You know, edk2 enforce the Elf section generation by link script

> \BaseTools\Scripts\GccBase.lds. In the below part of GccBase.lds, you can

> see edk2 has force linker to put .rodata into .text section. This is why I 
> hope

> to add SHF_EXECINSTR flag in the section check logic.

> >

>

> But why? How does that improve the current code?  And how can you be

> sure that nobody is using GenFw with a different linker? For example,

> RVCT uses GenFw, but does not use GccBase.lds.

>

> Could we not just drop this patch, please?

>

>

>

> >   .text : ALIGN(CONSTANT(COMMONPAGESIZE)) {

> > *(.text .text.* .stub .gnu.linkonce.t.*)

> > *(.rodata .rodata.* .gnu.linkonce.r.*)

> > *(.got .got.*)

> >

> > /*

> >  * The contents of AutoGen.c files are constant from the POV of the

> program,

> >  * but most of its contents end up in .data or .bss by default since 
> > few of

> >  * the variable definitions that get emitted are declared as CONST.

> >  */

> > *:AutoGen.obj(.data .data.* .bss .bss.*)

> >   }

> >

> > Below is a e

Re: [edk2] [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section type

[Shi, Steven]

Hi Ard,
You know, edk2 enforce the Elf section generation by link script 
\BaseTools\Scripts\GccBase.lds. In the below part of GccBase.lds, you can see 
edk2 has force linker to put .rodata into .text section. This is why I hope to 
add SHF_EXECINSTR flag in the section check logic.

  .text : ALIGN(CONSTANT(COMMONPAGESIZE)) {
*(.text .text.* .stub .gnu.linkonce.t.*)
*(.rodata .rodata.* .gnu.linkonce.r.*)
*(.got .got.*)

/*
 * The contents of AutoGen.c files are constant from the POV of the program,
 * but most of its contents end up in .data or .bss by default since few of
 * the variable definitions that get emitted are declared as CONST.
 */
*:AutoGen.obj(.data .data.* .bss .bss.*)
  }

Below is a edk2 Elf image typical example, and you can see there is no .rodata 
section, but a big .text section.
> readelf -S 
> /home/jshi19/edk2-fork/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe/DEBUG/UsbKbDxe.dll

Section Headers:
  [Nr] Name  Type Address   Offset
   Size  EntSize  Flags  Link  Info  Align
  [ 0]   NULL   
        0 0 0
  [ 1] .text PROGBITS 0240  00c0
   5a24  0008  AX   0 0 64
  [ 2] .rela.textRELA   8498
   44d0  0018   I   8 1 8
  [ 3] .data PROGBITS 5c80  5b00
   0a68    WA   0 0 64
  [ 4] .rela.dataRELA   c968
   04b0  0018   I   8 3 8
  [ 5] .rela RELA 6700  6580
   04b0  0018   A   0 0 8
  [ 6] .gnu_debuglinkPROGBITS   6a30
   0014     0 0 1
  [ 7] .shstrtab STRTAB     ce18
   0046     0 0 1
  [ 8] .symtab   SYMTAB     6a48
   11a0  0018   9   104 8
  [ 9] .strtab   STRTAB     7be8
   08ab     0 0 1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)


Anyway, I agree that it's better to enhance the Elf64Convert.c code comments to 
highlight that the GenFw tool has the big assumption of linking through 
GccBase.lds. What do you think?


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Friday, July 08, 2016 5:19 PM
> To: Shi, Steven <steven@intel.com>
> Cc: edk2-devel-01 <edk2-devel@lists.01.org>; Gao, Liming
> <liming@intel.com>; af...@apple.com; Justen, Jordan L
> <jordan.l.jus...@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>
> Subject: Re: [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf
> section type
> 
> On 8 July 2016 at 10:42, Shi, Steven <steven@intel.com> wrote:
> > Add SHF_EXECINSTR attribute flag in logic to check Elf Text and
> > Data section type.
> 
> Why?
> 
> > Also fix some typo in the code comments and
> > Debug strings.
> >
> > Contributed-under: TianoCore Contribution Agreement 1.0
> > Signed-off-by: Steven Shi <steven@intel.com>
> > ---
> >  BaseTools/Source/C/GenFw/Elf64Convert.c | 10 +-
> >  1 file changed, 5 insertions(+), 5 deletions(-)
> >  mode change 100644 => 100755
> BaseTools/Source/C/GenFw/Elf64Convert.c
> >
> > diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c
> b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > old mode 100644
> > new mode 100755
> > index 024a2a0..7c838f3
> > --- a/BaseTools/Source/C/GenFw/Elf64Convert.c
> > +++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > @@ -75,7 +75,7 @@ CleanUp64 (
> >);
> >
> >  //
> > -// Rename ELF32 strucutres to common names to help when porting to
> ELF64.
> > +// Rename ELF32 structures to common names to help when porting to
> ELF64.
> >  //
> >  typedef Elf64_Shdr Elf_Shdr;
> >  typedef Elf64_Ehdr Elf_Ehdr;
> > @@ -233,7 +233,7 @@ IsTextShdr (
> >Elf_Shdr *Shdr
> >)
> >  {
> > -  return (BOOLEAN) ((Shdr->sh_flags & (SHF_WRITE | 

[edk2] [PATCH v2 3/7] MdePkg: Enable new MS VA intrinsics for GNUC x86 64bits build

[Shi, Steven]

Both GCC and LLVM 3.8 64bits support new variable argument (VA)
intrinsics for Microsoft ABI, enable these new VA intrinsics for
GNUC family 64bits code build. These VA intrinsics are only
permitted use in 64bits code, so not use them in 32bits code build.
The original 32bits GNU VA intrinsics has the same calling conversion
as MS, so we don’t need change them.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 MdePkg/Include/Base.h | 27 +--
 1 file changed, 25 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 MdePkg/Include/Base.h

diff --git a/MdePkg/Include/Base.h b/MdePkg/Include/Base.h
old mode 100644
new mode 100755
index cbd9e55..5129b64
--- a/MdePkg/Include/Base.h
+++ b/MdePkg/Include/Base.h
@@ -588,9 +588,32 @@ struct _LIST_ENTRY {
 
 #define VA_COPY(Dest, Start)  __va_copy (Dest, Start)
 
-#elif defined(__GNUC__) && !defined(NO_BUILTIN_VA_FUNCS)
+
+#elif defined(__GNUC__) && !defined(NO_BUILTIN_VA_FUNCS) && !defined 
(MDE_CPU_IA32)
+//
+// 64bits build only. Use GCC built-in macros for variable argument lists.
+//
+///
+/// Both GCC and LLVM 3.8 64bits support new variable argument intrinsics for 
Microsoft ABI
+///
+
+///
+/// Variable used to traverse the list of arguments. This type can vary by
+/// implementation and could be an array or structure.
+///
+typedef __builtin_ms_va_list VA_LIST;
+
+#define VA_START(Marker, Parameter)  __builtin_ms_va_start (Marker, Parameter)
+
+#define VA_ARG(Marker, TYPE) ((sizeof (TYPE) < sizeof (UINTN)) ? 
(TYPE)(__builtin_va_arg (Marker, UINTN)) : (TYPE)(__builtin_va_arg (Marker, 
TYPE)))
+
+#define VA_END(Marker)   __builtin_ms_va_end (Marker)
+
+#define VA_COPY(Dest, Start) __builtin_ms_va_copy (Dest, Start)
+
+#elif defined(__GNUC__) && !defined(NO_BUILTIN_VA_FUNCS) && defined 
(MDE_CPU_IA32)
 //
-// Use GCC built-in macros for variable argument lists.
+// 32bits build only. Use GCC built-in macros for variable argument lists.
 //
 
 ///
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v2 0/7] Introduce three new toolchains in edk2

[Shi, Steven]

Please review my new commits in public branch:
https://github.com/shijunjing/edk2/commits/llvm_v2

I did below enhancements according feedback to 
V1(https://github.com/shijunjing/edk2/commits/review5):
1. Remove the Quark platform work around and code clean work from this serial 
patches.
2. Seperate three toolchains to three different patches.
3. Seperate GenFw update general part from x86 specific part and enhance the 
PIE/PIC commit log.
4. Not set -D EFI32, but use MDE_CPU_IA32 instead.


Shi, Steven (7):
  BaseTools-GenFw:Use EXE flag to check Elf section type
  BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code
  MdePkg: Enable new MS VA intrinsics for GNUC x86 64bits build
  BaseTools-Conf:Introduce GCC5 new toolchain for x86
  BaseTools-Conf:Introduce CLANG38 new toolchain for x86
  BaseTools-Conf:Introduce CLANGSCAN38 new toolchain for x86
  ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

 BaseTools/Conf/build_rule.template  |  17 +-
 BaseTools/Conf/tools_def.template   | 267 
 BaseTools/Source/C/GenFw/Elf64Convert.c | 115 --
 BaseTools/Source/C/GenFw/elf_common.h   |   9 +-
 MdePkg/Include/Base.h   |  27 ++-
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c |   2 +-
 6 files changed, 419 insertions(+), 18 deletions(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/Elf64Convert.c
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/elf_common.h
 mode change 100644 => 100755 MdePkg/Include/Base.h
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v2 4/7] BaseTools-Conf:Introduce GCC5 new toolchain for x86

[Shi, Steven]

GCC5 enable GCC Link Time Optimization (LTO) and code size
optimization (–Os) for aggressive code size improvement.
GCC5 X64 code is small code model + position independent
code (PIE).

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc) and
Quark (Quark.dsc).
Test compiler and linker version: GCC 5.3, GCC 5.4, GNU ld 2.26.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/build_rule.template |  9 
 BaseTools/Conf/tools_def.template  | 92 ++
 2 files changed, 101 insertions(+)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template

diff --git a/BaseTools/Conf/build_rule.template 
b/BaseTools/Conf/build_rule.template
old mode 100644
new mode 100755
index 91bcc18..25cf380
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -295,6 +295,10 @@
 "$(DLINK)" -o ${dst} $(DLINK_FLAGS) --start-group $(DLINK_SPATH) 
@$(STATIC_LIBRARY_FILES_LIST) --end-group $(DLINK2_FLAGS)
 "$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
 
+
+"$(DLINK)" -o ${dst} $(DLINK_FLAGS) 
-Wl,--start-group,$(DLINK_SPATH),@$(STATIC_LIBRARY_FILES_LIST) -Wl,--end-group 
$(DLINK2_FLAGS)
+"$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
+
 
 "$(DLINK)" $(DLINK_FLAGS) -o ${dst} $(DLINK_SPATH) --via 
$(STATIC_LIBRARY_FILES_LIST) $(DLINK2_FLAGS)
 
@@ -448,6 +452,11 @@
 "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
 "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(GENFW_FLAGS)
 
+
+"$(ASLCC)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASLCC_FLAGS) 
$(INC) ${src}
+"$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
+"$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(GENFW_FLAGS)
+
 
 "$(ASLCC)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj  $(ASLCC_FLAGS) 
$(INC) ${src}
 "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
old mode 100644
new mode 100755
index 2065fa3..e41aad14
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -366,6 +366,12 @@ DEFINE SOURCERY_CYGWIN_TOOLS = /cygdrive/c/Program 
Files/CodeSourcery/Sourcery G
 # Required to build platforms or ACPI tables:
 #   Intel(r) ACPI Compiler from
 #   https://acpica.org/downloads
+#   GCC5-Linux-  Requires:
+# GCC 5.x targeting x86_64-linux-gnu
+#Optional:
+# Required to build platforms or ACPI tables:
+#   Intel(r) ACPI Compiler from
+#   https://acpica.org/downloads
 #   CLANG35 -Linux,Windows-  Requires:
 # Clang v3.5 or later, and GNU binutils targeting 
aarch64-linux-gnu or arm-linux-gnueabi
 #Optional:
@@ -7410,3 +7416,89 @@ RELEASE_RVCTCYGWIN_ARM_CC_FLAGS  = "$(CCPATH_FLAG)" 
$(ARCHCC_FLAGS) $(PLATFORM_F
 # Build rule order
 #
 *_*_*_*_BUILDRULEORDER = nasm asm Asm ASM S s
+
+##
+#
+# GCC 5.x - This configuration is used to compile under Linux to produce
+# PE/COFF binaries using GCC 5.x with link-time optimization and image size 
optimization.
+# Tested with GNU Binutils 2.26, e.g. ld 2.26 and gcc-ar 2.26
+#
+##
+DEFINE GCC5_IA32_PREFIX  = ENV(GCC5_BIN)
+DEFINE GCC5_X64_PREFIX   = ENV(GCC5_BIN)
+
+*_GCC5_*_*_FAMILY= GCC
+*_GCC5_*_*_BUILDRULEFAMILY   = GCC5
+*_GCC5_*_MAKE_PATH   = DEF(GCC5_IA32_PREFIX)make
+*_GCC5_*_*_DLL   = ENV(GCC5_DLL)
+*_GCC5_*_ASL_PATH= DEF(UNIX_IASL_BIN)
+
+*_GCC5_*_PP_FLAGS= DEF(GCC_PP_FLAGS)
+*_GCC5_*_ASLPP_FLAGS = DEF(GCC_ASLPP_FLAGS)
+*_GCC5_*_ASLCC_FLAGS = DEF(GCC_ASLCC_FLAGS)
+*_GCC5_*_VFRPP_FLAGS = DEF(GCC_VFRPP_FLAGS)
+*_GCC5_*_APP_FLAGS   =
+*_GCC5_*_ASL_FLAGS   = DEF(IASL_FLAGS)
+*_GCC5_*_ASL_OUTFLAGS= DEF(IASL_OUTFLAGS)
+
+DEFINE GCC5_ALL_CC_FLAGS = -fshort-wchar -fno-strict-aliasing -Wall 
-Werror -Wno-array-bounds -Wno-address  -ffunction-sections -fdata-sections 
-fno-stack-protector -

[edk2] [PATCH v2 2/7] BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code

[Shi, Steven]

Add support to convert new Elf relocation types
(R_X86_64_PLT32, R_X86_64_GOTPCREL, R_X86_64_GOTPCRELX,
R_X86_64_REX_GOTPCRELX) to PeCoff, which are required by
position independent code (PIC) built Elf image.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Source/C/GenFw/Elf64Convert.c | 105 +---
 BaseTools/Source/C/GenFw/elf_common.h   |   9 ++-
 2 files changed, 105 insertions(+), 9 deletions(-)
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/elf_common.h

diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c 
b/BaseTools/Source/C/GenFw/Elf64Convert.c
index 7c838f3..fc241ab 100755
--- a/BaseTools/Source/C/GenFw/Elf64Convert.c
+++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
@@ -755,6 +755,11 @@ WriteSections64 (
 // Determine how to handle each relocation type based on the machine 
type.
 //
 if (mEhdr->e_machine == EM_X86_64) {
+  //
+  // See Relocation Types details semantics in Table 4.9 of
+  // SystemV x86_64 abi Draft Version 0.99.8
+  // 
https://github.com/hjl-tools/x86-psABI/wiki/x86-64-psABI-secure.pdf
+  //
   switch (ELF_R_TYPE(Rel->r_info)) {
   case R_X86_64_NONE:
 break;
@@ -763,24 +768,24 @@ WriteSections64 (
 // Absolute relocation.
 //
 VerboseMsg ("R_X86_64_64");
-VerboseMsg ("Offset: 0x%08X, Addend: 0x%016LX", 
-  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)), 
+VerboseMsg ("Offset: 0x%08X, Addend: 0x%016LX",
+  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
   *(UINT64 *)Targ);
 *(UINT64 *)Targ = *(UINT64 *)Targ - SymShdr->sh_addr + 
mCoffSectionsOffset[Sym->st_shndx];
 VerboseMsg ("Relocation:  0x%016LX", *(UINT64*)Targ);
 break;
   case R_X86_64_32:
 VerboseMsg ("R_X86_64_32");
-VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X", 
-  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)), 
+VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X",
+  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
   *(UINT32 *)Targ);
 *(UINT32 *)Targ = (UINT32)((UINT64)(*(UINT32 *)Targ) - 
SymShdr->sh_addr + mCoffSectionsOffset[Sym->st_shndx]);
 VerboseMsg ("Relocation:  0x%08X", *(UINT32*)Targ);
 break;
   case R_X86_64_32S:
 VerboseMsg ("R_X86_64_32S");
-VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X", 
-  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)), 
+VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X",
+  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
   *(UINT32 *)Targ);
 *(INT32 *)Targ = (INT32)((INT64)(*(INT32 *)Targ) - 
SymShdr->sh_addr + mCoffSectionsOffset[Sym->st_shndx]);
 VerboseMsg ("Relocation:  0x%08X", *(UINT32*)Targ);
@@ -790,14 +795,45 @@ WriteSections64 (
 // Relative relocation: Symbol - Ip + Addend
 //
 VerboseMsg ("R_X86_64_PC32");
-VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X", 
-  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)), 
+VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X",
+  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
   *(UINT32 *)Targ);
 *(UINT32 *)Targ = (UINT32) (*(UINT32 *)Targ
   + (mCoffSectionsOffset[Sym->st_shndx] - SymShdr->sh_addr)
   - (SecOffset - SecShdr->sh_addr));
 VerboseMsg ("Relocation:  0x%08X", *(UINT32 *)Targ);
 break;
+  case R_X86_64_PLT32:
+//
+// Relative relocation: L + A - P
+//
+VerboseMsg ("R_X86_64_PLT32");
+VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X",
+  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
+  *(UINT32 *)Targ);
+*(UINT32 *)Targ = (UINT32) (*(UINT32 *)Targ
+  + (mCoffSectionsOffset[Sym->st_shndx] - SymShdr->sh_addr)
+  - (SecOffset - SecShdr->sh_addr));
+VerboseMsg ("Relocation:  0x%08X", *(UINT32 *)Targ);
+break;
+  case R_X86_64_GOTPCREL:
+//
+// Relative relocation: G + GOT + A - P
+//
+VerboseMsg ("R_X86_64_GOTPCREL");
+break;
+  case R_X86_64_GOTPCRELX:
+//
+// Relative relocation: G + GOT + A - P
+//
+  

[edk2] [PATCH v2 7/7] ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

[Shi, Steven]

Use explicit CopyMem to replace compiler builtin to do the structure
values assignment.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
Reviewed-by: Jaben Carsey <jaben.car...@intel.com>

---
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

diff --git a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c 
b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
old mode 100644
new mode 100755
index 666ee9d..5c50797
--- a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
+++ b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
@@ -342,7 +342,7 @@ ShellCommandRunTftp (
 goto Error;
   }
 
-  Mtftp4ConfigData = DefaultMtftp4ConfigData;
+  CopyMem (, , sizeof 
(EFI_MTFTP4_CONFIG_DATA));
 
   //
   // Check the host IPv4 address
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH v2 5/7] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Shi, Steven]

LANG38 uses Clang3.8.0, enable LLVM Link Time Optimization (LTO)
and code size optimization flag (-Oz) by default for aggressive code
size improvement. CLANG38 X64 code is small code model + PIE.

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc).
Test compiler and linker version: LLVM 3.8, GNU ld 2.26.

Example steps to use the CLANG38 tool chain to build OVMF platform:
1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
extract it as ~/clang38).
2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so, this step is needed only for CLANG38)
3. Install new version linker with plugin support (e.g. ld 2.26 in
GNU Binutils 2.26 or Ubuntu16.04, this step is needed only for CLANG38)
$ cd edk2
$ git checkout llvm
$ export CLANG38_BIN=path/to/your/clang38/
(e.g. export CLANG38_BIN=~/clang38/bin/)
$ source edksetup.sh
$ make -C BaseTools/Source/C
$ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
-DDEBUG_ON_SERIAL_PORT
$ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
$ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
 -hda fat:.

If you want, you can build and install GNU Binutils 2.26 as below steps
in Ubuntu:
Download binutils-2.26 source code from http://ftp.gnu.org/gnu/binutils/
 and extract it to ~/binutils-2.26
$sudo apt-get install bison
$sudo apt-get install flex
Install other necessary binutils build tools if missing
$ mkdir build
$ cd build
$ ../binutils-2.26/configure --enable-gold --enable-plugins
--disable-werror --prefix=/usr
$ make -j 5
$ sudo make install

If you want, you can build LLVMgold.so as below steps
Download llvm-3.8.0 source code from http://www.llvm.org/releases/
3.8.0/llvm-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src
Download clang3.8.0 source code from http://www.llvm.org/releases/
3.8.0/cfe-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src/tools/clang
Refer http://clang.llvm.org/get_started.html to Install other necessary
clang build tools if missing
$ mkdir llvm38build
$ cd llvm38build
If your GNU Binutils 2.26 is in /home/jshi19/binutils-2.26,
$ cmake ../llvm-3.8.0.src -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release"
-DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_C_COMPILER="/usr/bin/gcc"
-DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
$ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/build_rule.template |  7 ++-
 BaseTools/Conf/tools_def.template  | 87 ++
 2 files changed, 92 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Conf/build_rule.template 
b/BaseTools/Conf/build_rule.template
index 25cf380..4530d92 100755
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -267,7 +267,10 @@
 
 
 "$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
-
+
+
+"$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
+
 
 "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
 
@@ -295,7 +298,7 @@
 "$(DLINK)" -o ${dst} $(DLINK_FLAGS) --start-group $(DLINK_SPATH) 
@$(STATIC_LIBRARY_FILES_LIST) --end-group $(DLINK2_FLAGS)
 "$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
 
-
+<Command.GCC5, Command.CLANG38>
 "$(DLINK)" -o ${dst} $(DLINK_FLAGS) 
-Wl,--start-group,$(DLINK_SPATH),@$(STATIC_LIBRARY_FILES_LIST) -Wl,--end-group 
$(DLINK2_FLAGS)
 "$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
 
diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
index e41aad14..092a393 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -378,6 +378,12 @@ DEFINE SOURCERY_CYGWIN_TOOLS = /cygdrive/c/Program 
Files/CodeSourcery/Sourcery G
 # Required to build platforms or ACPI tables:
 #   Intel(r) ACPI Compiler from
 #   https://acpica.org/downloads
+#   CLANG38  -Linux-  Requires:
+# Clang v3.8 or later, LLVMgold plugin and GNU 
binutils 2.26 targeting x86_64-linux-gnu
+#Optional:
+# Required to build platforms or ACPI tables:
+#   Intel(r) ACPI Compiler from
+#   https://acpica.org/downloads
 #   ELFGCC  -Linux-  Requires:
 # GCC(this tool chain uses whatever version of gcc 
and binutils that is installed in /usr/bin)
 #  

[edk2] [PATCH v2 1/7] BaseTools-GenFw:Use EXE flag to check Elf section type

[Shi, Steven]

Add SHF_EXECINSTR attribute flag in logic to check Elf Text and
Data section type. Also fix some typo in the code comments and
Debug strings.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Source/C/GenFw/Elf64Convert.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/Elf64Convert.c

diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c 
b/BaseTools/Source/C/GenFw/Elf64Convert.c
old mode 100644
new mode 100755
index 024a2a0..7c838f3
--- a/BaseTools/Source/C/GenFw/Elf64Convert.c
+++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
@@ -75,7 +75,7 @@ CleanUp64 (
   );
 
 //
-// Rename ELF32 strucutres to common names to help when porting to ELF64.
+// Rename ELF32 structures to common names to help when porting to ELF64.
 //
 typedef Elf64_Shdr Elf_Shdr;
 typedef Elf64_Ehdr Elf_Ehdr;
@@ -233,7 +233,7 @@ IsTextShdr (
   Elf_Shdr *Shdr
   )
 {
-  return (BOOLEAN) ((Shdr->sh_flags & (SHF_WRITE | SHF_ALLOC)) == SHF_ALLOC);
+  return (BOOLEAN) ((Shdr->sh_flags & (SHF_EXECINSTR | SHF_WRITE | SHF_ALLOC)) 
== (SHF_EXECINSTR | SHF_ALLOC));
 }
 
 STATIC
@@ -256,7 +256,7 @@ IsDataShdr (
   if (IsHiiRsrcShdr(Shdr)) {
 return FALSE;
   }
-  return (BOOLEAN) (Shdr->sh_flags & (SHF_WRITE | SHF_ALLOC)) == (SHF_ALLOC | 
SHF_WRITE);
+  return (BOOLEAN) (Shdr->sh_flags & (SHF_EXECINSTR | SHF_WRITE | SHF_ALLOC)) 
== (SHF_ALLOC | SHF_WRITE);
 }
 
 STATIC
@@ -661,9 +661,9 @@ WriteSections64 (
 
   default:
 //
-//  Ignore for unkown section type.
+//  Ignore for unknown section type.
 //
-VerboseMsg ("%s unknown section type %x. We directly copy this section 
into Coff file", mInImageName, (unsigned)Shdr->sh_type);
+VerboseMsg ("%s unknown section type %x. We directly ignore this 
section and NOT copy into Coff file", mInImageName, (unsigned)Shdr->sh_type);
 break;
   }
 }
-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH V2 0/7] Introduce three new toolchains in edk2

[Shi, Steven]

Hi Ard,
I've put v2 in the subject and change the branch name to 
https://github.com/shijunjing/edk2/commits/llvm_v2 . You can review all my 
serial patches in the branch link, and let me know if you need me send the 
patches in the mail list again.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Shi,
> Steven
> Sent: Friday, July 08, 2016 10:32 AM
> To: edk2-devel@lists.01.org; ard.biesheu...@linaro.org
> Cc: Kinney, Michael D <michael.d.kin...@intel.com>; Justen, Jordan L
> <jordan.l.jus...@intel.com>; af...@apple.com; Gao, Liming
> <liming@intel.com>
> Subject: [edk2] [PATCH V2 0/7] Introduce three new toolchains in edk2
> 
> Please review my new commits in public branch:
> https://github.com/shijunjing/edk2/commits/llvm_v2
> 
> I did below enhancements according feedback to
> V1(https://github.com/shijunjing/edk2/commits/review5):
> 1. Remove the Quark platform work around and code clean work from this
> serial patches.
> 2. Seperate three toolchains to three different patches.
> 3. Seperate GenFw update general part from x86 specific part and enhance
> the PIE/PIC commit log.
> 4. Not set -D EFI32, but use MDE_CPU_IA32 instead.
> 
> 
> Shi, Steven (7):
>   BaseTools-GenFw:Use EXE flag to check Elf section type
>   BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code
>   MdePkg: Enable new MS VA intrinsics for GNUC x86 64bits build
>   BaseTools-Conf:Introduce GCC5 new toolchain for x86
>   BaseTools-Conf:Introduce CLANG38 new toolchain for x86
>   BaseTools-Conf:Introduce CLANGSCAN38 new toolchain for x86
>   ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin
> 
>  BaseTools/Conf/build_rule.template  |  17 +-
>  BaseTools/Conf/tools_def.template   | 267
> 
>  BaseTools/Source/C/GenFw/Elf64Convert.c | 115 --
>  BaseTools/Source/C/GenFw/elf_common.h   |   9 +-
>  MdePkg/Include/Base.h   |  27 ++-
>  ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c |   2 +-
>  6 files changed, 419 insertions(+), 18 deletions(-)
>  mode change 100644 => 100755 BaseTools/Conf/build_rule.template
>  mode change 100644 => 100755 BaseTools/Conf/tools_def.template
>  mode change 100644 => 100755
> BaseTools/Source/C/GenFw/Elf64Convert.c
>  mode change 100644 => 100755
> BaseTools/Source/C/GenFw/elf_common.h
>  mode change 100644 => 100755 MdePkg/Include/Base.h
>  mode change 100644 => 100755
> ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
> 
> --
> 2.7.4
> 
> ___
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH V2 0/7] Introduce three new toolchains in edk2

[Shi, Steven]

Please review my new commits in public branch:
https://github.com/shijunjing/edk2/commits/llvm_v2

I did below enhancements according feedback to 
V1(https://github.com/shijunjing/edk2/commits/review5):
1. Remove the Quark platform work around and code clean work from this serial 
patches.
2. Seperate three toolchains to three different patches.
3. Seperate GenFw update general part from x86 specific part and enhance the 
PIE/PIC commit log.
4. Not set -D EFI32, but use MDE_CPU_IA32 instead.


Shi, Steven (7):
  BaseTools-GenFw:Use EXE flag to check Elf section type
  BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code
  MdePkg: Enable new MS VA intrinsics for GNUC x86 64bits build
  BaseTools-Conf:Introduce GCC5 new toolchain for x86
  BaseTools-Conf:Introduce CLANG38 new toolchain for x86
  BaseTools-Conf:Introduce CLANGSCAN38 new toolchain for x86
  ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

 BaseTools/Conf/build_rule.template  |  17 +-
 BaseTools/Conf/tools_def.template   | 267 
 BaseTools/Source/C/GenFw/Elf64Convert.c | 115 --
 BaseTools/Source/C/GenFw/elf_common.h   |   9 +-
 MdePkg/Include/Base.h   |  27 ++-
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c |   2 +-
 6 files changed, 419 insertions(+), 18 deletions(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/Elf64Convert.c
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/elf_common.h
 mode change 100644 => 100755 MdePkg/Include/Base.h
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 1/7] BaseTools: Enable three new toolchains

[Shi, Steven]

Hi Jordan,
I've separated this patch to 3 commits and post a public branch for them: 
https://github.com/shijunjing/edk2/commits/review6 . Please take a look.



Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Shi, Steven
> Sent: Monday, July 04, 2016 9:38 PM
> To: Justen, Jordan L <jordan.l.jus...@intel.com>; edk2-devel@lists.01.org;
> Gao, Liming <liming@intel.com>
> Cc: Kinney, Michael D <michael.d.kin...@intel.com>; af...@apple.com
> Subject: RE: [edk2] [PATCH 1/7] BaseTools: Enable three new toolchains
> 
> OK. I will separate this patch to 3 commits and post a public branch for them.
> 
> 
> Steven Shi
> Intel\SSG\STO\UEFI Firmware
> 
> Tel: +86 021-61166522
> iNet: 821-6522
> 
> 
> > -Original Message-
> > From: Justen, Jordan L
> > Sent: Thursday, June 30, 2016 4:55 AM
> > To: Shi, Steven <steven@intel.com>; edk2-devel@lists.01.org; Gao,
> > Liming <liming@intel.com>
> > Cc: Kinney, Michael D <michael.d.kin...@intel.com>; af...@apple.com
> > Subject: Re: [edk2] [PATCH 1/7] BaseTools: Enable three new toolchains
> >
> > This should be 3 separate commits. It should be obvious in the commit
> > subject line what toolchain is being added.
> >
> > Can you post a public branch with your commits? Maybe on a personal
> > github fork of the edk2 repo.
> >
> > On 2016-06-28 08:18:55, Shi, Steven wrote:
> > > Enable three new toolchains: CLANG38, CLANGSCAN38 and GCC53.
> > > (1)CLANG38 uses Clang3.8.0, enable LLVM Link Time Optimization (LTO)
> > > and code size optimization flag (-Oz) by default for aggressive code
> > > size improvement. CLANG38 X64 code is small code model + PIE.
> > > (2)CLANGSCAN38 is based on CLANG38 to seamlessly integrate Clang
> > > scan-build analyzer infrastructure into edk2 build infrastructure.
> >
> > I'm not sure we should add a separate toolchain for this.
> >
> > > (3)GCC53 use gcc 5.3, enable GCC Link Time Optimization (LTO) and
> > > code size optimization (–Os) for aggressive code size improvement.
> > > GCC53 X64 code is also small + PIE.
> >
> > As of GCC 5.0, the minor releases have been bumped left. Essentially,
> > GCC51, GCC52, etc don't make sense, and GCC5 or GCC6 should be used
> > now.
> >
> > https://gcc.gnu.org/releases.html
> >
> > So, will this work with all versions of GCC 5.*? If not, I suggest we
> > consider starting this support at GCC6.
> >
> > -Jordan
> >
> > >
> > > Contributed-under: TianoCore Contribution Agreement 1.0
> > > Signed-off-by: Steven Shi <steven@intel.com>
> > > ---
> > >  BaseTools/Conf/build_rule.template |  17 ++-
> > >  BaseTools/Conf/tools_def.template  | 267
> > +
> > >  2 files changed, 283 insertions(+), 1 deletion(-)
> > >  mode change 100644 => 100755 BaseTools/Conf/build_rule.template
> > >  mode change 100644 => 100755 BaseTools/Conf/tools_def.template
> > >
> > > diff --git a/BaseTools/Conf/build_rule.template
> > b/BaseTools/Conf/build_rule.template
> > > old mode 100644
> > > new mode 100755
> > > index 91bcc18..7f21813
> > > --- a/BaseTools/Conf/build_rule.template
> > > +++ b/BaseTools/Conf/build_rule.template
> > > @@ -137,6 +137,9 @@
> > >  
> > >  "$(CC)" $(CC_FLAGS) -o ${dst} $(INC) ${src}
> > >
> > > +
> > > +"$(SCAN)" $(SCAN_FLAGS) -o $(SCAN_DST) "$(CC)" $(CC_FLAGS) -o
> > ${dst} $(INC) ${src}
> > > +
> > >  [C-Code-File.COMMON.IPF]
> > >  
> > >  ?.c
> > > @@ -267,7 +270,10 @@
> > >
> > >  
> > >  "$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
> > > -
> > > +
> > > +<Command.CLANG38, Command.CLANGSCAN38>
> > > +"$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
> > > +
> > >  
> > >  "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
> > >
> > > @@ -295,6 +301,10 @@
> > >  "$(DLINK)" -o ${dst} $(DLINK_FLAGS) --start-group $(DLINK_SPATH)
> > @$(STATIC_LIBRARY_FILES_LIST) --end-group $(DLINK2_FLAGS)
> > >  "$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
> > >
> > > +<Command.GCC53, Command.CLANG38, Command.CLANGSCAN38>
> &

[edk2] [PATCH 0/7] Introduce three new toolchains in edk2

[Shi, Steven]

Please review my new commits in public branch:
https://github.com/shijunjing/edk2/commits/review6

I did below enhancements according feedback:
1. Remove the Quark platform work around and code clean work from this serial 
patches.
2. Seperate three toolchains to three different patches.
3. Seperate GenFw update general part from x86 specific part and enhance the 
PIE/PIC commit log.
4. Not set -D EFI32, but use MDE_CPU_IA32 instead.


Shi, Steven (7):
  BaseTools-GenFw:Use EXE flag to check Elf section type
  BaseTools-GenFw:Add new x86_64 Elf relocation types for PIC/PIE code
  MdePkg: Enable new MS VA intrinsics for GNUC x86 64bits build
  BaseTools-Conf:Introduce GCC5 new toolchain for x86
  BaseTools-Conf:Introduce CLANG38 new toolchain for x86
  BaseTools-Conf:Introduce CLANGSCAN38 new toolchain for x86
  ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

 BaseTools/Conf/build_rule.template  |  17 +-
 BaseTools/Conf/tools_def.template   | 267 
 BaseTools/Source/C/GenFw/Elf64Convert.c | 115 --
 BaseTools/Source/C/GenFw/elf_common.h   |   9 +-
 MdePkg/Include/Base.h   |  27 ++-
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c |   2 +-
 6 files changed, 419 insertions(+), 18 deletions(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/Elf64Convert.c
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/elf_common.h
 mode change 100644 => 100755 MdePkg/Include/Base.h
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 1/7] BaseTools: Enable three new toolchains

[Shi, Steven]

OK. I will separate this patch to 3 commits and post a public branch for them.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Justen, Jordan L
> Sent: Thursday, June 30, 2016 4:55 AM
> To: Shi, Steven <steven@intel.com>; edk2-devel@lists.01.org; Gao,
> Liming <liming@intel.com>
> Cc: Kinney, Michael D <michael.d.kin...@intel.com>; af...@apple.com
> Subject: Re: [edk2] [PATCH 1/7] BaseTools: Enable three new toolchains
> 
> This should be 3 separate commits. It should be obvious in the commit
> subject line what toolchain is being added.
> 
> Can you post a public branch with your commits? Maybe on a personal
> github fork of the edk2 repo.
> 
> On 2016-06-28 08:18:55, Shi, Steven wrote:
> > Enable three new toolchains: CLANG38, CLANGSCAN38 and GCC53.
> > (1)CLANG38 uses Clang3.8.0, enable LLVM Link Time Optimization (LTO)
> > and code size optimization flag (-Oz) by default for aggressive code
> > size improvement. CLANG38 X64 code is small code model + PIE.
> > (2)CLANGSCAN38 is based on CLANG38 to seamlessly integrate Clang
> > scan-build analyzer infrastructure into edk2 build infrastructure.
> 
> I'm not sure we should add a separate toolchain for this.
> 
> > (3)GCC53 use gcc 5.3, enable GCC Link Time Optimization (LTO) and
> > code size optimization (–Os) for aggressive code size improvement.
> > GCC53 X64 code is also small + PIE.
> 
> As of GCC 5.0, the minor releases have been bumped left. Essentially,
> GCC51, GCC52, etc don't make sense, and GCC5 or GCC6 should be used
> now.
> 
> https://gcc.gnu.org/releases.html
> 
> So, will this work with all versions of GCC 5.*? If not, I suggest we
> consider starting this support at GCC6.
> 
> -Jordan
> 
> >
> > Contributed-under: TianoCore Contribution Agreement 1.0
> > Signed-off-by: Steven Shi <steven@intel.com>
> > ---
> >  BaseTools/Conf/build_rule.template |  17 ++-
> >  BaseTools/Conf/tools_def.template  | 267
> +
> >  2 files changed, 283 insertions(+), 1 deletion(-)
> >  mode change 100644 => 100755 BaseTools/Conf/build_rule.template
> >  mode change 100644 => 100755 BaseTools/Conf/tools_def.template
> >
> > diff --git a/BaseTools/Conf/build_rule.template
> b/BaseTools/Conf/build_rule.template
> > old mode 100644
> > new mode 100755
> > index 91bcc18..7f21813
> > --- a/BaseTools/Conf/build_rule.template
> > +++ b/BaseTools/Conf/build_rule.template
> > @@ -137,6 +137,9 @@
> >  
> >  "$(CC)" $(CC_FLAGS) -o ${dst} $(INC) ${src}
> >
> > +
> > +"$(SCAN)" $(SCAN_FLAGS) -o $(SCAN_DST) "$(CC)" $(CC_FLAGS) -o
> ${dst} $(INC) ${src}
> > +
> >  [C-Code-File.COMMON.IPF]
> >  
> >  ?.c
> > @@ -267,7 +270,10 @@
> >
> >  
> >  "$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
> > -
> > +
> > +<Command.CLANG38, Command.CLANGSCAN38>
> > +"$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
> > +
> >  
> >  "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
> >
> > @@ -295,6 +301,10 @@
> >  "$(DLINK)" -o ${dst} $(DLINK_FLAGS) --start-group $(DLINK_SPATH)
> @$(STATIC_LIBRARY_FILES_LIST) --end-group $(DLINK2_FLAGS)
> >  "$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
> >
> > +<Command.GCC53, Command.CLANG38, Command.CLANGSCAN38>
> > +"$(DLINK)" -o ${dst} $(DLINK_FLAGS) -Wl,--start-
> group,$(DLINK_SPATH),@$(STATIC_LIBRARY_FILES_LIST) -Wl,--end-group
> $(DLINK2_FLAGS)
> > +"$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
> > +
> >  
> >  "$(DLINK)" $(DLINK_FLAGS) -o ${dst} $(DLINK_SPATH) --via
> $(STATIC_LIBRARY_FILES_LIST) $(DLINK2_FLAGS)
> >
> > @@ -448,6 +458,11 @@
> >  "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll
> $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
> >  "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll
> $(GENFW_FLAGS)
> >
> > +
> > +"$(ASLCC)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
> $(ASLCC_FLAGS) $(INC) ${src}
> > +"$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll
> $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
> > +"$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll
> $(GENFW_FLAG

Re: [edk2] [PATCH 4/7] QuarkPlatformPkg-AcpiPlatform: Downgrade the optimization to O1

[Shi, Steven]

OK, I will try to root cause it. And I will separate the Quark build enabling 
work from this serial patches as a new one.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Kinney, Michael D
> Sent: Thursday, June 30, 2016 4:56 AM
> To: af...@apple.com; Shi, Steven <steven@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>
> Cc: edk2-devel <edk2-devel@lists.01.org>; Gao, Liming
> <liming@intel.com>; Zimmer, Vincent <vincent.zim...@intel.com>;
> Doran, Mark <mark.do...@intel.com>; Mudusuru, Giri P
> <giri.p.mudus...@intel.com>
> Subject: RE: [edk2] [PATCH 4/7] QuarkPlatformPkg-AcpiPlatform:
> Downgrade the optimization to O1
> 
> Andrew,
> 
> I agree that we need to root cause and fix this code issue.
> 
> I also like idea of doing RELEASE/DEBUG/NOOPT builds to make sure code
> changes do not introduce build failures.
> 
> Mike
> 
> > -Original Message-----
> > From: af...@apple.com [mailto:af...@apple.com]
> > Sent: Wednesday, June 29, 2016 11:31 AM
> > To: Shi, Steven <steven@intel.com>
> > Cc: edk2-devel <edk2-devel@lists.01.org>; Gao, Liming
> <liming@intel.com>; Kinney,
> > Michael D <michael.d.kin...@intel.com>; Zimmer, Vincent
> <vincent.zim...@intel.com>;
> > Doran, Mark <mark.do...@intel.com>; Mudusuru, Giri P
> <giri.p.mudus...@intel.com>
> > Subject: Re: [edk2] [PATCH 4/7] QuarkPlatformPkg-AcpiPlatform:
> Downgrade the
> > optimization to O1
> >
> >
> > > On Jun 28, 2016, at 8:18 AM, Shi, Steven <steven@intel.com> wrote:
> > >
> > > Quark AcpiPlatform module cannot link successfully by CLANG38
> > > with Oz optimization level. Add a work around to downgrade the
> > > optimization to O1.
> > >
> >
> > We should fix the code too. These bugs are usually structure assignments in
> the scope
> > of a function that should be global, static, or done via CopyMem.
> >
> > I really think they should get fixed as I see a lot of Si code that fails 
> > to link
> when
> > you compile with optimizations disabled, -O0, with Xcode clang as the
> compiler falls
> > back to memset/memcpy and we were just getting lucky the optimizer
> removed it.
> >
> > Actually it would be nice if packages also got built NOOPT to help catch
> these classes
> > of errors.
> >
> > Thanks,
> >
> > Andrew Fish
> >
> > > Contributed-under: TianoCore Contribution Agreement 1.0
> > > Signed-off-by: Steven Shi <steven@intel.com>
> > > ---
> > > QuarkPlatformPkg/Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf | 4 
> > > 1 file changed, 4 insertions(+)
> > > mode change 100644 => 100755
> QuarkPlatformPkg/Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf
> > >
> > > diff --git a/QuarkPlatformPkg/Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf
> > b/QuarkPlatformPkg/Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf
> > > old mode 100644
> > > new mode 100755
> > > index dcf2b4a..f1bcfa1
> > > --- a/QuarkPlatformPkg/Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf
> > > +++ b/QuarkPlatformPkg/Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf
> > > @@ -200,3 +200,7 @@
> > >
> > > [Depex]
> > >   gEfiMpServiceProtocolGuid AND gEfiAcpiTableProtocolGuid
> > > +
> > > +[BuildOptions]
> > > +  GCC:*_CLANG38_IA32_DLINK_FLAGS  = -O1
> > > +  GCC:*_CLANGSCAN38_IA32_DLINK_FLAGS  = -O1
> > > --
> > > 2.7.4
> > >
> > > ___
> > > edk2-devel mailing list
> > > edk2-devel@lists.01.org
> > > https://lists.01.org/mailman/listinfo/edk2-devel

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [PATCH 5/7] QuarkSocPkg-MemoryInitPei: Enable compiler builtin and disable CLANG LTO

[Shi, Steven]

I will separate the Quark platform code cleanup work from this serial patches. 
Thanks.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Kinney, Michael D
> Sent: Thursday, June 30, 2016 2:10 AM
> To: af...@apple.com; Shi, Steven <steven@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>
> Cc: edk2-devel <edk2-devel@lists.01.org>; Gao, Liming
> <liming@intel.com>; Zimmer, Vincent <vincent.zim...@intel.com>;
> Doran, Mark <mark.do...@intel.com>; Mudusuru, Giri P
> <giri.p.mudus...@intel.com>
> Subject: RE: [PATCH 5/7] QuarkSocPkg-MemoryInitPei: Enable compiler
> builtin and disable CLANG LTO
> 
> Andrew,
> 
> I agree.  I will work with Steven on this module to improve its compiler
> compatibility.
> 
> Mike
> 
> > -Original Message-
> > From: af...@apple.com [mailto:af...@apple.com]
> > Sent: Wednesday, June 29, 2016 10:57 AM
> > To: Shi, Steven <steven@intel.com>
> > Cc: edk2-devel <edk2-devel@lists.01.org>; Gao, Liming
> <liming@intel.com>; Kinney,
> > Michael D <michael.d.kin...@intel.com>; Zimmer, Vincent
> <vincent.zim...@intel.com>;
> > Doran, Mark <mark.do...@intel.com>; Mudusuru, Giri P
> <giri.p.mudus...@intel.com>
> > Subject: Re: [PATCH 5/7] QuarkSocPkg-MemoryInitPei: Enable compiler
> builtin and disable
> > CLANG LTO
> >
> >
> > > On Jun 28, 2016, at 8:18 AM, Shi, Steven <steven@intel.com> wrote:
> > >
> > > The MRC use the memset() in the code and assume the compiler to offer
> > > such builtin fuctions. Add work around to permit compiler add builtin
> > > functions in this module. Also MRC mix some inline assembly,
> > > e.g. asm("rdtsc":"=a"(tscL),"=d"(tscH)),in the C code, and CLANG38
> > > could wrongly remove them if enable LTO with high optimization level.
> > > Add work around to simply disable the LTO in this module.
> > >
> >
> > Steven,
> >
> > If this code is in the edk2 it should follow the edk2 coding rules and we
> should clean
> > it up and not propagate the workarounds to new compilers. Not following
> the rules in
> > the 1st place is what made this code harder to port. Actually if you look at
> the code
> > it started off using VC++ compiler intrinsics and inline assembler syntax,
> and then it
> > had to add some hacky #ifdefs to compile with GCC. If the code just used
> BaseLib and
> > BaseMemoryLib it would not require compiler specific hacks.
> >
> > Why not fix the code to NOT depend on compiler intrinsics and use
> ZeroMem()?
> > /work/src/edk2/QuarkSocPkg/QuarkNorthCluster/MemoryInit(master)>git
> grep memcpy
> > Pei/meminit_utils.h:94:void *memcpy(void *d, const void *s, size_t n);
> >
> ~/work/src/edk2/QuarkSocPkg/QuarkNorthCluster/MemoryInit(master)>git
> grep memset
> > Pei/MemoryInitPei.inf:79:  # /Oi option to use the intrinsic memset
> function in source
> > code.
> > Pei/meminit.c::  memset((void *) (final_delay), 0x00, (size_t)
> > sizeof(final_delay));
> > Pei/meminit.c:1396:  memset((void *) (final_delay), 0x00, (size_t)
> > sizeof(final_delay));
> > Pei/meminit.c:1650:  memset((void *) (final_delay), 0x00, (size_t)
> > sizeof(final_delay));
> > Pei/meminit.c:1972:  memset((void *) (final_delay), 0x00, (size_t)
> > sizeof(final_delay));
> > Pei/meminit_utils.h:93:void *memset(void *d, int c, size_t n);
> >
> > All the inline assembly code is duplicating BaseLib functionality so why not
> fix it?
> >
> ~/work/src/edk2/QuarkSocPkg/QuarkNorthCluster/MemoryInit(master)>git
> grep -W  asm
> > Pei/meminit.c:2240:static void asm_wbinvd(void)
> > Pei/meminit.c-2241-{
> > Pei/meminit.c-2242-#if defined (SIM) || defined (GCC)
> > Pei/meminit.c:2243:  asm(
> > Pei/meminit.c-2244-"wbinvd;"
> > Pei/meminit.c-2245-  );
> > Pei/meminit.c-2246-#else
> > Pei/meminit.c:2247:  __asm wbinvd;
> > Pei/meminit.c-2248-#endif
> > Pei/meminit.c-2249-}
> > Pei/meminit.c-2250-
> > Pei/meminit.c-2251-// cache invalidate
> > Pei/meminit.c:2252:static void asm_invd(void)
> > Pei/meminit.c-2253-{
> > Pei/meminit.c-2254-#if defined (SIM) || defined (GCC)
> > Pei/meminit.c:2255:  asm(
> > Pei/meminit.c-2256-  "invd;"
> > Pei/meminit.c-2257-  );
> > Pei/meminit.c-2258-#else
> > Pei/meminit.c:2259:  __asm invd;
> > Pei/meminit.c-2260-#endif
> > Pei/meminit.c-2261-}
> > Pei/memi

Re: [edk2] [PATCH 3/7] MdePkg: Enable new MS VA intrinsics for GNUC 64bits build

[Shi, Steven]

Yes. MS VA intrinsic (__builtin_ms_va_*) can work on GCC44, and in fact, GCC44 
should have to use them for Uefi 64bits build. The current GCC44 wrongly use 
UNIX  VA intrinsics (__builtin_va_*) as default which cause generate wrong VA 
code if enable -O2 or higher optimization (e.g. -Os). I could fix them on GCC44 
as well in a separate patch later.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522

> -Original Message-
> From: Justen, Jordan L
> Sent: Thursday, June 30, 2016 5:47 AM
> To: Andrew Fish <af...@apple.com>
> Cc: Shi, Steven <steven@intel.com>; edk2-devel@lists.01.org; Gao,
> Liming <liming@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>
> Subject: Re: [edk2] [PATCH 3/7] MdePkg: Enable new MS VA intrinsics for
> GNUC 64bits build
> 
> On 2016-06-29 14:31:49, Andrew Fish wrote:
> >
> > > On Jun 29, 2016, at 2:02 PM, Jordan Justen <jordan.l.jus...@intel.com>
> wrote:
> > >
> > > On 2016-06-28 08:18:57, Shi, Steven wrote:
> > >> Both GCC and LLVM 3.8 64bits support new variable argument (VA)
> > >
> > > Which versions of GCC? Maybe we should define MS_VA_LIST on the
> > > command line for toolchains that can support this?
> > >
> >
> > Maybe I'm backwards but VA_LIST is by definition MS_VA_LIST. What
> > you lose access to is UNIX_VA_LIST?
> >
> > Thus for a compiler that uses the EFIAPI __builtin_va_arg is EFIABI.
> > For a compiler that produces Sys V ABI __builtin_va_arg is the Unix
> > ABI. I verified this assertion with the clang maintainers.
> >
> > VA_LIST always points to EFIAPI and that is why you have to decorate
> > var arg functions with EFIAPI to match VA_LIST.
> >
> > We could add a NATIVE_VA_LIST but for an compiler that natively does
> > EFIAPI it would be the same a VA_LIST. Thus it would be hard to
> > enforce, but it would let some one that knew what they are doing get
> > access to __builtin_va_arg. The problem is if you know what you are
> > doing you can decorate with EFIAPI so is adding NATIVE_VA_LIST
> > really solving any problem?
> >
> 
> My concern was more about if this patch would work on GCC44. I think
> Steven has been testing with GCC5+, so I thought it might be better to
> selectively enable usage of __builtin_ms_va_list for each toolchain.
> 
> -Jordan
> 
> > >
> > >> intrinsics for Microsoft ABI, enable these new VA intrinsics for
> > >> GNUC family 64bits code build. These VA intrinsics are only
> > >> permitted use in 64bits code, so not use them in 32bits code build.
> > >> The original 32bits GNU VA intrinsics has the same calling conversion
> > >> as MS, so we don’t need change them.
> > >>
> > >> Contributed-under: TianoCore Contribution Agreement 1.0
> > >> Signed-off-by: Steven Shi <steven@intel.com>
> > >> ---
> > >> MdePkg/Include/Base.h | 27 +--
> > >> 1 file changed, 25 insertions(+), 2 deletions(-)
> > >> mode change 100644 => 100755 MdePkg/Include/Base.h
> > >>
> > >> diff --git a/MdePkg/Include/Base.h b/MdePkg/Include/Base.h
> > >> old mode 100644
> > >> new mode 100755
> > >> index cbd9e55..05fccf3
> > >> --- a/MdePkg/Include/Base.h
> > >> +++ b/MdePkg/Include/Base.h
> > >> @@ -588,9 +588,32 @@ struct _LIST_ENTRY {
> > >>
> > >> #define VA_COPY(Dest, Start)  __va_copy (Dest, Start)
> > >>
> > >> -#elif defined(__GNUC__) && !defined(NO_BUILTIN_VA_FUNCS)
> > >> +
> > >> +#elif defined(__GNUC__) && !defined(NO_BUILTIN_VA_FUNCS)
> && !defined (EFI32)
> > >> +//
> > >> +// 64bits build only. Use GCC built-in macros for variable argument 
> > >> lists.
> > >> +//
> > >> +///
> > >> +/// Both GCC and LLVM 3.8 64bits support new variable argument
> intrinsics for Microsoft ABI
> > >> +///
> > >> +
> > >> +///
> > >> +/// Variable used to traverse the list of arguments. This type can vary
> by
> > >> +/// implementation and could be an array or structure.
> > >> +///
> > >> +typedef __builtin_ms_va_list VA_LIST;
> > >> +
> > >> +#define VA_START(Marker, Parameter)  __builtin_ms_va_start (Marker,
> Parameter)
> > >> +
> > >> +#define VA_ARG(Marker, TYPE) ((sizeof (TYPE) < sizeof (UINTN)) ?
> (TYPE)(__builtin_va_ar

Re: [edk2] [PATCH 2/7] BaseTools: Add missing Elf relocation type for LTO build

[Shi, Steven]

Yes. It should related to PIC/PIE code. I will correct my commit log later. 
Thank you let me know it.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Wednesday, June 29, 2016 10:42 PM
> To: Shi, Steven <steven@intel.com>
> Cc: edk2-devel-01 <edk2-devel@lists.01.org>; Gao, Liming
> <liming@intel.com>; Kinney, Michael D <michael.d.kin...@intel.com>;
> af...@apple.com
> Subject: Re: [edk2] [PATCH 2/7] BaseTools: Add missing Elf relocation type
> for LTO build
> 
> On 29 June 2016 at 15:59, Shi, Steven <steven@intel.com> wrote:
> > Ok, will separate it. Thank you to let me know it.
> >
> 
> 
> BTW, your commit log suggests that these relocation types are somehow
> related to LTO, but I think they are more related to PIC/PIE code,
> right? So even if you disable LTO, you may encounter these relocation
> types when using the new toolchain definitions?
> 
> 
> >> -Original Message-
> >> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> >> Sent: Wednesday, June 29, 2016 9:30 PM
> >> To: Shi, Steven <steven@intel.com>
> >> Cc: edk2-devel-01 <edk2-devel@lists.01.org>; Gao, Liming
> >> <liming@intel.com>; Kinney, Michael D
> <michael.d.kin...@intel.com>;
> >> af...@apple.com
> >> Subject: Re: [edk2] [PATCH 2/7] BaseTools: Add missing Elf relocation
> type
> >> for LTO build
> >>
> >> On 28 June 2016 at 17:18, Shi, Steven <steven@intel.com> wrote:
> >> > Add support to convert missing Elf relocation types
> >> > (R_X86_64_PLT32, R_X86_64_GOTPCREL, R_X86_64_REX_GOTPCRELX)
> >> > to PeCoff, which are required by LTO image.
> >> >
> >>
> >> Could you please put the changes that affect other architectures in a
> >> separate patch?
> >> The commit log suggests that only X64 is affected, but the patch
> >> touches other things as well.
> >>
> >> > Contributed-under: TianoCore Contribution Agreement 1.0
> >> > Signed-off-by: Steven Shi <steven@intel.com>
> >> > ---
> >> >  BaseTools/Source/C/GenFw/Elf64Convert.c | 98
> >> -
> >> >  BaseTools/Source/C/GenFw/elf_common.h   |  2 +-
> >> >  2 files changed, 86 insertions(+), 14 deletions(-)
> >> >  mode change 100644 => 100755
> >> BaseTools/Source/C/GenFw/Elf64Convert.c
> >> >  mode change 100644 => 100755
> >> BaseTools/Source/C/GenFw/elf_common.h
> >> >
> >> > diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c
> >> b/BaseTools/Source/C/GenFw/Elf64Convert.c
> >> > old mode 100644
> >> > new mode 100755
> >> > index 024a2a0..205360c
> >> > --- a/BaseTools/Source/C/GenFw/Elf64Convert.c
> >> > +++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
> >> > @@ -75,7 +75,7 @@ CleanUp64 (
> >> >);
> >> >
> >> >  //
> >> > -// Rename ELF32 strucutres to common names to help when porting to
> >> ELF64.
> >> > +// Rename ELF32 structures to common names to help when porting to
> >> ELF64.
> >> >  //
> >> >  typedef Elf64_Shdr Elf_Shdr;
> >> >  typedef Elf64_Ehdr Elf_Ehdr;
> >> > @@ -233,7 +233,7 @@ IsTextShdr (
> >> >Elf_Shdr *Shdr
> >> >)
> >> >  {
> >> > -  return (BOOLEAN) ((Shdr->sh_flags & (SHF_WRITE | SHF_ALLOC)) ==
> >> SHF_ALLOC);
> >> > +  return (BOOLEAN) ((Shdr->sh_flags & (SHF_EXECINSTR | SHF_WRITE |
> >> SHF_ALLOC)) == (SHF_EXECINSTR | SHF_ALLOC));
> >> >  }
> >> >
> >> >  STATIC
> >> > @@ -256,7 +256,7 @@ IsDataShdr (
> >> >if (IsHiiRsrcShdr(Shdr)) {
> >> >  return FALSE;
> >> >}
> >> > -  return (BOOLEAN) (Shdr->sh_flags & (SHF_WRITE | SHF_ALLOC)) ==
> >> (SHF_ALLOC | SHF_WRITE);
> >> > +  return (BOOLEAN) (Shdr->sh_flags & (SHF_EXECINSTR | SHF_WRITE |
> >> SHF_ALLOC)) == (SHF_ALLOC | SHF_WRITE);
> >> >  }
> >> >
> >> >  STATIC
> >> > @@ -661,9 +661,9 @@ WriteSections64 (
> >> >
> >> >default:
> >> >  //
> >> > -//  Ignore for unkown section type.
> >> > +//  Ignore for unknown section type.
> >

Re: [edk2] [PATCH 2/7] BaseTools: Add missing Elf relocation type for LTO build

[Shi, Steven]

Ok, will separate it. Thank you to let me know it.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Ard Biesheuvel [mailto:ard.biesheu...@linaro.org]
> Sent: Wednesday, June 29, 2016 9:30 PM
> To: Shi, Steven <steven@intel.com>
> Cc: edk2-devel-01 <edk2-devel@lists.01.org>; Gao, Liming
> <liming@intel.com>; Kinney, Michael D <michael.d.kin...@intel.com>;
> af...@apple.com
> Subject: Re: [edk2] [PATCH 2/7] BaseTools: Add missing Elf relocation type
> for LTO build
> 
> On 28 June 2016 at 17:18, Shi, Steven <steven@intel.com> wrote:
> > Add support to convert missing Elf relocation types
> > (R_X86_64_PLT32, R_X86_64_GOTPCREL, R_X86_64_REX_GOTPCRELX)
> > to PeCoff, which are required by LTO image.
> >
> 
> Could you please put the changes that affect other architectures in a
> separate patch?
> The commit log suggests that only X64 is affected, but the patch
> touches other things as well.
> 
> > Contributed-under: TianoCore Contribution Agreement 1.0
> > Signed-off-by: Steven Shi <steven@intel.com>
> > ---
> >  BaseTools/Source/C/GenFw/Elf64Convert.c | 98
> -
> >  BaseTools/Source/C/GenFw/elf_common.h   |  2 +-
> >  2 files changed, 86 insertions(+), 14 deletions(-)
> >  mode change 100644 => 100755
> BaseTools/Source/C/GenFw/Elf64Convert.c
> >  mode change 100644 => 100755
> BaseTools/Source/C/GenFw/elf_common.h
> >
> > diff --git a/BaseTools/Source/C/GenFw/Elf64Convert.c
> b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > old mode 100644
> > new mode 100755
> > index 024a2a0..205360c
> > --- a/BaseTools/Source/C/GenFw/Elf64Convert.c
> > +++ b/BaseTools/Source/C/GenFw/Elf64Convert.c
> > @@ -75,7 +75,7 @@ CleanUp64 (
> >);
> >
> >  //
> > -// Rename ELF32 strucutres to common names to help when porting to
> ELF64.
> > +// Rename ELF32 structures to common names to help when porting to
> ELF64.
> >  //
> >  typedef Elf64_Shdr Elf_Shdr;
> >  typedef Elf64_Ehdr Elf_Ehdr;
> > @@ -233,7 +233,7 @@ IsTextShdr (
> >Elf_Shdr *Shdr
> >)
> >  {
> > -  return (BOOLEAN) ((Shdr->sh_flags & (SHF_WRITE | SHF_ALLOC)) ==
> SHF_ALLOC);
> > +  return (BOOLEAN) ((Shdr->sh_flags & (SHF_EXECINSTR | SHF_WRITE |
> SHF_ALLOC)) == (SHF_EXECINSTR | SHF_ALLOC));
> >  }
> >
> >  STATIC
> > @@ -256,7 +256,7 @@ IsDataShdr (
> >if (IsHiiRsrcShdr(Shdr)) {
> >  return FALSE;
> >}
> > -  return (BOOLEAN) (Shdr->sh_flags & (SHF_WRITE | SHF_ALLOC)) ==
> (SHF_ALLOC | SHF_WRITE);
> > +  return (BOOLEAN) (Shdr->sh_flags & (SHF_EXECINSTR | SHF_WRITE |
> SHF_ALLOC)) == (SHF_ALLOC | SHF_WRITE);
> >  }
> >
> >  STATIC
> > @@ -661,9 +661,9 @@ WriteSections64 (
> >
> >default:
> >  //
> > -//  Ignore for unkown section type.
> > +//  Ignore for unknown section type.
> >  //
> > -VerboseMsg ("%s unknown section type %x. We directly copy this
> section into Coff file", mInImageName, (unsigned)Shdr->sh_type);
> > +VerboseMsg ("%s unknown section type %x. We directly ignore this
> section and NOT copy into Coff file", mInImageName, (unsigned)Shdr-
> >sh_type);
> >  break;
> >}
> >  }
> > @@ -763,24 +763,24 @@ WriteSections64 (
> >  // Absolute relocation.
> >  //
> >  VerboseMsg ("R_X86_64_64");
> > -VerboseMsg ("Offset: 0x%08X, Addend: 0x%016LX",
> > -  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
> > +VerboseMsg ("Offset: 0x%08X, Addend: 0x%016LX",
> > +  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
> >*(UINT64 *)Targ);
> >  *(UINT64 *)Targ = *(UINT64 *)Targ - SymShdr->sh_addr +
> mCoffSectionsOffset[Sym->st_shndx];
> >  VerboseMsg ("Relocation:  0x%016LX", *(UINT64*)Targ);
> >  break;
> >case R_X86_64_32:
> >  VerboseMsg ("R_X86_64_32");
> > -VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X",
> > -  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),
> > +VerboseMsg ("Offset: 0x%08X, Addend: 0x%08X",
> > +  (UINT32)(SecOffset + (Rel->r_offset - SecShdr->sh_addr)),

Re: [edk2] [PATCH 0/7] Introduce three new toolchains in edk2

[Shi, Steven]

Hi Cran,

You know to support X64 Microsoft variable argument (VA) ABI, we need two parts 
work together in edk2:

First is to the define "DEFIAPI=__attribute__((ms_abi)) in CC_FLAGS.

Second is to use below __builtin_ms_va_* builtins as the VA macro.

  __builtin_ms_va_list ap;

  __builtin_ms_va_start (ap, n);

  __builtin_ms_va_end (ap);



Only set -mabi=ms cannot really work. See the GCC example in below link.



As Andrew mentioned, we wish GCC or LLVM can natively support EFI/Windows ABIs, 
and then we don't need use special builtins and macro any longer. This request 
has been raised to GCC for ~5 years, see below link, but we has not convince 
GCC guys to implement it yet...



https://gcc.gnu.org/bugzilla/show_bug.cgi?id=50818







Steven Shi

Intel\SSG\STO\UEFI Firmware



Tel: +86 021-61166522

iNet: 821-6522





> -Original Message-

> From: af...@apple.com [mailto:af...@apple.com]

> Sent: Wednesday, June 29, 2016 10:22 AM

> To: Gao, Liming <liming@intel.com>

> Cc: Bruce Cran <br...@cran.org.uk>; Shi, Steven <steven@intel.com>;

> edk2-devel@lists.01.org; Kinney, Michael D <michael.d.kin...@intel.com>

> Subject: Re: [edk2] [PATCH 0/7] Introduce three new toolchains in edk2

>

>

> > On Jun 28, 2016, at 7:09 PM, Gao, Liming 
> > <liming@intel.com<mailto:liming@intel.com>> wrote:

> >

> > Bruce:

> >  Yes. We add EFIAPI in such function, and define "-

> DEFIAPI=__attribute__((ms_abi))" in GCC CC_FLAGS.

> >

>

> I don't know how the version of clang used was constructed, but clang is

> inherently a cross compiler and it can easily support multiple ABIs (System V

> and X64). It comes down to how it was constructed.

>

> Basically a flag like -arch x86_64 will be the system default ABI, but you can

> build other stuff for the architecture via -target x86_64-pc-win32-macho. So

> for example you might be able to enable -target x86_64-pc-win32-efl. So

> have the native EFI/Windows ABI, but make an ELF with dwarf for the

> debugger.

>

> If you don't want to use the __attribute__((ms_abi)) then you might be able

> to get that support in upstream. There might already be some support for

> cross building Windows Apps on Linux that could be leveraged. You just need

> the correctly ABI, and a source level debugging scheme that works on your

> build target.

>

> Thanks,

>

> Andrew Fish

>

> > Thanks

> > Liming

> >> -Original Message-

> >> From: Bruce Cran [mailto:br...@cran.org.uk]

> >> Sent: Wednesday, June 29, 2016 6:45 AM

> >> To: Shi, Steven <steven@intel.com<mailto:steven@intel.com>>; 
> >> edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>; Gao,

> >> Liming <liming@intel.com<mailto:liming....@intel.com>>

> >> Cc: Kinney, Michael D 
> >> <michael.d.kin...@intel.com<mailto:michael.d.kin...@intel.com>>; 
> >> af...@apple.com<mailto:af...@apple.com>

> >> Subject: Re: [edk2] [PATCH 0/7] Introduce three new toolchains in edk2

> >>

> >> On 6/28/16 9:18 AM, Shi, Steven wrote:

> >>> Introduce three new toolchains in edk2. The first two ones support Link

> >> Time Optimization (LTO) for aggressive code size improvement and the

> third

> >> one is for static analysis.

> >>>

> >>> CLANG38: Enable LLVM Link Time Optimization (LTO) and code size

> >> optimization flag (-Os) by default for aggressive code size improvement.

> >>> X64 code is small code model + position independent code (PIE).

> >>

> >> One problem I've run into trying to build with clang is that we have a

> >> set of files which are the same across platforms, and they use varargs.

> >>

> >> With gcc, we can just set -mabi=ms and be done with it, but it appears

> >> clang doesn't have that option.  Do you know of a way to set the default

> >> ABI with clang, or would we need to add an __attribute__((ms_abi))

> >> anywhere we need to use varargs?

> >>

> >> --

> >> Bruce Cran

> > ___

> > edk2-devel mailing list

> > edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>

> > https://lists.01.org/mailman/listinfo/edk2-devel


___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Segfault in GenFw when building driver with new CLANG38 toolchain

[Shi, Steven]

OK. How about GCC53?  Can GCC53 build your driver?


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Bruce Cran [mailto:br...@cran.org.uk]
> Sent: Wednesday, June 29, 2016 12:12 PM
> To: Shi, Steven <steven@intel.com>; edk2-devel@lists.01.org
> Subject: Re: Segfault in GenFw when building driver with new CLANG38
> toolchain
> 
> On 6/28/2016 9:51 PM, Shi, Steven wrote:
> 
> > Can you build the OVMF with CLANG38 in your side?
> > And if possible, pls send me your driver .dll Elf image and I can try to 
> > debug
> the GenFw convert in my side.
> 
> Yes, both DEBUG and RELEASE X64 builds of OVMF complete fine.
> 
> I wish I could send the driver binaries, but I'd need to get sign-off
> from Legal, and considering the delay and work involved, I'm not sure
> it's worth it. I'll see if I can find some time to play around with
> settings etc. to see if I can see what's going wrong.
> 
> Thanks.
> Bruce
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] Segfault in GenFw when building driver with new CLANG38 toolchain

[Shi, Steven]

Hi Cran,
Can you build the OVMF with CLANG38 in your side? 
And if possible, pls send me your driver .dll Elf image and I can try to debug 
the GenFw convert in my side.


Steven Shi
Intel\SSG\STO\UEFI Firmware

Tel: +86 021-61166522
iNet: 821-6522


> -Original Message-
> From: Bruce Cran [mailto:br...@cran.org.uk]
> Sent: Wednesday, June 29, 2016 11:37 AM
> To: edk2-devel@lists.01.org
> Cc: Shi, Steven <steven@intel.com>
> Subject: Segfault in GenFw when building driver with new CLANG38 toolchain
> 
> After applying Steven's (cc'd) patch set and building our driver with
> the CLANG38 toolchain, GenFw crashes with a segmentation fault.  Due to
> unresolved memset and memcpy symbols, I've added the flags "-fbuiltin
> -fno-lto".
> 
> #0  0x0040bccf in WriteRelocations64 () at Elf64Convert.c:975
> #1  0x00408219 in ConvertElf (FileBuffer=0x7fffde28,
> FileLength=0x7fffde24) at ElfConvert.c:207
> #2  0x00405189 in main (argc=0, argv=0x7fffe098) at
> GenFw.c:2031
> 
> 
> Elf64Convert.c:975:  VerboseMsg ("Relocation: 0x%08X",
> *(UINT64*)CoffFileGoTPcRelPtr);
> 
> (gdb) p CoffFileGoTPcRelPtr
> 
> $1 = (UINT64 *) 0x776f8160
> 
> (gdb) p *CoffFileGoTPcRelPtr
> 
> Cannot access memory at address 0x776f8160
> 
> 
> --
> Bruce

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH 0/7] Introduce three new toolchains in edk2

[Shi, Steven]

quot; -DCMAKE_VERBOSE_MAKEFILE=ON 
-DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_C_COMPILER="/usr/bin/gcc" 
-DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
$ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so

Shi, Steven (7):
  BaseTools: Enable three new toolchains
  BaseTools: Add missing Elf relocation type for LTO build
  MdePkg: Enable new MS VA intrinsics for GNUC 64bits build
  QuarkPlatformPkg-AcpiPlatform: Downgrade the optimization to O1
  QuarkSocPkg-MemoryInitPei: Enable compiler builtin and disable CLANG
LTO
  QuarkSocPkg-OhicPei: Fix wrong operator
  ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

 BaseTools/Conf/build_rule.template |  17 +-
 BaseTools/Conf/tools_def.template  | 267 +
 BaseTools/Source/C/GenFw/Elf64Convert.c|  98 +++-
 BaseTools/Source/C/GenFw/elf_common.h  |   2 +-
 MdePkg/Include/Base.h  |  27 ++-
 .../Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf |   4 +
 .../MemoryInit/Pei/MemoryInitPei.inf   |   3 +
 .../QuarkSouthCluster/Usb/Ohci/Pei/OhciReg.c   |   2 +-
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c|   2 +-
 9 files changed, 403 insertions(+), 19 deletions(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/Elf64Convert.c
 mode change 100644 => 100755 BaseTools/Source/C/GenFw/elf_common.h
 mode change 100644 => 100755 MdePkg/Include/Base.h
 mode change 100644 => 100755 
QuarkPlatformPkg/Acpi/Dxe/AcpiPlatform/AcpiPlatform.inf
 mode change 100644 => 100755 
QuarkSocPkg/QuarkNorthCluster/MemoryInit/Pei/MemoryInitPei.inf
 mode change 100644 => 100755 
QuarkSocPkg/QuarkSouthCluster/Usb/Ohci/Pei/OhciReg.c
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

-- 
2.7.4

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [PATCH 1/7] BaseTools: Enable three new toolchains

[Shi, Steven]

Enable three new toolchains: CLANG38, CLANGSCAN38 and GCC53.
(1)CLANG38 uses Clang3.8.0, enable LLVM Link Time Optimization (LTO)
and code size optimization flag (-Oz) by default for aggressive code
size improvement. CLANG38 X64 code is small code model + PIE.
(2)CLANGSCAN38 is based on CLANG38 to seamlessly integrate Clang
scan-build analyzer infrastructure into edk2 build infrastructure.
(3)GCC53 use gcc 5.3, enable GCC Link Time Optimization (LTO) and
code size optimization (–Os) for aggressive code size improvement.
GCC53 X64 code is also small + PIE.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven@intel.com>
---
 BaseTools/Conf/build_rule.template |  17 ++-
 BaseTools/Conf/tools_def.template  | 267 +
 2 files changed, 283 insertions(+), 1 deletion(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template

diff --git a/BaseTools/Conf/build_rule.template 
b/BaseTools/Conf/build_rule.template
old mode 100644
new mode 100755
index 91bcc18..7f21813
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -137,6 +137,9 @@
 
 "$(CC)" $(CC_FLAGS) -o ${dst} $(INC) ${src}
 
+
+"$(SCAN)" $(SCAN_FLAGS) -o $(SCAN_DST) "$(CC)" $(CC_FLAGS) -o ${dst} 
$(INC) ${src}
+
 [C-Code-File.COMMON.IPF]
 
 ?.c
@@ -267,7 +270,10 @@
 
 
 "$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
-
+
+<Command.CLANG38, Command.CLANGSCAN38>
+"$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
+
 
 "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
 
@@ -295,6 +301,10 @@
 "$(DLINK)" -o ${dst} $(DLINK_FLAGS) --start-group $(DLINK_SPATH) 
@$(STATIC_LIBRARY_FILES_LIST) --end-group $(DLINK2_FLAGS)
 "$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
 
+<Command.GCC53, Command.CLANG38, Command.CLANGSCAN38>
+"$(DLINK)" -o ${dst} $(DLINK_FLAGS) 
-Wl,--start-group,$(DLINK_SPATH),@$(STATIC_LIBRARY_FILES_LIST) -Wl,--end-group 
$(DLINK2_FLAGS)
+"$(OBJCOPY)" $(OBJCOPY_FLAGS) ${dst}
+
 
 "$(DLINK)" $(DLINK_FLAGS) -o ${dst} $(DLINK_SPATH) --via 
$(STATIC_LIBRARY_FILES_LIST) $(DLINK2_FLAGS)
 
@@ -448,6 +458,11 @@
 "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
 "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(GENFW_FLAGS)
 
+
+"$(ASLCC)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASLCC_FLAGS) 
$(INC) ${src}
+"$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
+"$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(GENFW_FLAGS)
+
 
 "$(ASLCC)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj  $(ASLCC_FLAGS) 
$(INC) ${src}
 "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll 
$(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
diff --git a/BaseTools/Conf/tools_def.template 
b/BaseTools/Conf/tools_def.template
old mode 100644
new mode 100755
index 2065fa3..d401031
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -366,12 +366,30 @@ DEFINE SOURCERY_CYGWIN_TOOLS = /cygdrive/c/Program 
Files/CodeSourcery/Sourcery G
 # Required to build platforms or ACPI tables:
 #   Intel(r) ACPI Compiler from
 #   https://acpica.org/downloads
+#   GCC53-Linux-  Requires:
+# GCC 5.3 targeting x86_64-linux-gnu
+#Optional:
+# Required to build platforms or ACPI tables:
+#   Intel(r) ACPI Compiler from
+#   https://acpica.org/downloads
 #   CLANG35 -Linux,Windows-  Requires:
 # Clang v3.5 or later, and GNU binutils targeting 
aarch64-linux-gnu or arm-linux-gnueabi
 #Optional:
 # Required to build platforms or ACPI tables:
 #   Intel(r) ACPI Compiler from
 #   https://acpica.org/downloads
+#   CLANG38  -Linux-  Requires:
+# Clang v3.8 or later, LLVMgold plugin and GNU 
binutils 2.26 targeting x86_64-linux-gnu
+#Optional:
+# Required to build platforms or ACPI tables:
+#   Intel(r) ACPI Compiler from
+#   https://acpica.org/downloads
+#   CLANGSCAN38 -Linux-  Requires:
+#