Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-14 Thread Ard Biesheuvel
On 13 August 2015 at 21:57, Bill Paul wp...@windriver.com wrote:
 Of all the gin joints in all the towns in all the world, Ard Biesheuvel had to
 walk into mine at 12:25:31 on Thursday 13 August 2015 and say:

 On 13 August 2015 at 21:14, David Woodhouse dw...@infradead.org wrote:
  On Thu, 2015-08-13 at 13:25 -0500, Scott Duplichan wrote:
  A while back I experimented with mingw as a Windows hosted gcc tool
  chain for EDK2. It is usable, but has limitations. From a 2014 email
  to this list:
 
  1) Image is big due to dead library code in final image.
  2) Default calling convention is different. This makes MINGW code
 
 generation differ from gcc-linux code generation.
 
  3) MINGW sizeof long differs from gnu-linux size of long, resulting
 
 in more code generation differences.
 
  These aren't new issues, surely? The GCC builds are still larger than
  the MSVC builds, largely due to CFLAGS as recently discussed but also
  because we don't yet have LTO for GCC, do we? And we don't use
  -ffunction-sections --gc-sections and other tricks so we can only drop
  unused things out at the C file granularity.

 Another reason to unify the GCC compiler and linker flags: we do use
 -ffunction-sections -fdata-sections and --gc-sections, but only for
 GCC44 - GCC49, and these flags are not inherited by UNIXGCC et al

 So I suppose the linker output is a PE/COFF application when using
 MinGW, and the BaseTools/ just skip the ELF conversion in this case? I
 wonder whether --gc-sections works well here, but I can have a look.

 For the record, I tend to use the UNIXGCC cross-build toolchain on my
 FreeBSD/amd64 9.1-RELEASE machine at work. The system compiler is gcc 4.2.1,
 and it and the system linker aren't quite sexy enough to build EDK2/OVMF on
 their own. (Newer releases might be better, but I'm not in a position to
 upgrade my work system right now.) Instead I've been using the mingw-gcc-
 build.py script to bootstrap a GCC MinGW-targeted toolchain, and it's worked
 fine for me so far.


OK, good to know.

 Yes, the MinGW toolchain does generate PE/COFF objects directly so you can
 skip the objcopy conversion step from ELF. You need a set of binutils to
 handle this off course, but the mingw-gcc-build.py bootstraps those for you
 too.


I am currently testing with my distro supplied MinGW, which is based on 4.8.3

 I recently experimented with updating the mingw-gcc-build.py script to use a
 newer version of GCC, in this case 4.9.3 and binutils 2.25. I had to update
 tools_def.template a little to get it to work. I was planning to submit this
 as a patch but was waiting for the dust around GCC rule tweaking to settle. So
 far it hasn't settled yet. :)


Well, MinGW is a really good data point, so if I end up proposing
another round of patches that unifies GCC support, I will take this
into account.

 The only real gotcha I ran into is the underscore convention. With MinGW,
 the rule seems to be that for IA32, all symbols get a leading underscore, but
 on X64 they do not. This is something that seems to have changed between GCC
 4.3.0 and 4.9.3. For example, currently (well, unless someone changed them
 recently), the following generic GCC rules exist:

 DEFINE GCC_IA32_X64_ASLDLINK_FLAGS = DEF(GCC_IA32_X64_DLINK_COMMON) --entry
 _ReferenceAcpiTable -u $(IMAGE_ENTRY_POINT)

 DEFINE GCC_IA32_X64_DLINK_FLAGS= DEF(GCC_IA32_X64_DLINK_COMMON) --entry
 _$(IMAGE_ENTRY_POINT) --file-alignment 0x20 --section-alignment 0x20 -Map $(D
 EST_DIR_DEBUG)/$(BASE_NAME).map

 The IA32_X64 in the names assumes that these rules apply for both targets, and
 you can see that they apply leading underscores to symbol names
 (_ReferenceAcpiTable, _$(IMAGE_ENTRY_POINT)). This worked for GCC 4.3.0, but
 GCC 4.9.3 only automatically applies the leading underscore for IA32, not for
 X64. So this rule can't be applied to both anymore.

 I'm not sure of the reason for the difference, but I tried several different
 GCC releases newer than 4.3.0 and they all had the same behavior. I suspect
 the newer behavior (X64 doesn't use leading underscores) is correct and the
 earlier behavior was buggy. Maybe someone who knows more about the MS
 toolchain can clear this up.


As far as I could dig up, the leading underscore convention used by
Microsoft is 32-bit only. I guess that means EDK2 was right all along
(unsurprisingly), MinGW got fixed along the way, and the best way
forward would be to use the correct convention for UNIXGCC/X64 builds
from now on, which would mean dropping support for pre-4.3.0

 In any case, when I first tried it, I found that IA32 OVMF builds produced
 working firmware images, but X64 OVMF builds did not. The underscore
 difference meant that the linker could not find _$(IMAGE_ENTRY_POINT) which
 lead to the generation of bogus images, though it did not lead to the build
 aborting right away. Instead there was a warning in the build log which took
 me a while to track down.

 To address this problem, I created 

Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-14 Thread David Woodhouse
On Thu, 2015-08-13 at 23:54 -0500, Scott Duplichan wrote:
 David Woodhouse [mailto:dw...@infradead.org] wrote:
 ]On Thu, 2015-08-13 at 13:25 -0500, Scott Duplichan wrote:
 ] A while back I experimented with mingw as a Windows hosted gcc tool
 ] chain for EDK2. It is usable, but has limitations. From a 2014 email
 ] to this list:
 ]
 ] 1) Image is big due to dead library code in final image.
 ] 2) Default calling convention is different. This makes MINGW code
 ]generation differ from gcc-linux code generation.
 ] 3) MINGW sizeof long differs from gnu-linux size of long, resulting
 ]in more code generation differences.
 ]
 ]These aren't new issues, surely? The GCC builds are still larger than
 ]the MSVC builds, largely due to CFLAGS as recently discussed but also
 ]because we don't yet have LTO for GCC, do we? And we don't use 
 ]-ffunction-sections --gc-sections and other tricks so we can only drop
 ]unused things out at the C file granularity.
 
 Issue 1) is something separate from missing GCC LTO, and affects only
 mingw: https://sourceware.org/bugzilla/show_bug.cgi?id=11539.

I note Nick is waiting for test results in that bug. Is anyone trying
the patch he just committed?

  As for
 GCC LTO, the 07/14/2015 patch didn't get enough reviews for approval. 

Pet peeve: There aren't 14 months in 2015. Use ISO standard dates (2015
-07-14) unless you *know* you're only talking to Americans, please.

Or http://permalink.gmane.org/gmane.comp.bios.tianocore.devel/17382 wou
ld be even better :)

 With equal settings, GCC is competitive with Microsoft on code size now.

Nice.

 ](I wonder if we could get the MSVC build running under wine... now
 ]*that* would be useful)
 
 Brian Johnson says this works (using the DDK3790 tool chain). The
 newer DDK7600 most likely works too. See the 05/27/2015 email:
 Re: [edk2] CorebootModulePkg: gcc reports conflicting types for
 'CbParseAcpiTable'

Hm, I don't seem to have that one. Found it at
https://sourceforge.net/p/edk2/mailman/message/34151871/ though.

Brian said it took 'a lot of one-time setup' to get the DDK compilers
running under wine, but that's precisely the kind of thing that could
be scripted, surely?

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-14 Thread Bill Paul
Of all the gin joints in all the towns in all the world, Ard Biesheuvel had to 
walk into mine at 00:55:26 on Friday 14 August 2015 and say:

 On 13 August 2015 at 21:57, Bill Paul wp...@windriver.com wrote:
  Of all the gin joints in all the towns in all the world, Ard Biesheuvel
  had to
  
  walk into mine at 12:25:31 on Thursday 13 August 2015 and say:
  On 13 August 2015 at 21:14, David Woodhouse dw...@infradead.org wrote:
   On Thu, 2015-08-13 at 13:25 -0500, Scott Duplichan wrote:
   A while back I experimented with mingw as a Windows hosted gcc tool
   chain for EDK2. It is usable, but has limitations. From a 2014 email
   to this list:
   
   1) Image is big due to dead library code in final image.
   2) Default calling convention is different. This makes MINGW code
   
  generation differ from gcc-linux code generation.
   
   3) MINGW sizeof long differs from gnu-linux size of long, resulting
   
  in more code generation differences.
   
   These aren't new issues, surely? The GCC builds are still larger than
   the MSVC builds, largely due to CFLAGS as recently discussed but also
   because we don't yet have LTO for GCC, do we? And we don't use
   -ffunction-sections --gc-sections and other tricks so we can only drop
   unused things out at the C file granularity.
  
  Another reason to unify the GCC compiler and linker flags: we do use
  -ffunction-sections -fdata-sections and --gc-sections, but only for
  GCC44 - GCC49, and these flags are not inherited by UNIXGCC et al
  
  So I suppose the linker output is a PE/COFF application when using
  MinGW, and the BaseTools/ just skip the ELF conversion in this case? I
  wonder whether --gc-sections works well here, but I can have a look.
  
  For the record, I tend to use the UNIXGCC cross-build toolchain on my
  FreeBSD/amd64 9.1-RELEASE machine at work. The system compiler is gcc
  4.2.1, and it and the system linker aren't quite sexy enough to build
  EDK2/OVMF on their own. (Newer releases might be better, but I'm not in
  a position to upgrade my work system right now.) Instead I've been using
  the mingw-gcc- build.py script to bootstrap a GCC MinGW-targeted
  toolchain, and it's worked fine for me so far.
 
 OK, good to know.
 
  Yes, the MinGW toolchain does generate PE/COFF objects directly so you
  can skip the objcopy conversion step from ELF. You need a set of
  binutils to handle this off course, but the mingw-gcc-build.py
  bootstraps those for you too.
 
 I am currently testing with my distro supplied MinGW, which is based on
 4.8.3

That should be ok. I tried a couple different cases including 4.6.x, 4.8.x and 
4.9.x which worked ok. I was able to bootstrap a GCC 5 compiler but it crashed 
with an internal compiler error somewhere during the OVMF build.
 
  I recently experimented with updating the mingw-gcc-build.py script to
  use a newer version of GCC, in this case 4.9.3 and binutils 2.25. I had
  to update tools_def.template a little to get it to work. I was planning
  to submit this as a patch but was waiting for the dust around GCC rule
  tweaking to settle. So far it hasn't settled yet. :)
 
 Well, MinGW is a really good data point, so if I end up proposing
 another round of patches that unifies GCC support, I will take this
 into account.

Okay.
 
  The only real gotcha I ran into is the underscore convention. With
  MinGW, the rule seems to be that for IA32, all symbols get a leading
  underscore, but on X64 they do not. This is something that seems to have
  changed between GCC 4.3.0 and 4.9.3. For example, currently (well,
  unless someone changed them recently), the following generic GCC rules
  exist:
  
  DEFINE GCC_IA32_X64_ASLDLINK_FLAGS = DEF(GCC_IA32_X64_DLINK_COMMON)
  --entry _ReferenceAcpiTable -u $(IMAGE_ENTRY_POINT)
  
  DEFINE GCC_IA32_X64_DLINK_FLAGS= DEF(GCC_IA32_X64_DLINK_COMMON)
  --entry _$(IMAGE_ENTRY_POINT) --file-alignment 0x20 --section-alignment
  0x20 -Map $(D EST_DIR_DEBUG)/$(BASE_NAME).map
  
  The IA32_X64 in the names assumes that these rules apply for both
  targets, and you can see that they apply leading underscores to symbol
  names
  (_ReferenceAcpiTable, _$(IMAGE_ENTRY_POINT)). This worked for GCC 4.3.0,
  but GCC 4.9.3 only automatically applies the leading underscore for
  IA32, not for X64. So this rule can't be applied to both anymore.
  
  I'm not sure of the reason for the difference, but I tried several
  different GCC releases newer than 4.3.0 and they all had the same
  behavior. I suspect the newer behavior (X64 doesn't use leading
  underscores) is correct and the earlier behavior was buggy. Maybe
  someone who knows more about the MS toolchain can clear this up.
 
 As far as I could dig up, the leading underscore convention used by
 Microsoft is 32-bit only. I guess that means EDK2 was right all along
 (unsurprisingly), MinGW got fixed along the way, and the best way
 forward would be to use the correct convention for UNIXGCC/X64 builds
 from now on, which would 

Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread Ard Biesheuvel
On 13 August 2015 at 08:27, Ard Biesheuvel ard.biesheu...@linaro.org wrote:
 On 12 August 2015 at 23:48, David Woodhouse dw...@infradead.org wrote:
 On Wed, 2015-08-12 at 09:08 +0200, Ard Biesheuvel wrote:
 Is there any reason these are kept out of sync? Are UNIXGCC and CYGGCC
 known to be widely used in some particular environment? If not, I
 think it makes sense to merge them, i.e., retain the UNIXGCC and
 CYGGCC toolchain names, but make them use the same options for IA32
 and X64 as GCC44 - GCC49 do. (UNIXGCC and CYGGCC are unversioned, so
 it is unclear which GCC version they expect anyway)

 That would seem to make sense.

 FWIW it doesn't actually build with MinGW (which is what UNIXGCC is)
 these days anyway. You might make it work with -fstack-check=specific:
 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67169


 OK noted

 However, building with i686-w64-mingw32-gcc produces lots of
 -Wint-to-pointer-cast issues, so I am not sure how this is supposed to
 work.
 Is this the wrong mingw version to use? I guess UINTN is typedef'ed to
 'unsigned long long' on LLP64, otherwise the assumption that
 sizeof(UINTN) == sizeof(VOID*) that is made throughout the EDK2 code
 base obviously does not hold.


OK, never mind. I should be using x86_64-w64-mingw32-gcc to build for X64
Sorry for the noise
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread Bruce Cran

On 8/12/15 11:47 PM, Gao, Liming wrote:


 Add -std=gnu89 to the CC flags.

This is the default for gcc 4.x, so it doesn't change anything for those tool 
chains (other than making the command line slightly longer). GCC5 however, 
defaults to -std=gnu11. By adding -std=gnu89, gcc5 will behave more like the 
current gcc 4.x compilers, and reject use of some C99 features not supported by 
the C99-challenged Microsoft tool chains that EDK2 coders must maintain 
compatibility with.


We might want to revisit this in the nearish future as people move to 
newer versions of Visual Studio: since VS 2012 Microsoft have been 
adding more support for C99. In VS 2013 they added loads of new 
functionality, and have improved it with 2015:


C99 Conformance: Visual Studio 2015 fully implements the C99 Standard 
Library, with the exception of any library features that depend on 
compiler features not yet supported by the Visual C++ compiler (for 
example, tgmath.h is not implemented).


--
Bruce
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread Scott Duplichan
Ard Biesheuvel [mailto:ard.biesheu...@linaro.org] wrote:

]Sent: Thursday, August 13, 2015 01:40 AM
]To: David Woodhouse dw...@infradead.org
]Cc: Justen, Jordan L jordan.l.jus...@intel.com; edk2-devel@lists.01.org 
edk2-de...@ml01.01.org; Liu, Yingke D ]yingke.d@intel.com; Gao, Liming 
liming@intel.com
]Subject: Re: [edk2] [RFC PATCH 0/4] unify GCC command line options
]
]On 13 August 2015 at 08:27, Ard Biesheuvel ard.biesheu...@linaro.org wrote:
] On 12 August 2015 at 23:48, David Woodhouse dw...@infradead.org wrote:
] On Wed, 2015-08-12 at 09:08 +0200, Ard Biesheuvel wrote:
] Is there any reason these are kept out of sync? Are UNIXGCC and CYGGCC
] known to be widely used in some particular environment? If not, I
] think it makes sense to merge them, i.e., retain the UNIXGCC and
] CYGGCC toolchain names, but make them use the same options for IA32
] and X64 as GCC44 - GCC49 do. (UNIXGCC and CYGGCC are unversioned, so
] it is unclear which GCC version they expect anyway)
]
] That would seem to make sense.
]
] FWIW it doesn't actually build with MinGW (which is what UNIXGCC is)
] these days anyway. You might make it work with -fstack-check=specific:
] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67169
]
]
] OK noted
]
] However, building with i686-w64-mingw32-gcc produces lots of
] -Wint-to-pointer-cast issues, so I am not sure how this is supposed to
] work.
] Is this the wrong mingw version to use? I guess UINTN is typedef'ed to
] 'unsigned long long' on LLP64, otherwise the assumption that
] sizeof(UINTN) == sizeof(VOID*) that is made throughout the EDK2 code
] base obviously does not hold.
]
]
]OK, never mind. I should be using x86_64-w64-mingw32-gcc to build for X64
]Sorry for the noise

A while back I experimented with mingw as a Windows hosted gcc tool
chain for EDK2. It is usable, but has limitations. From a 2014 email
to this list:

1) Image is big due to dead library code in final image.
2) Default calling convention is different. This makes MINGW code
   generation differ from gcc-linux code generation.
3) MINGW sizeof long differs from gnu-linux size of long, resulting
   in more code generation differences.
4) Gcc linker plug-in for link time code generation is not yet
   available for Windows.

To overcome these problems, I started using a non-mingw Windows hosted
build of gcc that targets x86_64-linux-gnu. This approach solves
problems 1-3. A patch solves problem 4. In addition, this approach is
easily extended to support arm, aarch64, and ia64.

Thanks,
Scott




___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread David Woodhouse
On Thu, 2015-08-13 at 21:25 +0200, Ard Biesheuvel wrote:

 Another reason to unify the GCC compiler and linker flags: we do use
 -ffunction-sections -fdata-sections and --gc-sections, but only for
 GCC44 - GCC49, and these flags are not inherited by UNIXGCC et al

Ah, I didn't realise we did. That's good. As long as it's having the
desired effect :)

 So I suppose the linker output is a PE/COFF application when using
 MinGW, and the BaseTools/ just skip the ELF conversion in this case? I
 wonder whether --gc-sections works well here, but I can have a look.
 
 Btw I managed to complete the OVMF/X86 build with Mingw without any
 reported errors regarding the stack protector, which I did not 
 disable afaik.
 What is the symptom you observed here?

If you use a lot of stack, MinGW GCC will emit a call to __chkstk_ms():
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67169

Although I only saw this in OpenSSL code, not elsewhere. Not sure why.
Perhaps nothing else uses enough stack to trigger it.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread Bill Paul
Of all the gin joints in all the towns in all the world, Ard Biesheuvel had to 
walk into mine at 12:25:31 on Thursday 13 August 2015 and say:

 On 13 August 2015 at 21:14, David Woodhouse dw...@infradead.org wrote:
  On Thu, 2015-08-13 at 13:25 -0500, Scott Duplichan wrote:
  A while back I experimented with mingw as a Windows hosted gcc tool
  chain for EDK2. It is usable, but has limitations. From a 2014 email
  to this list:
  
  1) Image is big due to dead library code in final image.
  2) Default calling convention is different. This makes MINGW code
  
 generation differ from gcc-linux code generation.
  
  3) MINGW sizeof long differs from gnu-linux size of long, resulting
  
 in more code generation differences.
  
  These aren't new issues, surely? The GCC builds are still larger than
  the MSVC builds, largely due to CFLAGS as recently discussed but also
  because we don't yet have LTO for GCC, do we? And we don't use
  -ffunction-sections --gc-sections and other tricks so we can only drop
  unused things out at the C file granularity.
 
 Another reason to unify the GCC compiler and linker flags: we do use
 -ffunction-sections -fdata-sections and --gc-sections, but only for
 GCC44 - GCC49, and these flags are not inherited by UNIXGCC et al
 
 So I suppose the linker output is a PE/COFF application when using
 MinGW, and the BaseTools/ just skip the ELF conversion in this case? I
 wonder whether --gc-sections works well here, but I can have a look.

For the record, I tend to use the UNIXGCC cross-build toolchain on my 
FreeBSD/amd64 9.1-RELEASE machine at work. The system compiler is gcc 4.2.1, 
and it and the system linker aren't quite sexy enough to build EDK2/OVMF on 
their own. (Newer releases might be better, but I'm not in a position to 
upgrade my work system right now.) Instead I've been using the mingw-gcc-
build.py script to bootstrap a GCC MinGW-targeted toolchain, and it's worked 
fine for me so far.

Yes, the MinGW toolchain does generate PE/COFF objects directly so you can 
skip the objcopy conversion step from ELF. You need a set of binutils to 
handle this off course, but the mingw-gcc-build.py bootstraps those for you 
too.

I recently experimented with updating the mingw-gcc-build.py script to use a 
newer version of GCC, in this case 4.9.3 and binutils 2.25. I had to update 
tools_def.template a little to get it to work. I was planning to submit this 
as a patch but was waiting for the dust around GCC rule tweaking to settle. So 
far it hasn't settled yet. :)

The only real gotcha I ran into is the underscore convention. With MinGW, 
the rule seems to be that for IA32, all symbols get a leading underscore, but 
on X64 they do not. This is something that seems to have changed between GCC 
4.3.0 and 4.9.3. For example, currently (well, unless someone changed them 
recently), the following generic GCC rules exist:

DEFINE GCC_IA32_X64_ASLDLINK_FLAGS = DEF(GCC_IA32_X64_DLINK_COMMON) --entry 
_ReferenceAcpiTable -u $(IMAGE_ENTRY_POINT)

DEFINE GCC_IA32_X64_DLINK_FLAGS= DEF(GCC_IA32_X64_DLINK_COMMON) --entry 
_$(IMAGE_ENTRY_POINT) --file-alignment 0x20 --section-alignment 0x20 -Map $(D
EST_DIR_DEBUG)/$(BASE_NAME).map

The IA32_X64 in the names assumes that these rules apply for both targets, and 
you can see that they apply leading underscores to symbol names 
(_ReferenceAcpiTable, _$(IMAGE_ENTRY_POINT)). This worked for GCC 4.3.0, but 
GCC 4.9.3 only automatically applies the leading underscore for IA32, not for 
X64. So this rule can't be applied to both anymore.

I'm not sure of the reason for the difference, but I tried several different 
GCC releases newer than 4.3.0 and they all had the same behavior. I suspect 
the newer behavior (X64 doesn't use leading underscores) is correct and the 
earlier behavior was buggy. Maybe someone who knows more about the MS 
toolchain can clear this up.

In any case, when I first tried it, I found that IA32 OVMF builds produced 
working firmware images, but X64 OVMF builds did not. The underscore 
difference meant that the linker could not find _$(IMAGE_ENTRY_POINT) which 
lead to the generation of bogus images, though it did not lead to the build 
aborting right away. Instead there was a warning in the build log which took 
me a while to track down.

To address this problem, I created specific GCC_X64_ASLDLINK_FLAGS and 
GCC_X64_DLINK_FLAGS rules which omit the leading underscore and set the 
UNIXGCC X64 case to use them instead of the common IA32/X64 one. This struck 
me as inelegant, but I couldn't think of what else to do at the time.

A slightly smaller gotcha is that the GCC MinGW bootstrap build insists on 
there being an include/mingw directory present in your intended installation 
path. It has to be there, even if it's empty. I changed the MakeDirs in mingw-
gcc-build.py to include this path.

I put my modified copies of mingw-gcc-build.py and tools_def.template at:

http://people.freebsd.org/~wpaul/edk2

Like I said, I 

Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread David Woodhouse
On Thu, 2015-08-13 at 13:25 -0500, Scott Duplichan wrote:
 A while back I experimented with mingw as a Windows hosted gcc tool
 chain for EDK2. It is usable, but has limitations. From a 2014 email
 to this list:

 1) Image is big due to dead library code in final image.
 2) Default calling convention is different. This makes MINGW code
generation differ from gcc-linux code generation.
 3) MINGW sizeof long differs from gnu-linux size of long, resulting
in more code generation differences.

These aren't new issues, surely? The GCC builds are still larger than
the MSVC builds, largely due to CFLAGS as recently discussed but also
because we don't yet have LTO for GCC, do we? And we don't use 
-ffunction-sections --gc-sections and other tricks so we can only drop
unused things out at the C file granularity.

We already have different calling conventions, and different size
'long', between GCC and MSVC builds. MinGW is consistent with the MSVC
builds for this — why would that be a problem? That was the whole
*point* in wanting to use MinGW, for me — to test a LLP64 build without
the pain of actually having to use Windows+MSVC.

(I wonder if we could get the MSVC build running under wine... now
*that* would be useful)

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread Ard Biesheuvel
On 13 August 2015 at 21:14, David Woodhouse dw...@infradead.org wrote:
 On Thu, 2015-08-13 at 13:25 -0500, Scott Duplichan wrote:
 A while back I experimented with mingw as a Windows hosted gcc tool
 chain for EDK2. It is usable, but has limitations. From a 2014 email
 to this list:

 1) Image is big due to dead library code in final image.
 2) Default calling convention is different. This makes MINGW code
generation differ from gcc-linux code generation.
 3) MINGW sizeof long differs from gnu-linux size of long, resulting
in more code generation differences.

 These aren't new issues, surely? The GCC builds are still larger than
 the MSVC builds, largely due to CFLAGS as recently discussed but also
 because we don't yet have LTO for GCC, do we? And we don't use
 -ffunction-sections --gc-sections and other tricks so we can only drop
 unused things out at the C file granularity.


Another reason to unify the GCC compiler and linker flags: we do use
-ffunction-sections -fdata-sections and --gc-sections, but only for
GCC44 - GCC49, and these flags are not inherited by UNIXGCC et al

So I suppose the linker output is a PE/COFF application when using
MinGW, and the BaseTools/ just skip the ELF conversion in this case? I
wonder whether --gc-sections works well here, but I can have a look.

Btw I managed to complete the OVMF/X86 build with Mingw without any
reported errors regarding the stack protector, which I did not disable
afaik.
What is the symptom you observed here?

 We already have different calling conventions, and different size
 'long', between GCC and MSVC builds. MinGW is consistent with the MSVC
 builds for this — why would that be a problem? That was the whole
 *point* in wanting to use MinGW, for me — to test a LLP64 build without
 the pain of actually having to use Windows+MSVC.


Indeed. The primary motivation for my recent involvement with the
toolchain configs is to help ensure that the representative sample of
toolchains we may propose for pre-commit compile tests sufficiently
covers the architectures and toolchains we care about. And MinGW would
be useful for this purpose as well, especially since it is a free
toolchain that implements LLP64.

-- 
Ard.

 (I wonder if we could get the MSVC build running under wine... now
 *that* would be useful)

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-13 Thread Scott Duplichan
David Woodhouse [mailto:dw...@infradead.org] wrote:

]Sent: Thursday, August 13, 2015 02:15 PM
]To: Scott Duplichan sc...@notabs.org; 'Ard Biesheuvel' 
ard.biesheu...@linaro.org
]Cc: 'Justen, Jordan L' jordan.l.jus...@intel.com; 'edk2-devel@lists.01.org' 
edk2-de...@ml01.01.org; 'Liu, ]Yingke D' yingke.d@intel.com; 'Gao, 
Liming' liming@intel.com
]Subject: Re: [edk2] [RFC PATCH 0/4] unify GCC command line options
]
]On Thu, 2015-08-13 at 13:25 -0500, Scott Duplichan wrote:
] A while back I experimented with mingw as a Windows hosted gcc tool
] chain for EDK2. It is usable, but has limitations. From a 2014 email
] to this list:
]
] 1) Image is big due to dead library code in final image.
] 2) Default calling convention is different. This makes MINGW code
]generation differ from gcc-linux code generation.
] 3) MINGW sizeof long differs from gnu-linux size of long, resulting
]in more code generation differences.
]
]These aren't new issues, surely? The GCC builds are still larger than
]the MSVC builds, largely due to CFLAGS as recently discussed but also
]because we don't yet have LTO for GCC, do we? And we don't use 
]-ffunction-sections --gc-sections and other tricks so we can only drop
]unused things out at the C file granularity.

Issue 1) is something separate from missing GCC LTO, and affects only
mingw: https://sourceware.org/bugzilla/show_bug.cgi?id=11539. As for
GCC LTO, the 07/14/2015 patch didn't get enough reviews for approval. 
With equal settings, GCC is competitive with Microsoft on code size now.

]We already have different calling conventions, and different size
]'long', between GCC and MSVC builds. MinGW is consistent with the MSVC
]builds for this — why would that be a problem? That was the whole
]*point* in wanting to use MinGW, for me — to test a LLP64 build without
]the pain of actually having to use Windows+MSVC.

I left out a part of the email chain that puts the quoted text in
context. I started off proposing Windows hosted GCC builds as a way
for Windows users to check their patches for warnings that only happen
with GCC. After Liming asked if mingw would produce exactly the same
warnings as Linux hosted GCC, I realized the answer was no. This is
due in part to the LLP64/LP64 difference between mingw and standard
Linux hosted GCC. So that difference is good for your need but bad 
for a Windows user trying to find warnings not caught by the Microsoft
tool chains.

](I wonder if we could get the MSVC build running under wine... now
]*that* would be useful)

Brian Johnson says this works (using the DDK3790 tool chain). The
newer DDK7600 most likely works too. See the 05/27/2015 email:
Re: [edk2] CorebootModulePkg: gcc reports conflicting types for
'CbParseAcpiTable'

Thanks,
Scott

]-- 
]David WoodhouseOpen Source Technology Centre
]david.woodho...@intel.com  Intel Corporation

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-12 Thread David Woodhouse
On Wed, 2015-08-12 at 09:08 +0200, Ard Biesheuvel wrote:
 Is there any reason these are kept out of sync? Are UNIXGCC and CYGGCC
 known to be widely used in some particular environment? If not, I
 think it makes sense to merge them, i.e., retain the UNIXGCC and
 CYGGCC toolchain names, but make them use the same options for IA32
 and X64 as GCC44 - GCC49 do. (UNIXGCC and CYGGCC are unversioned, so
 it is unclear which GCC version they expect anyway)

That would seem to make sense.

FWIW it doesn't actually build with MinGW (which is what UNIXGCC is)
these days anyway. You might make it work with -fstack-check=specific: 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67169


-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-12 Thread Scott Duplichan
Ard Biesheuvel [mailto:ard.biesheu...@linaro.org] wrote:

]Sent: Friday, August 07, 2015 10:05 AM
]To: edk2-de...@ml01.01.org; jordan.l.jus...@intel.com; yingke.d@intel.com
]Cc: Ard Biesheuvel ard.biesheu...@linaro.org
]Subject: [edk2] [RFC PATCH 0/4] unify GCC command line options
]
]This unifies all command line option defines in tools_def.txt, in order
]to reduce the maintenance burden.
]
]Note that this does not add or remove any GCC4x toolchains, it just folds
]the common DEFINEs into a single series of GCC4X defines.

Here is one suggestion:

Add -std=gnu89 to the CC flags.

This is the default for gcc 4.x, so it doesn't change anything for those tool
chains (other than making the command line slightly longer). GCC5 however,
defaults to -std=gnu11. By adding -std=gnu89, gcc5 will behave more like the
current gcc 4.x compilers, and reject use of some C99 features not supported
by the C99-challenged Microsoft tool chains that EDK2 coders must maintain
compatibility with.

A proposal to add a GCC5 tool chain definition was not approved, so it is
especially important that the GCC49 definition works as well as possible
with GCC5.

Thanks,
Scott

]Ard Biesheuvel (4):
]  BaseTools GCC: remove 4.9 specific linker alignment override
]  BaseTools GCC: unify warning flags for all GCC versions
]  BaseTools GCC: unify ARM CC flags for all GCC versions
]  BaseTools GCC: unify GCC toolchain command line options
]
] BaseTools/Conf/tools_def.template | 306 
] 1 file changed, 117 insertions(+), 189 deletions(-)
]
]-- 
]1.9.1


___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-12 Thread Gao, Liming
I agree to unify GCC option and add -std=gun89 flag. 

Thanks
Liming
-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Scott 
Duplichan
Sent: Thursday, August 13, 2015 9:25 AM
To: 'Ard Biesheuvel'; edk2-de...@ml01.01.org; Justen, Jordan L; Liu, Yingke D
Subject: Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

Ard Biesheuvel [mailto:ard.biesheu...@linaro.org] wrote:

]Sent: Friday, August 07, 2015 10:05 AM
]To: edk2-de...@ml01.01.org; jordan.l.jus...@intel.com; yingke.d@intel.com
]Cc: Ard Biesheuvel ard.biesheu...@linaro.org
]Subject: [edk2] [RFC PATCH 0/4] unify GCC command line options ] ]This unifies 
all command line option defines in tools_def.txt, in order ]to reduce the 
maintenance burden.
]
]Note that this does not add or remove any GCC4x toolchains, it just folds ]the 
common DEFINEs into a single series of GCC4X defines.

Here is one suggestion:

Add -std=gnu89 to the CC flags.

This is the default for gcc 4.x, so it doesn't change anything for those tool 
chains (other than making the command line slightly longer). GCC5 however, 
defaults to -std=gnu11. By adding -std=gnu89, gcc5 will behave more like the 
current gcc 4.x compilers, and reject use of some C99 features not supported by 
the C99-challenged Microsoft tool chains that EDK2 coders must maintain 
compatibility with.

A proposal to add a GCC5 tool chain definition was not approved, so it is 
especially important that the GCC49 definition works as well as possible with 
GCC5.

Thanks,
Scott

]Ard Biesheuvel (4):
]  BaseTools GCC: remove 4.9 specific linker alignment override ]  BaseTools 
GCC: unify warning flags for all GCC versions ]  BaseTools GCC: unify ARM CC 
flags for all GCC versions ]  BaseTools GCC: unify GCC toolchain command line 
options ] ] BaseTools/Conf/tools_def.template | 306  ] 1 
file changed, 117 insertions(+), 189 deletions(-) ]
]--
]1.9.1


___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-12 Thread Ard Biesheuvel
On 10 August 2015 at 10:00, Gao, Liming liming@intel.com wrote:
 Ard:
   This patch introduces GCC4X_ for common GCC option. It may be common for 
 GCC5, GCC6... So, how about use GCC_ prefix for all GCC common option?


OK, I have been looking into this. It is mostly possible to fold all
GCC 4.x options into the GCC_ defines. There are a couple that are
also used for UNIXGCC and CYGGCC with slightly different options:

I.e.,

DEFINE GCC_ALL_CC_FLAGS= -g -Os -fshort-wchar
-fno-strict-aliasing -Wall -Werror -Wno-array-bounds -c -include
AutoGen.h
DEFINE GCC_IA32_CC_FLAGS   = DEF(GCC_ALL_CC_FLAGS) -m32
-malign-double -freorder-blocks -freorder-blocks-and-partition -O2
-mno-stack-arg-probe

and

DEFINE GCC4X_ALL_CC_FLAGS= -g -fshort-wchar
-fno-strict-aliasing -Wall -Werror -Wno-array-bounds -Wno-address
-Wno-unused-but-set-variable -ffunction-sections -fdata-sections -c
-include AutoGen.h -DSTRING_ARRAY_NAME=$(BASE_NAME)Strings
DEFINE GCC4X_IA32_CC_FLAGS   = DEF(GCC4X_ALL_CC_FLAGS) -m32
-malign-double -fno-stack-protector -D EFI32

where the former is used by UNIXGCC and CYGGCC, and the latter are
shared between GCC44 - GCC49.

Is there any reason these are kept out of sync? Are UNIXGCC and CYGGCC
known to be widely used in some particular environment? If not, I
think it makes sense to merge them, i.e., retain the UNIXGCC and
CYGGCC toolchain names, but make them use the same options for IA32
and X64 as GCC44 - GCC49 do. (UNIXGCC and CYGGCC are unversioned, so
it is unclear which GCC version they expect anyway)

-- 
Ard.


 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Ard 
 Biesheuvel
 Sent: Friday, August 7, 2015 11:05 PM
 To: edk2-devel@lists.01.org; Justen, Jordan L; Liu, Yingke D
 Cc: Ard Biesheuvel
 Subject: [edk2] [RFC PATCH 0/4] unify GCC command line options

 This unifies all command line option defines in tools_def.txt, in order to 
 reduce the maintenance burden.

 Note that this does not add or remove any GCC4x toolchains, it just folds the 
 common DEFINEs into a single series of GCC4X defines.

 Ard Biesheuvel (4):
   BaseTools GCC: remove 4.9 specific linker alignment override
   BaseTools GCC: unify warning flags for all GCC versions
   BaseTools GCC: unify ARM CC flags for all GCC versions
   BaseTools GCC: unify GCC toolchain command line options

  BaseTools/Conf/tools_def.template | 306 
  1 file changed, 117 insertions(+), 189 deletions(-)

 --
 1.9.1

 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-10 Thread Gao, Liming
Ard:
  This patch introduces GCC4X_ for common GCC option. It may be common for 
GCC5, GCC6... So, how about use GCC_ prefix for all GCC common option?

Thanks
Liming
-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Ard 
Biesheuvel
Sent: Friday, August 7, 2015 11:05 PM
To: edk2-devel@lists.01.org; Justen, Jordan L; Liu, Yingke D
Cc: Ard Biesheuvel
Subject: [edk2] [RFC PATCH 0/4] unify GCC command line options

This unifies all command line option defines in tools_def.txt, in order to 
reduce the maintenance burden.

Note that this does not add or remove any GCC4x toolchains, it just folds the 
common DEFINEs into a single series of GCC4X defines.

Ard Biesheuvel (4):
  BaseTools GCC: remove 4.9 specific linker alignment override
  BaseTools GCC: unify warning flags for all GCC versions
  BaseTools GCC: unify ARM CC flags for all GCC versions
  BaseTools GCC: unify GCC toolchain command line options

 BaseTools/Conf/tools_def.template | 306 
 1 file changed, 117 insertions(+), 189 deletions(-)

--
1.9.1

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-10 Thread Ard Biesheuvel
On 10 August 2015 at 10:00, Gao, Liming liming@intel.com wrote:
 Ard:
   This patch introduces GCC4X_ for common GCC option. It may be common for 
 GCC5, GCC6... So, how about use GCC_ prefix for all GCC common option?


Yes, I agree that would be better. I will change it.

Thanks,
Ard.


 -Original Message-
 From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Ard 
 Biesheuvel
 Sent: Friday, August 7, 2015 11:05 PM
 To: edk2-devel@lists.01.org; Justen, Jordan L; Liu, Yingke D
 Cc: Ard Biesheuvel
 Subject: [edk2] [RFC PATCH 0/4] unify GCC command line options

 This unifies all command line option defines in tools_def.txt, in order to 
 reduce the maintenance burden.

 Note that this does not add or remove any GCC4x toolchains, it just folds the 
 common DEFINEs into a single series of GCC4X defines.

 Ard Biesheuvel (4):
   BaseTools GCC: remove 4.9 specific linker alignment override
   BaseTools GCC: unify warning flags for all GCC versions
   BaseTools GCC: unify ARM CC flags for all GCC versions
   BaseTools GCC: unify GCC toolchain command line options

  BaseTools/Conf/tools_def.template | 306 
  1 file changed, 117 insertions(+), 189 deletions(-)

 --
 1.9.1

 ___
 edk2-devel mailing list
 edk2-devel@lists.01.org
 https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


[edk2] [RFC PATCH 0/4] unify GCC command line options

2015-08-07 Thread Ard Biesheuvel
This unifies all command line option defines in tools_def.txt, in order
to reduce the maintenance burden.

Note that this does not add or remove any GCC4x toolchains, it just folds
the common DEFINEs into a single series of GCC4X defines.

Ard Biesheuvel (4):
  BaseTools GCC: remove 4.9 specific linker alignment override
  BaseTools GCC: unify warning flags for all GCC versions
  BaseTools GCC: unify ARM CC flags for all GCC versions
  BaseTools GCC: unify GCC toolchain command line options

 BaseTools/Conf/tools_def.template | 306 
 1 file changed, 117 insertions(+), 189 deletions(-)

-- 
1.9.1

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel