Re: [edk2] [Patch] BaseTools: process the files by the priority in BUILDRULEORDER

2016-01-18 Thread Laszlo Ersek 
Hello Zhu Yonghong,

On 01/18/16 08:36, Yonghong Zhu wrote:
> By the BUILDRULEORDER feature to process files listed in INF [Sources]
> sections in priority order, if a filename is listed with multiple
> extensions, the tools will use only the file that matches the first
> extension in the space separated list.
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Yonghong Zhu 
> ---
>  BaseTools/Source/Python/AutoGen/AutoGen.py | 29 -
>  1 file changed, 28 insertions(+), 1 deletion(-)

does this version address the issue that we found in the following threads?

http://thread.gmane.org/gmane.comp.bios.edk2.devel/5192/focus=5492
http://thread.gmane.org/gmane.comp.bios.edk2.devel/5485/focus=5501

Thanks!
Laszlo


> 
> diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py 
> b/BaseTools/Source/Python/AutoGen/AutoGen.py
> index abac477..e9f4888 100644
> --- a/BaseTools/Source/Python/AutoGen/AutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
> @@ -1,9 +1,9 @@
>  ## @file
>  # Generate AutoGen.h, AutoGen.c and *.depex files
>  #
> -# Copyright (c) 2007 - 2015, Intel Corporation. All rights reserved.
> +# Copyright (c) 2007 - 2016, Intel Corporation. All rights reserved.
>  # This program and the accompanying materials
>  # are licensed and made available under the terms and conditions of the BSD 
> License
>  # which accompanies this distribution.  The full text of the license may be 
> found at
>  # http://opensource.org/licenses/bsd-license.php
>  #
> @@ -2739,13 +2739,40 @@ class ModuleAutoGen(AutoGen):
>  
>  # add the file path into search path list for file including
>  if F.Dir not in self.IncludePathList and self.AutoGenVersion 
> >= 0x00010005:
>  self.IncludePathList.insert(0, F.Dir)
>  self._SourceFileList.append(F)
> +
> +self._MatchBuildRuleOrder(self._SourceFileList)
> +
> +for F in self._SourceFileList:
>  self._ApplyBuildRule(F, TAB_UNKNOWN_FILE)
>  return self._SourceFileList
>  
> +def _MatchBuildRuleOrder(self, FileList):
> +Order_Dict = {}
> +self._GetModuleBuildOption()
> +for SingleFile in FileList:
> +if self.BuildRuleOrder and SingleFile.Ext in self.BuildRuleOrder 
> and SingleFile.Ext in self.BuildRules:
> +key = SingleFile.Path.split(SingleFile.Ext)[0]
> +if key in Order_Dict:
> +Order_Dict[key].append(SingleFile.Ext)
> +else:
> +Order_Dict[key] = [SingleFile.Ext]
> +
> +RemoveList = []
> +for F in Order_Dict:
> +if len(Order_Dict[F]) > 1:
> +Order_Dict[F].sort(key=lambda i: 
> self.BuildRuleOrder.index(i))
> +for Ext in Order_Dict[F][1:]:
> +RemoveList.append(F + Ext)
> +   
> +for item in RemoveList:
> +FileList.remove(item)
> +
> +return FileList
> +
>  ## Return the list of unicode files
>  def _GetUnicodeFileList(self):
>  if self._UnicodeFileList == None:
>  if TAB_UNICODE_FILE in self.FileTypes:
>  self._UnicodeFileList = self.FileTypes[TAB_UNICODE_FILE]
> 

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [Patch] BaseTools: process the files by the priority in BUILDRULEORDER

2016-01-18 Thread Gao, Liming 
Reviewed-by: Liming Gao 

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of Yonghong 
Zhu
Sent: Monday, January 18, 2016 3:36 PM
To: edk2-devel@lists.01.org
Subject: [edk2] [Patch] BaseTools: process the files by the priority in 
BUILDRULEORDER

By the BUILDRULEORDER feature to process files listed in INF [Sources] sections 
in priority order, if a filename is listed with multiple extensions, the tools 
will use only the file that matches the first extension in the space separated 
list.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu 
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 29 -
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py 
b/BaseTools/Source/Python/AutoGen/AutoGen.py
index abac477..e9f4888 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -1,9 +1,9 @@
 ## @file
 # Generate AutoGen.h, AutoGen.c and *.depex files  # -# Copyright (c) 2007 - 
2015, Intel Corporation. All rights reserved.
+# Copyright (c) 2007 - 2016, Intel Corporation. All rights 
+reserved.
 # This program and the accompanying materials  # are licensed and made 
available under the terms and conditions of the BSD License  # which 
accompanies this distribution.  The full text of the license may be found at  # 
http://opensource.org/licenses/bsd-license.php
 #
@@ -2739,13 +2739,40 @@ class ModuleAutoGen(AutoGen):
 
 # add the file path into search path list for file including
 if F.Dir not in self.IncludePathList and self.AutoGenVersion 
>= 0x00010005:
 self.IncludePathList.insert(0, F.Dir)
 self._SourceFileList.append(F)
+
+self._MatchBuildRuleOrder(self._SourceFileList)
+
+for F in self._SourceFileList:
 self._ApplyBuildRule(F, TAB_UNKNOWN_FILE)
 return self._SourceFileList
 
+def _MatchBuildRuleOrder(self, FileList):
+Order_Dict = {}
+self._GetModuleBuildOption()
+for SingleFile in FileList:
+if self.BuildRuleOrder and SingleFile.Ext in self.BuildRuleOrder 
and SingleFile.Ext in self.BuildRules:
+key = SingleFile.Path.split(SingleFile.Ext)[0]
+if key in Order_Dict:
+Order_Dict[key].append(SingleFile.Ext)
+else:
+Order_Dict[key] = [SingleFile.Ext]
+
+RemoveList = []
+for F in Order_Dict:
+if len(Order_Dict[F]) > 1:
+Order_Dict[F].sort(key=lambda i: self.BuildRuleOrder.index(i))
+for Ext in Order_Dict[F][1:]:
+RemoveList.append(F + Ext)
+   
+for item in RemoveList:
+FileList.remove(item)
+
+return FileList
+
 ## Return the list of unicode files
 def _GetUnicodeFileList(self):
 if self._UnicodeFileList == None:
 if TAB_UNICODE_FILE in self.FileTypes:
 self._UnicodeFileList = self.FileTypes[TAB_UNICODE_FILE]
--
2.6.1.windows.1

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [Patch] BaseTools: process the files by the priority in BUILDRULEORDER

2016-01-17 Thread Yonghong Zhu 
By the BUILDRULEORDER feature to process files listed in INF [Sources]
sections in priority order, if a filename is listed with multiple
extensions, the tools will use only the file that matches the first
extension in the space separated list.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu 
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 29 -
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py 
b/BaseTools/Source/Python/AutoGen/AutoGen.py
index abac477..e9f4888 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -1,9 +1,9 @@
 ## @file
 # Generate AutoGen.h, AutoGen.c and *.depex files
 #
-# Copyright (c) 2007 - 2015, Intel Corporation. All rights reserved.
+# Copyright (c) 2007 - 2016, Intel Corporation. All rights reserved.
 # This program and the accompanying materials
 # are licensed and made available under the terms and conditions of the BSD 
License
 # which accompanies this distribution.  The full text of the license may be 
found at
 # http://opensource.org/licenses/bsd-license.php
 #
@@ -2739,13 +2739,40 @@ class ModuleAutoGen(AutoGen):
 
 # add the file path into search path list for file including
 if F.Dir not in self.IncludePathList and self.AutoGenVersion 
>= 0x00010005:
 self.IncludePathList.insert(0, F.Dir)
 self._SourceFileList.append(F)
+
+self._MatchBuildRuleOrder(self._SourceFileList)
+
+for F in self._SourceFileList:
 self._ApplyBuildRule(F, TAB_UNKNOWN_FILE)
 return self._SourceFileList
 
+def _MatchBuildRuleOrder(self, FileList):
+Order_Dict = {}
+self._GetModuleBuildOption()
+for SingleFile in FileList:
+if self.BuildRuleOrder and SingleFile.Ext in self.BuildRuleOrder 
and SingleFile.Ext in self.BuildRules:
+key = SingleFile.Path.split(SingleFile.Ext)[0]
+if key in Order_Dict:
+Order_Dict[key].append(SingleFile.Ext)
+else:
+Order_Dict[key] = [SingleFile.Ext]
+
+RemoveList = []
+for F in Order_Dict:
+if len(Order_Dict[F]) > 1:
+Order_Dict[F].sort(key=lambda i: self.BuildRuleOrder.index(i))
+for Ext in Order_Dict[F][1:]:
+RemoveList.append(F + Ext)
+   
+for item in RemoveList:
+FileList.remove(item)
+
+return FileList
+
 ## Return the list of unicode files
 def _GetUnicodeFileList(self):
 if self._UnicodeFileList == None:
 if TAB_UNICODE_FILE in self.FileTypes:
 self._UnicodeFileList = self.FileTypes[TAB_UNICODE_FILE]
-- 
2.6.1.windows.1

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [Patch] BaseTools: process the files by the priority in BUILDRULEORDER

2015-12-07 Thread Laszlo Ersek 
On 12/02/15 09:45, Yonghong Zhu wrote:
> By the BUILDRULEORDER feature to process files listed in INF [Sources]
> sections in priority order, if a filename is listed with multiple
> extensions, the tools will use only the file that matches the first
> extension in the space separated list.
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Yonghong Zhu 
> ---
>  Source/Python/AutoGen/AutoGen.py | 27 +++
>  1 file changed, 27 insertions(+)
> 
> diff --git a/Source/Python/AutoGen/AutoGen.py 
> b/Source/Python/AutoGen/AutoGen.py
> index fe56574..263224b 100644
> --- a/Source/Python/AutoGen/AutoGen.py
> +++ b/Source/Python/AutoGen/AutoGen.py
> @@ -2711,13 +2711,40 @@ class ModuleAutoGen(AutoGen):
>  
>  # add the file path into search path list for file including
>  if F.Dir not in self.IncludePathList and self.AutoGenVersion 
> >= 0x00010005:
>  self.IncludePathList.insert(0, F.Dir)
>  self._SourceFileList.append(F)
> +
> +self._MatchBuildRuleOrder(self._SourceFileList)
> +
> +for F in self._SourceFileList:
>  self._ApplyBuildRule(F, TAB_UNKNOWN_FILE)
>  return self._SourceFileList
>  
> +def _MatchBuildRuleOrder(self, FileList):
> +Order_Dict = {}
> +self._GetModuleBuildOption()
> +for SingleFile in FileList:
> +if self.BuildRuleOrder and SingleFile.Ext in self.BuildRuleOrder:
> +key = SingleFile.Path.split(SingleFile.Ext)[0]
> +if key in Order_Dict:
> +Order_Dict[key].append(SingleFile.Ext)
> +else:
> +Order_Dict[key] = [SingleFile.Ext]
> +
> +RemoveList = []
> +for F in Order_Dict:
> +if len(Order_Dict[F]) > 1:
> +Order_Dict[F].sort(key=lambda i: 
> self.BuildRuleOrder.index(i))
> +for Ext in Order_Dict[F][1:]:
> +RemoveList.append(F + Ext)
> +   
> +for item in RemoveList:
> +FileList.remove(item)
> +
> +return FileList
> +
>  ## Return the list of unicode files
>  def _GetUnicodeFileList(self):
>  if self._UnicodeFileList == None:
>  if TAB_UNICODE_FILE in self.FileTypes:
>  self._UnicodeFileList = self.FileTypes[TAB_UNICODE_FILE]
> 

This patch breaks the OVMF build.

$ build \
-a X64 \
-p OvmfPkg/OvmfPkgX64.dsc \
-t GCC48 \
-n 8 \
--report-file=.../build.ovmf.report \
--log=.../build.ovmf.log \
-b DEBUG \
-D HTTP_BOOT_ENABLE

The result I get is:

Processing meta-data 

build.py...
 : error F000: Nothing to build

[/home/lacos/src/upstream/edk2-git-svn/UefiCpuPkg/Library/BaseUefiCpuLib/BaseUefiCpuLib.inf]


- Failed -

Thanks
Laszlo
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [Patch] BaseTools: process the files by the priority in BUILDRULEORDER

2015-12-07 Thread Jordan Justen 
On 2015-12-07 05:52:42, Laszlo Ersek wrote:
> On 12/02/15 09:45, Yonghong Zhu wrote:
> > By the BUILDRULEORDER feature to process files listed in INF [Sources]
> > sections in priority order, if a filename is listed with multiple
> > extensions, the tools will use only the file that matches the first
> > extension in the space separated list.
> > 
> > Contributed-under: TianoCore Contribution Agreement 1.0
> > Signed-off-by: Yonghong Zhu 
> > ---
> >  Source/Python/AutoGen/AutoGen.py | 27 +++
> >  1 file changed, 27 insertions(+)
> > 
> > diff --git a/Source/Python/AutoGen/AutoGen.py 
> > b/Source/Python/AutoGen/AutoGen.py
> > index fe56574..263224b 100644
> > --- a/Source/Python/AutoGen/AutoGen.py
> > +++ b/Source/Python/AutoGen/AutoGen.py
> > @@ -2711,13 +2711,40 @@ class ModuleAutoGen(AutoGen):
> >  
> >  # add the file path into search path list for file 
> > including
> >  if F.Dir not in self.IncludePathList and 
> > self.AutoGenVersion >= 0x00010005:
> >  self.IncludePathList.insert(0, F.Dir)
> >  self._SourceFileList.append(F)
> > +
> > +self._MatchBuildRuleOrder(self._SourceFileList)
> > +
> > +for F in self._SourceFileList:
> >  self._ApplyBuildRule(F, TAB_UNKNOWN_FILE)
> >  return self._SourceFileList
> >  
> > +def _MatchBuildRuleOrder(self, FileList):
> > +Order_Dict = {}
> > +self._GetModuleBuildOption()
> > +for SingleFile in FileList:
> > +if self.BuildRuleOrder and SingleFile.Ext in 
> > self.BuildRuleOrder:
> > +key = SingleFile.Path.split(SingleFile.Ext)[0]
> > +if key in Order_Dict:
> > +Order_Dict[key].append(SingleFile.Ext)
> > +else:
> > +Order_Dict[key] = [SingleFile.Ext]
> > +
> > +RemoveList = []
> > +for F in Order_Dict:
> > +if len(Order_Dict[F]) > 1:
> > +Order_Dict[F].sort(key=lambda i: 
> > self.BuildRuleOrder.index(i))
> > +for Ext in Order_Dict[F][1:]:
> > +RemoveList.append(F + Ext)
> > +   
> > +for item in RemoveList:
> > +FileList.remove(item)
> > +
> > +return FileList
> > +
> >  ## Return the list of unicode files
> >  def _GetUnicodeFileList(self):
> >  if self._UnicodeFileList == None:
> >  if TAB_UNICODE_FILE in self.FileTypes:
> >  self._UnicodeFileList = self.FileTypes[TAB_UNICODE_FILE]
> > 
> 
> This patch breaks the OVMF build.
> 
> $ build \
> -a X64 \
> -p OvmfPkg/OvmfPkgX64.dsc \
> -t GCC48 \
> -n 8 \
> --report-file=.../build.ovmf.report \
> --log=.../build.ovmf.log \
> -b DEBUG \
> -D HTTP_BOOT_ENABLE
> 
> The result I get is:
> 
> Processing meta-data 
> 
> build.py...
>  : error F000: Nothing to build
> 
> [/home/lacos/src/upstream/edk2-git-svn/UefiCpuPkg/Library/BaseUefiCpuLib/BaseUefiCpuLib.inf]

Yonghong, Liming, can we revert this change until this can be fixed?
Laszlo investigated, and he thinks BUILDRULEORDER is causing a .asm
file to be used for the source on GCC builds. But, .asm should always
be ignored for GCC builds.

-Jordan
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [Patch] BaseTools: process the files by the priority in BUILDRULEORDER

2015-12-07 Thread Laszlo Ersek 
On 12/07/15 18:14, Jordan Justen wrote:
> On 2015-12-07 05:52:42, Laszlo Ersek wrote:
>> On 12/02/15 09:45, Yonghong Zhu wrote:
>>> By the BUILDRULEORDER feature to process files listed in INF [Sources]
>>> sections in priority order, if a filename is listed with multiple
>>> extensions, the tools will use only the file that matches the first
>>> extension in the space separated list.
>>>
>>> Contributed-under: TianoCore Contribution Agreement 1.0
>>> Signed-off-by: Yonghong Zhu 
>>> ---
>>>  Source/Python/AutoGen/AutoGen.py | 27 +++
>>>  1 file changed, 27 insertions(+)
>>>
>>> diff --git a/Source/Python/AutoGen/AutoGen.py 
>>> b/Source/Python/AutoGen/AutoGen.py
>>> index fe56574..263224b 100644
>>> --- a/Source/Python/AutoGen/AutoGen.py
>>> +++ b/Source/Python/AutoGen/AutoGen.py
>>> @@ -2711,13 +2711,40 @@ class ModuleAutoGen(AutoGen):
>>>  
>>>  # add the file path into search path list for file 
>>> including
>>>  if F.Dir not in self.IncludePathList and 
>>> self.AutoGenVersion >= 0x00010005:
>>>  self.IncludePathList.insert(0, F.Dir)
>>>  self._SourceFileList.append(F)
>>> +
>>> +self._MatchBuildRuleOrder(self._SourceFileList)
>>> +
>>> +for F in self._SourceFileList:
>>>  self._ApplyBuildRule(F, TAB_UNKNOWN_FILE)
>>>  return self._SourceFileList
>>>  
>>> +def _MatchBuildRuleOrder(self, FileList):
>>> +Order_Dict = {}
>>> +self._GetModuleBuildOption()
>>> +for SingleFile in FileList:
>>> +if self.BuildRuleOrder and SingleFile.Ext in 
>>> self.BuildRuleOrder:
>>> +key = SingleFile.Path.split(SingleFile.Ext)[0]
>>> +if key in Order_Dict:
>>> +Order_Dict[key].append(SingleFile.Ext)
>>> +else:
>>> +Order_Dict[key] = [SingleFile.Ext]
>>> +
>>> +RemoveList = []
>>> +for F in Order_Dict:
>>> +if len(Order_Dict[F]) > 1:
>>> +Order_Dict[F].sort(key=lambda i: 
>>> self.BuildRuleOrder.index(i))
>>> +for Ext in Order_Dict[F][1:]:
>>> +RemoveList.append(F + Ext)
>>> +   
>>> +for item in RemoveList:
>>> +FileList.remove(item)
>>> +
>>> +return FileList
>>> +
>>>  ## Return the list of unicode files
>>>  def _GetUnicodeFileList(self):
>>>  if self._UnicodeFileList == None:
>>>  if TAB_UNICODE_FILE in self.FileTypes:
>>>  self._UnicodeFileList = self.FileTypes[TAB_UNICODE_FILE]
>>>
>>
>> This patch breaks the OVMF build.
>>
>> $ build \
>> -a X64 \
>> -p OvmfPkg/OvmfPkgX64.dsc \
>> -t GCC48 \
>> -n 8 \
>> --report-file=.../build.ovmf.report \
>> --log=.../build.ovmf.log \
>> -b DEBUG \
>> -D HTTP_BOOT_ENABLE
>>
>> The result I get is:
>>
>> Processing meta-data 
>>
>> build.py...
>>  : error F000: Nothing to build
>>
>> [/home/lacos/src/upstream/edk2-git-svn/UefiCpuPkg/Library/BaseUefiCpuLib/BaseUefiCpuLib.inf]
> 
> Yonghong, Liming, can we revert this change until this can be fixed?
> Laszlo investigated, and he thinks BUILDRULEORDER is causing a .asm
> file to be used for the source on GCC builds.

More precisely, I'm fully certain that this patch causes the regression,
because I bisected it. :)

The matter of opinion part is how the problem should be fixed. (And I
tend to agree with your analysis there:)

> But, .asm should always
> be ignored for GCC builds.

Thanks
Laszlo

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

[edk2] [Patch] BaseTools: process the files by the priority in BUILDRULEORDER

2015-12-02 Thread Yonghong Zhu 
By the BUILDRULEORDER feature to process files listed in INF [Sources]
sections in priority order, if a filename is listed with multiple
extensions, the tools will use only the file that matches the first
extension in the space separated list.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu 
---
 Source/Python/AutoGen/AutoGen.py | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/Source/Python/AutoGen/AutoGen.py b/Source/Python/AutoGen/AutoGen.py
index fe56574..263224b 100644
--- a/Source/Python/AutoGen/AutoGen.py
+++ b/Source/Python/AutoGen/AutoGen.py
@@ -2711,13 +2711,40 @@ class ModuleAutoGen(AutoGen):
 
 # add the file path into search path list for file including
 if F.Dir not in self.IncludePathList and self.AutoGenVersion 
>= 0x00010005:
 self.IncludePathList.insert(0, F.Dir)
 self._SourceFileList.append(F)
+
+self._MatchBuildRuleOrder(self._SourceFileList)
+
+for F in self._SourceFileList:
 self._ApplyBuildRule(F, TAB_UNKNOWN_FILE)
 return self._SourceFileList
 
+def _MatchBuildRuleOrder(self, FileList):
+Order_Dict = {}
+self._GetModuleBuildOption()
+for SingleFile in FileList:
+if self.BuildRuleOrder and SingleFile.Ext in self.BuildRuleOrder:
+key = SingleFile.Path.split(SingleFile.Ext)[0]
+if key in Order_Dict:
+Order_Dict[key].append(SingleFile.Ext)
+else:
+Order_Dict[key] = [SingleFile.Ext]
+
+RemoveList = []
+for F in Order_Dict:
+if len(Order_Dict[F]) > 1:
+Order_Dict[F].sort(key=lambda i: self.BuildRuleOrder.index(i))
+for Ext in Order_Dict[F][1:]:
+RemoveList.append(F + Ext)
+   
+for item in RemoveList:
+FileList.remove(item)
+
+return FileList
+
 ## Return the list of unicode files
 def _GetUnicodeFileList(self):
 if self._UnicodeFileList == None:
 if TAB_UNICODE_FILE in self.FileTypes:
 self._UnicodeFileList = self.FileTypes[TAB_UNICODE_FILE]
-- 
2.6.1.windows.1

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel