Re: [edk2] [edk2-announce] Soft Feature Freeze starts today for edk2-stable201903

2019-02-23 Thread Gao, Liming
Lazlo:
  I agree. I don't think there is the rule break for the patch set of boot 
progress reporting. I also prefer to push them early next week. 

Thanks
Liming
> -Original Message-
> From: Laszlo Ersek [mailto:ler...@redhat.com]
> Sent: Saturday, February 23, 2019 4:20 AM
> To: Gao, Liming ; edk2-devel@lists.01.org
> Subject: Re: [edk2] [edk2-announce] Soft Feature Freeze starts today for 
> edk2-stable201903
> 
> Hi Liming,
> 
> On 02/22/19 15:25, Gao, Liming wrote:
> > Hi, all
> >   
> > https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning
> >  lists edk2-stable201903 tag planning. Now, we enter
> into Soft Feature Freeze phase. In this phase, the feature without 
> Reviewed-by or Acked-by tags will be delayed after the upcoming stable
> tag. The patch review can continue without break. Below is edk2-stable201903 
> tag planning.
> >
> > 2019-03-08 Beginning of development
> > 2019-02-22 Soft Feature Freeze
> > 2019-03-01 Hard Feature Freeze
> > 2019-03-08 Release
> 
> I got the last missing R-b from Ray for my patch set
> 
> [edk2] [PATCH v3 0/5] MdeModulePkg, OvmfPkg, ArmVirtPkg: more visible
> boot progress reporting
> 
> today (2019-02-22) at 12:50 in my time zone (CET = UTC+01:00). Before
> pushing the series, I asked Ray to confirm one last thing about his
> review. I consider the series properly reviewed, but due to this last
> question, I expect I should push it early next week.
> 
> IMO that satisfies the soft feature freeze definition:
> 
> https://github.com/tianocore/tianocore.github.io/wiki/SoftFeatureFreeze
> 
> "By the date of the soft feature freeze, developers must have sent their
> patches to the mailing list and received positive maintainer reviews
> (Reviewed-by or Acked-by tags). [...] Between the soft feature freeze
> and the hard feature freeze, previously reviewed and unit-tested
> features may be applied (or merged) to the master branch, for
> integration testing."
> 
> Do you (and others) agree?
> 
> If not, I won't insist -- while I'd certainly prefer these patches to be
> in the upcoming stable tag, if the community perceives it would mean
> bending the rules, I wouldn't want to do that. (Right now I don't think
> it would bend the rules).
> 
> Thanks,
> Laszlo
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [Patch V2] BaseTools: Add parameter check for the AsciiStringToUint64

2019-02-23 Thread Gao, Liming
Reviewed-by: Liming Gao 

> -Original Message-
> From: Feng, Bob C
> Sent: Saturday, February 23, 2019 5:42 PM
> To: edk2-devel@lists.01.org
> Cc: Feng, Bob C ; Gao, Liming 
> Subject: [Patch V2] BaseTools: Add parameter check for the AsciiStringToUint64
> 
> If the input parameter AsciiString length is greater
> than 255, the GenFv will hang.
> 
> This patch is to fix this issue.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Bob Feng 
> Cc: Liming Gao 
> ---
>  BaseTools/Source/C/Common/ParseInf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/BaseTools/Source/C/Common/ParseInf.c 
> b/BaseTools/Source/C/Common/ParseInf.c
> index 3907f44331..c5d79c89fa 100644
> --- a/BaseTools/Source/C/Common/ParseInf.c
> +++ b/BaseTools/Source/C/Common/ParseInf.c
> @@ -506,11 +506,11 @@ Returns:
>Index = 0;
> 
>//
>// Check input parameter
>//
> -  if (AsciiString == NULL || ReturnValue == NULL) {
> +  if (AsciiString == NULL || ReturnValue == NULL || strlen(AsciiString) > 
> 0xFF) {
>  return EFI_INVALID_PARAMETER;
>}
>while (AsciiString[Index] == ' ') {
>  Index ++;
>}
> --
> 2.18.0.windows.1

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [Patch V2] BaseTool: Fixed incremental rebuild issue.

2019-02-23 Thread Ard Biesheuvel
On Sat, 23 Feb 2019 at 11:14, Marcin Wojtas  wrote:
>
> Hi Laszlo,
>
> pt., 22 lut 2019 o 21:08 Laszlo Ersek  napisał(a):
> >
> > On 02/22/19 20:21, Ard Biesheuvel wrote:
> > > On Fri, 22 Feb 2019 at 08:26, Gao, Liming  wrote:
> > >>
> > >> Reviewed-by: Liming Gao 
> > >>
> > >
> > > Incremental builds are still broken for me, even with this patch. Is
> > > anyone else seeing the same?
> >
> > I am not; but I've been using Python3.4 for a while now (it is needed by
> > another package on my system, and then the build tools find it too).
> >
> > The regression is attributed to commit d943b0c339fe ("BaseTools: Handle
> > the bytes and str difference", 2019-02-01), which I believe is related
> > to the python2/3 conversion. So I assume python3.4 masks the problem.
> >
>
> In my setup with the latest BaseTools update this problem is gone. I'm
> using Python 3.5.2.
>

Thanks for confirming.

I did a full clean of my workspace, and now things are working for me
again as well.

Thanks,
___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [Patch V2] BaseTool: Fixed incremental rebuild issue.

2019-02-23 Thread Marcin Wojtas
Hi Laszlo,

pt., 22 lut 2019 o 21:08 Laszlo Ersek  napisał(a):
>
> On 02/22/19 20:21, Ard Biesheuvel wrote:
> > On Fri, 22 Feb 2019 at 08:26, Gao, Liming  wrote:
> >>
> >> Reviewed-by: Liming Gao 
> >>
> >
> > Incremental builds are still broken for me, even with this patch. Is
> > anyone else seeing the same?
>
> I am not; but I've been using Python3.4 for a while now (it is needed by
> another package on my system, and then the build tools find it too).
>
> The regression is attributed to commit d943b0c339fe ("BaseTools: Handle
> the bytes and str difference", 2019-02-01), which I believe is related
> to the python2/3 conversion. So I assume python3.4 masks the problem.
>

In my setup with the latest BaseTools update this problem is gone. I'm
using Python 3.5.2.

Thanks,
Marcin

>
> >>> -Original Message-
> >>> From: Feng, Bob C
> >>> Sent: Wednesday, February 20, 2019 11:22 PM
> >>> To: edk2-devel@lists.01.org
> >>> Cc: Feng, Bob C ; Gao, Liming 
> >>> Subject: [Patch V2] BaseTool: Fixed incremental rebuild issue.
> >>>
> >>> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1540
> >>>
> >>> This issue in introduced by commit
> >>> d943b0c339fe3d35ffdf9f580ccb7a55915c6854
> >>>
> >>> To convert bytes to string, we need to use bytes.decode()
> >>> instead of using str(bytes).
> >>>
> >>> If the source file is not a txt file, ignore that file.
> >>>
> >>> Contributed-under: TianoCore Contribution Agreement 1.1
> >>> Signed-off-by: Bob Feng 
> >>> Cc: Liming Gao 
> >>> ---
> >>> BaseTools/Source/Python/AutoGen/GenMake.py  | 16 
> >>> .../Source/Python/Workspace/DscBuildData.py | 17 -
> >>> 2 files changed, 16 insertions(+), 17 deletions(-)
> >>>
> >>> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> >>> b/BaseTools/Source/Python/AutoGen/GenMake.py
> >>> index 53c5b8577d..b441817b52 100644
> >>> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> >>> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> >>> @@ -1043,18 +1043,18 @@ cleanlib:
> >>> Fd.close()
> >>> except BaseException as X:
> >>> EdkLogger.error("build", FILE_OPEN_FAILURE, 
> >>> ExtraData=F.Path +
> >>> "\n\t" + str(X))
> >>> if len(FileContent) == 0:
> >>> continue
> >>> -
> >>> -if FileContent[0] == 0xff or FileContent[0] == 0xfe:
> >>> -FileContent = FileContent.decode('utf-16')
> >>> -else:
> >>> -try:
> >>> -FileContent = str(FileContent)
> >>> -except:
> >>> -pass
> >>> +try:
> >>> +if FileContent[0] == 0xff or FileContent[0] == 0xfe:
> >>> +FileContent = FileContent.decode('utf-16')
> >>> +else:
> >>> +FileContent = FileContent.decode()
> >>> +except:
> >>> +# The file is not txt file. for example .mcb file
> >>> +continue
> >>> IncludedFileList = gIncludePattern.findall(FileContent)
> >>>
> >>> for Inc in IncludedFileList:
> >>> Inc = Inc.strip()
> >>> # if there's macro used to reference header file, 
> >>> expand it
> >>> diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py
> >>> b/BaseTools/Source/Python/Workspace/DscBuildData.py
> >>> index 1ffefe6e7e..7221946062 100644
> >>> --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> >>> +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> >>> @@ -153,19 +153,18 @@ def GetDependencyList(FileStack, SearchPathList):
> >>> Fd.close()
> >>>
> >>> if len(FileContent) == 0:
> >>> continue
> >>>
> >>> -if FileContent[0] == 0xff or FileContent[0] == 0xfe:
> >>> -FileContent = FileContent.decode('utf-16')
> >>> -IncludedFileList = gIncludePattern.findall(FileContent)
> >>> -else:
> >>> -try:
> >>> -FileContent = str(FileContent)
> >>> -IncludedFileList = 
> >>> gIncludePattern.findall(FileContent)
> >>> -except:
> >>> -pass
> >>> +try:
> >>> +if FileContent[0] == 0xff or FileContent[0] == 0xfe:
> >>> +FileContent = FileContent.decode('utf-16')
> >>> +else:
> >>> +FileContent = FileContent.decode()
> >>> +except:
> >>> +# The file is not txt file. for example .mcb file
> >>> +continue
> >>> IncludedFileList = gIncludePattern.findall(FileContent)
> >>>
> >>> for Inc in IncludedFileList:
> >>> Inc = Inc.strip()
> >>> Inc = os.path.normpath(Inc)
> >>> --
> >>> 2.18.0.windows.1
> >>
> >> __

[edk2] [Patch V2] BaseTools: Add parameter check for the AsciiStringToUint64

2019-02-23 Thread Feng, Bob C
If the input parameter AsciiString length is greater
than 255, the GenFv will hang.

This patch is to fix this issue.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng 
Cc: Liming Gao 
---
 BaseTools/Source/C/Common/ParseInf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/C/Common/ParseInf.c 
b/BaseTools/Source/C/Common/ParseInf.c
index 3907f44331..c5d79c89fa 100644
--- a/BaseTools/Source/C/Common/ParseInf.c
+++ b/BaseTools/Source/C/Common/ParseInf.c
@@ -506,11 +506,11 @@ Returns:
   Index = 0;
 
   //
   // Check input parameter
   //
-  if (AsciiString == NULL || ReturnValue == NULL) {
+  if (AsciiString == NULL || ReturnValue == NULL || strlen(AsciiString) > 
0xFF) {
 return EFI_INVALID_PARAMETER;
   }
   while (AsciiString[Index] == ' ') {
 Index ++;
   }
-- 
2.18.0.windows.1

___
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


Re: [edk2] [Patch] BaseTools: Enable component override functionality

2019-02-23 Thread Feng, Bob C
Felix,

I agree. I filed a new  BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1557 
to track the document update.
I think this new behavior would be better to be recorded in the Build spec.

Thanks,
Bob
-Original Message-
From: Felix Polyudov [mailto:fel...@ami.com] 
Sent: Saturday, February 23, 2019 6:06 AM
To: Feng, Bob C ; edk2-devel@lists.01.org
Cc: Carsey, Jaben ; Gao, Liming 
Subject: RE: [edk2] [Patch] BaseTools: Enable component override functionality

Bob,

Do you know if DSC specification has been updated to document this new behavior?
If not, it should be updated.

-Original Message-
From: edk2-devel [mailto:edk2-devel-boun...@lists.01.org] On Behalf Of BobCF
Sent: Thursday, January 10, 2019 9:39 PM
To: edk2-devel@lists.01.org
Cc: Carsey Jaben; Liming Gao
Subject: [edk2] [Patch] BaseTools: Enable component override functionality

https://bugzilla.tianocore.org/show_bug.cgi?id=1449
This patch enable build tools to recognize that when two given files have the 
same GUID, file path and ARCH in Dsc, The later one's definition will be used.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng 
Cc: Liming Gao 
Cc: Carsey Jaben 
---
 .../Source/Python/Workspace/DscBuildData.py   | 24 ---
 .../Source/Python/Workspace/MetaFileParser.py |  5   
.../Source/Python/Workspace/MetaFileTable.py  |  7 --
 3 files changed, 25 insertions(+), 11 deletions(-)

diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py 
b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 7e82e8e934..f9805f58f5 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -704,36 +704,44 @@ class DscBuildData(PlatformBuildClassObject):
 if TAB_DEFAULT_STORES_DEFAULT not in self.DefaultStores:
 self.DefaultStores[TAB_DEFAULT_STORES_DEFAULT] = (0, 
TAB_DEFAULT_STORES_DEFAULT)
 GlobalData.gDefaultStores = sorted(self.DefaultStores.keys())
 return self.DefaultStores
 
+def OverrideDuplicateModule(self):
+RecordList = self._RawData[MODEL_META_DATA_COMPONENT, self._Arch]
+Macros = self._Macros
+Macros["EDK_SOURCE"] = GlobalData.gEcpSource
+Components = {}
+for Record in RecordList:
+ModuleId = Record[6]
+file_guid = self._RawData[MODEL_META_DATA_HEADER, self._Arch, 
None, ModuleId]
+file_guid_str = file_guid[0][2] if file_guid else "NULL"
+ModuleFile = PathClass(NormPath(Record[0], Macros), 
GlobalData.gWorkspace, Arch=self._Arch)
+if self._Arch != TAB_ARCH_COMMON and 
(file_guid_str,str(ModuleFile)) in Components:
+
self._RawData.DisableOverrideComponent(Components[(file_guid_str,str(ModuleFile))])
+Components[(file_guid_str,str(ModuleFile))] = ModuleId
+self._RawData._PostProcessed = False
 ## Retrieve [Components] section information
 @property
 def Modules(self):
 if self._Modules is not None:
 return self._Modules
-
+self.OverrideDuplicateModule()
 self._Modules = OrderedDict()
 RecordList = self._RawData[MODEL_META_DATA_COMPONENT, self._Arch]
 Macros = self._Macros
 Macros["EDK_SOURCE"] = GlobalData.gEcpSource
 for Record in RecordList:
-DuplicatedFile = False
-
 ModuleFile = PathClass(NormPath(Record[0], Macros), 
GlobalData.gWorkspace, Arch=self._Arch)
 ModuleId = Record[6]
 LineNo = Record[7]
 
 # check the file validation
 ErrorCode, ErrorInfo = ModuleFile.Validate('.inf')
 if ErrorCode != 0:
 EdkLogger.error('build', ErrorCode, File=self.MetaFile, 
Line=LineNo,
 ExtraData=ErrorInfo)
-# Check duplication
-# If arch is COMMON, no duplicate module is checked since all 
modules in all component sections are selected
-if self._Arch != TAB_ARCH_COMMON and ModuleFile in self._Modules:
-DuplicatedFile = True
 
 Module = ModuleBuildClassObject()
 Module.MetaFile = ModuleFile
 
 # get module private library instance @@ -792,12 +800,10 @@ class 
DscBuildData(PlatformBuildClassObject):
 else:
 OptionString = Module.BuildOptions[ToolChainFamily, 
ToolChain]
 Module.BuildOptions[ToolChainFamily, ToolChain] = 
OptionString + " " + Option
 
 RecordList = self._RawData[MODEL_META_DATA_HEADER, self._Arch, 
None, ModuleId]
-if DuplicatedFile and not RecordList:
-EdkLogger.error('build', FILE_DUPLICATED, File=self.MetaFile, 
ExtraData=str(ModuleFile), Line=LineNo)
 if RecordList:
 if len(RecordList) != 1:
 EdkLogger.error('build', OPTION_UNKNOWN, 'Only FILE_GUID 
can be listed in  section.