Re: [edk2] [PATCH] BaseTools: Add support for nested !include in FDF and DSC files Also added code to detect include loops and enhanced error reporting for included files

2015-08-24 Thread El-Haj-Mahmoud, Samer 
Reviewed-by: Samer El-Haj-Mahmoud mailto:[email protected]>>


From: Sheng, Cecil (HPS SW)
Sent: Tuesday, August 18, 2015 1:14 AM
To: [email protected]
Cc: El-Haj-Mahmoud, Samer ; Liu, Yingke D 

Subject: [edk2] [PATCH] BaseTools: Add support for nested !include in FDF and 
DSC files Also added code to detect include loops and enhanced error reporting 
for included files

Hi,

Sorry for the delayed update. Please review the attachment for changes to 
support nested include in DSC and FDF files. The performance impact in previous 
patch has been addressed.




Sincerely,

Cecil Sheng
ISS Firmware Development
HP Servers

--
___
edk2-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/edk2-devel

Re: [edk2] [PATCH] BaseTools: Add support for nested !include in FDF and DSC files Also added code to detect include loops and enhanced error reporting for included files

2015-07-13 Thread El-Haj-Mahmoud, Samer 
Our regression has found some issues with this patch. Will submit a new one 
soon.

Thanks,
--Samer


-Original Message-
From: Jordan Justen [mailto:[email protected]] 
Sent: Saturday, July 11, 2015 8:56 PM
To: El-Haj-Mahmoud, Samer; [email protected]
Cc: El-Haj-Mahmoud, Samer
Subject: Re: [edk2] [PATCH] BaseTools: Add support for nested !include in FDF 
and DSC files Also added code to detect include loops and enhanced error 
reporting for included files

Your subject line is too long. (151 characters)

BaseTools/Contributions.txt

https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format

Should this be 2 patches? ('Also added...')

On 2015-07-11 07:00:48, Samer El-Haj-Mahmoud wrote:
> From: Cecil Sheng 
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Samer El-Haj-Mahmoud 

Consider Cc'ing the package owner here in the commit message:

Cc: Yingke D Liu 

Package owners are documented in Maintainers.txt.

-Jordan

> ---
>  BaseTools/Source/Python/GenFds/FdfParser.py| 90 
> +++---
>  .../Source/Python/Workspace/MetaFileParser.py  | 12 +--
>  2 files changed, 86 insertions(+), 16 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py 
> b/BaseTools/Source/Python/GenFds/FdfParser.py
> index ffc54ab..e84bdb1 100644
> --- a/BaseTools/Source/Python/GenFds/FdfParser.py
> +++ b/BaseTools/Source/Python/GenFds/FdfParser.py
> @@ -2,6 +2,7 @@
>  # parse FDF file
>  #
>  #  Copyright (c) 2007 - 2014, Intel Corporation. All rights 
> reserved.
> +#  (C) Copyright 2015 Hewlett-Packard Development Company, L.P.
>  #
>  #  This program and the accompanying materials  #  are licensed and 
> made available under the terms and conditions of the BSD License @@ 
> -81,16 +82,31 @@ RegionSizeGuidPattern = 
> re.compile("\s*(?P\w+\.\w+)\s*\|\s*(?P\w+\.\
>  RegionOffsetPcdPattern = re.compile("\s*(?P\w+\.\w+)\s*$")
>  ShortcutPcdPattern = 
> re.compile("\s*\w+\s*=\s*(?P(?:0x|0X)?[a-fA-F0-9]+)\s*\|\s*(?P<
> name>\w+\.\w+)\s*")
>  
> -IncludeFileList = []
> +AllIncludeFileList = []
> +
> +# Get the closest parent
> +def GetParentAtLine (Line):
> +for Profile in AllIncludeFileList:
> +if Profile.IsLineInFile(Line):
> +return Profile
> +return None
> +
> +# Check include loop
> +def IsValidInclude (File, Line):
> +for Profile in AllIncludeFileList:
> +if Profile.IsLineInFile(Line) and Profile.FileName == File:
> +return False
> +
> +return True
>  
>  def GetRealFileLine (File, Line):
>  
>  InsertedLines = 0
> -for Profile in IncludeFileList:
> -if Line >= Profile.InsertStartLineNumber and Line < 
> Profile.InsertStartLineNumber + Profile.InsertAdjust + 
> len(Profile.FileLinesList):
> -return (Profile.FileName, Line - Profile.InsertStartLineNumber + 
> 1)
> -if Line >= Profile.InsertStartLineNumber + Profile.InsertAdjust + 
> len(Profile.FileLinesList):
> -InsertedLines += Profile.InsertAdjust + 
> len(Profile.FileLinesList)
> +for Profile in AllIncludeFileList:
> +if Profile.IsLineInFile(Line):
> +return Profile.GetLineInFile(Line)
> +elif Line >= Profile.InsertStartLineNumber and Profile.Level == 1:
> +   InsertedLines += Profile.GetTotalLines()
>  
>  return (File, Line - InsertedLines)
>  
> @@ -111,6 +127,7 @@ class Warning (Exception):
>  FileLineTuple = GetRealFileLine(File, Line)
>  self.FileName = FileLineTuple[0]
>  self.LineNumber = FileLineTuple[1]
> +self.OriginalLineNumber = Line
>  self.Message = Str
>  self.ToolName = 'FdfParser'
>  
> @@ -157,6 +174,38 @@ class IncludeFileProfile :
>  
>  self.InsertStartLineNumber = None
>  self.InsertAdjust = 0
> +self.IncludeFileList = []
> +self.Level = 1 # first level include file
> +
> +def GetTotalLines(self):
> +TotalLines = self.InsertAdjust + len(self.FileLinesList)
> +
> +for Profile in self.IncludeFileList:
> +  TotalLines += Profile.GetTotalLines()
> +
> +return TotalLines
> +
> +def IsLineInFile(self, Line):
> +if Line >= self.InsertStartLineNumber and Line < 
> self.InsertStartLineNumber + self.GetTotalLines():
> +return True
> +
> +return False
> +
> +def GetLineInFile(self, Line):
> +if not self.IsLineInFile (Line):
> +return (self.FileName, -1)
> +
> +InsertedLines = self.InsertStartLineNumber
> +
> +for Pro

Re: [edk2] [PATCH] BaseTools: Add support for nested !include in FDF and DSC files Also added code to detect include loops and enhanced error reporting for included files

2015-07-11 Thread Jordan Justen 
Your subject line is too long. (151 characters)

BaseTools/Contributions.txt

https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format

Should this be 2 patches? ('Also added...')

On 2015-07-11 07:00:48, Samer El-Haj-Mahmoud wrote:
> From: Cecil Sheng 
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Samer El-Haj-Mahmoud 

Consider Cc'ing the package owner here in the commit message:

Cc: Yingke D Liu 

Package owners are documented in Maintainers.txt.

-Jordan

> ---
>  BaseTools/Source/Python/GenFds/FdfParser.py| 90 
> +++---
>  .../Source/Python/Workspace/MetaFileParser.py  | 12 +--
>  2 files changed, 86 insertions(+), 16 deletions(-)
> 
> diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py 
> b/BaseTools/Source/Python/GenFds/FdfParser.py
> index ffc54ab..e84bdb1 100644
> --- a/BaseTools/Source/Python/GenFds/FdfParser.py
> +++ b/BaseTools/Source/Python/GenFds/FdfParser.py
> @@ -2,6 +2,7 @@
>  # parse FDF file
>  #
>  #  Copyright (c) 2007 - 2014, Intel Corporation. All rights reserved.
> +#  (C) Copyright 2015 Hewlett-Packard Development Company, L.P.
>  #
>  #  This program and the accompanying materials
>  #  are licensed and made available under the terms and conditions of the BSD 
> License
> @@ -81,16 +82,31 @@ RegionSizeGuidPattern = 
> re.compile("\s*(?P\w+\.\w+)\s*\|\s*(?P\w+\.\
>  RegionOffsetPcdPattern = re.compile("\s*(?P\w+\.\w+)\s*$")
>  ShortcutPcdPattern = 
> re.compile("\s*\w+\s*=\s*(?P(?:0x|0X)?[a-fA-F0-9]+)\s*\|\s*(?P\w+\.\w+)\s*")
>  
> -IncludeFileList = []
> +AllIncludeFileList = []
> +
> +# Get the closest parent
> +def GetParentAtLine (Line):
> +for Profile in AllIncludeFileList:
> +if Profile.IsLineInFile(Line):
> +return Profile
> +return None
> +
> +# Check include loop
> +def IsValidInclude (File, Line):
> +for Profile in AllIncludeFileList:
> +if Profile.IsLineInFile(Line) and Profile.FileName == File:
> +return False
> +
> +return True
>  
>  def GetRealFileLine (File, Line):
>  
>  InsertedLines = 0
> -for Profile in IncludeFileList:
> -if Line >= Profile.InsertStartLineNumber and Line < 
> Profile.InsertStartLineNumber + Profile.InsertAdjust + 
> len(Profile.FileLinesList):
> -return (Profile.FileName, Line - Profile.InsertStartLineNumber + 
> 1)
> -if Line >= Profile.InsertStartLineNumber + Profile.InsertAdjust + 
> len(Profile.FileLinesList):
> -InsertedLines += Profile.InsertAdjust + 
> len(Profile.FileLinesList)
> +for Profile in AllIncludeFileList:
> +if Profile.IsLineInFile(Line):
> +return Profile.GetLineInFile(Line)
> +elif Line >= Profile.InsertStartLineNumber and Profile.Level == 1:
> +   InsertedLines += Profile.GetTotalLines()
>  
>  return (File, Line - InsertedLines)
>  
> @@ -111,6 +127,7 @@ class Warning (Exception):
>  FileLineTuple = GetRealFileLine(File, Line)
>  self.FileName = FileLineTuple[0]
>  self.LineNumber = FileLineTuple[1]
> +self.OriginalLineNumber = Line
>  self.Message = Str
>  self.ToolName = 'FdfParser'
>  
> @@ -157,6 +174,38 @@ class IncludeFileProfile :
>  
>  self.InsertStartLineNumber = None
>  self.InsertAdjust = 0
> +self.IncludeFileList = []
> +self.Level = 1 # first level include file
> +
> +def GetTotalLines(self):
> +TotalLines = self.InsertAdjust + len(self.FileLinesList)
> +
> +for Profile in self.IncludeFileList:
> +  TotalLines += Profile.GetTotalLines()
> +
> +return TotalLines
> +
> +def IsLineInFile(self, Line):
> +if Line >= self.InsertStartLineNumber and Line < 
> self.InsertStartLineNumber + self.GetTotalLines():
> +return True
> +
> +return False
> +
> +def GetLineInFile(self, Line):
> +if not self.IsLineInFile (Line):
> +return (self.FileName, -1)
> +
> +InsertedLines = self.InsertStartLineNumber
> +
> +for Profile in self.IncludeFileList:
> +if Profile.IsLineInFile(Line):
> +return Profile.GetLineInFile(Line)
> +elif Line >= Profile.InsertStartLineNumber:
> +InsertedLines += Profile.GetTotalLines()
> +
> +return (self.FileName, Line - InsertedLines + 1)
> +
> +
>  
>  ## The FDF content class that used to record file data when parsing FDF
>  #
> @@ -565,10 +614,12 @@ class FdfParser:
>  #   @param  selfThe object pointer
>  #
>  def PreprocessIncludeFile(self):
> -
> +   # nested include support
> +Processed = False
>  while self.__GetNextToken():
>  
>  if self.__Token == '!include':
> +Processed = True
>  IncludeLine = self.CurrentLineNumber
>  IncludeOffset = self.CurrentOffsetWithinLine - 
> len('!include')
>